From 916b5513cf3cd8a067c418616e0373391b84a189 Mon Sep 17 00:00:00 2001 From: Peter Park Date: Fri, 18 Oct 2024 11:33:36 -0400 Subject: [PATCH 01/74] add radeon pro v710 to gpu arch specs (#192) * Add V710 specs gpg: using RSA key 22223038B47B3ED4B3355AB11B54779B4780494E gpg: Good signature from "Peter Park (MKMPETEPARK01) " [ultimate] add some specs add cols clean up extra line * fix graphics l1 cache description * update SGPR for RDNA2 and RDNA3 archs * update VGPR * Apply suggestions from code review * change l2 cache to 4 * Update docs/reference/gpu-arch-specs.rst --- docs/reference/gpu-arch-specs.rst | 116 +++++++++++++++++------------- 1 file changed, 66 insertions(+), 50 deletions(-) diff --git a/docs/reference/gpu-arch-specs.rst b/docs/reference/gpu-arch-specs.rst index 116649434..90fe32a13 100644 --- a/docs/reference/gpu-arch-specs.rst +++ b/docs/reference/gpu-arch-specs.rst @@ -233,6 +233,22 @@ The following tables provide an overview of the hardware specifications for AMD - L0 Instruction Cache (KiB) - VGPR File (KiB) - SGPR File (KiB) + * + - Radeon PRO V710 + - RDNA3 + - gfx1101 + - 28 + - 54 + - 32 + - 128 + - 56 + - 4 + - 256 + - 32 + - 16 + - 32 + - 768 + - 16 * - Radeon PRO W7900 Dual Slot - RDNA3 @@ -247,8 +263,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon PRO W7900 - RDNA3 @@ -263,8 +279,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon PRO W7800 - RDNA3 @@ -279,8 +295,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon PRO W7700 - RDNA3 @@ -295,8 +311,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon PRO W6800 - RDNA2 @@ -311,8 +327,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon PRO W6600 - RDNA2 @@ -327,8 +343,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon PRO V620 - RDNA2 @@ -343,8 +359,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon Pro W5500 - RDNA @@ -359,7 +375,7 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 + - 512 - 20 * - Radeon Pro VII @@ -414,8 +430,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon RX 7900 XT - RDNA3 @@ -430,8 +446,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon RX 7900 GRE - RDNA3 @@ -446,8 +462,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon RX 7800 XT - RDNA3 @@ -462,8 +478,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon RX 7700 XT - RDNA3 @@ -478,8 +494,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 384 - - 20 + - 768 + - 16 * - Radeon RX 7600 - RDNA3 @@ -494,8 +510,8 @@ The following tables provide an overview of the hardware specifications for AMD - 32 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6950 XT - RDNA2 @@ -510,8 +526,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6900 XT - RDNA2 @@ -526,8 +542,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6800 XT - RDNA2 @@ -542,8 +558,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6800 - RDNA2 @@ -558,8 +574,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6750 XT - RDNA2 @@ -574,8 +590,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6700 XT - RDNA2 @@ -590,8 +606,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6700 - RDNA2 @@ -606,8 +622,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6650 XT - RDNA2 @@ -622,8 +638,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6600 XT - RDNA2 @@ -638,8 +654,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon RX 6600 - RDNA2 @@ -654,8 +670,8 @@ The following tables provide an overview of the hardware specifications for AMD - 16 - 16 - 32 - - 256 - - 20 + - 512 + - 16 * - Radeon VII - GCN5.1 @@ -682,7 +698,7 @@ For more information about the terms used, see the **LLVM target name** -Argument to pass to clang in `--offload-arch` to compile code for the given +Argument to pass to clang in ``--offload-arch`` to compile code for the given architecture. **VRAM** @@ -717,13 +733,13 @@ data and instructions. Similar to the L3 Cache on CDNA/GCN architectures. **L2 Cache** -Size of the level 3 cache. Shared by all compute units on the same GCD. Caches +Size of the level 2 cache. Shared by all compute units on the same GCD. Caches data and instructions. **Graphics L1 Cache (RDNA only)** An additional cache level that only exists in RDNA architectures. Local to a -work group processor. +shader array. **L1 Vector Cache (CDNA/GCN only)** From be19ac1071c3d1aaa1900c53777c0346300f87ae Mon Sep 17 00:00:00 2001 From: Peter Park Date: Fri, 18 Oct 2024 11:39:09 -0400 Subject: [PATCH 02/74] ROCm 6.2.4 compatibility matrix (#186) * prep compat column (historical) and mi300x column * update historical compat matrix for 6.2.4 * update compat matrix for 6.2.4 * fix compat * fix thunk version * fix hipify ver --- .../compatibility-matrix-historical-6.0.csv | 234 +++++++++--------- docs/compatibility/compatibility-matrix.rst | 53 ++-- 2 files changed, 144 insertions(+), 143 deletions(-) diff --git a/docs/compatibility/compatibility-matrix-historical-6.0.csv b/docs/compatibility/compatibility-matrix-historical-6.0.csv index c19f1ed9b..942e173b6 100644 --- a/docs/compatibility/compatibility-matrix-historical-6.0.csv +++ b/docs/compatibility/compatibility-matrix-historical-6.0.csv @@ -1,117 +1,117 @@ -ROCm Version,6.2.2,6.2.1,6.2.0, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0 - :ref:`Operating systems & kernels `,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,, - ,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3, 22.04.2","Ubuntu 22.04.4, 22.04.3, 22.04.2" - ,,,,"Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5" - ,"RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.3, 9.2","RHEL 9.3, 9.2" - ,"RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8" - ,"SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4" - ,,,,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9 - ,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,,, - ,.. _architecture-support-compatibility-matrix-past-60:,,,,,,, - :doc:`Architecture `,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3 - ,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2 - ,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA - ,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3 - ,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2 - ,.. _gpu-support-compatibility-matrix-past-60:,,,,,,, - :doc:`GPU / LLVM target `,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100 - ,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030 - ,gfx942 [#mi300_622-past-60]_,gfx942 [#mi300_621-past-60]_,gfx942 [#mi300_620-past-60]_, gfx942 [#mi300_612-past-60]_, gfx942 [#mi300_611-past-60]_, gfx942 [#mi300_610-past-60]_, gfx942 [#mi300_602-past-60]_, gfx942 [#mi300_600-past-60]_ - ,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a - ,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908 - ,,,,,,,, - FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix-past-60:,,,,,,, - :doc:`PyTorch `,"2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13" - :doc:`TensorFlow `,"2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.14.0, 2.13.1, 2.12.1","2.14.0, 2.13.1, 2.12.1" - :doc:`JAX `,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26 - `ONNX Runtime `_,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.14.1,1.14.1 - ,,,,,,,, - THIRD PARTY COMMS,.. _thirdpartycomms-support-compatibility-matrix-past-60:,,,,,,, - `UCC `_,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.2.0,>=1.2.0 - `UCX `_,>=1.15.0,>=1.15.0,>=1.15.0,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1 - ,,,,,,,, - THIRD PARTY ALGORITHM,.. _thirdpartyalgorithm-support-compatibility-matrix-past-60:,,,,,,, - Thrust,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1 - CUB,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1 - ,,,,,,,, - KFD & USER SPACE [#kfd_support-past-60]_,.. _kfd-userspace-support-compatibility-matrix-past-60:,,,,,,, - Tested user space versions,"6.1.x, 6.0.x","6.1.x, 6.0.x","6.1.x, 6.0.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x, 5.6.x","6.2.x, 6.0.x, 5.7.x, 5.6.x" - ,,,,,,,, - ML & COMPUTER VISION,.. _mllibs-support-compatibility-matrix-past-60:,,,,,,, - :doc:`Composable Kernel `,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0 - :doc:`MIGraphX `,2.10.0,2.10.0,2.10.0,2.9.0,2.9.0,2.9.0,2.8.0,2.8.0 - :doc:`MIOpen `,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 - :doc:`MIVisionX `,3.0.0,3.0.0,3.0.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0 - :doc:`rocAL `,2.0.0,2.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 - :doc:`rocDecode `,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,N/A,N/A - :doc:`rocPyDecode `,0.1.0,0.1.0,0.1.0,N/A,N/A,N/A,N/A,N/A - :doc:`RPP `,1.8.0,1.8.0,1.8.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0 - ,,,,,,,, - COMMUNICATION,.. _commlibs-support-compatibility-matrix-past-60:,,,,,,, - :doc:`RCCL `,2.20.5,2.20.5,2.20.5,2.18.6,2.18.6,2.18.6,2.18.3,2.18.3 - ,,,,,,,, - MATH LIBS,.. _mathlibs-support-compatibility-matrix-past-60:,,,,,,, - `half `_ ,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0 - :doc:`hipBLAS `,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0 - :doc:`hipBLASLt `,0.8.0,0.8.0,0.8.0,0.7.0,0.7.0,0.7.0,0.6.0,0.6.0 - :doc:`hipFFT `,1.0.15,1.0.15,1.0.14,1.0.14,1.0.14,1.0.14,1.0.13,1.0.13 - :doc:`hipFORT `,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0 - :doc:`hipRAND `,2.11.0,2.11.0,2.11.0,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16 - :doc:`hipSOLVER `,2.2.0,2.2.0,2.2.0,2.1.1,2.1.1,2.1.0,2.0.0,2.0.0 - :doc:`hipSPARSE `,3.1.1,3.1.1,3.1.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0 - :doc:`hipSPARSELt `,0.2.1,0.2.1,0.2.1,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0 - :doc:`rocALUTION `,3.2.0,3.2.0,3.2.0,3.1.1,3.1.1,3.1.1,3.0.3,3.0.3 - :doc:`rocBLAS `,4.2.1,4.2.1,4.2.0,4.1.2,4.1.0,4.1.0,4.0.0,4.0.0 - :doc:`rocFFT `,1.0.29,1.0.29,1.0.28,1.0.27,1.0.27,1.0.26,1.0.25,1.0.23 - :doc:`rocRAND `,3.1.0,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.0,2.10.17 - :doc:`rocSOLVER `,3.26.0,3.26.0,3.26.0,3.25.0,3.25.0,3.25.0,3.24.0,3.24.0 - :doc:`rocSPARSE `,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.0.2,3.0.2 - :doc:`rocWMMA `,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0 - `Tensile `_,4.40.0,4.40.0,4.40.0,4.40.0,4.40.0,4.40.0,4.39.0,4.39.0 - ,,,,,,,, - PRIMITIVES,.. _primitivelibs-support-compatibility-matrix-past-60:,,,,,,, - :doc:`hipCUB `,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 - :doc:`hipTensor `,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0 - :doc:`rocPRIM `,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 - :doc:`rocThrust `,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0 - ,,,,,,,, - SUPPORT LIBS,,,,,,,, - `hipother `_,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 - `rocm-core `_,6.2.2,6.2.1,6.2.0,6.1.2,6.1.1,6.1.0,6.0.2,6.0.0 - `ROCT-Thunk-Interface `_,20240607.4.05,20240607.4.05,20240607.1.4246,20240125.5.08,20240125.5.08,20240125.3.30,20231016.2.245,20231016.2.245 - ,,,,,,,, - SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix-past-60:,,,,,,, - :doc:`AMD SMI `,24.6.3,24.6.3,24.6.2,24.5.1,24.5.1,24.4.1,23.4.2,23.4.2 - :doc:`ROCm Data Center Tool `,1.0.0,1.0.0,1.0.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0 - :doc:`rocminfo `,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 - :doc:`ROCm SMI `,7.3.0,7.3.0,7.3.0,7.2.0,7.0.0,7.0.0,6.0.2,6.0.0 - :doc:`ROCm Validation Suite `,rocm-6.2.2,rocm-6.2.1,rocm-6.2.0,rocm-6.1.2,rocm-6.1.1,rocm-6.1.0,rocm-6.0.2,rocm-6.0.0 - ,,,,,,,, - PERFORMANCE TOOLS,,,,,,,, - :doc:`Omniperf `,2.0.1,2.0.1,2.0.1,N/A,N/A,N/A,N/A,N/A - :doc:`Omnitrace `,1.11.2,1.11.2,1.11.2,N/A,N/A,N/A,N/A,N/A - :doc:`ROCm Bandwidth Test `,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0 - :doc:`ROCProfiler `,2.0.60202,2.0.60201,2.0.60200,2.0.60102,2.0.60101,2.0.60100,2.0.60002,2.0.60000 - :doc:`ROCprofiler-SDK `,0.4.0,0.4.0,0.4.0,N/A,N/A,N/A,N/A,N/A - :doc:`ROCTracer `,4.1.60202,4.1.60201,4.1.60200,4.1.60102,4.1.60101,4.1.60100,4.1.60002,4.1.60000 - ,,,,,,,, - DEVELOPMENT TOOLS,,,,,,,, - :doc:`HIPIFY `,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 - :doc:`ROCm CMake `,0.13.0,0.13.0,0.13.0,0.12.0,0.12.0,0.12.0,0.11.0,0.11.0 - :doc:`ROCdbgapi `,0.76.0,0.76.0,0.76.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0 - :doc:`ROCm Debugger (ROCgdb) `,14.2.0,14.2.0,14.2.0,14.1.0,14.1.0,14.1.0,13.2.0,13.2.0 - `rocprofiler-register `_,0.4.0,0.4.0,0.4.0,0.3.0,0.3.0,0.3.0,N/A,N/A - :doc:`ROCr Debug Agent `,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3 - ,,,,,,,, - COMPILERS,.. _compilers-support-compatibility-matrix-past-60:,,,,,,, - `clang-ocl `_,N/A,N/A,N/A,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0 - :doc:`hipCC `,1.1.1,1.1.1,1.1.1,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 - `Flang `_,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 - :doc:`llvm-project `,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 - `OpenMP `_,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 - ,,,,,,,, - RUNTIMES,.. _runtime-support-compatibility-matrix-past-60:,,,,,,, - :doc:`AMD CLR `,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 - :doc:`HIP `,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 - `OpenCL Runtime `_,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0 - :doc:`ROCR-Runtime `,1.14.0,1.14.0,1.13.0,1.13.0,1.13.0,1.13.0,1.12.0,1.12.0 +ROCm Version,6.2.4,6.2.2,6.2.1,6.2.0, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0 + :ref:`Operating systems & kernels `,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,, + ,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3, 22.04.2","Ubuntu 22.04.4, 22.04.3, 22.04.2" + ,,,,,"Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5" + ,"RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.4 [#red-hat94-past-60]_, 9.3, 9.2","RHEL 9.3, 9.2","RHEL 9.3, 9.2" + ,"RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8" + ,"SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4" + ,,,,,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9 + ,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,Oracle Linux 8.9 [#oracle89-past-60]_,,, + ,.. _architecture-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`Architecture `,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3 + ,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2 + ,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA + ,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3 + ,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2 + ,.. _gpu-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`GPU / LLVM target `,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100 + ,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030 + ,gfx942 [#mi300_624-past-60]_,gfx942 [#mi300_622-past-60]_,gfx942 [#mi300_621-past-60]_,gfx942 [#mi300_620-past-60]_, gfx942 [#mi300_612-past-60]_, gfx942 [#mi300_611-past-60]_, gfx942 [#mi300_610-past-60]_, gfx942 [#mi300_602-past-60]_, gfx942 [#mi300_600-past-60]_ + ,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a + ,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908 + ,,,,,,,,, + FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`PyTorch `,"2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13" + :doc:`TensorFlow `,"2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.14.0, 2.13.1, 2.12.1","2.14.0, 2.13.1, 2.12.1" + :doc:`JAX `,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26 + `ONNX Runtime `_,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.14.1,1.14.1 + ,,,,,,,,, + THIRD PARTY COMMS,.. _thirdpartycomms-support-compatibility-matrix-past-60:,,,,,,,, + `UCC `_,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.2.0,>=1.2.0 + `UCX `_,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1 + ,,,,,,,,, + THIRD PARTY ALGORITHM,.. _thirdpartyalgorithm-support-compatibility-matrix-past-60:,,,,,,,, + Thrust,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1 + CUB,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1 + ,,,,,,,,, + KFD & USER SPACE [#kfd_support-past-60]_,.. _kfd-userspace-support-compatibility-matrix-past-60:,,,,,,,, + Tested user space versions,"6.1.x, 6.0.x","6.1.x, 6.0.x","6.1.x, 6.0.x","6.1.x, 6.0.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x","6.2.x, 6.0.x, 5.7.x, 5.6.x","6.2.x, 6.0.x, 5.7.x, 5.6.x" + ,,,,,,,,, + ML & COMPUTER VISION,.. _mllibs-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`Composable Kernel `,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0 + :doc:`MIGraphX `,2.10.0,2.10.0,2.10.0,2.10.0,2.9.0,2.9.0,2.9.0,2.8.0,2.8.0 + :doc:`MIOpen `,3.2.0,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 + :doc:`MIVisionX `,3.0.0,3.0.0,3.0.0,3.0.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0 + :doc:`rocAL `,2.0.0,2.0.0,2.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 + :doc:`rocDecode `,0.6.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,N/A,N/A + :doc:`rocPyDecode `,0.1.0,0.1.0,0.1.0,0.1.0,N/A,N/A,N/A,N/A,N/A + :doc:`RPP `,1.8.0,1.8.0,1.8.0,1.8.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0 + ,,,,,,,,, + COMMUNICATION,.. _commlibs-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`RCCL `,2.20.5,2.20.5,2.20.5,2.20.5,2.18.6,2.18.6,2.18.6,2.18.3,2.18.3 + ,,,,,,,,, + MATH LIBS,.. _mathlibs-support-compatibility-matrix-past-60:,,,,,,,, + `half `_ ,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0 + :doc:`hipBLAS `,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0 + :doc:`hipBLASLt `,0.8.0,0.8.0,0.8.0,0.8.0,0.7.0,0.7.0,0.7.0,0.6.0,0.6.0 + :doc:`hipFFT `,1.0.16,1.0.15,1.0.15,1.0.14,1.0.14,1.0.14,1.0.14,1.0.13,1.0.13 + :doc:`hipFORT `,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0 + :doc:`hipRAND `,2.11.1,2.11.0,2.11.0,2.11.0,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16 + :doc:`hipSOLVER `,2.2.0,2.2.0,2.2.0,2.2.0,2.1.1,2.1.1,2.1.0,2.0.0,2.0.0 + :doc:`hipSPARSE `,3.1.1,3.1.1,3.1.1,3.1.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0 + :doc:`hipSPARSELt `,0.2.1,0.2.1,0.2.1,0.2.1,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0 + :doc:`rocALUTION `,3.2.1,3.2.0,3.2.0,3.2.0,3.1.1,3.1.1,3.1.1,3.0.3,3.0.3 + :doc:`rocBLAS `,4.2.4,4.2.1,4.2.1,4.2.0,4.1.2,4.1.0,4.1.0,4.0.0,4.0.0 + :doc:`rocFFT `,1.0.30,1.0.29,1.0.29,1.0.28,1.0.27,1.0.27,1.0.26,1.0.25,1.0.23 + :doc:`rocRAND `,3.1.1,3.1.0,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.0,2.10.17 + :doc:`rocSOLVER `,3.26.2,3.26.0,3.26.0,3.26.0,3.25.0,3.25.0,3.25.0,3.24.0,3.24.0 + :doc:`rocSPARSE `,3.2.1,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.0.2,3.0.2 + :doc:`rocWMMA `,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0 + `Tensile `_,4.40.0,4.40.0,4.40.0,4.40.0,4.40.0,4.40.0,4.40.0,4.39.0,4.39.0 + ,,,,,,,,, + PRIMITIVES,.. _primitivelibs-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`hipCUB `,3.2.1,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 + :doc:`hipTensor `,1.3.0,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0 + :doc:`rocPRIM `,3.2.2,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0 + :doc:`rocThrust `,3.1.1,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0 + ,,,,,,,,, + SUPPORT LIBS,,,,,,,,, + `hipother `_,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 + `rocm-core `_,6.2.4,6.2.2,6.2.1,6.2.0,6.1.2,6.1.1,6.1.0,6.0.2,6.0.0 + `ROCT-Thunk-Interface `_,20240607.5.7,20240607.5.7,20240607.4.05,20240607.1.4246,20240125.5.08,20240125.5.08,20240125.3.30,20231016.2.245,20231016.2.245 + ,,,,,,,,, + SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`AMD SMI `,24.6.3,24.6.3,24.6.3,24.6.2,24.5.1,24.5.1,24.4.1,23.4.2,23.4.2 + :doc:`ROCm Data Center Tool `,1.0.0,1.0.0,1.0.0,1.0.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0 + :doc:`rocminfo `,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 + :doc:`ROCm SMI `,7.3.0,7.3.0,7.3.0,7.3.0,7.2.0,7.0.0,7.0.0,6.0.2,6.0.0 + :doc:`ROCm Validation Suite `,rocm-6.2.4,rocm-6.2.2,rocm-6.2.1,rocm-6.2.0,rocm-6.1.2,rocm-6.1.1,rocm-6.1.0,rocm-6.0.2,rocm-6.0.0 + ,,,,,,,,, + PERFORMANCE TOOLS,,,,,,,,, + :doc:`Omniperf `,2.0.1,2.0.1,2.0.1,2.0.1,N/A,N/A,N/A,N/A,N/A + :doc:`Omnitrace `,1.11.2,1.11.2,1.11.2,1.11.2,N/A,N/A,N/A,N/A,N/A + :doc:`ROCm Bandwidth Test `,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0 + :doc:`ROCProfiler `,2.0.60204,2.0.60202,2.0.60201,2.0.60200,2.0.60102,2.0.60101,2.0.60100,2.0.60002,2.0.60000 + :doc:`ROCprofiler-SDK `,0.4.0,0.4.0,0.4.0,0.4.0,N/A,N/A,N/A,N/A,N/A + :doc:`ROCTracer `,4.1.60204,4.1.60202,4.1.60201,4.1.60200,4.1.60102,4.1.60101,4.1.60100,4.1.60002,4.1.60000 + ,,,,,,,,, + DEVELOPMENT TOOLS,,,,,,,,, + :doc:`HIPIFY `,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 + :doc:`ROCm CMake `,0.13.0,0.13.0,0.13.0,0.13.0,0.12.0,0.12.0,0.12.0,0.11.0,0.11.0 + :doc:`ROCdbgapi `,0.76.0,0.76.0,0.76.0,0.76.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0 + :doc:`ROCm Debugger (ROCgdb) `,14.2.0,14.2.0,14.2.0,14.2.0,14.1.0,14.1.0,14.1.0,13.2.0,13.2.0 + `rocprofiler-register `_,0.4.0,0.4.0,0.4.0,0.4.0,0.3.0,0.3.0,0.3.0,N/A,N/A + :doc:`ROCr Debug Agent `,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3 + ,,,,,,,,, + COMPILERS,.. _compilers-support-compatibility-matrix-past-60:,,,,,,,, + `clang-ocl `_,N/A,N/A,N/A,N/A,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0 + :doc:`hipCC `,1.1.1,1.1.1,1.1.1,1.1.1,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 + `Flang `_,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 + :doc:`llvm-project `,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 + `OpenMP `_,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483 + ,,,,,,,,, + RUNTIMES,.. _runtime-support-compatibility-matrix-past-60:,,,,,,,, + :doc:`AMD CLR `,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 + :doc:`HIP `,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830 + `OpenCL Runtime `_,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0 + :doc:`ROCR-Runtime `,1.14.0,1.14.0,1.14.0,1.13.0,1.13.0,1.13.0,1.13.0,1.12.0,1.12.0 diff --git a/docs/compatibility/compatibility-matrix.rst b/docs/compatibility/compatibility-matrix.rst index 98586d9b1..bf94e371f 100644 --- a/docs/compatibility/compatibility-matrix.rst +++ b/docs/compatibility/compatibility-matrix.rst @@ -16,11 +16,11 @@ You can also refer to the :ref:`past versions of ROCm compatibility matrix`,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04 + :ref:`Operating systems & kernels `,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04", ,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.4, 22.04.3" ,,,"Ubuntu 20.04.6, 20.04.5" ,"RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4 [#red-hat94]_, 9.3, 9.2" @@ -37,13 +37,13 @@ You can also refer to the :ref:`past versions of ROCm compatibility matrix`,gfx1100,gfx1100,gfx1100 ,gfx1030,gfx1030,gfx1030 - ,gfx942 [#mi300_622]_,gfx942 [#mi300_621]_, gfx942 [#mi300_610]_ + ,gfx942 [#mi300_624]_,gfx942 [#mi300_622]_, gfx942 [#mi300_610]_ ,gfx90a,gfx90a,gfx90a ,gfx908,gfx908,gfx908 ,,, FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix:,, :doc:`PyTorch `,"2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.1, 2.0, 1.13" - :doc:`TensorFlow `,"2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15, 2.14, 2.13" + :doc:`TensorFlow `,"2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15.0, 2.14.0, 2.13.1" :doc:`JAX `,0.4.26,0.4.26,0.4.26 `ONNX Runtime `_,1.17.3,1.17.3,1.17.3 ,,, @@ -75,49 +75,49 @@ You can also refer to the :ref:`past versions of ROCm compatibility matrix`_ ,1.12.0,1.12.0,1.12.0 :doc:`hipBLAS `,2.2.0,2.2.0,2.1.0 :doc:`hipBLASLt `,0.8.0,0.8.0,0.7.0 - :doc:`hipFFT `,1.0.15,1.0.15,1.0.14 + :doc:`hipFFT `,1.0.16,1.0.15,1.0.14 :doc:`hipFORT `,0.4.0,0.4.0,0.4.0 - :doc:`hipRAND `,2.11.0,2.11.0,2.10.16 + :doc:`hipRAND `,2.11.1,2.11.0,2.10.16 :doc:`hipSOLVER `,2.2.0,2.2.0,2.1.0 :doc:`hipSPARSE `,3.1.1,3.1.1,3.0.1 :doc:`hipSPARSELt `,0.2.1,0.2.1,0.1.0 - :doc:`rocALUTION `,3.2.0,3.2.0,3.1.1 - :doc:`rocBLAS `,4.2.1,4.2.1,4.1.0 - :doc:`rocFFT `,1.0.29,1.0.29,1.0.26 - :doc:`rocRAND `,3.1.0,3.1.0,3.0.1 - :doc:`rocSOLVER `,3.26.0,3.26.0,3.25.0 - :doc:`rocSPARSE `,3.2.0,3.2.0,3.1.2 + :doc:`rocALUTION `,3.2.1,3.2.0,3.1.1 + :doc:`rocBLAS `,4.2.4,4.2.1,4.1.0 + :doc:`rocFFT `,1.0.30,1.0.29,1.0.26 + :doc:`rocRAND `,3.1.1,3.1.0,3.0.1 + :doc:`rocSOLVER `,3.26.2,3.26.0,3.25.0 + :doc:`rocSPARSE `,3.2.1,3.2.0,3.1.2 :doc:`rocWMMA `,1.5.0,1.5.0,1.4.0 `Tensile `_,4.40.0,4.40.0,4.40.0 ,,, PRIMITIVES,.. _primitivelibs-support-compatibility-matrix:,, - :doc:`hipCUB `,3.2.0,3.2.0,3.1.0 + :doc:`hipCUB `,3.2.1,3.2.0,3.1.0 :doc:`hipTensor `,1.3.0,1.3.0,1.2.0 - :doc:`rocPRIM `,3.2.0,3.2.0,3.1.0 - :doc:`rocThrust `,3.1.0,3.1.0,3.0.1 + :doc:`rocPRIM `,3.2.2,3.2.0,3.1.0 + :doc:`rocThrust `,3.1.1,3.1.0,3.0.1 ,,, SUPPORT LIBS,,, `hipother `_,6.2.41134,6.2.41134,6.1.40091 - `rocm-core `_,6.2.2,6.2.1,6.1.0 - `ROCT-Thunk-Interface `_,20240607.4.05,20240607.4.05,20240125.3.30 + `rocm-core `_,6.2.4,6.2.2,6.1.0 + `ROCT-Thunk-Interface `_,20240607.5.7,20240607.5.7,20240125.3.30 ,,, SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix:,, :doc:`AMD SMI `,24.6.3,24.6.3,24.4.1 :doc:`ROCm Data Center Tool `,1.0.0,1.0.0,0.3.0 :doc:`rocminfo `,1.0.0,1.0.0,1.0.0 :doc:`ROCm SMI `,7.3.0,7.3.0,7.0.0 - :doc:`ROCm Validation Suite `,rocm-6.2.2,rocm-6.2.1,rocm-6.1.0 + :doc:`ROCm Validation Suite `,rocm-6.2.4,rocm-6.2.2,rocm-6.1.0 ,,, PERFORMANCE TOOLS,,, :doc:`Omniperf `,2.0.1,2.0.1,N/A :doc:`Omnitrace `,1.11.2,1.11.2,N/A :doc:`ROCm Bandwidth Test `,1.4.0,1.4.0,1.4.0 - :doc:`ROCProfiler `,2.0.60202,2.0.60201,2.0.60100 + :doc:`ROCProfiler `,2.0.60204,2.0.60202,2.0.60100 :doc:`ROCprofiler-SDK `,0.4.0,0.4.0,N/A - :doc:`ROCTracer `,4.1.60202,4.1.60201,4.1.60100 + :doc:`ROCTracer `,4.1.60204,4.1.60202,4.1.60100 ,,, DEVELOPMENT TOOLS,,, - :doc:`HIPIFY `,18.0.0.24355,18.0.0.24355,17.0.0.24103 + :doc:`HIPIFY `,18.0.0.24392,18.0.0.24355,17.0.0.24103 :doc:`ROCm CMake `,0.13.0,0.13.0,0.12.0 :doc:`ROCdbgapi `,0.76.0,0.76.0,0.71.0 :doc:`ROCm Debugger (ROCgdb) `,14.2.0,14.2.0,14.1.0 @@ -127,9 +127,9 @@ You can also refer to the :ref:`past versions of ROCm compatibility matrix`_,N/A,N/A,0.5.0 :doc:`hipCC `,1.1.1,1.1.1,1.0.0 - `Flang `_,18.0.0.24355,18.0.0.24355,17.0.0.24103 - :doc:`llvm-project `,18.0.0.24355,18.0.0.24355,17.0.0.24103 - `OpenMP `_,18.0.0.24355,18.0.0.24355,17.0.0.24103 + `Flang `_,18.0.0.24392,18.0.0.24355,17.0.0.24103 + :doc:`llvm-project `,18.0.0.24392,18.0.0.24355,17.0.0.24103 + `OpenMP `_,18.0.0.24392,18.0.0.24355,17.0.0.24103 ,,, RUNTIMES,.. _runtime-support-compatibility-matrix:,, :doc:`AMD CLR `,6.2.41134,6.2.41134,6.1.40091 @@ -142,8 +142,8 @@ You can also refer to the :ref:`past versions of ROCm compatibility matrix Date: Fri, 18 Oct 2024 12:21:38 -0400 Subject: [PATCH 03/74] ROCm 6.2.4 release notes (#184) * prep 6.2.4 release notes * add mathlibs * add detail component changes * rm non-updated linnks * fix sentence * fix rocthrust v * rm offline installer * condense * add leo/ram fdback words * update documentation section * add rocm on radeon note * update os support note wording * update release * update version and GA date to 10-17 * update 6.2.4 rn * update wording * add link to v710 * update wording * update templ * simplify note * words os note words * change URLs to latest * update link to supported GPUs * Update versions.md 6.2.4 date to Oct 18 * Update conf.py release note date to Oct 18 --------- Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> --- RELEASE.md | 495 +++++++----------- docs/conf.py | 6 +- docs/release/versions.md | 1 + .../templates/extra_components/6.2.4.md | 26 + tools/autotag/templates/highlights/6.2.4.md | 46 ++ tools/autotag/templates/support/6.2.4.md | 5 + .../templates/upcoming_changes/6.2.4.md | 9 + 7 files changed, 272 insertions(+), 316 deletions(-) create mode 100644 tools/autotag/templates/extra_components/6.2.4.md create mode 100644 tools/autotag/templates/highlights/6.2.4.md create mode 100644 tools/autotag/templates/support/6.2.4.md create mode 100644 tools/autotag/templates/upcoming_changes/6.2.4.md diff --git a/RELEASE.md b/RELEASE.md index 5ba7126db..935d6997b 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,104 +1,62 @@ -# ROCm 6.2.2 release notes +# ROCm 6.2.4 release notes -These release notes provide a summary of notable changes since the previous ROCm release. +The release notes provide a summary of notable changes since the previous ROCm release. + +- [Release highlights](#release-highlights) + +- [Operating system and hardware support changes](#operating-system-and-hardware-support-changes) + +- [ROCm components versioning](#rocm-components) + +- [Detailed component changes](#detailed-component-changes) + +- [ROCm known issues](#rocm-known-issues) + +- [ROCm upcoming changes](#rocm-upcoming-changes) ```{note} -As ROCm 6.2.2 was released shortly after 6.2.1, the changes between these versions -are minimal. For a comprehensive overview of recent updates, the ROCm 6.2.1 release -notes are appended to the end of this document. - -For detailed information about the changes in ROCm 6.2.1, refer to the appended -section: [ROCm 6.2.1 release notes](rocm-6-2-1-release-notes). +If you’re using Radeon™ PRO or Radeon GPUs for graphics workloads, +continue to use ROCm 6.2.3. See the [Use ROCm on Radeon +GPUs](https://rocm.docs.amd.com/projects/radeon/en/latest/index.html) +documentation to verify compatibility and system requirements. ``` -The [Compatibility matrix](https://rocm.docs.amd.com/en/docs-6.2.2/compatibility/compatibility-matrix.html) -provides the full list of supported hardware, operating systems, ecosystems, third-party components, and ROCm components -for each ROCm release. - -Release notes for previous ROCm releases are available in earlier versions of the documentation. -See the [ROCm documentation release history](https://rocm.docs.amd.com/en/latest/release/versions.html). - ## Release highlights -The following is a significant fix introduced in ROCm 6.2.2. +The following are notable new features and improvements in ROCm 6.2.4. For changes to individual components, see +[Detailed component changes](#detailed-component-changes). -### Fixed Instinct MI300X error recovery failure +#### ROCm documentation updates -Improved the reliability of AMD Instinct MI300X accelerators in scenarios involving -uncorrectable errors. Previously, error recovery did not occur as expected, -potentially leaving the system in an undefined state. This fix ensures that error -recovery functions as expected, maintaining system stability. +ROCm documentation continues to be updated to provide clearer and more comprehensive guidance for +a wider variety of user needs and use cases. -See the [original issue](#instinct-mi300x-gpu-recovery-failure-on-uncorrectable-errors) -noted in the ROCm 6.2.1 release notes. +* Added a new GPU cluster networking guide. See + [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html). ---- + This documentation provides guidelines on validating network configurations + in single-node and multi-node environments to attain optimal speed and bandwidth + in AMD Instinct-powered clusters. -## ROCm 6.2.1 release notes +* Updated the HIP runtime documentation. -The ROCm 6.2.1 release notes document newly added ecosystem support, ROCm Offline Installer Creator updates, -and improvements to several ROCm libraries and tools. + * Added a new section on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hipgraph.html). -- [Release highlights](release-highlights) + * Added a new section about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/stream_ordered_allocator.html). -- [Operating system and hardware support changes](operating-system-and-hardware-support-changes) + * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hip_porting_driver_api.html) section. -- [ROCm components versioning](rocm-components) +## Operating system and hardware support changes -- [Detailed component changes](detailed-component-changes) +ROCm 6.2.4 adds support for the [AMD Radeon PRO V710](https://www.amd.com/en/products/accelerators/radeon-pro/amd-radeon-pro-v710.html) GPU. See +[Supported GPUs](https://advanced-micro-devices-demo--287.com.readthedocs.build/projects/install-on-linux-internal/en/287/reference/system-requirements.html) +for more information. -- [ROCm known issues](rocm-known-issues) +This release maintains the same operating system support as 6.2.2. -- [ROCm upcoming changes](rocm-upcoming-changes) +## ROCm components -### Release highlights - -The following are notable new features and improvements in ROCm 6.2.1. For changes to individual components, see [Detailed component changes](#detailed-component-changes). - -#### rocAL major version change - -The new version of rocAL introduces many new features, but does not modify any of the existing public API functions. However, the version number was incremented from 1.3 to 2.0. -Applications linked to version 1.3 must be recompiled to link against version 2.0. - -See [the rocAL detailed changes](#rocal-2-0-0) for more information. - -#### New support for FBGEMM (Facebook General Matrix Multiplication) - -As of ROCm 6.2.1, ROCm supports Facebook General Matrix Multiplication (FBGEMM) and the related FBGEMM_GPU library. - -FBGEMM is a low-precision, high-performance CPU kernel library for convolution and matrix multiplication. It is used for server-side inference and as a back end for PyTorch quantized operators. FBGEMM_GPU includes a collection of PyTorch GPU operator libraries for training and inference. For more information, see the ROCm [Model acceleration libraries guide](https://rocm.docs.amd.com/en/docs-6.2.1/how-to/llm-fine-tuning-optimization/model-acceleration-libraries.html) -and [PyTorch's FBGEMM GitHub repository](https://github.com/pytorch/FBGEMM). - -#### ROCm Offline Installer Creator changes - -The [ROCm Offline Installer Creator 6.2.1](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.1/install/rocm-offline-installer.html) introduces several new features and improvements including: - -* Logging support for create and install logs -* More stringent checks for Linux versions and distributions -* Updated prerequisite repositories -* Fixed CTest issues - -#### ROCm documentation changes - -There have been no changes to supported hardware or operating systems from ROCm 6.2.0 to ROCm 6.2.1. - -* The Programming Model Reference and Understanding the Programming Model topics in HIP have been consolidated into one topic, -[HIP programming model (conceptual)](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.1/understand/programming_model.html). -* The [HIP virtual memory management](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.1/how-to/virtual_memory.html) and [HIP virtual memory management API](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.1/reference/virtual_memory_reference.html) topics have been added. - -```{note} -The ROCm documentation, like all ROCm projects, is open source and available on GitHub. To contribute to ROCm documentation, see the [ROCm documentation contribution guidelines](https://rocm.docs.amd.com/en/latest/contribute/contributing.html). -``` - -### Operating system and hardware support changes - -ROCm 6.2.1 adds support for Ubuntu 24.04.1 (kernel: 6.8 [GA]). - -See the [Compatibility matrix](https://rocm.docs.amd.com/en/docs-6.2.1/compatibility/compatibility-matrix.html) for the full list of supported operating systems and hardware architectures. - -### ROCm components - -The following table lists the versions of ROCm components for ROCm 6.2.1, including any version changes from 6.2.0 to 6.2.1. +The following table lists the versions of ROCm components for ROCm 6.2.4, including any version changes from 6.2.2 to 6.2.4. Click the component's updated version to go to a detailed list of its changes. Click to go to the component's source code on GitHub. @@ -121,51 +79,51 @@ Click the component's updated version to go to a detailed list of its changes. C Libraries Machine learning and computer vision - Composable Kernel + Composable Kernel 1.1.0 - - MIGraphX + MIGraphX 2.10 - + - MIOpen + MIOpen 3.2.0 - - MIVisionX + MIVisionX 3.0.0 - - rocAL - 1.0.0 ⇒ 2.0.0 - rocAL + 2.0.0 + - rocDecode + rocDecode 0.6.0 - - rocPyDecode + rocPyDecode 0.1.0 - - RPP + RPP 1.8.0 - @@ -173,9 +131,9 @@ Click the component's updated version to go to a detailed list of its changes. C Communication - RCCL - 2.20.5 ⇒ 2.20.5 - RCCL + 2.20.5 + @@ -183,99 +141,99 @@ Click the component's updated version to go to a detailed list of its changes. C Math - hipBLAS + hipBLAS 2.2.0 - - hipBLASLt + hipBLASLt 0.8.0 - - hipFFT - 1.0.15 - hipFFT + 1.0.15 ⇒ 1.0.16 + - hipfort + hipfort 0.4.0 - - hipRAND - 2.11.0 - hipRAND + 2.11.0 ⇒ 2.11.1 + - hipSOLVER + hipSOLVER 2.2.0 - - hipSPARSE + hipSPARSE 3.1.1 - - hipSPARSELt + hipSPARSELt 0.2.1 - - rocALUTION - 3.2.0 - rocALUTION + 3.2.0 ⇒ 3.2.1 + - rocBLAS - 4.1.2 ⇒ 4.2.1 - rocBLAS + 4.2.1 ⇒ 4.2.4 + - rocFFT - 1.0.28 ⇒ 1.0.29 - rocFFT + 1.0.29 ⇒ 1.0.30 + - rocRAND - 3.1.0 - rocRAND + 3.1.0 ⇒ 3.1.1 + - rocSOLVER - 3.26.0 - rocSOLVER + 3.26.0 ⇒ 3.26.2 + - rocSPARSE - 3.2.0 - rocSPARSE + 3.2.0 ⇒ 3.2.1 + - rocWMMA + rocWMMA 1.5.0 - Tensile 4.41.0 - @@ -283,27 +241,27 @@ Click the component's updated version to go to a detailed list of its changes. C Primitives - hipCUB - 3.2.0 - hipCUB + 3.2.0 ⇒ 3.2.1 + - hipTensor + hipTensor 1.3.0 - - rocPRIM - 3.2.0 ⇒ 3.2.1 - rocPRIM + 3.2.1 ⇒ 3.2.2 + - rocThrust - 3.1.0 - rocThrust + 3.1.0 ⇒ 3.1.1 + @@ -311,33 +269,33 @@ Click the component's updated version to go to a detailed list of its changes. C Tools System management - AMD SMI - 24.6.2 ⇒ 24.6.3 - AMD SMI + 24.6.3 + - rocminfo + rocminfo 1.0.0 - - ROCm Data Center Tool + ROCm Data Center Tool 1.0.0 - - ROCm SMI - 7.3.0 ⇒ 7.3.0 - ROCm SMI + 7.3.0 + - ROCm Validation Suite + ROCm Validation Suite 1.0.0 - @@ -345,40 +303,40 @@ Click the component's updated version to go to a detailed list of its changes. C Performance - Omniperf + Omniperf 2.0.1 - - Omnitrace - 1.11.2 ⇒ 1.11.2 - Omnitrace + 1.11.2 + - ROCm Bandwidth + ROCm Bandwidth Test 1.4.0 - - ROCProfiler + ROCProfiler 2.0.0 - - ROCprofiler-SDK + ROCprofiler-SDK 0.4.0 - - ROCTracer + ROCTracer 4.1.0 - @@ -386,219 +344,130 @@ Click the component's updated version to go to a detailed list of its changes. C Development - HIPIFY - 18.0.0 ⇒ 18.0.0 - HIPIFY + 18.0.0 + - ROCdbgapi + ROCdbgapi 0.76.0 - - ROCm CMake + ROCm CMake 0.13.0 - - ROCm Debugger (ROCgdb) + ROCm Debugger (ROCgdb) 14.2 - - ROCr Debug Agent + ROCr Debug Agent 2.0.3 - Compilers - HIPCC + HIPCC 1.1.1 - - llvm-project + llvm-project 18.0.0 - Runtimes - HIP - 6.2 ⇒ 6.2.1 - HIP + 6.2.4 + - ROCr Runtime + ROCr Runtime 1.14.0 - -### Detailed component changes +## Detailed component changes The following sections describe key changes to ROCm components. -#### **AMD SMI** (24.6.3) +### Hardware architecture support updates -##### Changes +Updated the following math and primitives libraries to pre-enable support for +an upcoming hardware architecture. -* Added `amd-smi static --ras` on Guest VMs. Guest VMs can view enabled/disabled RAS features on Host cards. +* hipCUB (3.2.1) -##### Removals +* hipFFT (1.0.16) -* Removed `amd-smi metric --ecc` & `amd-smi metric --ecc-blocks` on Guest VMs. Guest VMs do not support getting current ECC counts from the Host cards. +* hipRAND (2.11.1) -##### Resolved issues +* rocALUTION (3.2.1) -* Fixed TypeError in `amd-smi process -G`. -* Updated CLI error strings to handle empty and invalid GPU/CPU inputs. -* Fixed Guest VM showing passthrough options. -* Fixed firmware formatting where leading 0s were missing. +* rocBLAS (4.2.4) -#### **HIP** (6.2.1) +* rocFFT (1.0.30) -##### Resolved issues +* rocPRIM (3.2.2) -* Soft hang when using `AMD_SERIALIZE_KERNEL` -* Memory leak in `hipIpcCloseMemHandle` +* rocRAND (3.1.1) -#### **HIPIFY** (18.0.0) +* rocSOLVER (3.26.2) -##### Changes +* rocSPARSE (3.2.1) -* Added CUDA 12.5.1 support -* Added cuDNN 9.2.1 support -* Added LLVM 18.1.8 support -* Added `hipBLAS` 64-bit APIs support -* Added Support for math constants `math_constants.h` +* rocThrust (3.1.1) -#### **Omnitrace** (1.11.2) +### **rocFFT** (1.0.30)[*](#hardware-architecture-support-updates) -##### Known issues +#### Optimized -Perfetto can no longer open Omnitrace proto files. Loading Perfetto trace output `.proto` files in the latest version of `ui.perfetto.dev` can result in a dialog with the message, "Oops, something went wrong! Please file a bug." The information in the dialog will refer to an "Unknown field type." The workaround is to open the files with the previous version of the Perfetto UI found at [https://ui.perfetto.dev/v46.0-35b3d9845/#!/](https://ui.perfetto.dev/v46.0-35b3d9845/#!/). +* Implemented 1D kernels for factorizable sizes greater than 1024 and less than 2048. -See [issue #3767](https://github.com/ROCm/ROCm/issues/3767) on GitHub. +#### Resolved issues -#### **RCCL** (2.20.5) +* Fixed plan creation failure on some even-length real-complex transforms that use Bluestein's algorithm. -##### Known issues +### **rocSOLVER** (3.26.2)[*](#hardware-architecture-support-updates) -On systems running Linux kernel 6.8.0, such as Ubuntu 24.04, Direct Memory Access (DMA) transfers between the GPU and NIC are disabled and impacts multi-node RCCL performance. -This issue was reproduced with RCCL 2.20.5 (ROCm 6.2.0 and 6.2.1) on systems with Broadcom Thor-2 NICs and affects other systems with RoCE networks using Linux 6.8.0 or newer. -Older RCCL versions are also impacted. +#### Resolved issues -This issue will be addressed in a future ROCm release. +* Fixed synchronization issue in STEIN. -See [issue #3772](https://github.com/ROCm/ROCm/issues/3772) on GitHub. +## ROCm known issues -#### **rocAL** (2.0.0) +ROCm known issues are tracked on [GitHub](https://github.com/ROCm/ROCm/labels/Verified%20Issue). -##### Changes - -* The new version of rocAL introduces many new features, but does not modify any of the existing public API functions.However, the version number was incremented from 1.3 to 2.0. - Applications linked to version 1.3 must be recompiled to link against version 2.0. -* Added development and test packages. -* Added C++ rocAL audio unit test and Python script to run and compare the outputs. -* Added Python support for audio decoders. -* Added Pytorch iterator for audio. -* Added Python audio unit test and support to verify outputs. -* Added rocDecode for HW decode. -* Added support for: - * Audio loader and decoder, which uses libsndfile library to decode wav files - * Audio augmentation - PreEmphasis filter, Spectrogram, ToDecibels, Resample, NonSilentRegionDetection, MelFilterBank - * Generic augmentation - Slice, Normalize - * Reading from file lists in file reader - * Downmixing audio channels during decoding - * TensorTensorAdd and TensorScalarMultiply operations - * Uniform and Normal distribution nodes -* Image to tensor updates -* ROCm install - use case graphics removed - -##### Known issues - -* Dependencies are not installed with the rocAL package installer. Dependencies must be installed with the prerequisite setup script provided. See the [rocAL README on GitHub](https://github.com/ROCm/rocAL/blob/docs/6.2.1/README.md#prerequisites-setup-script) for details. - -#### **rocBLAS** (4.2.1) - -##### Removals - -* Removed Device_Memory_Allocation.pdf link in documentation. - -##### Resolved issues - -* Fixed error/warning message during `rocblas_set_stream()` call. - -#### **rocFFT** (1.0.29) - -##### Optimizations - -* Implemented 1D kernels for factorizable sizes less than 1024. - -#### **ROCm SMI** (7.3.0) - -##### Optimizations - -* Improved handling of UnicodeEncodeErrors with non UTF-8 locales. Non UTF-8 locales were causing crashes on UTF-8 special characters. - -##### Resolved issues - -* Fixed an issue where the Compute Partition tests segfaulted when AMDGPU was loaded with optional parameters. - -##### Known issues - -* When setting CPX as a partition mode, there is a DRM node limit of 64. This is a known limitation when multiple drivers are using the DRM nodes. The `ls /sys/class/drm` command can be used to see the number of DRM nodes, and the following steps can be used to remove unnecessary drivers: - - 1. Unload AMDGPU: `sudo rmmod amdgpu`. - 2. Remove any unnecessary drivers using `rmmod`. For example, to remove an AST driver, run `sudo rmmod ast`. - 3. Reload AMDGPU using `modprobe`: `sudo modprobe amdgpu`. - -#### **rocPRIM** (3.2.1) - -##### Optimizations - -* Improved performance of `block_reduce_warp_reduce` when warp size equals block size. - -### ROCm known issues - -ROCm known issues are tracked on [GitHub](https://github.com/ROCm/ROCm/labels/Verified%20Issue). Known issues related to -individual components are listed in the [Detailed component changes](detailed-component-changes) section. - -#### Instinct MI300X GPU recovery failure on uncorrectable errors - -For the AMD Instinct MI300X accelerator, GPU recovery resets triggered by uncorrectable errors (UE) might not complete -successfully, which can result in the system being left in an undefined state. A system reboot is needed to recover from -this state. Additionally, error logging might fail in these situations, hindering diagnostics. - -This issue is under investigation and will be resolved in a future ROCm release. - -See [issue #3766](https://github.com/ROCm/ROCm/issues/3766) on GitHub. - -### ROCm upcoming changes +## ROCm upcoming changes The following changes to the ROCm software stack are anticipated for future releases. -#### rocm-llvm-alt +### rocm-llvm-alt The `rocm-llvm-alt` package will be removed in an upcoming release. Users relying on the functionality provided by the closed-source compiler should transition to the open-source compiler. Once the `rocm-llvm-alt` package is removed, any compilation requesting functionality provided by the closed-source compiler will result in a Clang warning: "*[AMD] proprietary optimization compiler has been removed*". -#### rccl-rdma-sharp-plugins +### rccl-rdma-sharp-plugins The RCCL plugin package, `rccl-rdma-sharp-plugins`, will be removed in an upcoming ROCm release. diff --git a/docs/conf.py b/docs/conf.py index f20f6cd76..cca8f0ad3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -30,15 +30,15 @@ if os.environ.get("READTHEDOCS", "") == "True": project = "ROCm Documentation" author = "Advanced Micro Devices, Inc." copyright = "Copyright (c) 2024 Advanced Micro Devices, Inc. All rights reserved." -version = "6.2.2" -release = "6.2.2" +version = "6.2.4" +release = "6.2.4" setting_all_article_info = True all_article_info_os = ["linux", "windows"] all_article_info_author = "" # pages with specific settings article_pages = [ - {"file": "about/release-notes", "os": ["linux", "windows"], "date": "2024-09-27"}, + {"file": "about/release-notes", "os": ["linux", "windows"], "date": "2024-10-18"}, {"file": "how-to/deep-learning-rocm", "os": ["linux"]}, {"file": "how-to/rocm-for-ai/index", "os": ["linux"]}, {"file": "how-to/rocm-for-ai/install", "os": ["linux"]}, diff --git a/docs/release/versions.md b/docs/release/versions.md index 6d1e5b5fd..3c01f5d72 100644 --- a/docs/release/versions.md +++ b/docs/release/versions.md @@ -8,6 +8,7 @@ | Version | Release date | | ------- | ------------ | +| [6.2.4](https://rocm.docs.amd.com/en/docs-6.2.4/) | October 18, 2024 | | [6.2.2](https://rocm.docs.amd.com/en/docs-6.2.2/) | September 27, 2024 | | [6.2.1](https://rocm.docs.amd.com/en/docs-6.2.1/) | September 20, 2024 | | [6.2.0](https://rocm.docs.amd.com/en/docs-6.2.0/) | August 2, 2024 | diff --git a/tools/autotag/templates/extra_components/6.2.4.md b/tools/autotag/templates/extra_components/6.2.4.md new file mode 100644 index 000000000..4dee22b54 --- /dev/null +++ b/tools/autotag/templates/extra_components/6.2.4.md @@ -0,0 +1,26 @@ +### Hardware architecture support updates + +Updated the following math and primitives libraries to pre-enable support for +an upcoming hardware architecture. + +* hipCUB (3.2.1) + +* hipFFT (1.0.16) + +* hipRAND (2.11.1) + +* rocALUTION (3.2.1) + +* rocBLAS (4.2.4) + +* rocFFT (1.0.30) + +* rocPRIM (3.2.2) + +* rocRAND (3.1.1) + +* rocSOLVER (3.26.2) + +* rocSPARSE (3.2.1) + +* rocThrust (3.1.1) diff --git a/tools/autotag/templates/highlights/6.2.4.md b/tools/autotag/templates/highlights/6.2.4.md new file mode 100644 index 000000000..da45a17d3 --- /dev/null +++ b/tools/autotag/templates/highlights/6.2.4.md @@ -0,0 +1,46 @@ +These release notes provide a summary of notable changes since the previous ROCm release. + +- [Release highlights](#release-highlights) + +- [Operating system and hardware support changes](#operating-system-and-hardware-support-changes) + +- [ROCm components versioning](#rocm-components) + +- [Detailed component changes](#detailed-component-changes) + +- [ROCm known issues](#rocm-known-issues) + +- [ROCm upcoming changes](#rocm-upcoming-changes) + +```{note} +ROCm 6.2.3 is supported on systems using AMD Radeon™ or Radeon PRO workstation +GPUs for graphics workloads. If you’re using ROCm in this context, use ROCm +version 6.2.3. See the [Use ROCm on Radeon +GPUs](https://rocm.docs.amd.com/projects/radeon/en/latest/index.html) +documentation to verify compatibility and system requirements. +``` + +## Release highlights + +The following are notable new features and improvements in ROCm 6.2.4. For changes to individual components, see +[Detailed component changes](#detailed-component-changes). + +#### ROCm documentation updates + +ROCm documentation continues to be updated to provide clearer and more comprehensive guidance for +a wider variety of user needs and use cases. + +* Added a new GPU cluster networking topic. See + [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/docs-6.2.4/index.html). + + This documentation provides guidelines on validating network configurations + in single-node and multi-node environments to attain optimal speed and bandwidth + in AMD Instinct-powered clusters. + +* Updated the HIP runtime documentation. + + * Added a new topic on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hipgraph.html). + + * Added a new topic about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/stream_ordered_allocator.html). + + * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hip_porting_driver_api.html) topic. diff --git a/tools/autotag/templates/support/6.2.4.md b/tools/autotag/templates/support/6.2.4.md new file mode 100644 index 000000000..cb5b1f3a2 --- /dev/null +++ b/tools/autotag/templates/support/6.2.4.md @@ -0,0 +1,5 @@ +## Operating system and hardware support changes + +ROCm 6.2.4 adds support for the AMD Radeon PRO V710 GPU. See +[Supported GPUs](https://rocm.docs.amd.com/projects/install-on-linux-internal/en/docs-6.2.4/reference/system-requirements.html#supported-gpus) +for more information. diff --git a/tools/autotag/templates/upcoming_changes/6.2.4.md b/tools/autotag/templates/upcoming_changes/6.2.4.md new file mode 100644 index 000000000..24e7e6696 --- /dev/null +++ b/tools/autotag/templates/upcoming_changes/6.2.4.md @@ -0,0 +1,9 @@ +The following changes to the ROCm software stack are anticipated for future releases. + +### rocm-llvm-alt + +The `rocm-llvm-alt` package will be removed in an upcoming release. Users relying on the functionality provided by the closed-source compiler should transition to the open-source compiler. Once the `rocm-llvm-alt` package is removed, any compilation requesting functionality provided by the closed-source compiler will result in a Clang warning: "*[AMD] proprietary optimization compiler has been removed*". + +### rccl-rdma-sharp-plugins + +The RCCL plugin package, `rccl-rdma-sharp-plugins`, will be removed in an upcoming ROCm release. From 8e3d51c31d59dbdb9912bac5f54c12758456c8b2 Mon Sep 17 00:00:00 2001 From: alexxu-amd <159800977+alexxu-amd@users.noreply.github.com> Date: Fri, 25 Oct 2024 14:41:40 -0400 Subject: [PATCH 04/74] Sync change from ROCm to ROCm-internal (#194) * Fix Radeon link and point at R6.1.3 as absolute link (#3757) * Update ROCm manifest to 6.2.1 * Update ROCm branch name * Add 6.2.1 to version list (#3770) * Add links to GH issues in 6.2.1 release notes (#3769) * add MAD page * link to GitHub issues in release notes known issues * update templates for 6.2.1 * Revert "add MAD page" This reverts commit 9cce72bba306286c7eb317d592645d4e0e1b27aa. * update wordlist for spellcheck linter * add rccl note * update rocal version change heading to be more obvious * make rocal note more specific * fix missing space * fix capitalization * Update RCCL known issue wording (#3775) * add MAD page * fix wording in RCCL known issue * Revert "add MAD page" This reverts commit c81d0f3b0a3620305b11de8745686c86b060b006. * update llvm version for 6.2.1 (#3779) * Fix broken links in 6.2.1 release notes (#3782) * External CI: Replace libomp dependencies with aomp (#3781) Add roctracer dependency for hipBLAS and rocWMMA testing * External CI: Add rocprofiler v1 and v2 smoke tests (#3784) * External CI: ROCgdb smoke tests (#3785) - Since this is an autotools project and not cmake, build and test on gfx942 system instead of separating into two jobs. Pipeline time is short anyway. - Follow build instructions to update build flags and to incorporate the ROCdbgapi. - Results are not parsed and graphed, but the log contents are printed at the end. This was helpful for debugging and will be kept in the pipeline, as the make check-gdb command's output was not helpful on its own. * External CI: rocPyDecode Smoke Test (#3786) * External CI: omniperf pipeline (#3788) - Referred to public documentation, source, and iterative attempts to create and improve build and test pipeline. - ctest failures are due to the test node not having expected marketing name string and override not working. - The fix should be on the omniperf repo side of things, so this pull request should be fine as is. * External CI: create omniperf pipeline IDs, update nightly build (#3790) * Fixed greater than to be less than in rocFFT changes * fix footnote for 6.1.0 (#3791) * fix footnote for 6.1.0 * fix empty columns in historical KFD title * External CI: Publish wheel as artifact for rocPyDecode (#3796) * External CI: fix hip-tests symlink creation (#3799) * Docs: Add Ubuntu 24.04.1 (#3801) * add ubuntu 24.04.1 * add 24.04.1 to bottom os section * fix heading and template * Update compatibility-matrix.rst for OpenMP version * Update compatibility-matrix-historical-6.0.csv for OpenMP version * rm ubuntu 24.04.1 from 6.2.0 * Update docs/compatibility/compatibility-matrix.rst Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * rm duplicate ubuntu in historical --------- Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * External CI: fixes for rocMLIR and nightly build (#3800) * External CI: fix symlinks for rocMLIR and nightly build * add pipeline IDs for hip-tests * fix hip-test ID typo * remove llvm-alt license (#3727) * remove llvm-alt license * fix linting error * External CI: enable ROCR-Runtime tests (#3809) * External CI: default branches for hip-tests, omniperf (#3811) * External CI: torch and torchvision smoke tests (#3810) * External CI: torch and torchvision smoke tests - Fixed issues with package name and version for the vision wheel that prevented it from installing. A patch is used until my pull request in vision repo is merged. - Referred to rocAutomation scripts to pick which test scripts to run out of the many in the torch and vision repo, and iteratively tested suggested scripts to see which ones completed in a timely manner. - Leveraging pytest-azurepipelines module to automatically parse and graph results from these tests. * External CI: omnitrace build pipeline (#3812) * External CI: omnitrace build pipeline starter - Adding initial set of dependencies and build flags. * External CI: omnitrace build pipeline - Add bison, rccl, texinfo dependencies based on build failures. - Add AMDGPU_TARGETS flag - Add ROCm binaries to PATH for clang-format and other tools used. * Fix indentation --------- Co-authored-by: Daniel Su * External CI: AMDMIGraphX Build Fix (#3814) - Swap to default gcc on OS to resolve build errors from recent commits. - Added libdnnl-dev dependency from iterative attempts with compiler change. - Referred to the passing GitHub checks to observe the compilers that was used. - Build CK jit lib and include in AMDMIGraphX build. * External CI: test fixes w/ roctracer, list omniperf as partially succeeding (#3815) * External CI: rpp tests (#3816) * External CI: Build pipeline for rocprofiler-sdk (#3819) * External CI: Pipeline for rocprofiler-sdk * Add rocprofiler dependency * External CI: rocprofiler-sdk build pipeline --------- Co-authored-by: Daniel Su * External CI: Fix/add missing pipeline IDs (#3818) * External CI: omnitrace tests (#3822) * Update tags to 6.2.2 (#3827) * External CI: add roctracer to roc/hipSOLVER test deps (#3825) * External CI: add rocprofiler-sdk pipeline IDs (#3824) * External CI: AMDMIGraphX Smoke Tests (#3830) Co-authored-by: Daniel Su * External CI: MIOpen tests (#3837) * Point to release history instead of deprecated changelog (#3836) * External CI: filter out hipTensor extended tests (#3838) * added revised note re. radeon gpus (#3839) * Restructured the contributions section. (#3715) * testing if this file is editable * changed 'kebob-case' to 'dash-case' * Restructured the page to be more straightforward and provide additional repo information * forgot to save * Moved the topic sentence * Wrong accent on the a in diataxis * Removed the feedback info from contributing and moved it to Feedback * fixed spelling errors * fixed some wording and removed second person text * consolidated Build and Structure into Contribute; edited toolchai to (hopefully) conform to style guide; updated toc * updated the titles in the toc * made changes based on feedback * it's better when you save * removed structure and build; fixed something for the linter * added rst to wordlist * added customizations to wordlist * Add links to gpu cluster network guides (#3763) * Add links to gpu cluster network guides * Add newline character to eof * Make link absolute * add dynamic branch in toc * remove unnecessary page clean up * clean up index/toc * make multi-node topics adjacent --------- Co-authored-by: Peter Park * updated the radeon note (#3850) * External CI: Fix rocPyDecode wheel creation (#3852) - Set values for expected environment variables. - Accompanying changes required in rocPyDecode repo. Pull request will be made. * External CI: pytorch vision patch removal (#3855) My pull request applying this patch was merged upstream, so this is no longer needed and will break the pipeline since it can no longer be applied. * Build(deps): Bump rocm-docs-core from 1.8.1 to 1.8.2 in /docs/sphinx (#3807) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.1 to 1.8.2. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/v1.8.2/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.1...v1.8.2) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated the radeon note, as it were (#3857) * updated the radeon note, as it were * updated the note again * Set devops team as codeowners for rocm-build (#3860) * Set ext CI as codeowners for rocm-build * Update CODEOWNERS to rocm-devops * External CI: Add option to pull mainline branch for dependencies (#3689) * External CI: Add option to pull mainline branch for dependencies * Missing parameter for mainline branch dependencies. * External CI: mainline branch definitions * Removed MIGraphX optimization page (#3848) * External CI: add a global variable to control gfx942 tests (#3864) * External CI: update component default/mainline branches (#3871) * External CI: Stop building gfx90a (#3872) Save on VM resources until infrastructure has test targets. * External CI: add libstdc++-12 to rocMLIR (#3874) * Add building doc section (#3873) * External CI: programmatically get latest aqlprofile (#3876) * External CI: use ctest for rocm-examples (#3877) * External CI: Tensile pipeline (#3884) * add oversubscription conceptual doc (#3885) add mitigiation steps add to toc move page for build move doc fix spelling update doc update oversubscription update order fix spelling add oversubscription to wordlist move oversubscription topic to bottom of toc and index * add oversubscription conceptual doc (#3885) (cherry picked from commit d0ecf51b0c9202475e2abe90a45b50df0de6d7ae) * External CI: Add pipeline to build upstream boost (#3896) * Update bitsandbytes branch in docs (#3898) * Documentation: Add reference to precision-support floating-point types (#3899) * External CI: use Boost template for MIOpen (#3903) * External CI: create rocprofiler-systems pipeline (#3906) * External CI: omnitrace/rocprof-sys pipeline IDs (#3908) * External CI: MIOpen parse test results (#3913) * External CI: Use pip to install latest cmake on test system (#3915) * added a link to the compatibility matrix (#3904) * added a link to the compatibility matrix * removed quotes * docs: Remove invalid amd_iommu=on parameter Per kernel-parameters.txt, there is no "on" option for amd_iommu. While intel_iommu has it, amd_iommu is automatically on unless specified otherwise. For more info, see these 2 links: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt https://github.com/torvalds/linux/blob/75aa74d52f43e75d0beb20572f98529071b700e5/drivers/iommu/amd/init.c#L3481 Signed-off-by: Kent Russell * External CI: hipBLASLt build now requires python packaging module (#3926) https://github.com/ROCm/hipBLASLt/pull/1250/files#diff-fee2e6f068b33fca3a1dc49392de8848dbf05c3f4632b680abb1052523e5a30fR35 * External CI: Moved location of upstream pytorch build scripts (#3930) https://github.com/pytorch/pytorch/pull/138103 * External CI: disable rocMLIR tests (#3931) * External CI: disable rocMLIR tests * roctracer AMDGPU_TARGETS flag * External CI: create a GPU diagnostics template (#3932) * External CI: Add CK into pytorch build environment (#3934) * External CI: add support to disable individual component tests (#3938) * External CI: AMDMIGraphX greater-equal pip dependencies (#3939) * Build(deps): Bump rocm-docs-core from 1.8.2 to 1.8.3 in /docs/sphinx (#3933) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.2 to 1.8.3. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.2...v1.8.3) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * External CI: rocDecode add libva-amdgpu-dev dependency (#3940) * External CI: enumerate GPUs in gpu-diagnostics (#3942) * External CI: move gpu-diag directly before tests (#3943) * External CI: fix HIP_PIPELINE_ID (#3944) --------- Signed-off-by: dependabot[bot] Signed-off-by: Kent Russell Co-authored-by: Jeffrey Novotny Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> Co-authored-by: Wang, Yanyao Co-authored-by: Yanyao Wang Co-authored-by: Peter Park Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> Co-authored-by: Joseph Macaranas <145489236+amd-jmacaran@users.noreply.github.com> Co-authored-by: Daniel Su Co-authored-by: Sandra Polifroni Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com> Co-authored-by: Michael Benavidez Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: MKKnorr Co-authored-by: Kent Russell Co-authored-by: Joseph Greathouse --- .azuredevops/ci-builds/aomp-mainline.yml | 42 +++ .../ci-builds/{aomp.yml => aomp-staging.yml} | 5 +- .azuredevops/components/AMDMIGraphX.yml | 126 +++++-- .azuredevops/components/MIOpen.yml | 159 +++++++-- .azuredevops/components/MIVisionX.yml | 5 +- .azuredevops/components/ROCR-Runtime.yml | 128 ++++++- .azuredevops/components/ROCgdb.yml | 94 ++++- .../components/ROCmValidationSuite.yml | 7 +- .azuredevops/components/Tensile.yml | 169 +++++++++ .azuredevops/components/amdsmi.yml | 5 +- .azuredevops/components/aomp.yml | 3 +- .azuredevops/components/composable_kernel.yml | 7 +- .azuredevops/components/copyHIP.yml | 2 +- .azuredevops/components/hip-tests.yml | 8 +- .azuredevops/components/hipBLAS-common.yml | 2 +- .azuredevops/components/hipBLAS.yml | 7 +- .azuredevops/components/hipBLASLt.yml | 10 +- .azuredevops/components/hipCUB.yml | 5 +- .azuredevops/components/hipFFT.yml | 7 +- .azuredevops/components/hipRAND.yml | 5 +- .azuredevops/components/hipSOLVER.yml | 6 +- .azuredevops/components/hipSPARSE.yml | 6 +- .azuredevops/components/hipSPARSELt.yml | 4 +- .azuredevops/components/hipTensor.yml | 7 +- .azuredevops/components/hipfort.yml | 5 +- .azuredevops/components/omniperf.yml | 166 +++++++++ .azuredevops/components/omnitrace.yml | 139 ++++++++ .azuredevops/components/rccl.yml | 5 +- .azuredevops/components/rdc.yml | 7 +- .azuredevops/components/rocAL.yml | 6 +- .azuredevops/components/rocALUTION.yml | 6 +- .azuredevops/components/rocBLAS.yml | 6 +- .azuredevops/components/rocDecode.yml | 6 +- .azuredevops/components/rocFFT.yml | 5 +- .azuredevops/components/rocMLIR.yml | 13 +- .azuredevops/components/rocPRIM.yml | 5 +- .azuredevops/components/rocPyDecode.yml | 92 ++++- .azuredevops/components/rocRAND.yml | 5 +- .azuredevops/components/rocSOLVER.yml | 6 +- .azuredevops/components/rocSPARSE.yml | 6 +- .azuredevops/components/rocThrust.yml | 5 +- .azuredevops/components/rocWMMA.yml | 8 +- .azuredevops/components/rocm-examples.yml | 42 ++- .../components/rocm_bandwidth_test.yml | 4 +- .azuredevops/components/rocm_smi_lib.yml | 5 +- .azuredevops/components/rocminfo.yml | 5 +- .azuredevops/components/rocprofiler-sdk.yml | 93 +++++ .../components/rocprofiler-systems.yml | 139 ++++++++ .azuredevops/components/rocprofiler.yml | 62 +++- .azuredevops/components/rocr_debug_agent.yml | 3 +- .azuredevops/components/roctracer.yml | 7 +- .azuredevops/components/rpp.yml | 117 +++++- .azuredevops/dependencies/boost.yml | 65 ++++ .azuredevops/dependencies/grpc.yml | 2 +- .azuredevops/dependencies/gtest.yml | 2 +- .azuredevops/dependencies/lapack.yml | 2 +- .azuredevops/nightly/pytorch.yml | 245 ++++++++++++- .azuredevops/nightly/rocm-nightly.yml | 13 +- .azuredevops/tag-builds/Tensile.yml | 29 ++ .azuredevops/tag-builds/boost.yml | 23 ++ .azuredevops/tag-builds/omniperf.yml | 29 ++ .azuredevops/tag-builds/omnitrace.yml | 29 ++ .azuredevops/tag-builds/rocprofiler-sdk.yml | 29 ++ .../tag-builds/rocprofiler-systems.yml | 29 ++ .../templates/steps/artifact-download.yml | 87 ++++- .../templates/steps/build-autotools.yml | 5 +- .../steps/dependencies-aqlprofile.yml | 22 +- .../templates/steps/dependencies-boost.yml | 35 ++ .../templates/steps/dependencies-other.yml | 2 +- .../templates/steps/dependencies-rocm.yml | 26 ++ .../templates/steps/gpu-diagnostics.yml | 54 +++ .azuredevops/variables-global.yml | 34 +- .github/CODEOWNERS | 1 + .wordlist.txt | 4 + README.md | 6 +- default.xml | 2 +- docs/about/license.md | 3 +- docs/compatibility/compatibility-matrix.rst | 2 + docs/conceptual/ai-migraphx-optimization.md | 333 ------------------ docs/conceptual/oversubscription.rst | 34 ++ docs/contribute/building.md | 13 +- docs/contribute/contributing.md | 123 ++----- docs/contribute/doc-structure.md | 219 ------------ docs/contribute/feedback.md | 22 +- docs/contribute/toolchain.md | 39 +- docs/data/contribute/commit-status.png | Bin 0 -> 65570 bytes .../model-quantization.rst | 2 +- .../single-gpu-fine-tuning-and-inference.rst | 2 +- docs/how-to/rocm-for-ai/install.rst | 2 +- docs/how-to/system-optimization/mi100.md | 6 +- docs/how-to/system-optimization/mi200.md | 6 +- docs/how-to/system-optimization/mi300x.rst | 6 +- docs/how-to/system-optimization/w6000-v620.md | 2 +- docs/index.md | 3 +- docs/reference/gpu-arch-specs.rst | 2 + docs/reference/precision-support.rst | 2 + docs/sphinx/_toc.yml.in | 19 +- docs/sphinx/requirements.in | 2 +- docs/sphinx/requirements.txt | 2 +- tools/autotag/components.xml | 2 +- tools/autotag/templates/highlights/5.3.0.md | 2 +- 101 files changed, 2480 insertions(+), 930 deletions(-) create mode 100644 .azuredevops/ci-builds/aomp-mainline.yml rename .azuredevops/ci-builds/{aomp.yml => aomp-staging.yml} (92%) create mode 100644 .azuredevops/components/Tensile.yml create mode 100644 .azuredevops/components/omniperf.yml create mode 100644 .azuredevops/components/omnitrace.yml create mode 100644 .azuredevops/components/rocprofiler-sdk.yml create mode 100644 .azuredevops/components/rocprofiler-systems.yml create mode 100644 .azuredevops/dependencies/boost.yml create mode 100644 .azuredevops/tag-builds/Tensile.yml create mode 100644 .azuredevops/tag-builds/boost.yml create mode 100644 .azuredevops/tag-builds/omniperf.yml create mode 100644 .azuredevops/tag-builds/omnitrace.yml create mode 100644 .azuredevops/tag-builds/rocprofiler-sdk.yml create mode 100644 .azuredevops/tag-builds/rocprofiler-systems.yml create mode 100644 .azuredevops/templates/steps/dependencies-boost.yml create mode 100644 .azuredevops/templates/steps/gpu-diagnostics.yml delete mode 100644 docs/conceptual/ai-migraphx-optimization.md create mode 100644 docs/conceptual/oversubscription.rst delete mode 100644 docs/contribute/doc-structure.md create mode 100644 docs/data/contribute/commit-status.png diff --git a/.azuredevops/ci-builds/aomp-mainline.yml b/.azuredevops/ci-builds/aomp-mainline.yml new file mode 100644 index 000000000..4cc732587 --- /dev/null +++ b/.azuredevops/ci-builds/aomp-mainline.yml @@ -0,0 +1,42 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +resources: + repositories: + - repository: aomp_repo + type: github + endpoint: ROCm + name: ROCm/aomp + ref: amd-mainline-open + - repository: aomp-extras_repo + type: github + endpoint: ROCm + name: ROCm/aomp-extras + ref: amd-mainline-open + - repository: flang_repo + type: github + endpoint: ROCm + name: ROCm/flang + ref: amd-mainline-open + - repository: llvm-project_repo + type: github + endpoint: ROCm + name: ROCm/llvm-project + ref: amd-mainline-open + pipelines: + - pipeline: rocr-runtime_pipeline + source: \ROCR-Runtime + trigger: + branches: + include: + - amd-master +# this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml + parameters: + checkoutRepo: aomp_repo diff --git a/.azuredevops/ci-builds/aomp.yml b/.azuredevops/ci-builds/aomp-staging.yml similarity index 92% rename from .azuredevops/ci-builds/aomp.yml rename to .azuredevops/ci-builds/aomp-staging.yml index 335108adc..57a55cbf3 100644 --- a/.azuredevops/ci-builds/aomp.yml +++ b/.azuredevops/ci-builds/aomp-staging.yml @@ -27,7 +27,10 @@ resources: pipelines: - pipeline: rocr-runtime_pipeline source: \ROCR-Runtime - trigger: true + trigger: + branches: + include: + - amd-staging # this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime trigger: none diff --git a/.azuredevops/components/AMDMIGraphX.yml b/.azuredevops/components/AMDMIGraphX.yml index 6b05f0c90..5e9d2a7e3 100644 --- a/.azuredevops/components/AMDMIGraphX.yml +++ b/.azuredevops/components/AMDMIGraphX.yml @@ -8,30 +8,31 @@ parameters: - name: aptPackages type: object default: - - git - cmake - - ninja-build + - git + - libdnnl-dev - libdrm-dev - - libnuma-dev - - python3-pip - - python3-venv - - libtbb-dev - - nlohmann-json3-dev - libmsgpack-dev - - libsqlite3-dev + - libnuma-dev - libprotobuf-dev + - libsqlite3-dev + - libtbb-dev + - ninja-build + - nlohmann-json3-dev - protobuf-compiler + - python3-pip - python3-pybind11 + - python3-venv - name: pipModules type: object default: - https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz - - onnx==1.14.1 - - numpy==1.21.6 - - typing==3.7.4 - - pytest==6.0.1 - - packaging==23.0 - - protobuf==3.20.2 + - onnx>=1.14.1 + - numpy>=1.21.6 + - typing>=3.7.4 + - pytest>=6.0.1 + - packaging>=23.0 + - protobuf>=3.20.2 - name: rocmDependencies type: object default: @@ -50,6 +51,26 @@ parameters: - rocminfo - rocMLIR - rocprofiler-register + - roctracer +- name: rocmTestDependencies + type: object + default: + - aomp + - aomp-extras + - clr + - composable_kernel + - hipBLAS + - hipBLAS-common + - hipBLASLt + - llvm-project + - MIOpen + - rocm-cmake + - ROCR-Runtime + - rocBLAS + - rocminfo + - rocMLIR + - rocprofiler-register + - roctracer jobs: - job: AMDMIGraphX @@ -63,8 +84,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -95,17 +114,84 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml parameters: extraBuildFlags: >- - -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ - -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_BUILD_TYPE=Release -DGPU_TARGETS=$(JOB_GPU_TARGET) -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm -DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include - -DMIGRAPHX_USE_COMPOSABLEKERNEL=OFF -DBUILD_TESTING=ON -GNinja -# REFERENCE: https://github.com/ROCm/composable_kernel/issues/782 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: gpuTarget: $(JOB_GPU_TARGET) + +- job: AMDMIGraphX_testing + dependsOn: AMDMIGraphX + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + - name: MIGRAPHX_TRACE_BENCHMARKING + value: 1 + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds +# half version should be fixed to 5.6.0 + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencySource: fixed + fixedComponentName: half + fixedPipelineIdentifier: ${{ variables.HALF560_PIPELINE_ID }} + skipLibraryLinking: true + skipLlvmSymlink: true + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmTestDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: CMake@1 + displayName: MIGraphXTest CMake Flags + inputs: + cmakeArgs: >- + -DCMAKE_BUILD_TYPE=Release + -DGPU_TARGETS=$(JOB_GPU_TARGET) + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm + -DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include + -DBUILD_TESTING=ON + -DMIGRAPHX_ENABLE_C_API_TEST=ON + .. + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - task: Bash@3 + displayName: Build and run MIGraphX tests + inputs: + targetType: inline + workingDirectory: build + script: make -j$(nproc) check diff --git a/.azuredevops/components/MIOpen.yml b/.azuredevops/components/MIOpen.yml index dce0edef0..92b0bd899 100644 --- a/.azuredevops/components/MIOpen.yml +++ b/.azuredevops/components/MIOpen.yml @@ -8,15 +8,24 @@ parameters: - name: aptPackages type: object default: - - software-properties-common - - python3-pip - cmake - - ninja-build - - libsqlite3-dev - libbz2-dev - - nlohmann-json3-dev - - libgtest-dev - libdrm-dev + - libeigen3-dev + - libgmock-dev + - libgtest-dev + - libsqlite3-dev + - libstdc++-12-dev + - libzstd-dev + - ninja-build + - nlohmann-json3-dev + - python3-pip + - software-properties-common + - zstd +- name: pipModules + type: object + default: + - cget - name: rocmDependencies type: object default: @@ -35,6 +44,24 @@ parameters: - clr - rocminfo - roctracer +- name: rocmTestDependencies + type: object + default: + - clr + - composable_kernel + - half + - hipBLAS + - hipBLAS-common + - hipBLASLt + - llvm-project + - rocBLAS + - rocm-cmake + - rocminfo + - rocMLIR + - ROCR-Runtime + - rocprofiler-register + - rocRAND + - roctracer jobs: - job: MIOpen @@ -48,28 +75,17 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: aptPackages: ${{ parameters.aptPackages }} - # The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+) - # Upgrade boost from apt by following https://launchpad.net/~mhier/+archive/ubuntu/libboost-latest - - task: Bash@3 - displayName: 'update boost version' - inputs: - targetType: inline - script: sudo add-apt-repository ppa:mhier/libboost-latest -y - - task: Bash@3 - displayName: 'install boost' - inputs: - targetType: inline - script: sudo apt-get --yes install libboost1.83-dev libboost-system1.83-dev libboost-filesystem1.83-dev + pipModules: ${{ parameters.pipModules }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters: checkoutRepo: ${{ parameters.checkoutRepo }} + # The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-boost.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml parameters: dependencyList: ${{ parameters.rocmDependencies }} @@ -85,7 +101,8 @@ jobs: extraBuildFlags: >- -DMIOPEN_BACKEND=HIP -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ - -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/boost + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) -DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF -DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF -DCMAKE_BUILD_TYPE=Release @@ -94,3 +111,103 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: gpuTarget: $(JOB_GPU_TARGET) + +- job: MIOpen_testing + dependsOn: MIOpen + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + # The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-boost.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmTestDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds +# MIOpen depends on a specific version of frugally-deep which is forked here: https://github.com/ROCm/frugally-deep +# https://github.com/ROCm/frugally-deep/blob/master/INSTALL.md + - task: Bash@3 + displayName: Add Python site-packages binaries to path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.prependpath]$USER_BASE/bin" + - task: Bash@3 + displayName: Install FunctionalPlus + inputs: + targetType: inline + script: cget install Dobiasd/FunctionalPlus + - task: Bash@3 + displayName: Remove Python site-packages binaries from path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: git clone frugally-deep + inputs: + targetType: inline + script: git clone https://github.com/ROCm/frugally-deep --depth=1 --shallow-submodules + workingDirectory: $(Build.SourcesDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: + componentName: frugally-deep + cmakeBuildDir: $(Build.SourcesDirectory)/frugally-deep/build + installDir: $(Build.SourcesDirectory)/bin + extraBuildFlags: -DCMAKE_PREFIX_PATH=$(Build.SourcesDirectory)/cget/cget/pkg/Dobiasd__FunctionalPlus/install + - task: CMake@1 + displayName: 'MIOpen Test CMake Flags' + inputs: + cmakeArgs: >- + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Build.SourcesDirectory)/bin;$(Build.SourcesDirectory)/cget/cget/pkg/Dobiasd__FunctionalPlus/install;$(Agent.BuildDirectory)/boost + -DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocm + -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ + -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang + -DMIOPEN_BACKEND=HIP + -DMIOPEN_TEST_FLAGS=" --disable-verification-cache" + -DCMAKE_BUILD_TYPE=release + -DBUILD_DEV=OFF + -DMIOPEN_USE_MLIR=ON + -DMIOPEN_GPU_SYNC=OFF + .. + - task: Bash@3 + displayName: 'MIOpen Test Build' + inputs: + targetType: inline + script: | + cmake --build . --target tests -- -j$(nproc) + workingDirectory: $(Build.SourcesDirectory)/build + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: MIOpen + reloadAMDGPU: true diff --git a/.azuredevops/components/MIVisionX.yml b/.azuredevops/components/MIVisionX.yml index 5082d3971..4af1a2212 100644 --- a/.azuredevops/components/MIVisionX.yml +++ b/.azuredevops/components/MIVisionX.yml @@ -86,8 +86,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -120,7 +118,7 @@ jobs: - job: MIVisionX_testing dependsOn: MIVisionX - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -162,6 +160,7 @@ jobs: mkdir mivisionx-tests cd mivisionx-tests cmake /opt/rocm/share/mivisionx/test + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: MIVisionX diff --git a/.azuredevops/components/ROCR-Runtime.yml b/.azuredevops/components/ROCR-Runtime.yml index b342ee080..73b7cfa34 100644 --- a/.azuredevops/components/ROCR-Runtime.yml +++ b/.azuredevops/components/ROCR-Runtime.yml @@ -8,15 +8,21 @@ parameters: - name: aptPackages type: object default: - - libelf-dev - g++ - libdrm-dev + - libelf-dev - libnuma-dev - name: rocmDependencies type: object default: - llvm-project - rocprofiler-register +- name: rocmTestDependencies + type: object + default: + - llvm-project + - rocm_smi_lib + - rocprofiler-register jobs: - job: ROCR_Runtime @@ -50,3 +56,123 @@ jobs: -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DBUILD_SHARED_LIBS=ON - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + +- job: ROCR_Runtime_testing + dependsOn: ROCR_Runtime + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + - task: Bash@3 + displayName: Install libhwloc5 + inputs: + targetType: 'inline' + script: | + wget http://ftp.us.debian.org/debian/pool/main/h/hwloc/libhwloc5_1.11.12-3_amd64.deb + wget http://ftp.us.debian.org/debian/pool/main/h/hwloc/libhwloc-dev_1.11.12-3_amd64.deb + sudo apt install -y --allow-downgrades ./libhwloc5_1.11.12-3_amd64.deb ./libhwloc-dev_1.11.12-3_amd64.deb + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmTestDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + parameters: + runRocminfo: false + - task: Bash@3 + displayName: Build kfdtest + inputs: + targetType: 'inline' + workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/kfdtest + script: | + mkdir build && cd build + cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm .. + make + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: kfdtest + testExecutable: BIN_DIR=$(Build.SourcesDirectory)/libhsakmt/tests/kfdtest/build ./run_kfdtest.sh + testParameters: '-p core --gtest_output=xml:./test_output.xml --gtest_color=yes' + testDir: $(Build.SourcesDirectory)/libhsakmt/tests/kfdtest/scripts + - task: Bash@3 + displayName: Build rdmatest app + inputs: + targetType: 'inline' + workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/app + script: | + cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DLIBHSAKMT_PATH=$(Agent.BuildDirectory)/rocm -DDRM_AMDGPU_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include . + cmake --build . + - task: Bash@3 + displayName: Build rdmatest driver + inputs: + targetType: 'inline' + workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/drv + script: | + sed -i 's/HSAKMT_PAGE_SHIFT/PAGE_SHIFT/g' amdp2ptest.c + sed -i 's/"MIT"/"GPL"/' amdp2ptest.c + RDMA_HEADER_DIR=/usr/src/amdgpu-*/include make all + - task: Bash@3 + displayName: Install rdmatest driver + inputs: + targetType: 'inline' + workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/drv + script: | + sudo rmmod amdp2ptest.ko + sudo insmod amdp2ptest.ko + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rdmatest + testExecutable: yes | ./rdma_test + testParameters: '' + testDir: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/app + testPublishResults: false + - task: Bash@3 + displayName: Build rocrtst + inputs: + targetType: 'inline' + workingDirectory: $(Build.SourcesDirectory)/rocrtst/suites/test_common + script: | + sudo rm -rf $(Agent.BuildDirectory)/external/llvm-project + mkdir -p $(Agent.BuildDirectory)/external/llvm-project/clang/lib + sudo ln -s $(Agent.BuildDirectory)/rocm/llvm/lib/clang/20/include $(Agent.BuildDirectory)/external/llvm-project/clang/lib/Headers + mkdir build && cd build + cmake .. \ + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm \ + -DTARGET_DEVICES=$(JOB_GPU_TARGET) \ + -DROCM_DIR=$(Agent.BuildDirectory)/rocm \ + -DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin \ + -DOPENCL_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin + make + make rocrtst_kernels + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocrtst + testExecutable: ./rocrtst64 + testParameters: '--gtest_filter="-rocrtstNeg.Memory_Negative_Tests" --gtest_output=xml:./test_output.xml --gtest_color=yes' + testDir: $(Build.SourcesDirectory)/rocrtst/suites/test_common/build/$(JOB_GPU_TARGET) diff --git a/.azuredevops/components/ROCgdb.yml b/.azuredevops/components/ROCgdb.yml index 5665bbbee..d41a80802 100644 --- a/.azuredevops/components/ROCgdb.yml +++ b/.azuredevops/components/ROCgdb.yml @@ -5,24 +5,47 @@ parameters: - name: checkoutRef type: string default: '' +# reference: https://github.com/ROCm/ROCgdb/blob/amd-staging/README-ROCM.md - name: aptPackages type: object default: - - libgmp-dev - - libmpfr-dev - - texinfo - bison + - dejagnu - flex + - libbabeltrace-dev + - libexpat-dev + - libgmp-dev + - liblzma-dev + - libmpfr-dev + - ncurses-dev + - texinfo + - zlib1g-dev +- name: rocmDependencies + type: object + default: + - clr + - llvm-project + - ROCdbgapi + - rocminfo + - rocprofiler-register + - ROCR-Runtime jobs: -- job: rocgdb +- job: ROCgdb + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml - pool: - vmImage: ${{ variables.BASE_BUILD_POOL }} + - name: PKG_CONFIG_PATH + value: $(Agent.BuildDirectory)/rocm/share/pkgconfig + pool: $(JOB_TEST_POOL) workspace: clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -31,5 +54,64 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters: checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml + parameters: + configureFlags: >- + --program-prefix=roc + --enable-64-bit-bfd + --enable-targets="x86_64-linux-gnu,amdgcn-amd-amdhsa" + --disable-ld + --disable-gas + --disable-gdbserver + --disable-sim + --enable-tui + --disable-gdbtk + --disable-shared + --disable-gprofng + --with-expat + --with-system-zlib + --without-guile + --with-babeltrace + --with-lzma + --with-python=python3 + --with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm + LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib" + makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + - task: Bash@3 + displayName: Setup test environment + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + echo "##vso[task.prependpath]/opt/rocm/bin" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - task: Bash@3 + displayName: check-gdb + continueOnError: true + inputs: + targetType: inline + script: make check-gdb TESTS=gdb.rocm/simple.exp + workingDirectory: $(Build.SourcesDirectory) + - task: Bash@3 + displayName: print gdb log + inputs: + targetType: inline + script: find -name gdb.log -exec cat {} \; + workingDirectory: $(Build.SourcesDirectory) diff --git a/.azuredevops/components/ROCmValidationSuite.yml b/.azuredevops/components/ROCmValidationSuite.yml index 8517dc387..7faa864b1 100644 --- a/.azuredevops/components/ROCmValidationSuite.yml +++ b/.azuredevops/components/ROCmValidationSuite.yml @@ -40,9 +40,11 @@ parameters: - llvm-project - rocBLAS - rocm_smi_lib + - rocminfo - rocprofiler-register - ROCR-Runtime - rocRAND + - roctracer jobs: - job: ROCmValidationSuite @@ -63,8 +65,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -97,7 +97,7 @@ jobs: - job: ROCmValidationSuite_testing dependsOn: ROCmValidationSuite - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -131,6 +131,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: ROCmValidationSuite diff --git a/.azuredevops/components/Tensile.yml b/.azuredevops/components/Tensile.yml new file mode 100644 index 000000000..bd3b01fdb --- /dev/null +++ b/.azuredevops/components/Tensile.yml @@ -0,0 +1,169 @@ +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: aptPackages + type: object + default: + - python3-pip + - cmake + - libmsgpack-dev + - libboost-program-options-dev +- name: pipModules + type: object + default: + - tox + - pytest +- name: rocmDependencies + type: object + default: + - aomp + - clr + - llvm-project + - rocm-cmake + - rocm-core + - rocminfo + - rocm_smi_lib + - rocprofiler-register + - ROCR-Runtime + +jobs: +- job: Tensile + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: + vmImage: ${{ variables.BASE_BUILD_POOL }} + workspace: + clean: all + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: Create wheel file + inputs: + targetType: inline + script: python3 setup.py bdist_wheel + workingDirectory: $(Build.SourcesDirectory) + env: + ROCM_PATH: $(Agent.BuildDirectory)/rocm + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml + parameters: + sourceDir: $(Build.SourcesDirectory)/dist + contentsString: '*.whl' + targetDir: $(Build.ArtifactStagingDirectory) + clean: false + - task: PublishPipelineArtifact@1 + displayName: 'wheel file Publish' + retryCountOnTaskFailure: 3 + inputs: + targetPath: $(Build.ArtifactStagingDirectory) + +- job: Tensile_testing + dependsOn: Tensile + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - task: DownloadPipelineArtifact@2 + displayName: 'Download Pipeline Wheel Files' + inputs: + itemPattern: '**/*.whl' + targetPath: $(Agent.BuildDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: pip install + inputs: + targetType: inline + script: find -name *.whl -exec pip install {} \; + workingDirectory: $(Agent.BuildDirectory) + - task: Bash@3 + displayName: Setup test environment + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + - task: Bash@3 + displayName: Add Python site-packages binaries to path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.prependpath]$USER_BASE/bin" + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add ROCm compilers to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - task: Bash@3 + displayName: tox test + inputs: + targetType: inline + script: tox run -v -e ci -- -m pre_checkin + workingDirectory: $(Build.SourcesDirectory) + - task: Bash@3 + displayName: Remove Python site-packages binaries from path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm binaries from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm compilers from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')" diff --git a/.azuredevops/components/amdsmi.yml b/.azuredevops/components/amdsmi.yml index b6d8a2ec8..5b10addef 100644 --- a/.azuredevops/components/amdsmi.yml +++ b/.azuredevops/components/amdsmi.yml @@ -35,7 +35,7 @@ jobs: - job: amdsmi_testing dependsOn: amdsmi - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -53,6 +53,9 @@ jobs: aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + parameters: + runRocminfo: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: amdsmi diff --git a/.azuredevops/components/aomp.yml b/.azuredevops/components/aomp.yml index e8bbcdfe5..9a443ac3c 100644 --- a/.azuredevops/components/aomp.yml +++ b/.azuredevops/components/aomp.yml @@ -413,7 +413,7 @@ jobs: - job: aomp_testing dependsOn: aomp - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -472,6 +472,7 @@ jobs: Contents: FileCheck TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests/bin retryCount: 3 + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - task: Bash@3 displayName: Test AOMP continueOnError: true diff --git a/.azuredevops/components/composable_kernel.yml b/.azuredevops/components/composable_kernel.yml index b0202474e..3550ec728 100644 --- a/.azuredevops/components/composable_kernel.yml +++ b/.azuredevops/components/composable_kernel.yml @@ -48,8 +48,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -90,6 +88,8 @@ jobs: -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_HIP_FLAGS="-Wno-missing-include-dirs" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DCK_BUILD_JIT_LIB=ON + -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release -DGPU_TARGETS=$(JOB_GPU_TARGET) -GNinja @@ -99,7 +99,7 @@ jobs: - job: composable_kernel_testing dependsOn: composable_kernel - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -135,6 +135,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - task: Bash@3 displayName: Iterate through test scripts inputs: diff --git a/.azuredevops/components/copyHIP.yml b/.azuredevops/components/copyHIP.yml index a925202ed..a1770d393 100644 --- a/.azuredevops/components/copyHIP.yml +++ b/.azuredevops/components/copyHIP.yml @@ -25,7 +25,7 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml parameters: componentName: HIP - pipelineId: $(hip-pipeline-id) + pipelineId: $(HIP_PIPELINE_ID) - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml parameters: sourceDir: $(Agent.BuildDirectory)/rocm diff --git a/.azuredevops/components/hip-tests.yml b/.azuredevops/components/hip-tests.yml index 9ea74926e..49dc6959c 100644 --- a/.azuredevops/components/hip-tests.yml +++ b/.azuredevops/components/hip-tests.yml @@ -52,8 +52,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -89,9 +87,9 @@ jobs: gpuTarget: $(JOB_GPU_TARGET) - job: hip_tests_testing - timeoutInMinutes: 180 + timeoutInMinutes: 240 dependsOn: hip_tests - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -130,8 +128,10 @@ jobs: inputs: targetType: inline script: | + sudo rm -rf /opt/rocm sudo mkdir -p /opt/rocm/bin sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hip_tests diff --git a/.azuredevops/components/hipBLAS-common.yml b/.azuredevops/components/hipBLAS-common.yml index 2fb39a591..775ec8ba3 100644 --- a/.azuredevops/components/hipBLAS-common.yml +++ b/.azuredevops/components/hipBLAS-common.yml @@ -29,7 +29,7 @@ jobs: - name: ROCM_PATH value: $(Agent.BuildDirectory)/rocm - template: /.azuredevops/variables-global.yml - pool: + pool: vmImage: ${{ variables.BASE_BUILD_POOL }} workspace: clean: all diff --git a/.azuredevops/components/hipBLAS.yml b/.azuredevops/components/hipBLAS.yml index 050b4f488..e71232cbc 100644 --- a/.azuredevops/components/hipBLAS.yml +++ b/.azuredevops/components/hipBLAS.yml @@ -16,7 +16,6 @@ parameters: - libgtest-dev - wget - python3-pip - - libomp-dev - name: rocmDependencies type: object default: @@ -46,6 +45,7 @@ parameters: - rocBLAS - rocSPARSE - rocSOLVER + - roctracer jobs: - job: hipBLAS @@ -59,8 +59,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -98,7 +96,7 @@ jobs: - job: hipBLAS_testing dependsOn: hipBLAS - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -132,6 +130,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipBLAS diff --git a/.azuredevops/components/hipBLASLt.yml b/.azuredevops/components/hipBLASLt.yml index c60cb8e40..ee610bfdc 100644 --- a/.azuredevops/components/hipBLASLt.yml +++ b/.azuredevops/components/hipBLASLt.yml @@ -16,13 +16,13 @@ parameters: - python3-pip - python3-venv - gfortran - - libomp-dev - libblas-dev - ccache - name: pipModules type: object default: - joblib + - packaging - name: rocmDependencies type: object default: @@ -66,6 +66,8 @@ jobs: value: $(Agent.BuildDirectory)/rocm/bin/hipcc - name: PATH value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin + - name: DAY_STRING + value: $[format('{0:ddMMyyyy}', pipeline.startTime)] pool: ${{ variables.ULTRA_BUILD_POOL }} workspace: clean: all @@ -73,8 +75,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -149,7 +149,7 @@ jobs: - job: hipBLASLt_testing dependsOn: hipBLASLt - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -165,6 +165,7 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml parameters: @@ -183,6 +184,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipBLASLt diff --git a/.azuredevops/components/hipCUB.yml b/.azuredevops/components/hipCUB.yml index 29a7f07e0..e81c8be82 100644 --- a/.azuredevops/components/hipCUB.yml +++ b/.azuredevops/components/hipCUB.yml @@ -43,8 +43,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -78,7 +76,7 @@ jobs: - job: hipCUB_testing dependsOn: hipCUB - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -112,6 +110,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipCUB diff --git a/.azuredevops/components/hipFFT.yml b/.azuredevops/components/hipFFT.yml index 9460f4b7b..38212a5cf 100644 --- a/.azuredevops/components/hipFFT.yml +++ b/.azuredevops/components/hipFFT.yml @@ -14,11 +14,11 @@ parameters: - libgtest-dev - libfftw3-dev - python3-pip - - libomp-14-dev # rocm dependencies should match dependencies-rocm.yml - name: rocmDependencies type: object default: + - aomp - rocRAND - hipRAND - llvm-project @@ -55,8 +55,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -96,7 +94,7 @@ jobs: - job: hipFFT_testing dependsOn: hipFFT - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -130,6 +128,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipFFT diff --git a/.azuredevops/components/hipRAND.yml b/.azuredevops/components/hipRAND.yml index c3d341f8f..767c9f7ab 100644 --- a/.azuredevops/components/hipRAND.yml +++ b/.azuredevops/components/hipRAND.yml @@ -45,8 +45,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -82,7 +80,7 @@ jobs: - job: hipRAND_testing dependsOn: hipRAND - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -116,6 +114,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipRAND diff --git a/.azuredevops/components/hipSOLVER.yml b/.azuredevops/components/hipSOLVER.yml index 7e7f6ee1f..60f2141d1 100644 --- a/.azuredevops/components/hipSOLVER.yml +++ b/.azuredevops/components/hipSOLVER.yml @@ -42,6 +42,7 @@ parameters: - ROCR-Runtime - rocSOLVER - rocSPARSE + - roctracer jobs: - job: hipSOLVER @@ -56,8 +57,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -102,7 +101,7 @@ jobs: - job: hipSOLVER_testing dependsOn: hipSOLVER - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -136,6 +135,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipSOLVER diff --git a/.azuredevops/components/hipSPARSE.yml b/.azuredevops/components/hipSPARSE.yml index 849c27a97..c433371f1 100644 --- a/.azuredevops/components/hipSPARSE.yml +++ b/.azuredevops/components/hipSPARSE.yml @@ -37,6 +37,7 @@ parameters: - rocprofiler-register - ROCR-Runtime - rocSPARSE + - roctracer jobs: - job: hipSPARSE @@ -51,8 +52,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -96,7 +95,7 @@ jobs: - job: hipSPARSE_testing dependsOn: hipSPARSE - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -130,6 +129,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipSPARSE diff --git a/.azuredevops/components/hipSPARSELt.yml b/.azuredevops/components/hipSPARSELt.yml index 869412822..67b8ba808 100644 --- a/.azuredevops/components/hipSPARSELt.yml +++ b/.azuredevops/components/hipSPARSELt.yml @@ -15,7 +15,6 @@ parameters: - python3-pip - gfortran - libgfortran5 - - libomp-dev - libopenblas-dev - name: pipModules type: object @@ -124,7 +123,7 @@ jobs: - job: hipSPARSELt_testing dependsOn: hipSPARSELt - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -152,6 +151,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipSPARSELt diff --git a/.azuredevops/components/hipTensor.yml b/.azuredevops/components/hipTensor.yml index 7cd71702b..d12cfaac3 100644 --- a/.azuredevops/components/hipTensor.yml +++ b/.azuredevops/components/hipTensor.yml @@ -42,8 +42,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -79,7 +77,7 @@ jobs: - job: hipTensor_testing timeoutInMinutes: 90 dependsOn: hipTensor - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -113,8 +111,9 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: hipTensor testDir: '$(Agent.BuildDirectory)/rocm/bin/hiptensor' - testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' + testParameters: '-E ".*-extended" -VV --output-on-failure --force-new-ctest-process --output-junit test_output.xml' diff --git a/.azuredevops/components/hipfort.yml b/.azuredevops/components/hipfort.yml index 034f36c14..3a3e70387 100644 --- a/.azuredevops/components/hipfort.yml +++ b/.azuredevops/components/hipfort.yml @@ -51,8 +51,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -94,7 +92,7 @@ jobs: - job: hipfort_testing dependsOn: hipfort - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -139,6 +137,7 @@ jobs: sudo rm -rf /opt/rocm sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm workingDirectory: $(Build.SourcesDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - task: Bash@3 displayName: 'Test hipfort' inputs: diff --git a/.azuredevops/components/omniperf.yml b/.azuredevops/components/omniperf.yml new file mode 100644 index 000000000..818d7ca0d --- /dev/null +++ b/.azuredevops/components/omniperf.yml @@ -0,0 +1,166 @@ +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: aptPackages + type: object + default: + - cmake + - python3-pip +- name: pipModules + type: object + default: + - astunparse==1.6.2 + - colorlover + - dash>=1.12.0 + - matplotlib + - numpy>=1.17.5 + - pandas>=1.4.3 + - pymongo + - pyyaml + - tabulate + - tqdm + - dash-svg + - dash-bootstrap-components + - kaleido + - setuptools + - plotille + - mock + - pytest + - pytest-cov + - pytest-xdist +- name: rocmDependencies + type: object + default: + - clr + - llvm-project + - rocm-cmake + - rocm-core + - rocminfo + - ROCR-Runtime + - rocprofiler + - rocprofiler-register + - roctracer + +jobs: +- job: omniperf + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: + vmImage: ${{ variables.BASE_BUILD_POOL }} + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) + +- job: omniperf_testing + dependsOn: omniperf + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + - name: PYTHON_VERSION + value: 3.10 + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add ROCm compilers to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: + extraBuildFlags: >- + -DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET) + -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang + -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DCMAKE_BUILD_TYPE=Release + -DENABLE_TESTS=ON + -DINSTALL_TESTS=ON + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: omniperf + testDir: $(Build.BinariesDirectory)/libexec/omniperf + testExecutable: export OMNIPERF_ARCH_OVERRIDE="MI300X"; ctest + - task: Bash@3 + displayName: Remove ROCm binaries from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm compilers from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')" diff --git a/.azuredevops/components/omnitrace.yml b/.azuredevops/components/omnitrace.yml new file mode 100644 index 000000000..cdba2ba59 --- /dev/null +++ b/.azuredevops/components/omnitrace.yml @@ -0,0 +1,139 @@ +# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: aptPackages + type: object + default: + - autoconf + - autotools-dev + - bison + - build-essential + - bzip2 + - clang + - cmake + - environment-modules + - g++-12 + - libdrm-dev + - libfabric-dev + - libiberty-dev + - libpapi-dev + - libpfm4-dev + - libtool + - libopenmpi-dev + - m4 + - openmpi-bin + - software-properties-common + - python3-pip + - texinfo + - zlib1g-dev +- name: pipModules + type: object + default: + - numpy + - perfetto + - dataclasses +- name: rocmDependencies + type: object + default: + - clr + - llvm-project + - rccl + - rocm-core + - rocm_smi_lib + - rocminfo + - ROCR-Runtime + - rocprofiler + - rocprofiler-register + - roctracer + +jobs: +- job: omnitrace + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: ${{ variables.MEDIUM_BUILD_POOL }} + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: ROCm symbolic link + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add ROCm compilers to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: +# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html + extraBuildFlags: >- + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DOMNITRACE_BUILD_TESTING=ON + -DOMNITRACE_BUILD_DYNINST=ON + -DOMNITRACE_BUILD_LIBUNWIND=ON + -DDYNINST_BUILD_TBB=ON + -DDYNINST_BUILD_ELFUTILS=ON + -DDYNINST_BUILD_LIBIBERTY=ON + -DDYNINST_BUILD_BOOST=ON + -DOMNITRACE_USE_PAPI=ON + -DOMNITRACE_USE_MPI=ON + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + multithreadFlag: -- -j32 + - task: Bash@3 + displayName: Set up omnitrace env + inputs: + targetType: inline + script: source share/omnitrace/setup-env.sh + workingDirectory: build + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: omnitrace + - task: Bash@3 + displayName: Remove ROCm binaries from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm compilers from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) diff --git a/.azuredevops/components/rccl.yml b/.azuredevops/components/rccl.yml index 845598500..0a7b59578 100644 --- a/.azuredevops/components/rccl.yml +++ b/.azuredevops/components/rccl.yml @@ -60,8 +60,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -102,7 +100,7 @@ jobs: - job: rccl_testing timeoutInMinutes: 120 dependsOn: rccl - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -136,6 +134,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rccl diff --git a/.azuredevops/components/rdc.yml b/.azuredevops/components/rdc.yml index a4382e36e..f51b83a9c 100644 --- a/.azuredevops/components/rdc.yml +++ b/.azuredevops/components/rdc.yml @@ -60,8 +60,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -113,7 +111,7 @@ jobs: - job: rdc_testing dependsOn: rdc - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -131,6 +129,8 @@ jobs: aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml parameters: ${{ if eq(parameters.checkoutRef, '') }}: @@ -156,6 +156,7 @@ jobs: sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rdcd /usr/sbin/rdcd echo $(Agent.BuildDirectory)/rocm/lib/rdc/grpc/lib | sudo tee /etc/ld.so.conf.d/grpc.conf sudo ldconfig -v + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - task: Bash@3 displayName: Test rdc inputs: diff --git a/.azuredevops/components/rocAL.yml b/.azuredevops/components/rocAL.yml index 11ac334e2..82c8c785f 100644 --- a/.azuredevops/components/rocAL.yml +++ b/.azuredevops/components/rocAL.yml @@ -53,6 +53,7 @@ parameters: - half - llvm-project - MIVisionX + - rocminfo - rocprofiler-register - ROCR-Runtime - rpp @@ -70,8 +71,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - task: Bash@3 displayName: 'Register libjpeg-turbo packages' @@ -157,7 +156,7 @@ jobs: - job: rocAL_testing dependsOn: rocAL - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -220,6 +219,7 @@ jobs: mkdir rocAL-tests cd rocAL-tests cmake $(Agent.BuildDirectory)/rocm/share/rocal/test + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocAL diff --git a/.azuredevops/components/rocALUTION.yml b/.azuredevops/components/rocALUTION.yml index a596da4a5..58c4eacb8 100644 --- a/.azuredevops/components/rocALUTION.yml +++ b/.azuredevops/components/rocALUTION.yml @@ -43,6 +43,7 @@ parameters: - ROCR-Runtime - rocRAND - rocSPARSE + - roctracer jobs: - job: rocALUTION @@ -59,8 +60,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -97,7 +96,7 @@ jobs: - job: rocALUTION_testing dependsOn: rocALUTION - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -131,6 +130,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocALUTION diff --git a/.azuredevops/components/rocBLAS.yml b/.azuredevops/components/rocBLAS.yml index a8331966c..8d18715de 100644 --- a/.azuredevops/components/rocBLAS.yml +++ b/.azuredevops/components/rocBLAS.yml @@ -52,6 +52,7 @@ parameters: - rocprofiler-register - rocm_smi_lib - ROCR-Runtime + - roctracer jobs: - job: rocBLAS @@ -73,8 +74,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -120,7 +119,7 @@ jobs: - job: rocBLAS_testing dependsOn: rocBLAS - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -155,6 +154,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocBLAS diff --git a/.azuredevops/components/rocDecode.yml b/.azuredevops/components/rocDecode.yml index e94a7e411..46b24ac2d 100644 --- a/.azuredevops/components/rocDecode.yml +++ b/.azuredevops/components/rocDecode.yml @@ -17,7 +17,7 @@ parameters: - libavformat-dev - libavutil-dev - libstdc++-12-dev - - libva-dev + - libva-amdgpu-dev - mesa-amdgpu-va-drivers - libdrm-dev - name: rocmDependencies @@ -35,6 +35,7 @@ parameters: default: - clr - llvm-project + - rocminfo - rocprofiler-register - ROCR-Runtime @@ -88,7 +89,7 @@ jobs: - job: rocDecode_testing dependsOn: rocDecode - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -128,6 +129,7 @@ jobs: mkdir rocDecode-tests cd rocDecode-tests cmake /opt/rocm/share/rocdecode/test + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocDecode diff --git a/.azuredevops/components/rocFFT.yml b/.azuredevops/components/rocFFT.yml index 06274c702..5d1d1f2bb 100644 --- a/.azuredevops/components/rocFFT.yml +++ b/.azuredevops/components/rocFFT.yml @@ -55,8 +55,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -95,7 +93,7 @@ jobs: - job: rocFFT_testing dependsOn: rocFFT - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -129,6 +127,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocFFT diff --git a/.azuredevops/components/rocMLIR.yml b/.azuredevops/components/rocMLIR.yml index 8b4979ccc..ecfa0f2b5 100644 --- a/.azuredevops/components/rocMLIR.yml +++ b/.azuredevops/components/rocMLIR.yml @@ -13,6 +13,7 @@ parameters: - git - python3-pip - libdrm-dev + - libstdc++-12-dev - name: pipModules type: object default: @@ -28,7 +29,7 @@ parameters: - ROCR-Runtime jobs: -- job: rocMLIR_library +- job: rocMLIR variables: - group: common - template: /.azuredevops/variables-global.yml @@ -66,16 +67,19 @@ jobs: # compiling and running test on the test system together - job: rocMLIR_testing + dependsOn: rocMLIR + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml - pool: ${{ variables.GFX942_TEST_POOL }} + pool: $(JOB_TEST_POOL) workspace: clean: all strategy: matrix: gfx942: JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -105,7 +109,9 @@ jobs: displayName: ROCm symbolic link inputs: targetType: inline - script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml parameters: installEnabled: false @@ -117,6 +123,7 @@ jobs: -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) -DROCM_TEST_CHIPSET=$(JOB_GPU_TARGET) -GNinja + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocMLIR diff --git a/.azuredevops/components/rocPRIM.yml b/.azuredevops/components/rocPRIM.yml index ef6f35cdb..a620c4749 100644 --- a/.azuredevops/components/rocPRIM.yml +++ b/.azuredevops/components/rocPRIM.yml @@ -42,8 +42,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -77,7 +75,7 @@ jobs: - job: rocPRIM_testing dependsOn: rocPRIM - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -111,6 +109,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocPRIM diff --git a/.azuredevops/components/rocPyDecode.yml b/.azuredevops/components/rocPyDecode.yml index 2383dabd5..ee9ecccf5 100644 --- a/.azuredevops/components/rocPyDecode.yml +++ b/.azuredevops/components/rocPyDecode.yml @@ -52,8 +52,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -73,6 +71,13 @@ jobs: # manual build case: triggered by ROCm/ROCm repo ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - task: Bash@3 + displayName: 'ROCm symbolic link' + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml parameters: extraBuildFlags: >- @@ -85,3 +90,86 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: gpuTarget: $(JOB_GPU_TARGET) + publish: false + - task: Bash@3 + displayName: Create wheel file + inputs: + targetType: inline + script: | + export ROCM_PATH=$(Agent.BuildDirectory)/rocm + export HIP_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include/hip + python3 setup.py bdist_wheel + workingDirectory: $(Build.SourcesDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml + parameters: + sourceDir: $(Build.SourcesDirectory)/dist + contentsString: '*.whl' + targetDir: $(Build.ArtifactStagingDirectory) + clean: false + - task: PublishPipelineArtifact@1 + displayName: 'wheel file Publish' + retryCountOnTaskFailure: 3 + inputs: + targetPath: $(Build.ArtifactStagingDirectory) + +- job: rocPyDecode_testing + dependsOn: rocPyDecode + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - task: Bash@3 + displayName: Setup test environment + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + cd $(Build.SourcesDirectory) + sudo pip install . + cmake -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) . + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocPyDecode + testDir: $(Build.SourcesDirectory) +# sudo required for pip install but screws up permissions for next pipeline run + - task: Bash@3 + displayName: Clean up test environment + inputs: + targetType: inline + script: | + sudo rm -rf $(Build.SourcesDirectory)/* diff --git a/.azuredevops/components/rocRAND.yml b/.azuredevops/components/rocRAND.yml index a3178779c..81b320a77 100644 --- a/.azuredevops/components/rocRAND.yml +++ b/.azuredevops/components/rocRAND.yml @@ -45,8 +45,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -79,7 +77,7 @@ jobs: - job: rocRAND_testing dependsOn: rocRAND - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -113,6 +111,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocRAND diff --git a/.azuredevops/components/rocSOLVER.yml b/.azuredevops/components/rocSOLVER.yml index 286d80a26..326ce2618 100644 --- a/.azuredevops/components/rocSOLVER.yml +++ b/.azuredevops/components/rocSOLVER.yml @@ -44,6 +44,7 @@ parameters: - rocprofiler-register - ROCR-Runtime - rocSPARSE + - roctracer jobs: - job: rocSOLVER @@ -57,8 +58,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -112,7 +111,7 @@ jobs: - job: rocSOLVER_testing dependsOn: rocSOLVER - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -146,6 +145,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocSOLVER diff --git a/.azuredevops/components/rocSPARSE.yml b/.azuredevops/components/rocSPARSE.yml index 5426e7acd..cca206ae2 100644 --- a/.azuredevops/components/rocSPARSE.yml +++ b/.azuredevops/components/rocSPARSE.yml @@ -40,6 +40,7 @@ parameters: - rocminfo - rocprofiler-register - ROCR-Runtime + - roctracer jobs: - job: rocSPARSE @@ -55,8 +56,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -106,7 +105,7 @@ jobs: - job: rocSPARSE_testing timeoutInMinutes: 90 dependsOn: rocSPARSE - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -140,6 +139,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocSPARSE diff --git a/.azuredevops/components/rocThrust.yml b/.azuredevops/components/rocThrust.yml index 39136d0e6..6fbd32c2d 100644 --- a/.azuredevops/components/rocThrust.yml +++ b/.azuredevops/components/rocThrust.yml @@ -47,8 +47,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -82,7 +80,7 @@ jobs: - job: rocThrust_testing dependsOn: rocThrust - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -116,6 +114,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocThrust diff --git a/.azuredevops/components/rocWMMA.yml b/.azuredevops/components/rocWMMA.yml index bd8a4c4b5..ff08ff99d 100644 --- a/.azuredevops/components/rocWMMA.yml +++ b/.azuredevops/components/rocWMMA.yml @@ -16,10 +16,10 @@ parameters: - googletest - libfftw3-dev - git - - libomp-dev - name: rocmDependencies type: object default: + - aomp - rocm-cmake - llvm-project - ROCR-Runtime @@ -42,6 +42,7 @@ parameters: - rocminfo - rocprofiler-register - ROCR-Runtime + - roctracer jobs: - job: rocWMMA @@ -55,8 +56,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -93,7 +92,7 @@ jobs: - job: rocWMMA_testing timeoutInMinutes: 90 dependsOn: rocWMMA - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -127,6 +126,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocWMMA diff --git a/.azuredevops/components/rocm-examples.yml b/.azuredevops/components/rocm-examples.yml index f27ced15d..c01e74688 100644 --- a/.azuredevops/components/rocm-examples.yml +++ b/.azuredevops/components/rocm-examples.yml @@ -60,6 +60,7 @@ parameters: - rocSOLVER - rocSPARSE - rocThrust + - roctracer jobs: - job: rocm_examples @@ -73,8 +74,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -116,7 +115,7 @@ jobs: - job: rocm_examples_testing dependsOn: rocm_examples - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -135,9 +134,9 @@ jobs: parameters: aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters: - gpuTarget: $(JOB_GPU_TARGET) + checkoutRepo: ${{ parameters.checkoutRepo }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml parameters: ${{ if eq(parameters.checkoutRef, '') }}: @@ -152,20 +151,19 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds - - task: Bash@3 - displayName: Unload and reload AMDGPU - inputs: - targetType: inline - script: | - sudo modprobe -r amdgpu - sudo modprobe amdgpu - - task: Bash@3 - displayName: Iterate through examples - inputs: - targetType: inline - script: | - for file in *; do - echo Now running: $file - ./$file | tee -a $(TEST_LOG_FILE) - done - workingDirectory: $(Agent.BuildDirectory)/rocm/examples + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: + # https://github.com/ROCm/HIP/issues/2203 + extraBuildFlags: >- + -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DROCM_ROOT=$(Agent.BuildDirectory)/rocm + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + -DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET) + -DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocm-examples + testDir: $(Build.SourcesDirectory)/build + reloadAMDGPU: true diff --git a/.azuredevops/components/rocm_bandwidth_test.yml b/.azuredevops/components/rocm_bandwidth_test.yml index 854619c9b..3eaa089ac 100644 --- a/.azuredevops/components/rocm_bandwidth_test.yml +++ b/.azuredevops/components/rocm_bandwidth_test.yml @@ -27,6 +27,7 @@ parameters: - name: rocmTestDependencies type: object default: + - rocminfo - rocprofiler-register - ROCR-Runtime @@ -72,7 +73,7 @@ jobs: - job: rocm_bandwidth_test_testing dependsOn: rocm_bandwidth_test - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -99,6 +100,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocm_bandwidth_test diff --git a/.azuredevops/components/rocm_smi_lib.yml b/.azuredevops/components/rocm_smi_lib.yml index 0cd9e86ef..3a1468f42 100644 --- a/.azuredevops/components/rocm_smi_lib.yml +++ b/.azuredevops/components/rocm_smi_lib.yml @@ -29,7 +29,7 @@ jobs: - job: rocm_smi_lib_testing dependsOn: rocm_smi_lib - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -44,6 +44,9 @@ jobs: steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + parameters: + runRocminfo: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocm_smi_lib diff --git a/.azuredevops/components/rocminfo.yml b/.azuredevops/components/rocminfo.yml index f13d35f98..223a02090 100644 --- a/.azuredevops/components/rocminfo.yml +++ b/.azuredevops/components/rocminfo.yml @@ -49,7 +49,7 @@ jobs: - job: rocminfo_testing dependsOn: rocminfo - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -72,6 +72,9 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + parameters: + runRocminfo: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocminfo diff --git a/.azuredevops/components/rocprofiler-sdk.yml b/.azuredevops/components/rocprofiler-sdk.yml new file mode 100644 index 000000000..acf0ba574 --- /dev/null +++ b/.azuredevops/components/rocprofiler-sdk.yml @@ -0,0 +1,93 @@ +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: aptPackages + type: object + default: + - cmake + - python3-pip + - libdrm-dev + - libdw-dev + - libelf-dev +- name: pipModules + type: object + default: + - black + - clang-format + - clang-tidy + - cmake + - cmake-format + - dataclasses + - numpy + - otf2 + - pandas + - perfetto + - pycobertura + - pytest + - pyyaml +- name: rocmDependencies + type: object + default: + - clr + - llvm-project + - rccl + - rocm-cmake + - rocm-core + - rocminfo + - ROCR-Runtime + - rocprofiler-register + - roctracer + +jobs: +- job: rocprofilersdk + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: + vmImage: ${{ variables.BASE_BUILD_POOL }} + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: + extraBuildFlags: >- + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DROCPROFILER_BUILD_TESTS=ON + -DROCPROFILER_BUILD_SAMPLES=OFF + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + multithreadFlag: -- -j2 + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) diff --git a/.azuredevops/components/rocprofiler-systems.yml b/.azuredevops/components/rocprofiler-systems.yml new file mode 100644 index 000000000..73856a559 --- /dev/null +++ b/.azuredevops/components/rocprofiler-systems.yml @@ -0,0 +1,139 @@ +# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: aptPackages + type: object + default: + - autoconf + - autotools-dev + - bison + - build-essential + - bzip2 + - clang + - cmake + - environment-modules + - g++-12 + - libdrm-dev + - libfabric-dev + - libiberty-dev + - libpapi-dev + - libpfm4-dev + - libtool + - libopenmpi-dev + - m4 + - openmpi-bin + - software-properties-common + - python3-pip + - texinfo + - zlib1g-dev +- name: pipModules + type: object + default: + - numpy + - perfetto + - dataclasses +- name: rocmDependencies + type: object + default: + - clr + - llvm-project + - rccl + - rocm-core + - rocm_smi_lib + - rocminfo + - ROCR-Runtime + - rocprofiler + - rocprofiler-register + - roctracer + +jobs: +- job: rocprofiler_systems + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: ${{ variables.MEDIUM_BUILD_POOL }} + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml + parameters: + checkoutRepo: ${{ parameters.checkoutRepo }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + # CI case: download latest default branch build + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + # manual build case: triggered by ROCm/ROCm repo + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: ROCm symbolic link + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add ROCm compilers to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml + parameters: +# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html + extraBuildFlags: >- + -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm + -DROCPROFSYS_BUILD_TESTING=ON + -DROCPROFSYS_BUILD_DYNINST=ON + -DROCPROFSYS_BUILD_LIBUNWIND=ON + -DDYNINST_BUILD_TBB=ON + -DDYNINST_BUILD_ELFUTILS=ON + -DDYNINST_BUILD_LIBIBERTY=ON + -DDYNINST_BUILD_BOOST=ON + -DROCPROFSYS_USE_PAPI=ON + -DROCPROFSYS_USE_MPI=ON + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + multithreadFlag: -- -j32 + - task: Bash@3 + displayName: Set up rocprofiler-systems env + inputs: + targetType: inline + script: source share/rocprofiler-systems/setup-env.sh + workingDirectory: build + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocprofiler-systems + - task: Bash@3 + displayName: Remove ROCm binaries from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm compilers from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')" + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) diff --git a/.azuredevops/components/rocprofiler.yml b/.azuredevops/components/rocprofiler.yml index 023b2dd3e..4d976659b 100644 --- a/.azuredevops/components/rocprofiler.yml +++ b/.azuredevops/components/rocprofiler.yml @@ -57,8 +57,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -92,6 +90,66 @@ jobs: -DENABLE_LDCONFIG=OFF -DUSE_PROF_API=1 -DGPU_TARGETS=$(JOB_GPU_TARGET) + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) + multithreadFlag: -- -j32 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: gpuTarget: $(JOB_GPU_TARGET) + +- job: rocprofiler_testing + dependsOn: rocprofiler + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - task: Bash@3 + displayName: Setup test environment + inputs: + targetType: inline + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocprofilerV1 + testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1/test" ./run.sh + testDir: $(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1 + testParameters: '' + testPublishResults: false + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rocprofilerV2 + testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests" share/rocprofiler/tests/runUnitTests + testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' + testDir: $(Agent.BuildDirectory)/rocm diff --git a/.azuredevops/components/rocr_debug_agent.yml b/.azuredevops/components/rocr_debug_agent.yml index cdf8ecd31..23263cfa0 100644 --- a/.azuredevops/components/rocr_debug_agent.yml +++ b/.azuredevops/components/rocr_debug_agent.yml @@ -72,7 +72,7 @@ jobs: - job: rocr_debug_agent_testing dependsOn: rocr_debug_agent - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -114,6 +114,7 @@ jobs: cmakeBuildDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test' cmakeSourceDir: '.' installEnabled: false + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: rocr_debug_agent diff --git a/.azuredevops/components/roctracer.yml b/.azuredevops/components/roctracer.yml index 624eef6ec..e54c1a4cf 100644 --- a/.azuredevops/components/roctracer.yml +++ b/.azuredevops/components/roctracer.yml @@ -30,6 +30,7 @@ parameters: default: - clr - llvm-project + - rocminfo - rocprofiler-register - ROCR-Runtime @@ -48,8 +49,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -77,6 +76,7 @@ jobs: -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DGPU_TARGETS=$(JOB_GPU_TARGET) + -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) -GNinja - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: @@ -84,7 +84,7 @@ jobs: - job: roctracer_testing dependsOn: roctracer - condition: succeeded() + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) variables: - group: common - template: /.azuredevops/variables-global.yml @@ -118,6 +118,7 @@ jobs: dependencySource: staging ${{ elseif ne(parameters.checkoutRef, '') }}: dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml parameters: componentName: roctracer diff --git a/.azuredevops/components/rpp.yml b/.azuredevops/components/rpp.yml index 88a4cb760..7d3c36a80 100644 --- a/.azuredevops/components/rpp.yml +++ b/.azuredevops/components/rpp.yml @@ -9,18 +9,39 @@ parameters: type: object default: - cmake - - libomp-dev # needed to pass flag step + - libopencv-dev + - libsndfile1-dev + - libstdc++-12-dev + - imagemagick - ninja-build - clang +- name: pipModules + type: object + default: + - openpyxl + - pandas + - sphinx - name: rocmDependencies type: object default: - - aomp # needed to pass build step + - aomp - clr - half - llvm-project - rocminfo - ROCR-Runtime +- name: rocmTestDependencies + type: object + default: + - aomp + - clr + - half + - hipTensor + - llvm-project + - rocm-cmake + - rocminfo + - rocprofiler-register + - ROCR-Runtime jobs: - job: rpp @@ -35,8 +56,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters: @@ -69,3 +88,93 @@ jobs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml parameters: gpuTarget: $(JOB_GPU_TARGET) + +- job: rpp_testing + dependsOn: rpp + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + - name: LD_LIBRARY_PATH + value: $(Agent.BuildDirectory)/rocm/lib;$(Agent.BuildDirectory)/rocm/llvm/lib + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml + parameters: + gpuTarget: $(JOB_GPU_TARGET) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmTestDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + ${{ if eq(parameters.checkoutRef, '') }}: + dependencySource: staging + ${{ elseif ne(parameters.checkoutRef, '') }}: + dependencySource: tag-builds + # Dependencies from: https://github.com/ROCm/rpp/blob/develop/utilities/test_suite/README.md + - task: Bash@3 + displayName: Build and install Turbo JPEG + inputs: + targetType: 'inline' + script: | + sudo apt-get install nasm + sudo apt-get install wget + git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git + cd libjpeg-turbo + mkdir build + cd build + cmake -DCMAKE_INSTALL_PREFIX=/usr \ + -DCMAKE_BUILD_TYPE=RELEASE \ + -DENABLE_STATIC=FALSE \ + -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \ + -DWITH_JPEG8=TRUE \ + .. + make -j$nproc + sudo make install + - task: Bash@3 + displayName: Build and install Nifti + inputs: + targetType: 'inline' + script: | + git clone https://github.com/NIFTI-Imaging/nifti_clib.git + cd nifti_clib + mkdir build + cd build + cmake .. + sudo make -j$nproc install + - task: Bash@3 + displayName: Build rpp tests + inputs: + targetType: 'inline' + script: | + sudo rm -rf /opt/rocm + sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm + mkdir rpp-tests + cd rpp-tests + cmake /opt/rocm/share/rpp/test \ + -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ \ + -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml + parameters: + componentName: rpp + testExecutable: 'export PATH=$(Agent.BuildDirectory)/rocm/llvm/bin:$PATH; CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang CMAKE_VERBOSE_MAKEFILE=ON VERBOSE=1 ctest' + testDir: 'rpp-tests' + - script: sudo rm /opt/rocm diff --git a/.azuredevops/dependencies/boost.yml b/.azuredevops/dependencies/boost.yml new file mode 100644 index 000000000..16cc1a699 --- /dev/null +++ b/.azuredevops/dependencies/boost.yml @@ -0,0 +1,65 @@ +parameters: +- name: checkoutRepo + type: string + default: 'self' +- name: checkoutRef + type: string + default: '' +- name: boostVersion + type: string + default: '' +- name: aptPackages + type: object + default: + - git +- name: rocmDependencies + type: object + default: + - llvm-project + +jobs: +- job: boost + variables: + - group: common + - template: /.azuredevops/variables-global.yml + pool: + vmImage: ${{ variables.BASE_BUILD_POOL }} + workspace: + clean: all + steps: + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - task: Bash@3 + displayName: 'git clone boost' + inputs: + targetType: inline + script: git clone -b ${{ parameters.boostVersion }} https://github.com/boostorg/boost --depth=1 --recurse-submodules + workingDirectory: $(Build.SourcesDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmDependencies }} + dependencySource: staging + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add ROCm compilers to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" + - task: Bash@3 + displayName: 'Build Boost with clang' + inputs: + targetType: inline + script: | + export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/clang + export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ + ./bootstrap.sh --with-toolset=clang --prefix=$(Build.BinariesDirectory) + ./b2 --toolset=clang threading=multi link=shared --prefix=$(Build.BinariesDirectory) cxxflags="-std=c++20" + ./b2 install + workingDirectory: $(Build.SourcesDirectory)/boost + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml diff --git a/.azuredevops/dependencies/grpc.yml b/.azuredevops/dependencies/grpc.yml index d9ea15fb2..dd6b65345 100644 --- a/.azuredevops/dependencies/grpc.yml +++ b/.azuredevops/dependencies/grpc.yml @@ -20,7 +20,7 @@ jobs: variables: - group: common - template: /.azuredevops/variables-global.yml - pool: + pool: vmImage: ${{ variables.BASE_BUILD_POOL }} workspace: clean: all diff --git a/.azuredevops/dependencies/gtest.yml b/.azuredevops/dependencies/gtest.yml index dcdffe698..5d18d9baa 100644 --- a/.azuredevops/dependencies/gtest.yml +++ b/.azuredevops/dependencies/gtest.yml @@ -20,7 +20,7 @@ jobs: variables: - group: common - template: /.azuredevops/variables-global.yml - pool: + pool: vmImage: ${{ variables.BASE_BUILD_POOL }} workspace: clean: all diff --git a/.azuredevops/dependencies/lapack.yml b/.azuredevops/dependencies/lapack.yml index b2729d22d..c1a15a138 100644 --- a/.azuredevops/dependencies/lapack.yml +++ b/.azuredevops/dependencies/lapack.yml @@ -21,7 +21,7 @@ jobs: variables: - group: common - template: /.azuredevops/variables-global.yml - pool: + pool: vmImage: ${{ variables.BASE_BUILD_POOL }} workspace: clean: all diff --git a/.azuredevops/nightly/pytorch.yml b/.azuredevops/nightly/pytorch.yml index b38497a34..5d69beb91 100644 --- a/.azuredevops/nightly/pytorch.yml +++ b/.azuredevops/nightly/pytorch.yml @@ -11,7 +11,6 @@ parameters: - ca-certificates - bc - bridge-utils - - cmake - devscripts - dkms - doxygen @@ -62,9 +61,12 @@ parameters: - ffmpeg - libopenblas-dev - liblapack-dev + - libswscale-dev + - libavformat-dev - name: pipModules type: object default: + - cmake - astunparse - expecttest!=0.2.0 - hypothesis @@ -87,7 +89,10 @@ parameters: # list for vision - auditwheel - future -# list from https://github.com/pytorch/builder/blob/main/manywheel/build_rocm.sh + - pytest + - pytest-azurepipelines + - pillow +# list from https://github.com/pytorch/pytorch/blob/main/.ci/manywheel/build_rocm.sh - name: rocmDependencies type: object default: @@ -117,6 +122,25 @@ parameters: - hipCUB - rocThrust - hipBLAS-common + - composable_kernel +- name: rocmTestDependencies + type: object + default: + - rocminfo +# Reference on what tests to run for torchvision found in private repo: +# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_pytorch_test1.sh#L54 +# Will iterate through this list using pytest +- name: torchTestList + type: object + default: + - nn + - torch +# - cuda seg faults and might need cuda installed on test system +# - ops takes too long + - unary_ufuncs + - binary_ufuncs + - autograd +# - inductor/torchinductor takes too long trigger: none pr: none @@ -136,13 +160,10 @@ jobs: amd-staging-gfx942: ROCM_BRANCH: amd-staging JOB_GPU_TARGET: gfx942 - amd-staging-gfx90a: - ROCM_BRANCH: amd-staging - JOB_GPU_TARGET: gfx90a variables: - group: common - template: /.azuredevops/variables-global.yml -# various flags/parameters expected by bash scripts in pytorch builder repo +# various flags/parameters expected by bash scripts in pytorch repo's .ci directory - name: ROCM_VERSION value: 6.3.0 - name: ROCM_PATH @@ -163,7 +184,7 @@ jobs: workspace: clean: all steps: -# copy environment setup from https://github.com/pytorch/builder/blob/main/manywheel/Dockerfile +# copy environment setup from https://github.com/pytorch/pytorch/blob/main/.ci/docker/manywheel/Dockerfile # but instead of centos, use ubuntu environment - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml - task: Bash@3 @@ -200,18 +221,19 @@ jobs: targetType: inline script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm - checkout: self - - task: Bash@3 - displayName: git clone pytorch builder - inputs: - targetType: inline - script: git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules - workingDirectory: $(Build.SourcesDirectory) - task: Bash@3 displayName: git clone upstream pytorch inputs: targetType: inline script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules workingDirectory: $(Build.SourcesDirectory) +# builder clone still needed due to run_tests.sh at end of build_common.sh call + - task: Bash@3 + displayName: git clone pytorch builder + inputs: + targetType: inline + script: git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules + workingDirectory: $(Build.SourcesDirectory)/pytorch - task: Bash@3 displayName: Install patchelf inputs: @@ -264,8 +286,8 @@ jobs: PYTORCH_BUILD_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1) PYTORCH_BUILD_NUMBER=$(date -u +%Y%m%d) SKIP_ALL_TESTS=1 - bash ./manywheel/build_rocm.sh - workingDirectory: $(Build.SourcesDirectory)/builder + bash ./.ci/manywheel/build_rocm.sh + workingDirectory: $(Build.SourcesDirectory)/pytorch - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml parameters: sourceDir: /remote/wheelhouserocm$(ROCM_VERSION) @@ -300,7 +322,10 @@ jobs: inputs: targetType: inline script: >- - BUILD_VERSION=$(ROCM_BRANCH).$(JOB_GPU_TARGET)-$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)$(date -u +%Y%m%d) + TORCH_PACKAGE_NAME=torch.$(ROCM_BRANCH).$(JOB_GPU_TARGET) + TORCHVISION_PACKAGE_NAME=torchvision.$(ROCM_BRANCH).$(JOB_GPU_TARGET) + PYTORCH_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)post$(date -u +%Y%m%d) + BUILD_VERSION=$(cat $(Build.SourcesDirectory)/vision/version.txt | cut -da -f1)post$(date -u +%Y%m%d) python3 setup.py bdist_wheel workingDirectory: $(Build.SourcesDirectory)/vision - task: Bash@3 @@ -319,3 +344,191 @@ jobs: retryCountOnTaskFailure: 3 inputs: targetPath: $(Build.BinariesDirectory) + +- job: torchvision_testing + dependsOn: pytorch + condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName']))) + variables: + - group: common + - template: /.azuredevops/variables-global.yml + - name: PYTORCH_TEST_WITH_ROCM + value: 1 + pool: $(JOB_TEST_POOL) + workspace: + clean: all + strategy: + matrix: + gfx942: + JOB_GPU_TARGET: gfx942 + JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }} + steps: + - task: Bash@3 + displayName: 'Register libjpeg-turbo packages' + inputs: + targetType: inline + script: | + sudo mkdir --parents --mode=0755 /etc/apt/keyrings + wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null + echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list + sudo apt update + apt-cache show libjpeg-turbo-official | grep Version + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml + parameters: + aptPackages: ${{ parameters.aptPackages }} + pipModules: ${{ parameters.pipModules }} + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml + - task: DownloadPipelineArtifact@2 + displayName: 'Download Pipeline Wheel Files' + inputs: + itemPattern: '**/*$(JOB_GPU_TARGET)*.whl' + targetPath: $(Agent.BuildDirectory) + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml + parameters: + dependencySource: staging + - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml + parameters: + dependencyList: ${{ parameters.rocmTestDependencies }} + gpuTarget: $(JOB_GPU_TARGET) + dependencySource: staging + skipLlvmSymlink: true +# get sources to run test scripts + - task: Bash@3 + displayName: git clone upstream pytorch + inputs: + targetType: inline + script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules + workingDirectory: $(Build.SourcesDirectory) + - task: Bash@3 + displayName: git clone pytorch vision + inputs: + targetType: inline + script: git clone https://github.com/pytorch/vision.git --depth=1 --recurse-submodules + workingDirectory: $(Build.SourcesDirectory) + - task: Bash@3 + displayName: Install Wheel Files + inputs: + targetType: inline + script: find . -name "*.whl" -exec pip install --no-index --find-links=. --no-dependencies -v {} \; + workingDirectory: $(Agent.BuildDirectory) + - task: Bash@3 + displayName: Show Updated pip List + inputs: + targetType: inline + script: pip list -v + workingDirectory: $(Agent.BuildDirectory) + - task: Bash@3 + displayName: Add ROCm binaries to PATH + inputs: + targetType: inline + script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" + - task: Bash@3 + displayName: Add Python site-packages binaries to path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.prependpath]$USER_BASE/bin" + - task: Bash@3 + displayName: Add torch libs to ldconfig + inputs: + targetType: inline + script: | + echo $(python3 -m site --user-site)/torch/lib | sudo tee /etc/ld.so.conf.d/torch.conf + sudo ldconfig -v + ldconfig -p +# https://pytorch.org/get-started/locally/#linux-verification +# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#testing-the-pytorch-installation + - task: Bash@3 + displayName: Simple Import Torch Tests + inputs: + targetType: inline + script: | + python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure' + python3 -c 'import torch; print(torch.cuda.is_available())' + python3 -c 'import torch; x = torch.rand(5, 3); print(x)' +# Test artifact build script has too many if statements for different environments +# Based off the snippet of interest for this environment, with some adjustments +# https://github.com/pytorch/pytorch/blob/main/.ci/pytorch/build.sh#L335-L375 +# Removing in-line comments since it does not fit with the yaml markup + - task: Bash@3 + displayName: Build Pytorch Test Artifacts + continueOnError: true + inputs: + targetType: inline + script: | + CUSTOM_TEST_ARTIFACT_BUILD_DIR="build/custom_test_artifacts" + CUSTOM_TEST_USE_ROCM=ON + CUSTOM_TEST_MODULE_PATH="${PWD}/cmake/public" + mkdir -pv "${CUSTOM_TEST_ARTIFACT_BUILD_DIR}" + + CUSTOM_OP_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-op-build" + CUSTOM_OP_TEST="${PWD}/test/custom_operator" + python --version + SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')" + + mkdir -p "$CUSTOM_OP_BUILD" + pushd "$CUSTOM_OP_BUILD" + cmake "$CUSTOM_OP_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \ + -DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM" + make VERBOSE=1 + popd + + JIT_HOOK_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/jit-hook-build" + JIT_HOOK_TEST="$PWD/test/jit_hooks" + python --version + SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')" + mkdir -p "$JIT_HOOK_BUILD" + pushd "$JIT_HOOK_BUILD" + cmake "$JIT_HOOK_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \ + -DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM" + make VERBOSE=1 + popd + + CUSTOM_BACKEND_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-backend-build" + CUSTOM_BACKEND_TEST="${PWD}/test/custom_backend" + python --version + mkdir -p "$CUSTOM_BACKEND_BUILD" + pushd "$CUSTOM_BACKEND_BUILD" + cmake "$CUSTOM_BACKEND_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \ + -DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM" + make VERBOSE=1 + popd + workingDirectory: $(Build.SourcesDirectory)/pytorch + - ${{ each torchTest in parameters.torchTestList }}: + - task: Bash@3 + displayName: Test ${{ torchTest }} + continueOnError: true + inputs: + targetType: inline + workingDirectory: $(Build.SourcesDirectory)/pytorch + ${{ if contains(torchTest, '/') }}: + script: pytest test/${{ split(torchTest, '/')[0] }}/test_${{ split(torchTest, '/')[1] }}.py + ${{ else }}: + script: pytest test/test_${{ torchTest }}.py +# Reference on what tests to run for torchvision found in private repo: +# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_torchvision.sh#L51 + - task: Bash@3 + displayName: Test vision/transforms + continueOnError: true + inputs: + targetType: inline + script: pytest test/test_transforms.py + workingDirectory: $(Build.SourcesDirectory)/vision + - task: Bash@3 + displayName: Uninstall Wheel Files + inputs: + targetType: inline + script: find . -name "*.whl" -exec pip uninstall -y {} \; + workingDirectory: $(Agent.BuildDirectory) + - task: Bash@3 + displayName: Remove Python site-packages binaries from path + inputs: + targetType: inline + script: | + USER_BASE=$(python3 -m site --user-base) + echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')" + - task: Bash@3 + displayName: Remove ROCm binaries from PATH + inputs: + targetType: inline + script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')" diff --git a/.azuredevops/nightly/rocm-nightly.yml b/.azuredevops/nightly/rocm-nightly.yml index 41dc9a636..8d13aa191 100644 --- a/.azuredevops/nightly/rocm-nightly.yml +++ b/.azuredevops/nightly/rocm-nightly.yml @@ -1,5 +1,5 @@ parameters: -# currently excludes clr and rocm-examples +# currently excludes clr - name: rocmDependencies type: object default: @@ -10,6 +10,7 @@ parameters: - composable_kernel - half - HIP + - hip-tests - hipBLAS - hipBLAS-common - hipBLASLt @@ -25,6 +26,7 @@ parameters: - llvm-project - MIOpen - MIVisionX + - omniperf - rccl - rdc - rocAL @@ -36,6 +38,7 @@ parameters: - ROCgdb - rocm-cmake - rocm-core + - rocm-examples - rocminfo - rocMLIR - ROCmValidationSuite @@ -43,7 +46,10 @@ parameters: - rocm_smi_lib - rocPRIM - rocprofiler-register + - rocprofiler-sdk + - rocprofiler-systems - rocprofiler + - rocPyDecode - ROCR-Runtime - rocRAND - rocr_debug_agent @@ -76,8 +82,6 @@ jobs: matrix: gfx942: JOB_GPU_TARGET: gfx942 - gfx90a: - JOB_GPU_TARGET: gfx90a steps: - task: DeleteFiles@1 displayName: 'Cleanup checkout space' @@ -97,6 +101,7 @@ jobs: dependencyList: ${{ parameters.rocmDependencies }} dependencySource: staging skipLibraryLinking: true + skipLlvmSymlink: true gpuTarget: $(JOB_GPU_TARGET) - script: df -h displayName: System disk space after ROCm @@ -113,7 +118,7 @@ jobs: - script: du -sh $(Build.ArtifactStagingDirectory) displayName: Compressed ROCm size - task: PublishPipelineArtifact@1 - displayName: 'Public ROCm Nightly Artifact' + displayName: 'Publish ROCm Nightly Artifact' retryCountOnTaskFailure: 3 inputs: targetPath: '$(Build.ArtifactStagingDirectory)' diff --git a/.azuredevops/tag-builds/Tensile.yml b/.azuredevops/tag-builds/Tensile.yml new file mode 100644 index 000000000..871c20b31 --- /dev/null +++ b/.azuredevops/tag-builds/Tensile.yml @@ -0,0 +1,29 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: checkoutRef + type: string + default: refs/tags/$(LATEST_RELEASE_TAG) + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + - repository: release_repo + type: github + endpoint: ROCm + name: ROCm/Tensile + ref: ${{ parameters.checkoutRef }} + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/Tensile.yml + parameters: + checkoutRepo: release_repo + checkoutRef: ${{ parameters.checkoutRef }} diff --git a/.azuredevops/tag-builds/boost.yml b/.azuredevops/tag-builds/boost.yml new file mode 100644 index 000000000..1411679e2 --- /dev/null +++ b/.azuredevops/tag-builds/boost.yml @@ -0,0 +1,23 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: boostVersion + type: string + default: 'master' + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_DEPENDENCIES_PATH }}/boost.yml + parameters: + boostVersion: ${{ parameters.boostVersion }} diff --git a/.azuredevops/tag-builds/omniperf.yml b/.azuredevops/tag-builds/omniperf.yml new file mode 100644 index 000000000..a1a396103 --- /dev/null +++ b/.azuredevops/tag-builds/omniperf.yml @@ -0,0 +1,29 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: checkoutRef + type: string + default: refs/tags/$(LATEST_RELEASE_TAG) + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + - repository: release_repo + type: github + endpoint: ROCm + name: ROCm/omniperf + ref: ${{ parameters.checkoutRef }} + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/omniperf.yml + parameters: + checkoutRepo: release_repo + checkoutRef: ${{ parameters.checkoutRef }} diff --git a/.azuredevops/tag-builds/omnitrace.yml b/.azuredevops/tag-builds/omnitrace.yml new file mode 100644 index 000000000..63a5ba34f --- /dev/null +++ b/.azuredevops/tag-builds/omnitrace.yml @@ -0,0 +1,29 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: checkoutRef + type: string + default: refs/tags/$(LATEST_RELEASE_TAG) + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + - repository: release_repo + type: github + endpoint: ROCm + name: ROCm/omnitrace + ref: ${{ parameters.checkoutRef }} + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/omnitrace.yml + parameters: + checkoutRepo: release_repo + checkoutRef: ${{ parameters.checkoutRef }} diff --git a/.azuredevops/tag-builds/rocprofiler-sdk.yml b/.azuredevops/tag-builds/rocprofiler-sdk.yml new file mode 100644 index 000000000..91708e8df --- /dev/null +++ b/.azuredevops/tag-builds/rocprofiler-sdk.yml @@ -0,0 +1,29 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: checkoutRef + type: string + default: refs/tags/$(LATEST_RELEASE_TAG) + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + - repository: release_repo + type: github + endpoint: ROCm + name: ROCm/rocprofiler-sdk + ref: ${{ parameters.checkoutRef }} + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-sdk.yml + parameters: + checkoutRepo: release_repo + checkoutRef: ${{ parameters.checkoutRef }} diff --git a/.azuredevops/tag-builds/rocprofiler-systems.yml b/.azuredevops/tag-builds/rocprofiler-systems.yml new file mode 100644 index 000000000..559b3362c --- /dev/null +++ b/.azuredevops/tag-builds/rocprofiler-systems.yml @@ -0,0 +1,29 @@ +variables: +- group: common +- template: /.azuredevops/variables-global.yml + +parameters: +- name: checkoutRef + type: string + default: refs/tags/$(LATEST_RELEASE_TAG) + +resources: + repositories: + - repository: pipelines_repo + type: github + endpoint: ROCm + name: ROCm/ROCm + - repository: release_repo + type: github + endpoint: ROCm + name: ROCm/rocprofiler-systems + ref: ${{ parameters.checkoutRef }} + +trigger: none +pr: none + +jobs: + - template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-systems.yml + parameters: + checkoutRepo: release_repo + checkoutRef: ${{ parameters.checkoutRef }} diff --git a/.azuredevops/templates/steps/artifact-download.yml b/.azuredevops/templates/steps/artifact-download.yml index 80395d76f..31e38b0b4 100644 --- a/.azuredevops/templates/steps/artifact-download.yml +++ b/.azuredevops/templates/steps/artifact-download.yml @@ -9,6 +9,10 @@ parameters: - name: useDefaultBranch type: boolean default: true +# useMainlineBranch only processed if useDefaultBranch is false +- name: useMainlineBranch + type: boolean + default: false - name: latestFromBranch type: boolean default: true @@ -22,13 +26,14 @@ parameters: type: object default: AMDMIGraphX: develop - amdsmi: develop + amdsmi: amd-staging aomp-extras: aomp-dev aomp: aomp-dev clr: amd-staging composable_kernel: develop half: rocm HIP: amd-staging + hip-tests: amd-staging hipBLAS: develop hipBLASLt: develop hipBLAS-common: develop @@ -44,8 +49,10 @@ parameters: llvm-project: amd-staging MIOpen: develop MIVisionX: develop + omniperf: amd-staging + omnitrace: amd-staging rccl: develop - rdc: develop + rdc: amd-staging rocAL: develop rocALUTION: develop rocBLAS: develop @@ -54,16 +61,19 @@ parameters: rocFFT: develop ROCgdb: amd-staging rocm-cmake: develop - rocm-core: master + rocm-core: amd-staging rocm-examples: develop rocminfo: amd-staging rocMLIR: develop ROCmValidationSuite: master rocm_bandwidth_test: master - rocm_smi_lib: develop + rocm_smi_lib: amd-staging rocPRIM: develop - rocprofiler-register: amd-mainline rocprofiler: amd-staging + rocprofiler-register: amd-staging + rocprofiler-sdk: amd-staging + rocprofiler-systems: amd-staging + rocPyDecode: develop ROCR-Runtime: amd-staging rocRAND: develop rocr_debug_agent: amd-staging @@ -73,18 +83,81 @@ parameters: roctracer: amd-staging rocWMMA: develop rpp: develop +- name: mainlineBranchList + type: object + default: + AMDMIGraphX: mainline + amdsmi: amd-mainline + aomp-extras: amd-mainline-open + aomp: amd-mainline-open + clr: amd-mainline + composable_kernel: mainline + half: rocm + HIP: amd-mainline + hip-tests: amd-mainline + hipBLAS: mainline + hipBLASLt: mainline + hipBLAS-common: mainline + hipCUB: mainline + hipFFT: mainline + hipfort: mainline + HIPIFY: amd-mainline + hipRAND: mainline + hipSOLVER: mainline + hipSPARSE: mainline + hipSPARSELt: mainline + hipTensor: mainline + llvm-project: amd-mainline-open + MIOpen: mainline + MIVisionX: mainline + omniperf: amd-mainline + omnitrace: amd-mainline + rccl: mainline + rdc: amd-mainline + rocAL: master # needs the yaml file + rocALUTION: mainline + rocBLAS: mainline + ROCdbgapi : amd-mainline + rocDecode: mainline + rocFFT: mainline + ROCgdb: amd-mainline-rocgdb-15 # + rocm-cmake: mainline + rocm-core: amd-master + rocm-examples: develop # no mainline + rocminfo: amd-master + rocMLIR: mainline # needs the yaml file + ROCmValidationSuite: mainline + rocm_bandwidth_test: master + rocm_smi_lib: amd-mainline + rocPRIM: mainline + rocprofiler: amd-master + rocprofiler-register: amd-mainline + rocprofiler-sdk: amd-mainline + rocprofiler-systems: amd-mainline + rocPyDecode: mainline + ROCR-Runtime: amd-master + rocRAND: mainline + rocr_debug_agent: amd-mainline + rocSOLVER: mainline + rocSPARSE: mainline + rocThrust: mainline + roctracer: amd-master + rocWMMA: mainline + rpp: mainline - name: allowPartiallySucceededBuilds type: object default: - amdsmi - aomp - HIPIFY + - MIVisionX + - omniperf - rccl - rdc - rocm-cmake - rocm_smi_lib - rocFFT - - MIVisionX + - rpp # BELOW REQUIRED IF useDefaultBranch false - name: branchName type: string @@ -104,6 +177,8 @@ steps: buildVersionToDownload: latestFromBranch # default is 'latest' ${{ if eq(parameters.useDefaultBranch, true) }}: branchName: refs/heads/${{ parameters.defaultBranchList[parameters.componentName] }} + ${{ elseif eq(parameters.useMainlineBranch, true) }}: + branchName: refs/heads/${{ parameters.mainlineBranchList[parameters.componentName] }} ${{ else }}: branchName: ${{ parameters.branchName }} ${{ if containsValue(parameters.allowPartiallySucceededBuilds, parameters.componentName) }}: diff --git a/.azuredevops/templates/steps/build-autotools.yml b/.azuredevops/templates/steps/build-autotools.yml index f2b565783..3425e3d7b 100644 --- a/.azuredevops/templates/steps/build-autotools.yml +++ b/.azuredevops/templates/steps/build-autotools.yml @@ -11,6 +11,9 @@ parameters: - name: installDir type: string default: '$(Build.BinariesDirectory)' +- name: makeCallPrefix + type: string + default: '' steps: - task: Bash@3 @@ -23,7 +26,7 @@ steps: displayName: '${{ parameters.componentName }} make' inputs: targetType: inline - script: make -j$(nproc) + script: ${{ parameters.makeCallPrefix }} make -j$(nproc) workingDirectory: ${{ parameters.buildDir }} - task: Bash@3 displayName: '${{ parameters.componentName }} make install' diff --git a/.azuredevops/templates/steps/dependencies-aqlprofile.yml b/.azuredevops/templates/steps/dependencies-aqlprofile.yml index af1f9339f..42ca8dac6 100644 --- a/.azuredevops/templates/steps/dependencies-aqlprofile.yml +++ b/.azuredevops/templates/steps/dependencies-aqlprofile.yml @@ -8,20 +8,22 @@ parameters: - name: repositoryUrl type: object default: - staging: https://repo.radeon.com/rocm/apt/6.2/pool/main/h/hsa-amd-aqlprofile - tag-builds: https://repo.radeon.com/rocm/apt/6.2/pool/main/h/hsa-amd-aqlprofile -- name: packageName - type: object - default: - staging: hsa-amd-aqlprofile_1.0.0.60200.60200-66~22.04_amd64.deb - tag-builds: hsa-amd-aqlprofile_1.0.0.60200.60200-66~22.04_amd64.deb + staging: https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/ # end slash is important for curl! + tag-builds: https://repo.radeon.com/rocm/apt/$(TAGGED_RELEASE)/pool/main/h/hsa-amd-aqlprofile/ steps: +- task: Bash@3 + displayName: Get aqlprofile package name + inputs: + targetType: inline + script: | + export packageName=$(curl -s ${{ parameters.repositoryUrl[parameters.dependencySource] }} | grep -oP "href=\"\K[^\"]*$(lsb_release -rs)[^\"]*\.deb") + echo "##vso[task.setvariable variable=packageName;isreadonly=true]$packageName" - task: Bash@3 displayName: 'Download aqlprofile' inputs: targetType: inline - script: wget -nv ${{ parameters.repositoryUrl[parameters.dependencySource] }}/${{ parameters.packageName[parameters.dependencySource] }} + script: wget -nv ${{ parameters.repositoryUrl[parameters.dependencySource] }}$(packageName) workingDirectory: '$(Pipeline.Workspace)' - task: Bash@3 displayName: 'Extract aqlprofile' @@ -29,7 +31,7 @@ steps: targetType: inline script: | mkdir hsa-amd-aqlprofile - dpkg-deb -R ${{ parameters.packageName[parameters.dependencySource] }} hsa-amd-aqlprofile + dpkg-deb -R $(packageName) hsa-amd-aqlprofile workingDirectory: '$(Pipeline.Workspace)' - task: Bash@3 displayName: 'Copy aqlprofile files' @@ -43,5 +45,5 @@ steps: displayName: 'Clean up aqlprofile' inputs: targetType: inline - script: rm -rf hsa-amd-aqlprofile ${{ parameters.packageName[parameters.dependencySource] }} + script: rm -rf hsa-amd-aqlprofile $(packageName) workingDirectory: '$(Pipeline.Workspace)' diff --git a/.azuredevops/templates/steps/dependencies-boost.yml b/.azuredevops/templates/steps/dependencies-boost.yml new file mode 100644 index 000000000..d944f7f66 --- /dev/null +++ b/.azuredevops/templates/steps/dependencies-boost.yml @@ -0,0 +1,35 @@ +steps: +- task: DownloadPipelineArtifact@2 + displayName: Download Boost + inputs: + buildType: specific + project: ROCm-CI + definition: $(BOOST_DEPENDENCY_PIPELINE_ID) + targetPath: $(Pipeline.Workspace)/d +- task: ExtractFiles@1 + displayName: Extract Boost + inputs: + archiveFilePatterns: '$(Pipeline.Workspace)/d/**/*.tar.gz' + destinationFolder: $(Agent.BuildDirectory)/boost + cleanDestinationFolder: true + overwriteExistingFiles: true +- task: DeleteFiles@1 + displayName: Cleanup Compressed Boost + inputs: + SourceFolder: $(Pipeline.Workspace)/d + Contents: '**/*.tar.gz' + RemoveDotFiles: true +- task: Bash@3 + displayName: 'List Boost files' + inputs: + targetType: inline + script: ls -1R $(Agent.BuildDirectory)/boost +- task: Bash@3 + displayName: 'Link Boost shared libraries' + inputs: + targetType: inline + script: | + echo $(Agent.BuildDirectory)/boost/lib | sudo tee /etc/ld.so.conf.d/boost.conf + sudo cat /etc/ld.so.conf.d/boost.conf + sudo ldconfig -v + ldconfig -p diff --git a/.azuredevops/templates/steps/dependencies-other.yml b/.azuredevops/templates/steps/dependencies-other.yml index 8aa77e22f..69ec8e9cb 100644 --- a/.azuredevops/templates/steps/dependencies-other.yml +++ b/.azuredevops/templates/steps/dependencies-other.yml @@ -34,4 +34,4 @@ steps: displayName: 'pip install ...' inputs: targetType: inline - script: pip install ${{ join(' ', parameters.pipModules) }} + script: pip install -v ${{ join(' ', parameters.pipModules) }} diff --git a/.azuredevops/templates/steps/dependencies-rocm.yml b/.azuredevops/templates/steps/dependencies-rocm.yml index 216a190bf..438ea01e7 100644 --- a/.azuredevops/templates/steps/dependencies-rocm.yml +++ b/.azuredevops/templates/steps/dependencies-rocm.yml @@ -9,6 +9,7 @@ parameters: default: staging values: - staging + - mainline - tag-builds - fixed - name: extractToMnt @@ -37,6 +38,7 @@ parameters: composable_kernel: $(COMPOSABLE_KERNEL_PIPELINE_ID) half: $(HALF_PIPELINE_ID) HIP: $(HIP_PIPELINE_ID) + hip-tests: $(HIP_TESTS_PIPELINE_ID) hipBLAS: $(HIPBLAS_PIPELINE_ID) hipBLAS-common: $(HIPBLAS_COMMON_PIPELINE_ID) hipBLASLt: $(HIPBLASLT_PIPELINE_ID) @@ -52,6 +54,8 @@ parameters: llvm-project: $(LLVM_PROJECT_PIPELINE_ID) MIOpen: $(MIOpen_PIPELINE_ID) MIVisionX: $(MIVISIONX_PIPELINE_ID) + omniperf: $(OMNIPERF_PIPELINE_ID) + omnitrace: $(OMNITRACE_PIPELINE_ID) rccl: $(RCCL_PIPELINE_ID) rdc: $(RDC_PIPELINE_ID) rocAL: $(ROCAL_PIPELINE_ID) @@ -71,7 +75,10 @@ parameters: rocm_smi_lib: $(ROCM_SMI_LIB_PIPELINE_ID) rocPRIM: $(ROCPRIM_PIPELINE_ID) rocprofiler-register: $(ROCPROFILER_REGISTER_PIPELINE_ID) + rocprofiler-sdk: $(ROCPROFILER_SDK_PIPELINE_ID) + rocprofiler-systems: $(ROCPROFILER_SYSTEMS_PIPELINE_ID) rocprofiler: $(ROCPROFILER_PIPELINE_ID) + rocPyDecode: $(ROCPYDECODE_PIPELINE_ID) ROCR-Runtime: $(ROCR_RUNTIME_PIPELINE_ID) rocRAND: $(ROCRAND_PIPELINE_ID) rocr_debug_agent: $(ROCR_DEBUG_AGENT_PIPELINE_ID) @@ -93,6 +100,7 @@ parameters: composable_kernel: $(COMPOSABLE_KERNEL_TAGGED_PIPELINE_ID) half: $(HALF_TAGGED_PIPELINE_ID) HIP: $(HIP_TAGGED_PIPELINE_ID) + hip-tests: $(HIP_TESTS_TAGGED_PIPELINE_ID) hipBLAS: $(HIPBLAS_TAGGED_PIPELINE_ID) hipBLAS-common: $(HIPBLAS_COMMON_TAGGED_PIPELINE_ID) hipBLASLt: $(HIPBLASLT_TAGGED_PIPELINE_ID) @@ -108,6 +116,8 @@ parameters: llvm-project: $(LLVM_PROJECT_TAGGED_PIPELINE_ID) MIOpen: $(MIOpen_TAGGED_PIPELINE_ID) MIVisionX: $(MIVISIONX_TAGGED_PIPELINE_ID) + omniperf: $(OMNIPERF_TAGGED_PIPELINE_ID) + omnitrace: $(OMNITRACE_TAGGED_PIPELINE_ID) rccl: $(RCCL_TAGGED_PIPELINE_ID) rdc: $(RDC_TAGGED_PIPELINE_ID) rocAL: $(ROCAL_TAGGED_PIPELINE_ID) @@ -127,7 +137,10 @@ parameters: rocm_smi_lib: $(ROCM_SMI_LIB_TAGGED_PIPELINE_ID) rocPRIM: $(ROCPRIM_TAGGED_PIPELINE_ID) rocprofiler-register: $(ROCPROFILER_REGISTER_TAGGED_PIPELINE_ID) + rocprofiler-sdk: $(ROCPROFILER_SDK_TAGGED_PIPELINE_ID) + rocprofiler-systems: $(ROCPROFILER_SYSTEMS_PIPELINE_ID) rocprofiler: $(ROCPROFILER_TAGGED_PIPELINE_ID) + rocPyDecode: $(ROCPYDECODE_TAGGED_PIPELINE_ID) ROCR-Runtime: $(ROCR_RUNTIME_TAGGED_PIPELINE_ID) rocRAND: $(ROCRAND_TAGGED_PIPELINE_ID) rocr_debug_agent: $(ROCR_DEBUG_AGENT_TAGGED_PIPELINE_ID) @@ -166,13 +179,17 @@ parameters: - hipRAND - hipSPARSELt - hipTensor + - omnitrace - rccl - rocALUTION - rocBLAS - rocFFT - rocm-examples - rocPRIM + - rocprofiler-sdk + - rocprofiler-systems - rocprofiler + - rocPyDecode - rocRAND - rocSOLVER - rocSPARSE @@ -186,6 +203,7 @@ parameters: - hipSPARSE - MIOpen - MIVision + - omniperf - rocAL - ROCmValidationSuite @@ -205,6 +223,10 @@ steps: ${{ if eq(parameters.dependencySource, 'staging') }}: pipelineId: ${{ parameters.stagingPipelineIdentifiers[ split(dependency, ':')[0] ] }} latestFromBranch: ${{ parameters.latestFromBranch }} + ${{ elseif eq(parameters.dependencySource, 'mainline') }}: + pipelineId: ${{ parameters.stagingPipelineIdentifiers[ split(dependency, ':')[0] ] }} + useMainlineBranch: true + latestFromBranch: ${{ parameters.latestFromBranch }} ${{ elseif eq(parameters.dependencySource, 'tag-builds') }}: pipelineId: ${{ parameters.taggedPipelineIdentifiers[ split(dependency, ':')[0] ] }} latestFromBranch: false @@ -221,6 +243,10 @@ steps: ${{ if eq(parameters.dependencySource, 'staging') }}: pipelineId: ${{ parameters.stagingPipelineIdentifiers[dependency] }} latestFromBranch: ${{ parameters.latestFromBranch }} + ${{ elseif eq(parameters.dependencySource, 'mainline') }}: + pipelineId: ${{ parameters.stagingPipelineIdentifiers[dependency] }} + useMainlineBranch: true + latestFromBranch: ${{ parameters.latestFromBranch }} ${{ elseif eq(parameters.dependencySource, 'tag-builds') }}: pipelineId: ${{ parameters.taggedPipelineIdentifiers[dependency] }} latestFromBranch: false diff --git a/.azuredevops/templates/steps/gpu-diagnostics.yml b/.azuredevops/templates/steps/gpu-diagnostics.yml new file mode 100644 index 000000000..d2e8f210e --- /dev/null +++ b/.azuredevops/templates/steps/gpu-diagnostics.yml @@ -0,0 +1,54 @@ +# Diagnostics for GPU-enabled systems +parameters: +- name: runRocminfo + type: boolean + default: true + +steps: +- ${{ if eq(parameters.runRocminfo, true) }}: + - task: Bash@3 + displayName: 'rocminfo' + continueOnError: true + inputs: + targetType: inline + script: $(Agent.BuildDirectory)/rocm/bin/rocminfo + - task: Bash@3 + displayName: 'rocm_agent_enumerator' + continueOnError: true + inputs: + targetType: inline + script: $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator +- task: Bash@3 + displayName: 'List DRI devices' + continueOnError: true + inputs: + targetType: inline + script: ls -la /dev/dri/by-path/ +- task: Bash@3 + displayName: 'List amdgpu/rocm/mesa packages' + continueOnError: true + inputs: + targetType: inline + script: apt list --installed | grep -E 'amdgpu|rocm|mesa' +- task: Bash@3 + displayName: 'List GPU processes' + continueOnError: true + inputs: + targetType: inline + script: | + ls /sys/class/kfd/kfd/proc/ + sudo lsof | grep amdgpu +- task: Bash@3 + displayName: 'System snapshot' + continueOnError: true + inputs: + targetType: inline + script: top -bn1 +- task: Bash@3 + displayName: 'List dmesg' + continueOnError: true + inputs: + targetType: inline + script: | + echo 'rocm-ci: $(Build.DefinitionName) $(System.DefinitionId)' | sudo tee /dev/kmsg + sudo dmesg diff --git a/.azuredevops/variables-global.yml b/.azuredevops/variables-global.yml index 21974f9db..13f3031dd 100644 --- a/.azuredevops/variables-global.yml +++ b/.azuredevops/variables-global.yml @@ -79,6 +79,10 @@ variables: value: 93 - name: HIP_TAGGED_PIPELINE_ID value: 31 +- name: HIP_TESTS_PIPELINE_ID + value: 233 +- name: HIP_TESTS_TAGGED_PIPELINE_ID + value: 220 - name: HIPBLAS_COMMON_PIPELINE_ID value: 223 - name: HIPBLAS_COMMON_TAGGED_PIPELINE_ID @@ -159,6 +163,14 @@ variables: value: 80 - name: MIVISIONX_TAGGED_PIPELINE_ID value: 18 +- name: OMNIPERF_PIPELINE_ID + value: 241 +- name: OMNIPERF_TAGGED_PIPELINE_ID + value: 242 +- name: OMNITRACE_PIPELINE_ID + value: 253 +- name: OMNITRACE_TAGGED_PIPELINE_ID + value: 252 - name: RCCL_GFX942_TEST_PIPELINE_ID value: 184 - name: RCCL_PIPELINE_ID @@ -219,6 +231,10 @@ variables: value: 22 - name: ROCM_EXAMPLES_GFX942_TEST_PIPELINE_ID value: 204 +- name: ROCM_EXAMPLES_PIPELINE_ID + value: 216 +- name: ROCM_EXAMPLES_TAGGED_PIPELINE_ID + value: 245 - name: ROCM_SMI_LIB_PIPELINE_ID value: 96 - name: ROCM_SMI_LIB_TAGGED_PIPELINE_ID @@ -243,14 +259,26 @@ variables: value: 20 - name: ROCPROFILER_GFX942_TEST_PIPELINE_ID value: 190 -- name: ROCPROFILER_PIPELINE_ID - value: 143 - name: ROCPROFILER_REGISTER_PIPELINE_ID value: 1 - name: ROCPROFILER_REGISTER_TAGGED_PIPELINE_ID value: 25 +- name: ROCPROFILER_SDK_PIPELINE_ID + value: 246 +- name: ROCPROFILER_SDK_TAGGED_PIPELINE_ID + value: 234 +- name: ROCPROFILER_SYSTEMS_PIPELINE_ID + value: 255 +- name: ROCPROFILER_SYSTEMS_TAGGED_PIPELINE_ID + value: 254 +- name: ROCPROFILER_PIPELINE_ID + value: 143 - name: ROCPROFILER_TAGGED_PIPELINE_ID value: 28 +- name: ROCPYDECODE_PIPELINE_ID + value: 239 +- name: ROCPYDECODE_TAGGED_PIPELINE_ID + value: 232 - name: ROCR_DEBUG_AGENT_PIPELINE_ID value: 136 - name: ROCR_DEBUG_AGENT_TAGGED_PIPELINE_ID @@ -305,3 +333,5 @@ variables: value: 78 - name: RPP_TAGGED_PIPELINE_ID value: 39 +- name: BOOST_DEPENDENCY_PIPELINE_ID + value: 250 diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 59f713e31..7cb275391 100755 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -5,3 +5,4 @@ docs/ @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation *.rst @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation # External CI /.azuredevops/ @ROCm/external-ci +tools/rocm-build/ @ROCm/rocm-devops diff --git a/.wordlist.txt b/.wordlist.txt index e748ca6a7..580890845 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -272,6 +272,7 @@ OpenMPI OpenSSL OpenVX OpenXLA +Oversubscription PCC PCI PCIe @@ -490,6 +491,7 @@ cuLIB cuRAND cuSOLVER cuSPARSE +customizations cTDP dataset datasets @@ -619,6 +621,7 @@ openmp openssl optimizers os +oversubscription pageable parallelization parameterization @@ -698,6 +701,7 @@ rocsolver rocsparse rocthrust roctracer +rst runtime runtimes sL diff --git a/README.md b/README.md index 09a901eb1..2f42b747b 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ The following example shows how to use the repo tool to download the ROCm source ```bash mkdir -p ~/ROCm/ cd ~/ROCm/ -~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.0.x +~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.2.x ~/bin/repo sync ``` @@ -76,7 +76,7 @@ The Build time will reduce significantly if we limit the GPU Architecture/s agai mkdir -p ~/WORKSPACE/ # Or any folder name other than WORKSPACE cd ~/WORKSPACE/ -export ROCM_VERSION=6.2.0 +export ROCM_VERSION=6.2.2 # Or 6.2.0 or 6.2.1 ~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.2.x -m tools/rocm-build/rocm-${ROCM_VERSION}.xml ~/bin/repo sync @@ -199,4 +199,4 @@ cmake --build build --target=doc ## Older ROCm releases For release information for older ROCm releases, refer to the -[CHANGELOG](./CHANGELOG.md). +[ROCm release history](https://rocm.docs.amd.com/en/latest/release/versions.html). diff --git a/default.xml b/default.xml index a7f1254e1..d3088683d 100644 --- a/default.xml +++ b/default.xml @@ -1,7 +1,7 @@ - diff --git a/docs/about/license.md b/docs/about/license.md index 3e4f9b2da..cfa637b73 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -91,8 +91,7 @@ additional licenses. Please review individual repositories for more information. Open sourced ROCm components are released via public GitHub repositories, packages on [https://repo.radeon.com](https://repo.radeon.com) and other distribution channels. -Proprietary products are only available on [https://repo.radeon.com](https://repo.radeon.com). Currently, only -one component of ROCm, `rocm-llvm-alt` is governed by a proprietary license. +Proprietary products are only available on [https://repo.radeon.com](https://repo.radeon.com). Proprietary components are organized in a proprietary subdirectory in the package repositories to distinguish from open sourced packages. diff --git a/docs/compatibility/compatibility-matrix.rst b/docs/compatibility/compatibility-matrix.rst index bf94e371f..873d47192 100644 --- a/docs/compatibility/compatibility-matrix.rst +++ b/docs/compatibility/compatibility-matrix.rst @@ -10,6 +10,8 @@ Use this matrix to view the ROCm compatibility and system requirements across su You can also refer to the :ref:`past versions of ROCm compatibility matrix`. +Accelerators and GPUs listed in the following table support compute workloads (no display information or graphics). If you’re using ROCm with AMD Radeon or Radeon Pro GPUs for graphics workloads, see the `Use ROCm on Radeon GPU documentation `_ to verify compatibility and system requirements. + .. |br| raw:: html
diff --git a/docs/conceptual/ai-migraphx-optimization.md b/docs/conceptual/ai-migraphx-optimization.md deleted file mode 100644 index 5a5d1bf9a..000000000 --- a/docs/conceptual/ai-migraphx-optimization.md +++ /dev/null @@ -1,333 +0,0 @@ - - - - - - -# Inference optimization with MIGraphX - -The following sections cover inferencing and introduces [MIGraphX](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/). - -## Inference - -The inference is where capabilities learned during deep-learning training are put to work. It refers to using a fully trained neural network to make conclusions (predictions) on unseen data that the model has never interacted with before. Deep-learning inferencing is achieved by feeding new data, such as new images, to the network, giving the Deep Neural Network a chance to classify the image. - -Taking our previous example of MNIST, the DNN can be fed new images of handwritten digit images, allowing the neural network to classify digits. A fully trained DNN should make accurate predictions about what an image represents, and inference cannot happen without training. - -## MIGraphX introduction - -MIGraphX is a graph compiler focused on accelerating the machine-learning inference that can target AMD GPUs and CPUs. MIGraphX accelerates the machine-learning models by leveraging several graph-level transformations and optimizations. These optimizations include: - -* Operator fusion -* Arithmetic simplifications -* Dead-code elimination -* Common subexpression elimination (CSE) -* Constant propagation - -After doing all these transformations, MIGraphX emits code for the AMD GPU by calling to MIOpen or rocBLAS or creating HIP kernels for a particular operator. MIGraphX can also target CPUs using DNNL or ZenDNN libraries. - -MIGraphX provides easy-to-use APIs in C++ and Python to import machine models in ONNX or TensorFlow. Users can compile, save, load, and run these models using the MIGraphX C++ and Python APIs. Internally, MIGraphX parses ONNX or TensorFlow models into internal graph representation where each operator in the model gets mapped to an operator within MIGraphX. Each of these operators defines various attributes such as: - -* Number of arguments -* Type of arguments -* Shape of arguments - -After optimization passes, all these operators get mapped to different kernels on GPUs or CPUs. - -After importing a model into MIGraphX, the model is represented as `migraphx::program`. `migraphx::program` is made up of `migraphx::module`. The program can consist of several modules, but it always has one main_module. Modules are made up of `migraphx::instruction_ref`. Instructions contain the `migraphx::op` and arguments to the operator.   - -## Installing MIGraphX - -There are three options to get started with MIGraphX installation. MIGraphX depends on ROCm libraries; assume that the machine has ROCm installed. - -### Option 1: installing binaries - -To install MIGraphX on Debian-based systems like Ubuntu, use the following command: - -```bash -sudo apt update && sudo apt install -y migraphx -``` - -The header files and libraries are installed under `/opt/rocm-\`, where \ is the ROCm version. - -### Option 2: building from source - -There are two ways to build the MIGraphX sources. - -* [Use the ROCm build tool](https://github.com/ROCm/AMDMIGraphX#use-the-rocm-build-tool-rbuild) - This approach uses `[rbuild](https://github.com/ROCm/rbuild)` to install the prerequisites and build the libraries with just one command. - - or - -* [Use CMake](https://github.com/ROCm/AMDMIGraphX#use-cmake-to-build-migraphx) - This approach uses a script to install the prerequisites, then uses CMake to build the source. - -For detailed steps on building from source and installing dependencies, refer to the following `README` file: - -[https://github.com/ROCm/AMDMIGraphX#building-from-source](https://github.com/ROCm/AMDMIGraphX#building-from-source) - -### Option 3: use docker - -To use Docker, follow these steps: - -1. The easiest way to set up the development environment is to use Docker. To build Docker from scratch, first clone the MIGraphX repository by running: - - ```bash - git clone --recursive https://github.com/ROCm/AMDMIGraphX - ``` - -2. The repository contains a Dockerfile from which you can build a Docker image as: - - ```bash - docker build -t migraphx . - ``` - -3. Then to enter the development environment, use Docker run: - - ```bash - docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX --group-add video -it migraphx - ``` - -The Docker image contains all the prerequisites required for the installation, so users can go to the folder `/code/AMDMIGraphX` and follow the steps mentioned in [Option 2: Building from Source](#option-2-building-from-source). - -## MIGraphX example - -MIGraphX provides both C++ and Python APIs. The following sections show examples of both using the Inception v3 model. To walk through the examples, fetch the Inception v3 ONNX model by running the following: - -```py -import torch -import torchvision.models as models -inception = models.inception_v3(pretrained=True) -torch.onnx.export(inception,torch.randn(1,3,299,299), "inceptioni1.onnx") -``` - -This will create `inceptioni1.onnx`, which can be imported in MIGraphX using C++ or Python API. - -### MIGraphX Python API - -Follow these steps: - -1. To import the MIGraphX module in Python script, set `PYTHONPATH` to the MIGraphX libraries installation. If binaries are installed using steps mentioned in [Option 1: Installing Binaries](#option-1-installing-binaries), perform the following action: - - ```bash - export PYTHONPATH=$PYTHONPATH:/opt/rocm/ - ``` - -2. The following script shows the usage of Python API to import the ONNX model, compile it, and run inference on it. Set `LD_LIBRARY_PATH` to `/opt/rocm/` if required. - - ```py - # import migraphx and numpy - import migraphx - import numpy as np - # import and parse inception model - model = migraphx.parse_onnx("inceptioni1.onnx") - # compile model for the GPU target - model.compile(migraphx.get_target("gpu")) - # optionally print compiled model - model.print() - # create random input image - input_image = np.random.rand(1, 3, 299, 299).astype('float32') - # feed image to model, 'x.1` is the input param name - results = model.run({'x.1': input_image}) - # get the results back - result_np = np.array(results[0]) - # print the inferred class of the input image - print(np.argmax(result_np)) - ``` - - Find additional examples of Python API in the `/examples` directory of the MIGraphX repository. - -## MIGraphX C++ API - -Follow these steps: - -1. The following is a minimalist example that shows the usage of MIGraphX C++ API to load ONNX file, compile it for the GPU, and run inference on it. To use MIGraphX C++ API, you only need to load the `migraphx.hpp` file. This example runs inference on the Inception v3 model. - - ```c++ - #include - #include - #include - #include - #include - #include - - int main(int argc, char** argv) - { - migraphx::program prog; - migraphx::onnx_options onnx_opts; - // import and parse onnx file into migraphx::program - prog = parse_onnx("inceptioni1.onnx", onnx_opts); - // print imported model - prog.print(); - migraphx::target targ = migraphx::target("gpu"); - migraphx::compile_options comp_opts; - comp_opts.set_offload_copy(); - // compile for the GPU - prog.compile(targ, comp_opts); - // print the compiled program - prog.print(); - // randomly generate input image - // of shape (1, 3, 299, 299) - std::srand(unsigned(std::time(nullptr))); - std::vector input_image(1*299*299*3); - std::generate(input_image.begin(), input_image.end(), std::rand); - // users need to provide data for the input - // parameters in order to run inference - // you can query into migraph program for the parameters - migraphx::program_parameters prog_params; - auto param_shapes = prog.get_parameter_shapes(); - auto input = param_shapes.names().front(); - // create argument for the parameter - prog_params.add(input, migraphx::argument(param_shapes[input], input_image.data())); - // run inference - auto outputs = prog.eval(prog_params); - // read back the output - float* results = reinterpret_cast(outputs[0].data()); - float* max = std::max_element(results, results + 1000); - int answer = max - results; - std::cout << "answer: " << answer << std::endl; - } - ``` - -2. To compile this program, you can use CMake and you only need to link the `migraphx::c` library to use MIGraphX's C++ API. The following is the `CMakeLists.txt` file that can build the earlier example: - - ```cmake - cmake_minimum_required(VERSION 3.5) - project (CAI) - - set (CMAKE_CXX_STANDARD 14) - set (EXAMPLE inception_inference) - - list (APPEND CMAKE_PREFIX_PATH /opt/rocm/hip /opt/rocm) - find_package (migraphx) - - message("source file: " ${EXAMPLE}.cpp " ---> bin: " ${EXAMPLE}) - add_executable(${EXAMPLE} ${EXAMPLE}.cpp) - - target_link_libraries(${EXAMPLE} migraphx::c) - ``` - -3. To build the executable file, run the following from the directory containing the `inception_inference.cpp` file: - - ```bash - mkdir build - cd build - cmake .. - make -j$(nproc) - ./inception_inference - ``` - -:::{note} - Set `LD_LIBRARY_PATH` to `/opt/rocm/lib` if required during the build. Additional examples can be found in the MIGraphX repository under the `/examples/` directory. -::: - -## Tuning MIGraphX - -MIGraphX uses MIOpen kernels to target AMD GPU. For the model compiled with MIGraphX, tune MIOpen to pick the best possible kernel implementation. The MIOpen tuning results in a significant performance boost. Tuning can be done by setting the environment variable `MIOPEN_FIND_ENFORCE=3`. - -:::{note} - The tuning process can take a long time to finish. -::: - -**Example:** The average inference time of the inception model example shown previously over 100 iterations using untuned kernels is 0.01383ms. After tuning, it reduces to 0.00459ms, which is a 3x improvement. This result is from ROCm v4.5 on a MI100 GPU. - -:::{note} - The results may vary depending on the system configurations. -::: - -For reference, the following code snippet shows inference runs for only the first 10 iterations for both tuned and untuned kernels: - -```console -### UNTUNED ### -iterator : 0 -Inference complete -Inference time: 0.063ms -iterator : 1 -Inference complete -Inference time: 0.008ms -iterator : 2 -Inference complete -Inference time: 0.007ms -iterator : 3 -Inference complete -Inference time: 0.007ms -iterator : 4 -Inference complete -Inference time: 0.007ms -iterator : 5 -Inference complete -Inference time: 0.008ms -iterator : 6 -Inference complete -Inference time: 0.007ms -iterator : 7 -Inference complete -Inference time: 0.028ms -iterator : 8 -Inference complete -Inference time: 0.029ms -iterator : 9 -Inference complete -Inference time: 0.029ms - -### TUNED ### -iterator : 0 -Inference complete -Inference time: 0.063ms -iterator : 1 -Inference complete -Inference time: 0.004ms -iterator : 2 -Inference complete -Inference time: 0.004ms -iterator : 3 -Inference complete -Inference time: 0.004ms -iterator : 4 -Inference complete -Inference time: 0.004ms -iterator : 5 -Inference complete -Inference time: 0.004ms -iterator : 6 -Inference complete -Inference time: 0.004ms -iterator : 7 -Inference complete -Inference time: 0.004ms -iterator : 8 -Inference complete -Inference time: 0.004ms -iterator : 9 -Inference complete -Inference time: 0.004ms -``` - -### YModel - -The best inference performance through MIGraphX is conditioned upon having tuned kernel configurations stored in a `/home` local User Database (DB). If a user were to move their model to a different server or allow a different user to use it, they would have to run through the MIOpen tuning process again to populate the next User DB with the best kernel configurations and corresponding solvers. - -Tuning is time consuming, and if the users have not performed tuning, they would see discrepancies between expected or claimed inference performance and actual inference performance. This has led to repetitive and time-consuming tuning tasks for each user. - -MIGraphX introduces a feature, known as YModel, that stores the kernel config parameters found during tuning into a `.mxr` file. This ensures the same level of expected performance, even when a model is copied to a different user/system. - -The YModel feature is available starting from ROCm 5.4.1 and UIF 1.1. - -#### YModel example - -Through the `migraphx-driver` functionality, you can generate `.mxr` files with tuning information stored inside it by passing additional `--binary --output model.mxr` to `migraphx-driver` along with the rest of the necessary flags. - -For example, to generate `.mxr` file from the ONNX model, use the following: - -```bash -./path/to/migraphx-driver compile --onnx resnet50.onnx --enable-offload-copy --binary --output resnet50.mxr -``` - -To run generated `.mxr` files through `migraphx-driver`, use the following: - -```bash -./path/to/migraphx-driver run --migraphx resnet50.mxr --enable-offload-copy -``` - -Alternatively, you can use the MIGraphX C++ or Python API to generate `.mxr` files. - -![Generating an MXR file](../data/conceptual/image018.png "Generating an MXR file") diff --git a/docs/conceptual/oversubscription.rst b/docs/conceptual/oversubscription.rst new file mode 100644 index 000000000..83865876a --- /dev/null +++ b/docs/conceptual/oversubscription.rst @@ -0,0 +1,34 @@ +.. meta:: + :description: Learn what causes oversubscription. + :keywords: warning, log, gpu, performance penalty, help + +******************************************************************* +Oversubscription of hardware resources in AMD Instinct accelerators +******************************************************************* + +When an AMD Instinct™ MI series accelerator enters an oversubscribed state, the ``amdgpu`` driver outputs the following +message. + +``amdgpu: Runlist is getting oversubscribed. Expect reduced ROCm performance.`` + +Oversubscription occurs when application demands exceed the available hardware resources. In an oversubscribed +state, the hardware scheduler tries to manage resource usage in a round-robin fashion. However, +this can result in reduced performance, as resources might be occupied by applications or queues not actively +submitting work. The granularity of hardware resources occupied by an inactive queue can be in the order of +milliseconds, during which the accelerator or GPU is effectively blocked and unable to process work submitted by other +queues. + +What triggers oversubscription? +=============================== + +The system enters an oversubscribed state when one of the following conditions is met: + +* **Hardware queue limit exceeded**: The number of user-mode compute queues requested by applications exceeds the + hardware limit of 24 queues for current Instinct accelerators. + +* **Virtual memory context slots exceeded**: The number of user processes exceeds the number of available virtual memory + context slots, which is 11 for current Instinct accelerators. + +* **Multiple processes using cooperative workgroups**: More than one process attempts to use the cooperative workgroup + feature, leading to resource contention. + diff --git a/docs/contribute/building.md b/docs/contribute/building.md index e8557f5bc..97801832b 100644 --- a/docs/contribute/building.md +++ b/docs/contribute/building.md @@ -7,17 +7,14 @@ # Building documentation -You can build our documentation via GitHub (in a pull request) or locally (using the command line or -Visual Studio (VS) Code. - ## GitHub -If you open a pull request on the `develop` branch of a ROCm repository and scroll to the bottom of -the page, there is a summary panel. Next to the line -`docs/readthedocs.com:advanced-micro-devices-demo`, there is a `Details` link. If you click this, it takes -you to the Read the Docs build for your pull request. +If you open a pull request and scroll down to the summary panel, +there is a commit status section. Next to the line +`docs/readthedocs.com:advanced-micro-devices-demo`, there is a `Details` link. +If you click this, it takes you to the Read the Docs build for your pull request. -![Screenshot of the GitHub documentation build link](../data/contribute/github-docs-build.png) +![GitHub PR commit status](../data/contribute/commit-status.png) If you don't see this line, click `Show all checks` to get an itemized view. diff --git a/docs/contribute/contributing.md b/docs/contribute/contributing.md index fcb7ebd34..01fd662f0 100644 --- a/docs/contribute/contributing.md +++ b/docs/contribute/contributing.md @@ -4,113 +4,68 @@ -# Contribute to ROCm documentation +# Contributing to the ROCm documentation -All ROCm projects are GitHub-based, so if you want to contribute, you can do so by: +The ROCm documentation, like all of ROCm, is open source and available on GitHub. You can contribute to the ROCm documentation by forking the appropriate repository, making your changes, and opening a pull request. -* [Submitting a pull request in the appropriate GitHub repository](#submit-a-pull-request) -* [Creating an issue in the appropriate GitHub repository](#create-an-issue) -* [Suggesting a new feature](#suggest-a-new-feature) +To provide feedback on the ROCm documentation, including submitting an issue or suggesting a feature, see [Providing feedback about the ROCm documentation](./feedback.md). -```{important} -By creating a pull request (PR), you agree to allow your contribution to be licensed under the terms of the -LICENSE.txt file in the corresponding repository. Different repositories may use different licenses. -``` +## The ROCm repositories -## Submit a pull request +The repositories for ROCm and all ROCm components are available on GitHub. -To make edits to our documentation via PR, follow these steps: +| Module | Documentation location | +| --- | --- | +| ROCm framework | [https://github.com/ROCm/ROCm/tree/develop/docs](https://github.com/ROCm/ROCm/tree/develop/docs) | +| ROCm installation for Linux | [https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs](https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs) | +| ROCm HIP SDK installation for Windows | [https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs](https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs) | -1. Identify the repository and the file you want to update. For example, to update this page, you would - need to modify content located in this file: - `https://github.com/ROCm/ROCm/blob/develop/docs/contribute/contributing.md` +Individual components have their own repositories with their own documentation in their own `docs` folders. -2. (optional, but recommended) Fork the repository. +The sub-folders within the `docs` folders across ROCm are typically structured as follows: -3. Clone the repository locally and (optionally) add your fork. Select the green 'Code' button and copy - the URL (e.g., `git@github.com:ROCm/ROCm.git`). +| Sub-folder name | Documentation type | +|-------|----------| +| `install` | Installation instructions, build instructions, and prerequisites | +| `conceptual` | Important concepts | +| `how-to` | How to implement specific use cases | +| `tutorials` | Tutorials | +| `reference` | API references and other reference resources | - * From your terminal, run: +## Editing and adding to the documentation - ```bash - git clone git@github.com:ROCm/ROCm.git - ``` +The ROCm documentation is written in [reStructuredText (rst)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html) and [Github-flavoured Markdown](https://github.github.com/gfm/), and follows the [Google developer documentation style guide](https://developers.google.com/style/highlights). reStructuredText is preferred when adding content to the documentation. - * Optionally add your fork to this local copy of the repository by running: +To edit or add to the documentation: - ```bash - git add remote - ``` +1. Fork the repository you want to add to or edit. +2. Clone your fork locally. +3. Create a new local branch cut from the `develop` branch of the repository. +4. Make your changes to the documentation. - To get the URL of your fork, go to your GitHub profile, select the fork and click the green 'Code' - button (the same process you followed to get the main GitHub repository URL). - -4. Change directory into your local copy of the repository, and run ``git pull`` (or ``git pull origin develop``) to ensure your local copy has the most recent content. - -5. Create and checkout a new branch using the following command: +5. Optionally, build the documentation locally before creating a pull request by running the following commands from within the `docs` folder: ```bash - git checkout -b - ``` - -6. Change directory into the `./docs` folder and make any documentation changes locally using your preferred code editor. Follow the guidelines listed on the - [documentation structure](./doc-structure.md) page. - - ```{note} - Spell checking is performed for pull requests by {doc}`ROCm Docs Core`. To ensure your PR passes spell checking you might need at add new words or acronyms to the `.wordlist.txt` file as described in {doc}`Spell Check`. - ``` - -7. Optionally run a local test build of the documentation to ensure the content builds and looks as expected. In your terminal, run the following commands from within the `./docs` folder of your cloned repository: - - ```bash pip3 install -r sphinx/requirements.txt # You only need to run this command once python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html ``` - The build output files are located in the `docs/_build` folder. To preview your build, open the index file - (`docs/_build/html/index.html`) file. For more information, see [Building documentation](building.md). To learn - more about our build tools, see [Documentation toolchain](toolchain.md). + The output files will be located in the `docs/_build` folder. Open `docs/_build/html/index.html` to view the documentation. -8. Commit your changes and push them to GitHub by running: + For more information on ROCm build tools, see [Documentation toolchain](toolchain.md). +6. Push your changes. A GitHub link will be returned in the output of the `git push` command. Open this link in a browser to create the pull request. - ```bash - git add # To add all modified files, you can use: git add . - git commit -m "my-updates" - git push - ``` + The documentation is built as part of the checks on pull request, along with spell checking and linting. Scroll to the bottom of your pull request to view all the checks. - After pushing, you will get a GitHub link in the terminal output. Copy this link and paste it into a - browser to create your PR. + Verify that the linking and spell checking have passed, and that the documentation was built successfully. New words or acronyms can be added to the [wordlist file](https://github.com/ROCm/rocm-docs-core/blob/develop/.wordlist.txt) as needed. -## Create an issue + The Read The Docs build of your pull request can be accessed by clicking on the Details link next to the Read The Docs build check. Verify that your changes are in the build and look as expected. -1. To create a new GitHub issue, select the 'Issues' tab in the appropriate repository - (e.g., https://github.com/ROCm/ROCm/issues). -2. Use the search bar to make sure the issue doesn't already exist. -3. If your issue is not already listed, select the green 'New issue' button to the right of the page. Select - the type of issue and fill in the resulting template. + Your pull request will be reviewed by a member of the ROCm documentation team. -### General issue guidelines +See the [GitHub documentation](https://docs.github.com/en) for information on how to fork and clone a repository, and how to create and push a local branch. -* Use your best judgement for issue creation. If your issue is already listed, upvote the issue and - comment or post to provide additional details, such as how you reproduced this issue. -* If you're not sure if your issue is the same, err on the side of caution and file your issue. - You can add a comment to include the issue number (and link) for the similar issue. If we evaluate - your issue as being the same as the existing issue, we'll close the duplicate. -* If your issue doesn't exist, use the issue template to file a new issue. - * When filing an issue, be sure to provide as much information as possible, including script output so - we can collect information about your configuration. This helps reduce the time required to - reproduce your issue. - * Check your issue regularly, as we may require additional information to successfully reproduce the - issue. - -## Suggest a new feature - -Use the [GitHub Discussion forum](https://github.com/ROCm/ROCm/discussions) -(Ideas category) to propose new features. Our maintainers are happy to provide direction and -feedback on feature development. - -## Future development workflow - -The current ROCm development workflow is GitHub-based. If, in the future, we change this platform, -the tools and links may change. In this instance, we will update contribution guidelines accordingly. +```{important} +By creating a pull request (PR), you agree to allow your contribution to be licensed under the terms of the +LICENSE.txt file in the corresponding repository. Different repositories can use different licenses. +``` diff --git a/docs/contribute/doc-structure.md b/docs/contribute/doc-structure.md deleted file mode 100644 index 47c057021..000000000 --- a/docs/contribute/doc-structure.md +++ /dev/null @@ -1,219 +0,0 @@ -# Documentation structure - -Our documentation follows the Pitchfork folder structure. Most documentation files are stored in the -`/docs` folder. Some special files (such as release, contributing, and changelog) are stored in the root -(`/`) folder. - -All images are stored in the `/docs/data` folder. An image's file path mirrors that of the documentation -file where it is used. - -Our naming structure uses kebab case; for example, `my-file-name.rst`. - -## Supported formats and syntax - -Our documentation includes both Markdown and RST files. We are gradually transitioning existing -Markdown to RST in order to more effectively meet our documentation needs. When contributing, -RST is preferred; if you must use Markdown, use GitHub-flavored Markdown. - -We use [Sphinx Design](https://sphinx-design.readthedocs.io/en/latest/index.html) syntax and compile -our API references using [Doxygen](https://www.doxygen.nl/). - -The following table shows some common documentation components and the syntax convention we -use for each: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ComponentRST syntax
Code blocks - -```rst - -.. code-block:: language-name - - My code block. - - -``` - -
Cross-referencing internal files - -```rst - -:doc:`Title <../path/to/file/filename>` - -``` - -
External links - -```rst - -`link name `_ - -``` - -
Headings - -```rst - -****************** -Chapter title (H1) -****************** - -Section title (H2) -=============== - -Subsection title (H3) ---------------------- - -Sub-subsection title (H4) -^^^^^^^^^^^^^^^^^^^^ - - -``` - -
Images - -```rst - -.. image:: image1.png - -``` - -
Internal links - -```rst - -1. Add a tag to the section you want to reference: - -.. _my-section-tag: section-1 - -Section 1 -========== - -2. Link to your tag: - -As shown in :ref:`section-1`. - -``` - -
Lists - -```rst - -# Ordered (numbered) list item - -* Unordered (bulleted) list item - -``` - -
Math (block) - -```rst - -.. math:: - - A = \begin{pmatrix} - 0.0 & 1.0 & 1.0 & 3.0 \\ - 4.0 & 5.0 & 6.0 & 7.0 \\ - \end{pmatrix} - -``` - -
Math (inline) - -```rst - -:math:`2 \times 2 ` - -``` - -
Notes - -```rst - -.. note:: - - My note here. - -``` - -
Tables - -```rst - -.. csv-table:: Optional title here - :widths: 30, 70 #optional column widths - :header: "entry1 header", "entry2 header" - - "entry1", "entry2" - -``` - -
- -## Language and style - -We use the -[Google developer documentation style guide](https://developers.google.com/style/highlights) to -guide our content. - -Font size and type, page layout, white space control, and other formatting -details are controlled via -[rocm-docs-core](https://github.com/ROCm/rocm-docs-core). If you want to notify us -of any formatting issues, create a pull request in our -[rocm-docs-core](https://github.com/ROCm/rocm-docs-core) GitHub repository. - -## Building our documentation - - -To learn how to build our documentation, refer to -[Building documentation](./building.md). diff --git a/docs/contribute/feedback.md b/docs/contribute/feedback.md index fe12c0785..2adf8a78e 100644 --- a/docs/contribute/feedback.md +++ b/docs/contribute/feedback.md @@ -4,12 +4,24 @@ -# Providing feedback +# Providing feedback about the ROCm documentation -Your feedback is welcome. You can provide feedback either through GitHub Discussions or GitHub Issues. +Feedback about the ROCm documentation is welcome. You can provide feedback about the ROCm documentation either through GitHub Discussions or GitHub Issues. -Use [GitHub Discussions](https://github.com/ROCm/ROCm/discussions) to ask questions, view announcements, and communicate with other members of the community. +## Participating in discussions through GitHub Discussions -Use [GitHub Issues](https://github.com/ROCm/ROCm/issues) to submit issues you find with ROCm or with the ROCm documentation. +You can ask questions, view announcements, suggest new features, and communicate with other members of the community through [GitHub Discussions](https://github.com/ROCm/ROCm/discussions). -For information about contributing to the ROCm repository and creating a pull request (PR), see [Contributing](./contributing.md). +## Submitting issues through GitHub Issues + +You can submit issues through [GitHub Issues](https://github.com/ROCm/ROCm/issues). + +When creating a new issue, follow the following guidelines: + +1. Always do a search to see if the same issue already exists. If the issue already exists, upvote it, and comment or post to provide any additional details you might have. +2. If you find an issue that is similar to your issue, log your issue, then add a comment that includes a link to the similar issue, as well as its issue number. +3. Always provide as much information as possible. This helps reduce the time required to reproduce the issue. + +After creating your issue, make sure to check it regularly for any requests for additional information. + +For information about contributing content to the ROCm documentation, see [Contributing to the ROCm documentation](./contributing.md). diff --git a/docs/contribute/toolchain.md b/docs/contribute/toolchain.md index 077f9aed4..0b4f72982 100644 --- a/docs/contribute/toolchain.md +++ b/docs/contribute/toolchain.md @@ -6,60 +6,47 @@ # ROCm documentation toolchain -Our documentation relies on several open source toolchains and sites. +The ROCm documentation relies on several open source toolchains and sites. -## `rocm-docs-core` +## rocm-docs-core [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) is an AMD-maintained -project that applies customization for our documentation. This project is the tool most ROCm -repositories use as part of the documentation build. It is also available as a -[pip package on PyPI](https://pypi.org/project/rocm-docs-core/). +project that applies customizations for the ROCm documentation. This project is the tool most ROCm repositories use as part of their documentation build pipeline. It is available as a [pip package on PyPI](https://pypi.org/project/rocm-docs-core/). See the user and developer guides for rocm-docs-core at {doc}`rocm-docs-core documentation`. ## Sphinx -[Sphinx](https://www.sphinx-doc.org/en/master/) is a documentation generator originally used for -Python. It is now widely used in the open source community. +[Sphinx](https://www.sphinx-doc.org/en/master/) is a documentation generator originally used for Python. It is now widely used in the open source community. ### Sphinx External ToC -[Sphinx External ToC](https://sphinx-external-toc.readthedocs.io/en/latest/intro.html) is a Sphinx -extension used for ROCm documentation navigation. This tool generates a navigation menu on the left +[Sphinx External ToC](https://sphinx-external-toc.readthedocs.io/en/latest/intro.html) is a Sphinx extension used for ROCm documentation navigation. This tool generates a navigation menu on the left based on a YAML file (`_toc.yml.in`) that contains the table of contents. ### Sphinx-book-theme -[Sphinx-book-theme](https://sphinx-book-theme.readthedocs.io/en/latest/) is a Sphinx theme that -defines the base appearance for ROCm documentation. ROCm documentation applies some -customization, such as a custom header and footer on top of the Sphinx Book Theme. +[Sphinx-book-theme](https://sphinx-book-theme.readthedocs.io/en/latest/) is a Sphinx theme that defines the base appearance for ROCm documentation. ROCm documentation applies some customization, such as a custom header and footer, on top of the Sphinx Book Theme. ### Sphinx Design -[Sphinx design](https://sphinx-design.readthedocs.io/en/latest/index.html) is a Sphinx extension that -adds design functionality. ROCm documentation uses Sphinx Design for grids, cards, and synchronized -tabs. +[Sphinx design](https://sphinx-design.readthedocs.io/en/latest/index.html) is a Sphinx extension that adds design functionality. ROCm documentation uses Sphinx Design for grids, cards, and synchronized tabs. ## Doxygen -[Doxygen](https://www.doxygen.nl/) is a documentation generator that extracts information from inline -code. ROCm projects typically use Doxygen for public API documentation (unless the upstream project -uses a different tool). +[Doxygen](https://www.doxygen.nl/) is a documentation generator that extracts information from in-code comments. It is used for API documentation. ## Breathe -[Breathe](https://www.breathe-doc.org/) is a Sphinx plugin to integrate Doxygen content. +[Breathe](https://www.breathe-doc.org/) is a Sphinx plugin for integrating Doxygen content. ## MyST -[Markedly Structured Text (MyST)](https://myst-tools.org/docs/spec) is an extended flavor of -Markdown ([CommonMark](https://commonmark.org/)) influenced by reStructuredText (RST) and -Sphinx. It's integrated into ROCm documentation by the Sphinx extension -[`myst-parser`](https://myst-parser.readthedocs.io/en/latest/). -A MyST syntax cheat sheet is available on the [Jupyter reference](https://jupyterbook.org/en/stable/reference/cheatsheet.html) site. +[Markedly Structured Text (MyST)](https://myst-tools.org/docs/spec) is an extended flavor of Markdown ([CommonMark](https://commonmark.org/)) influenced by reStructuredText (rst) and Sphinx. It is integrated into the ROCm documentation with the [`myst-parser`](https://myst-parser.readthedocs.io/en/latest/) Sphinx extension. + +See the [MyST syntax cheat sheet](https://jupyterbook.org/en/stable/reference/cheatsheet.html) at the Jupyter Book site. ## Read the Docs -[Read the Docs](https://docs.readthedocs.io/en/stable/) is the service that builds and hosts the HTML -documentation generated using Sphinx to our end users. +[Read the Docs](https://docs.readthedocs.io/en/stable/) is the service that builds and hosts the HTML version of the ROCm documentation. diff --git a/docs/data/contribute/commit-status.png b/docs/data/contribute/commit-status.png new file mode 100644 index 0000000000000000000000000000000000000000..d6f7818151eeefaf2e013858ae09beeb56c5dbe5 GIT binary patch literal 65570 zcmce+V|Zj;(>9!BCdo`Vu_m_dq=SiVV`AI3ZF^#y9ox2Tn{V#>x#oJlKi|La{rc#m zyVu^UYpq?os&-YKXNZiHFf0@%6bJ|itf+{990pG8C%(3eCYsuP9p!$q+hlM zI`$@3mUs##76u@apFikW@uc2Bti3f;6Om|Ktu)j6r9sf z*PQH;6>107@&7=|efxn|Cj%y8oD;YnMuzt-w+JuW9;KCxLJ(X#K35k!e%*~0 zlpil6(ck|EKfmkBn#+d>F9&Guk2jC;A$A?7{c)g^QPODb!@8{baBC44*ui%#qKR;i zc!JOQ-|+wYfg@_Nhm{JKca)p_|Grd#+bYTU};loBC%{^RS_F#Yev7QLZhCofd@ zw?#6$Y%S?uw1#e#z|jAzjd!!*S4?752a76PrK_A@vFa%cb={sSf>g)L@nSB2rDn>Y zz7@SYvMHJ-uiwXS128%o7v!f@p)svig@j~kP5N4q|Mca~sbRum1czL+&d$_v0<6ya zXXGKpz%{a>vX#PyxejxXE01Q=Sdy;2MxwvVpS}cap(%X$oZS5(pWzFUJh@){y0qbt zI@Tj+{~Jj^>hnOpQh`Gd`_Cw8JOI8=Q&HcoL(|)=?CEput_IlE_XE`%F@riMms8nz zM;Ca0Wh)9!J*40c>(t8D)ptGxOo^4T{n>!8+dKC+i60%howFT&a+&WunS`IhYEOja z@3OeA{^!m+E0}UD<%BiGCf_xg5xj#Mu#i=s9eM|JoNc(mKzpg1b8>LG@`Y0S0GD;N zS64z68C&X|sBXy9{<&ALq(#O0KhuZu7jFkNk*?s$;S&@|uur^pRrVdqWN#w$ z&qdr08RDRg zksew=RhE%svFiPI%diIUll8RBerAsORyjHbhRaODdGf!;>>OS=!y5alMpQe4-}lci zVF-GKq$gj5s5g3#rbD(FE0|YZXN3yq@s?qiO_5p*U7>%oz||Q;1RsFoHqpuB)gek_%@U0Ab%{Iw+MHnfK-n;R%9-#{zBUn( zY?mGa7{52DbuKmlmpHEj_WKAC9#lQce4UaVSgJmi&ondkx)Iem=G!c-ee^i6AsFws z-rh9ye6&WFZeEYA=02>hJX50aZfwXPrr82BavaMpj{_rHoIM_?yO^)J*fkl?m~K`LT$$b)>s{PEg{X1tRDmKPAfY+<}9Uff;+#l1L^@RkdKbSP%SgEdpNXvk zuq~;z_iM9QY{&=#<6aVPre@kzMt)HIH>O<)$W10Z^iAI-;<^R3j)G0LEH4 zOF$>C*X3BDmbnLLzY9&ePFt;u0nQG6rDUhjj;rk}1y6D=X4>j`cAldQ=fL$sqCBuN z(B6zZ#)2^oi-y!)UY;5k9aROD&%jZsz0wNJKU!XCl)I7=rKJh86)pPhamG@2Uv5+7 zGNo|eMFcC&(5|$YPPtPA6Hr?2A7jb5lUbepvOF;*C2u;`r%A>YjaNNjV6mFzChr{@ zPSr*#!Yk|>Yx7i3o|7$?gh8`!Or*TiP-lO_m9GeenucBs62HDHfjDBKC{_wvZG~tL zBt4cH;MEJ?8$Cx-K8T8;oN}}c@HBO5Vj7}p@ zfkn7op6TxS(eM)?eG%_`-vYgjwm3LQA9zrKCLNAvU~Le8Xay)ufL(tHz5~{M_jRX4 zjTJCmR-Q(x=i*iAq`W4je7}7K+v-_;1QW38hAx5Q6{Y8 z>H)q$B*)^#I~)Lpml!7OXJeuUngJG&>1XW#5P40XXpSuh)7gD39F@(~FEN z4O%U)l>4#6+ZD%>?$5xI+gp;NSxmdCCKp<9_y{kyvmSsGF(y09FIBE7EU~CIFL=U7 zpRdy>htTV%SvNZfYgA@Vg-ag@(U@#@&=Ihj zc%kJlt?p8^^Op{M$TrJbP;n=KMghN3!!}VCg;tr|Zswb%Q>G&-*E2m3nk`q%$__bc z(wCWRftZ@w_{!i^>nO}2RL9n6E=i5;4Y6Q2R#XDc{D61Ts3}y;=H0k%9GW4=ySshh zJUozQ3j0jY$_7nU%J}#;#Z=Q|m0x{jP4v_la%;8_3E1)6)Gr9lwadG~BD`b-waC|k zvAFOT8r$n-^)uO1M(Xe`O!3MEQLO%BdBEaq%tyYF+2z4SFMFH*?)go!L@C#F?OsC7 z4Z%-WaBb_CiJ{W<856NZmMWo^ixu8_L3T%7M#_X!fkzVB<8?GB=L*A}4dH(+_MNdt z^l2&RmzCSvt(|k1`}Mg!g4LN{K%XGPc*YB5CtZIuf?M&SQv~V${k{s5I$xgC_~c^- zBaZ*H(p;)lxt-tI&eNLKowdGHWHQ`K6)qDS=_VB&wnJwDu z{hP}|$@N!qM(wL(4N37c)K2)3wgMD(Va7?)gxiGK?YVqNSP2vb53&=MUfs!_min0R z3RNsk;JJN8UG$R?Y?KNkb>db?#Hp zh4-2+!?uhu#N79=cMca%7}}6?{oT$X%t*qQPcfq>R#MR7(8#>iMG_+b50S+>Kj6#3 zrCgw00U(>3+!l9I$7**)XsTsRIv;)hMw22F@UK1~zd z;_Uqky{*$TBGjU2ckjyf<<3pPH?oVR6dF%l>&H!?FwxNjmCp8T7Mm7`KwIC}386CV zgRpVNXeC9%AGR)}Sy5bmDXBwCevoWw7NLGXYNbA0-q3tlB`C6jM3rM?UO)La%iFU8 zqm_zZuK));*^Q+f^ci+5vLi<*DGRar_MPQS`@30R`iyei@SPD!@}hLSXJ*Lbv*+Z* zM1~Kc|5h^bHYEr$R#^KwRHL+>0NlKG?_1`vU@q5u0YOtPt-}4yE;1t}>%CQ-3V}C% zwfRN`*U45v3N@MVq6f_UoPC|P9q@!u*(`WvG$i^5ZU(DXi8-A)CFqhZ96}A`LA8+- zTvWYvBj$5#Nb9|UtU={2XA}IGT2?*za_T(_=Kl6I87_{~?bU8&(PlM{8!9pJl&8>z zn)RLv1i2QRBAn$C`x9A6B*A<-eQGQ{SZ^Q3lLbWCqTJ-!0`&!oeCseR*z<(dDX{)J z>u#iqj428XP9B5DO+FZ$XZn?CORfx9qCU-RHu>J<3a$b~++|KSl>&TA1 zC~M1X0Q|%RT33{{mc{NcF&YwhPxHQbh>Vq;0!3bVI47qSEU8}p$Vv?%fre1sZNAMS z_{WW5pnv-6sB-09_XfV7-Brp1uB%}izq|YW-5FFZ^4G9bd-NCB zv`B#;*fzmD>*f(X)EA*;+-6HIbKgkjjOT{2%LNHjS($`o?bw?vvB*nMD@(MDQDjEd z8E<+6w%f(rG( z3gkCSF5&TfFe)GxipO_oZ@5x#t1DQjww}AJ?<{wj{RH=42t2*AdV45EIdT2?^zJvE z#OAW_S;rtKABMfoO3R~OpgD`>3ST{(Shb)CCt46#;8??ptDthHZ*FXCFggXaodkrjY!4GrWVRIE$y|>^ zG)dChrn=W^7TLAZIdg*+^(@2A83MR``bU0(#@=`#G4YPOy813xr!a@AcpwrF5Q=w%hU7hR!p6 z*wjT+89VqY&lLlOxdX8?&Z{_>d&PbrLc_`3dos%j@evWh-z=3 z-k#0`qxtpMvX>LBND`I$pD{W#H6X@a?Z>N)5px~@#?b!T7RaTd`x@uY6sSghu$gd! zkc4!n@+jFoHwS_S!qDVu%zo}yD4(B@*c;3!v&BwPUm-r871#VWEuBwC30}f7Fz$eR z)aO?3B}*Lg(=L((GF{_GJPE(sd`GwDHPVhtMPWJ(V+Mm_ z8Pnd0%WhvWiDJeUAcO zs4bOb9tQMaG;u_0e>j8Vwz7m{ErdqIIY#?1S13~aE|uInti-(>P`1+1)s84!zT4X) zr=40|!tAIxE57Xzk>U|+(%+Wu@L{f&fzfz%pqh|a5(oTVt|K0A+{0Kp)vk@0U4OGN zp3KzIP*r`cB8k4#C)$m9)p@Fdh>QduwyuN-2L zLIyi5nqKTM_7~b>*Bviua{MU*X+$ly%%L-Z{NCZoD;9{b=jkUl>l~W||7+$-lHyi7G9;D%d?||^=n7zm z=&Z_W%HN@f01D)HRp!hL#9?q`CT9%L+n!*H)ebE(`^~CpZ!fuH=(K7R&YN;r0u#3? zM)K*HDGn67a6`JH7;KFeSVsK|If0@cTJ9g< zQo!bLvJ9R|AaiO({n%ts(3M~V6ft&kUO}zV3}4MkLx0JXOuy6880X2j+~#6*)zp2$yaA?hgIv@*|VBgWVt9kK(q+8GvEn)8*@SLU>6T@x!7r zjGGM{B%-w4Z70IGhDXf)I7Q$s69cXTRGMyJv+7iN@W>ogwr7BRdc&>J;~zlN+LkN8 z?L)0W`$*htAd!o8@@q!w$a(!rQ!W^K$VPiW@DOZ~*-~iCj+&kQz7&lq*Wu+pLA&QT z+RW;gwv`GVXuS(fn~Vt=fXfAw6%9vZ&iIdYuWAyKqFrq5o&skSM!TC_SZ1w9)*GF% zd5JcTj-fE=HMXxpT?J>YWf^lg;(B1Eq;NXR*JOy&*xQ{$VY_!FJ^`sxYAx0ND=3Lg z9$`_Z2LgPgV56spkNarh!>|65XqqNGIvg3NBcl>L|I;MVj%oeOZxYw8$nUygq}o{X z9<^q#R*G}i9HlpEKY1|GgbYQF%z+b*Mmp+1WA?*`%brmt}{T8_G4{R4rj zuP-*b02R$wHv-Wx%c2nr6lr3+j}w8@jvo2y3&iWKu_z3FZ9KNyKuFujm|NkC)<8AEi)iqSs)V7Hl%=X3xyVX@Qj%j*z^a z1T$2Bc%D16383&cJMnU{aSz@U3d_(#+&E}xmhooqsd(aYr8EDv{qe^{+e)}CtkBKL zmNGqhgfWvEGWcxw4b_Vy^-z>6baD3&$5YWfX=OJmtoHsI!b(tzr=cNzbc^a4V{2O* z{$CnXCy9364l8npFLjKD1=)tUzt`^LPY4BI=*69#_>?`}n_2FJ-Uho6Ty}K=n?oRA zl9t}Cm=f1Xi9iWc&}M9KnbLTIRwpG@2T~r+<;JmK4)0I5?R7ldB`K!$?j2o0O7JsA zil2(4_Ki<9tr=v$Z@hAkOh7U$wi2xIYzQ@`K!h7lnj&|KJil8#(Kkic1;?5f4*Vul zeF+~HYgd1Rl6-E-DV@wD=V>s4@}`dqry?X_K}`Gy8pt#^*e($?dwAGo$2P0m3d6b3 zp~FOfUtyCXTHnH1f>*>9PrZlfR)j8jM^0Y}nM>#?^R(9b}?7&?kr%6ery$D#?1>6r;2rl^vq18M@N5fEA$0G~A@_;_b+MffD6H=qUT2Y$g>iia+}hj~P=*Gf|2#ZS-P0MmbD zZg&*cU57Bx=k8v76Ky|#`(9E4uok&?nic*RE0PXmK#1oGhMNNm$#nOwU|!j{dU6C% zrz{-EGlg~^@)UAei~Nf~1$$L8(6jK*CAlLD+d4wI=4H~hbO`70O%!0A>&_IILbspa ze21bl++%7GNP5H(Z=4ejmeXVF39JN=&Px#QsC1v5Ed5>jaUlLP2ku{u-C$!S+Crv% zediETG?1Er|6Ya&!aFi7`PA|-u<$E3B4*_y=J}hAc3`dfvny(Cd-7O!#=qEC?~VeA zFJjLeBvyCz#jB2%Iy-Dc?xo~LO1KHY7yJ#-BVgA3$KirUPUM>I?q85eU`d9>H&*U>n!duz-wQ^rdxIZ%Z(BTeA&B zNy+tnfwT&0%YOCz|3jM%4AseIT}$bZYLj<$9@}$EW4O z@{sG&EBE^g&*8rV40I!6&1xPr`^r-1ew8a;7b$p5U5QJDWmvhjI9?8{O#So_Vz9|s zQR!d~XX@Ed8Y2z&ZFb_!JX-LDB0ouKeZ<2!=4N3nA7GsE716xKz|zAli>pg$VRS5F z{>;!*E~hgw^UEa`SOb4I9Dh*~7;dBdM?;USl#GO8ATY{z`WE6IR0HG7iJ@DjL$tBX z<6}o3_DG|-td}<*`!_d)*x)L*OJ7F8}NSl5VaCWP&i@cV?oI^I-VNNWdIWA44(igyRFfA8!; zYtU=Cm62h~LPL|a?EpA_nl+LZIe1X8t7RJ!BLKaqPaXzh^VnugXVKgIc(z`Cr3m`? zHBhXPa{1jy!9OuBgM_l9U^?aeiaT!NOpS4)_x*CRV2#Iug#u1_k!WvvA)NrVH)Ysw zXS#sNf-Om;4PNK?Sa&Rut!Y{cQEOwasSB)X2QoVW{N6kUBa{Ohzh)`ced*On-<>4^ML zB|Rv1R(`8CK=>h(=mgHkyT2Ly?Rg<68igv8HA>sDaC^!I&qr$+@uzfG!#hak$sylz zmjTuLzFPw++l-?K?yEU6(>S&Belzc|_F|V6^#09CyM0?Le12RM`)^;PmjjH&N&lj; z&tgRO<1ZKB`ixb4@uvmf4t|fxfeDxz?K0O&Sc$5eYWr^*=p^i%Dnlz*W;;GVUyB|9 z%(trj(dWKbt9RXoUd~BdUeb~@Lu==)msf-Y>Fxaq7pE@3G_Hxr5OC#N_Mz!?1rS?j zq(k?7p~FV3SiF%2wpy0A+DxQFN3$rpB`YCpzuI`olcCM+`qdjEULFKVdKvIQ1uK!e zc)i0)rD21tmjTMP((i|*4cb>i!8gGjG0~Uc-f@kL(eHNavy@k-{Ev2fcVtL}?8BgnP}S)syImGZE2k zsoK+AQwjQXVf`V`pmT#9bW!y|NOU0IPn2{Gkb*5-khqQ^HWQm7Dp`IKDuN>_*NpF* z94F8Q^^c|$L{PCmy(phXQnchr%S~4vt`<&CZKRzI&9g0{TGL+>9nkD&YqgCI0U@y*EToK~I=`^-%e2OG)`H2BDP)Efg%Umr78dqBiUxhN zd;H_iyXW^5QstW+QC@Wnx_s9KBYMm(ss$WM*gqh@+njL!bx-+WcNs82r=B5|B>wR! z7^;({-}`27yo3@8+`Gd1F-Wl?fMVndOB8txGnP)`8{`}tK_9Sw^YM&eqMnT}tXKhd zCamzRGB*>|cWD^um-qQbf%};gppBfmoAsE0(Gg(d_O&2~ z-vPF$-;Ii`G}_A*HjI^blG5XF^_2{|LAw{^@L~?qgrV#&Q)W1pp;c%o<1h|taM3v+ zo>os(+aYCj69#JghF%|2Caf}2ksF->7PeLd$C zFadslL0FM2Ili`se~g&)V;&p1tdyzH!yD4YD5~0O#?~|SfqjIq?5b4z$u%{9<~`w^ z%&~$^9T}~nL&CO%9fdpn-IcFCvksd%rrjYzLu$Nlyyr{Wi35FK;fu47(+yu*riff5 zVFJa{D>H~db)n9O>}`oMoW+w`9h9km&zy1Ox!OP|vE8Gq1=u+3i7cO4s9Rp-*JrZc zfOEZ-l6-g87dKx;?PKNI<`Hj1?3R--PcoIJCb`VFgbV0$tb>m3er zQ|I|#kBd9j{!BKNK69HqyTPb&7YXJ`JC;=l`C%-j>W5K9h1TOom(*y?bYH$P5<>ZW{kH zPfq7Z671)xr(+IZq+2$Y({cHG(K61Yq2>`tQI~3L;EF*t^DP7B!@j+_vC!)T8XY4Eo8{+CNNm8z=JaYF{K!dy$}=H*JR zc(Cy+%Qdj}Xs>vx4%WvAj?D(*0$C4&%?cCI9^SxZIkO*qwJZ{;p>As@{;N$jGzGe0BP^ zBMqlb#Y^2JpizU7fZ33%_*?wd%;k5cd+=Q8Lrk&}(GCAk_Q-k@#KlBZ$rJfG`m7oC z#A;%KFJDNS=FEKDo^^aDjPfumqpg?X#b8$^8$7U5pV_0EcD^;QD`~7^@gr8uny4nM z?!2&j1gY(Ck zrxea#4itzmM0n#T)Y6lyD7%Fk`jne&pYSRjn!Ych{$klHM%@geF=a&{|0I%$w3&+n z;l`?@+4!voQ$8QX|D@Z&Ui_a7zfeh+B;n`j6&GK$Ax1siW`bOQCK8oo^4$Ye>!>}B zOy|&XFgpuC^OHd{%HP~J6FEOY`jxV9{k0SWdT7jJHNn}w1dy~U*HRt*HK*_8d-A1yR4bT4h*V*|MzigrihAChDQ?S8zb;-wCDfy%qrFPDrg)VHLhs3dgC zxJ@fCm#nF^RUX(vI}VqrAtkwED+Z;ix)fEa`q$k2n|wZH{{EC1R?8^0SFYN+4-cZC zC@A~cwGxMc_yW0G*?PyDHw7~EZ`a8Ghcw7{vGZSrGpG?n zB39OF#5d%imfDlMhl z-``(&7@L?ln9dh}y&AwB&Em;AJUmP**w=s-=Wsfc6fz`HuJ{8;Wck-wa)Vdz_5tQ) z<)7LEvxE~>P~7W7)7JlZf0Ib2^ASs+(6O|PoHFkHP4~GUdPc^cnVGn1y@4954S<&0 zS^bGw(x+$;sOy@l>IP493*^6>6+rlp4oc?`QBZ8SSDR3+-S3x-+;7HMn_61HWMpJ` zb#(AufjE9$-ddHyl?(DJD&M^G@(62ze!hbUHE#NKA1rm>J1!upH(i*;?83tQ0kh9Gu$OQ3OYeX^%{0w{>wc9rT1G&Lrnn zmi75J!+Qh<1Q6t3k+2;?CwtUY>8%jCOVo2-zs$K2knP9YId=6m|1>0C7Y)k?GSZAk zE+S{akx=vOS}P!L@xT^z^IV>@7QZoB!CtkNj~A31{rIW*+7kHD`Ma@$LJ#8>e4v6G zt-;?%3jT$HGXCNLTZcZzfBe|nN8(nV7SG)?^(Sya8 zE=>R9z&_-bJH_<$FdajEJaPQRjE;-|V88Cc?<@n62h$J(Hi zx>L3&4$lu1cJivb6G4bg)>~kqx!F_3beItm>1;6z+Flv2ukPJFJ))wbU9+>uNJvN$ zl9H7lhxQy25)vZf;)GOG;cabPTAoh^>z*$b;X*riox9xu(D@28{1yEJKd?OlcYoIV zzZ@354Nsh|x5iy$RfeK=CX03-8RzQJ#Hvo~;`ju3NoN+T3>xgLHOP>+VM^QEGZpI1 z&@DH~H@@Atx7f3M;S$^8FMyFT=e${pTsvKR$0-+g?W=S+NCD?uY&8Pk!9w+5DrQe& zL&BDncvXb@xL;5b9~<{8%4z1`0u|N4sZ=Db$avK#Ie;|d_R$XeZ7g`@va(d^*= z9+ko;wHWn>*nry|m<3`$z34(!5kW$a4)gf(!szjAA1*|gM5|q((-YX#)I{g`>Rh5w z%>SuAc6Rp9FtoX|^XoHw{m0*Kk7jy%dpFO{#^TB47u!8JTb(Z2aoXdd^bozI4*RmA z)TGAY)-Rg#m`oMGNM=o@q8EX+Lro_Etuf-1q*FURd556ocH#x@tV#IoV;DDL;`0*g zKC6sH`nO$n7>gF?Ja%*z2C1w=dLQv&t7kCEH+Dm;)ddaOigoHzx|3|Og;Ih{F3r0s zsxk$YgyM0!!Y^B2ieg)XzL#!aENA7$>!0ln)sfa_41d+fHaK zM0fSoVz6@S@2{oc)yA1Gx@7tam;HwfE^Aoqf?VJqmnQeh;~@7<)rg;t>*b3jHD`3Y z?U!dc9%D>Z;R6}uxcD{N`VV)wo`=@uMJB2T-WN@E<*%Q(t%-`7S}qxsi;K(d z!2uE$)`(dpP^i&z4V}l6%c}i}q_X3kW3AQcAVFCRpM(TLQE_N|Tu2uA({)@<${Q<^ zJS^7YgdI<2vC3)n76OrwszZu3TdwXaIoWVr<2}sw=?hYjFzsj4>N2&3opS`mBj`)N zy_dljG=3t9R_uLL6jLt4{46+Zz2_*57}IuOrG&ZPJ{-t!oH%Fd=1is@pNrwm^_f|4TP54({FX1Jok1%% z*a(r4`nft_YOAzeBrfR6w>{q+Jt&47e;G4xQ_kZ; z;olA_c|KLa&t1@i3+$lsdiUYqXEwhO$J2AiRykW&szVCH{#J`?VXoU12(&vdSgUKQ zcMo76X|&tbM@PtkT)KZj3x~3%czy(K{mhu9iB@jc=t{&mvrMzpQ?!DiloCKhdHq7@ z9!6oM3Z%VwYs&mW8d+`hBz#-!YQ~k;+M8PUX)VuU<^5*qmrGJ^}04B z(Cc2}D8}L7RrW$$A;=2`Jn^`$kPBA!)zInY!d;4*g#{f;N2sGg)KwGPEnqqp6tCL1 zk7I~c9`V$kJkrCNYYZRIHg-xv5_1>g&S@W94A`47_)B{&^)7t*DSK|W?&m9c=Hvc) zBLtW6&2V7CivIU)qko@(&GAWvN@COd17{?J+egmmqY1W~uk;T`UEDO!*XV=Sv#HfR zkKAlh^E)-&c+A_A4ewZK{kD|csa#o!>V{^6Yt?fT$Jb{{w{bKuOfHUa9bKyptoQiI ztraJRyk!NlcX!3AYpCo2yF7GG6Q~_+4F1CB3NN51hp?%GbLv(eDV@nQo6xO=putK9 z!0XhkA#dZRlEHIYzaPluPxFrQk@}unoo^ckfWdl$Kq8gazYX@RD>9eoDwhocTB$#G z&?T~xNE^PE_8d<2et||#SC8DyZ$P0hUrn%)f0iAYjWO&gyp-_6nqQfAojIysI8xht z_aZVWwbxs%FJ@iFyiKL^Rbs}PvARVrkV+eVOoE(hcWsW9Y@AihHsQoYbUNL3_ni7= zSaJ8w)e*T;GQGR^s*wOk~wQ1MUyyLuj3Dy+>}7lb)&fJjF5 zO3}O<)845>Siuy`x`?1OblI}`AE}LVW6%hBCyIM*!Lcj+Sr0v2=1dQGNVL-28f2$* z2+cHCJLm{X7SOD%n`y1%Hcya^oLZYJM;C%5`NxT`#0?Q6vRmLzbVbMo+=~z5m=T{b zrC2r(;%vFT6({vG{Y^<(x-(%fDbdQZ%$qr?xNti=Sk@EDi$SGUo2=65nT8cR7gF~r zny%momd5v^nYc{a%3_jxMsw5ai?;(d1U*ykLtgb87N1uG@}2 z4A~%|P-5>Ppyy|?ATF{kwhrEaHUdy@ag0NOPz;+n~bEQPVVJ6q+d^9|3|Jrsy zO{Fk>b29|F%*<&!q0(zJ4=$3-yFZVoHmPkQ+Yn_iimf;1E2DdBxX$}FtMMzZ8GtOA z6?a^mxt1L(E23H51H!${x4zbGCpMF}WSjS{q4IJlxsVRBEGu;|)~bJUiZO1}0gKJP zFMx%xW3zC=&AMm$C)qr9%DXC$1|y=V;@OoYi+|L{xsRO|T^Nsw&>x)1pqktA%0(O1 zzUq_K(X7?D=RJA$b~{MnSgK14*4POECa+*c)yau!Srnr;ea0JfLG7jWy@if~=<^xo z49QLkcMT?Z$k48O>&}K12QF-dZBn{Q7a$vhM))4lrZP#%D>XVU#n@~g4EU1foGZ!EgEA1=#$NIyxs__sso?o|?%>K| zjmLQULl{Zo(&UDVF=<@KJ1FK77}TUYE0?3CslkMqelpuH2_0HY>)FmxRV1p3OKoaN z=N7fcE#O75$YLVJUCS?p!!=y)hfcj5xwB>|^(S3Y=wMI!q54MU()3L3J0Fg}?!3QE zEy>JC2bj|E+k~4%!olN4)XdR|HUi>=?OZwOMceAB&##adY~h{hIE)R6T&FLbG`Q=d z{eDSRcNaZ%;EYEXiH-{oAzPz5aHM|fydZNp*CRi6_UQPx>t~Pe5PM!A5H=6|)Y;{c zp+v;lCi=o-J%aLV%=tG!K^KWn_o6P6E)TjSD8ON}Jx~}seM_*ja!RE^3t3)AWAm}t zEtZ1YftY%g-T~4^nMY~~uW@OQ4nq&G)ei)4)LHo(@{z_eRaYlsS^Y}sE7#CrXm8z- zz1LrFO@)~mWzsWql_xl?ZK-*v)kZ_#jugyS!ccLbhFZ}s#~QvujaL}V4~%?s>n&z& zWL*v+Idd0XY6*6^L9JyU4-8yemV0XtV7I zx+A3A<>wHOBN0AL(&-sCd$T(;LKm0+S$!j+}O0Xmj z>wTp$Us6`IN%n)1L|YDh>>6zr6WtN|y+*wV$f~az^S7aKiiCpGJTlvA;aO%?^T>}2 z=F8Rf3=9N6tF+KuI+5UFmbHue01a4(9!aMLdd!H?bao^`Su!fB-qF$U&Q4x$ zMC#pVV;KLjGwC!-kHdKvL^JsaM$o)hdif7nsBwu;C+ALhk@}9US-)@4{q}e8P_#uz~gx^gkkdYgcd(j|& z>y|(8>s_`{Qj)4`mv^q>_qR4&K5n;Zk7d*v1!_2eqRk}7q>eLBTfMRhx62P&$;|H3 ziMHWbyUj&IWIxl7yJgDWT6BC>*6$dVPWy2RrIN6)F#CFFfH#?Mv9-WS!sbjRMVz}&`APp~6gcZh$ z9ER+sjsKD-MM7*S*LxbEow|5Z{!BSr%i4{3^%OC(LRsCzP0Z;c-c$K3z>EP7FWLhQ zB|SyW7c)ywh$n@>-taeN*(uqFl&!idzumjI!{ugNZcgrzRe4s8HdBqGdpYekhbRAW zY>7r6Lgxx2X%b5jpX)Ja8*2|3d^h&k{M=HlEDlt|+3V6s66yejIUlupO5J_=B(;5+ zORxKL&azNSkIhG?eJx&BNEGlG5=Vr-b+-#yJqad7Nkt?{7VcJ2$GA6>RC-Ay=0wTU z-bHZ*^I-5Fpy1@w2+`v}#+#Qfd@?b)1*e%4H%E&@3xFXEgfijw3>t&o7HTI*(%`7sRVp^xw4>eLGvnwtbsovN!Bf!1_;mT`6Q^Jm8rt zMqR~PTi_sL7M`>6w5LEtepF19zA7HVbpS8$}NkbO28RlbfdX3YF(_yQtYwwvSM$T}F>g6^Ylx`IeJ;tx0_4muC zd~R)*6i;4k#ox;yc(`^EW0;7xv&O&T^vr#fW~+Sd00@UTkea1(pG6k%O&2PYg_}CF zK;z=#W^-A`W@SyBueE}ffD4=(em0}@4GeUwtfC4`686>_e-bmx&ak^3_j8^3I)Cb~ zYA&zCO}avvJ!09uQyD(6D&tg4k#}V0u>eRU<>G6|oItuXRi1=ZI4>~oW|?lTC;l{e z%?}kyZ89^ClwpWx{{j~q4g!Jb>-SxKHe(%dc4?*W*W7`{Q6A6OflVs?CH||D)WzWN z_3$QF{rL-PzaXL}*rDy<@LaYD3mQR1ZnkW^U4$;|C=rJ0LM#iwP^1QYCn<=bU5(uG z(N!JaAn`+-GJI2C;0f|!+TS@A#khj8cq=gPNB#S+_R;jtCRs0|J@NI&=lV`j$W)>Q zYA}kqoV8KBeBCJe!fzvrpb7pXqUiMvgRV&Sh^s6LeO%t_7I|#1LKb|Y31(XdBKj=Y z%gxgT7L>Pwq-@VDQGzOV-(qznFyH9Jm0Wht4*Y_EFrgClpsHddt;%Iv3b6>$ zRkS=Zn0Fq&g~S-ew+vHLM|)4m`e@5e_P+|X4lWbLRt2{HasjyY$#`v;)Ek3t%!4|w zuWvo;%q#-+xke(-rjR!l2Qn@2kzsKC%vM;MkUzR(st|>4EyV}sJ<*p{1&uNgk5KW+yf1bh4W7B>ALnvqi!`wLJ zbVv*-&%e7MEhqAy)5gVbN1t?RcI$Y!BdN?l;$RN)OM8gR2g^@dYA5tX<8Uf|kvg&V zC=U9?mdKomKwWs|C%o8Uz}VrBWtz!RiW2Rl$rpEKU6q7i>7&XP?n0E# zF4CRihnJ?aFbn+=xpoA2MbNO{sM=RJUu#S%vv#Q8b+xs9@__c1+lH$kg`2ED^bLy_ z5*QxCi%Uf5zrXW79sl9R9f?z`G7D785y@YuidWbs2@PWhj|hY*_W$`wrq^H_(NGO zzi7ligI2e@$%Sj}h{5K&a&S}xKH%+M*3lR5o9|@@FGD@QFoGDyVQO7xZynZH7Ps(C zNo9;eXxVABrDh^u#DS|bMxvIywar!M=sFg3RE3Cvnb}e)|5d<;%%UYIK>Ch|=bZ@r zyKs{ydw9(YH^Uv^u@Wv`#i^!XNdtaUD4Hl&C&h^p48zgU5#;dB_4?+9GU>gXT14*3 z>9aAfrG+CUC1so%2n2f6U*n=n{wgoir53%yREYFn7l@+rDM7bt?>BCG`6TJn++K4} zR50c{xy@m#)|o^#!p*jg^cGOCDa@C+dN6~C3MJaFqNDrgt_DT*FQ1(itz9vd?49Kh z)!Gd)z$vIN3*+xdm>96D%`uq4wxw+H96++P9ZuZkQ9hbK!MA12XR>;#Ni4YHj;<}2 zA6L#%RJOYDFqkZac~c5McBC@iDluqr2s`^kdS(P69qI*PdL&+MAH(Y;uA$X#wF+B{ zFO4LP0QO4l@?kVueA(9N2vkpg|AmjjrP;7l<*;CN-v%eiDCCXOZpwSfmWTV|NY z#l=TqNPz_iEQ*h(2cH0a4WA3pTQY!9${L{z`(d=JcM@2+QQ z9txEgju9$7@Dfgl{$|-D`9`(wfC>A`b)ljR)_p8zXTg zf5K7Op=i;K?JCNA4Kb1u>^NPJ`Z?y?Kj(+){rb-Kj;Ys7`5hc9r;C=6AgI7eGty&t zwyXyYLn(wpJ%{4N@Ty?cY6(?1!IzVNEu5LY^iAAR9oo_6ns5Q$fj;$R-E%0wOFxhE zrjkRYU!7;TDn<#k30W4X089no!}@M9S;)}cfS%e&D~MD?MUn^e54o6pswA2@ zC8fiu77Gg4P+>KIIlGYEVu?AThr>}DIXVA@GRvx8BRn1u#LrM_@{e~nAxr}<}Kgqa$>yRPMnWm|g~+sPf^a|TYJ z{L*+Us|)-~^Y!*(W@XOd@d5gVdOp2PlbjKyR=N?@D4d_FTw zjxswgPMKkv{=kv*D953%_o9zOTo+2Y)0HPwtadhBkEOUMZ*9Q>>t2KP*wRaNS49_^m~A?0+;gHJyRN4CVwSsTqZ4twyAL_J zCGsTk$ItuRS!B0OiClP~Kx9?Mn^LxdiglWeSr8`e&T5h8`%%DA@sN6H`W}g z8iY>Jp?w#o?~l}WD9XZR0rnw!3O(EQz#XSh3BC)~ro?drraAKAv`?4>x;?W>1^Zi( zc_%63`L>zcDRWqX2@!W^_ouNP$3duu8Sl}{dGCUR$OMk}rhd*5l$&$R696Q1G9-i+ zZd&4$TSCY^m6Vid9m0}QQ_+x-1(lVR>&U%n?c9g4p@2}fiaGk5 zL*)4)NCZ6#xyGBjrNFc2yxq3iZur}namxht01PdA_y(iTR*ZqDMn}lRU?mFMt09L+ z8nNnp2q8gvl))T`ye#gB8yw7fU7!N?e@5pGGg>65EJ(P+l%26(cwvIdOR*%{so(6kL^w=~Hk1PmhTy5$%qFO!#Me?}Pdgf-Vl;x5){-yID*x=}{FeVl ziBBFSBf}j1t}f`0Fa|*Uk@p2xDfG3s&mw;=e>H2ZDISvje{8aIk2mO~$)Y?;@vKF9 zeO^$k+R12YsIdR-=95uDo_SJ%->SLebVg;V{$a@)P*76r2nKtkAR?DaR<7HL^ z4gAie#7qN`0dah!HTk_DyT*vuBx~=;7^^*59+zPnyaLckY4`|28n7QrwWj>ZJf?hk zsyrNRhj6o0rEnOAOb3w!U`+UWifHC0&V{B4Y&M6&(J);(RtDV>v;`U^!rNu*8?gr0 zpR7(2v}xsQv$I_h>nx*>+;e@3bYA0%>EE=Sw_7VB5##^KuZ>lA`(&`?qb1#w1cv15 zP%kfwMj4#SmiDQv$FNn*tG;R6<$jwO3azf`-)6IB@&kKHC=rn1Og zu86$>hYZJ_KOQpsWo}u$z_h*phK}*)z=jq3t2O={30QJ~0h-lP>$!P)yfn(;$dC${ z%hd6%*o*QvuM$`4H0)j0rlWJ5E#Oo|0)a>N*3b)kwIRwE3lyCS#a!h}J&32ow3Cm9 zI~_aOEgsW$1O?a2asTC5?CN*!@gO~gz%{;CA(HV5dvNZgx`b>qORcREOT#Cn5ET69 zgV|*a6+ThN`rq;5FdE-6e~_0#`4-a>Ji-_h9`AhjBwTQx?vov_tC_bMH_pe1*3FvC z(?;gM=$A-LnX)rON*Ript=cQ)WW7sZW7gR+gW6=2vG7})L~>AnFBxgI3cbuQzF~v~lseD2yoO2E9F+1-qR3qn%H28`R+W_FtnFl;SaZ?2CIkx=*SF&v*W2Mx$K^+ z?zBbEsuO&iaNdi$@V(LoH8kz%SkPGDn$SEPcZyx&PBjKVhCpLtXq~7dfguUK0J+_T zJk*4D{L$}{py;5Z%B)YCZrvezsz7@mZC`5BUz^UuhNAK{B^aLSqBNQFNSh;Vq!T2| z*v!sIW-#2(`*2U0M*n_y3m&thA-;xM>^r;dubW*Ay6?E|M`f+dW98^)1=Kn}XfF4V80q9E;c4oS*^!5)ozVqr%bit@R@}BsL-=9`^ zjz{LB%AS}_etwBb=*@_?gq5Z3syl*h=HzUnFBQ~YDsMNMCv)!Af>?Zndb}z7-RjG#YBcp&Mbk`h8~sBJFg*+ zmMgm}7vH?tG9L$uacc>hvn!-L`FXLEmo?-Y31DHYK_1o!rOZ^SJyc!6B$CetSj~i>+ho(_lkns8vBU1S}gg~JKVf)$J+>&_~H`x zqdjd>ODNut)Nuym?`4wNAs&!piKV(!z-SMzj*$mN={97S*5#|MDyuBbum zRMHE>0rV&v0%->|rmWS0e1k3X7naH-S@t*QCbvDDIok{Uu9wBhVFw0cf5yJxOE4)q zn-it_NcU&=15EgC0Mp{NBJEZFvLpvCjhyHxO_WIv&;me#ix1=c?1bxOhJN{PI^AYB zY&T{B41CEmnz1?3xQ1@ z5485k@@~uP>F@E-vG?1bjf~e_yJuns*EkYuWo)(jD*8+hj~3!ydg4dd6DZYZLDa3D znA)8UwYbdhxb<-)#bc}RK@qBvKEA2CILVspX2~k0J_$ZyA%hA{>Ep z(poPZ!L3mmF*~!r)THI*@D~FM%oi3GkWo;CYD`AC+^?yGgoH#zMXU6>KK%Lfhl+ur zpscKnMZ&sE=kJGbQ`+*Tr=&Rf!7?w z=QL2~;4_evl$wK)IK$I*lNpj%7@aI?9xMcq#`wpI>|)G19vv2mdfJo***nF+Kj_al zJvyPrKx*qW6CV&88((_C#VAj(eNj(n@_zX`oX99118_cD8_kX6mnxJ?U+C}=0FP%D z+OD^Ejux|Nw>aV{9(e6DjN*0{BUhC>sJ*S?1 zXYpalr5xMUt&HqT)p%P`xApwp`7vU@nY-klBHP+P*fD^hXY1xC>G;XnK~<*tcU_Sn{0Pak6%&} zb^@KcBKM`J%{l0u51~aL1ty<<*Pl<*IQRR@o_UNUPnXnlt<`?{nx||sn#g{S6PJjW z7wi1_MTxW-;l$Je;f;-42smO$`@fXsCsI8gHlr!nDSg;=3>Nd4>4L4ya+VHfp4qmx zw}{&|kUR$x%vYx3t^``UAKC4khHJ?FP8!HQS?yt1zRhi-Rl_|d23f?wz`%HFE`opu zg~;(^OnD9V^JUw0eHcHVBqSI_;<>k~23??^+v)9Lqu^(X&v!)MdAVXUa zUN*o;b^n4S_t&ZWUw?iUueA1Yyb_R(8R-n`&+J!2*{+8GBq0%Fxk55NzI2H2R}bLK z>1+`gDe2)v_7d1QYy{}N@l1Ic8HD~wBA6lZWU3HF8~_fxJKez|_s;I_3o{JWzXXRB zCXdd|Lo6$!vvacRT3id+IMIAxLatvfu?DXUEl8uIlznJf3N6Xv05358n{);H^ zKltNUYlSk6H8Iift5v zlKPjDA^+GQ|3dwz&LMC8@IixHXzlqs`M&I>6DMlpyr@8+A;R_r0pWj*`{y_epEpi442LF3BVE=p} z{q`T-{(ruF|7{=sIe!1{mY#}Hk*|sU^uRdji!c%l)L-aR1WK{ zVa5)Kpan}aX(4)pK?-P%8JY6Vw{-$L*q{DwUsRN;ZN>g|@fZ*V3f>S07nt+DTu=M? z`JG=5GfwA-ea2~T455CqlLaUdIcNEdRbu)~26^bqpy z9VI1Y0)zGtmaeCr=G6TDe$m(R97Fy~B~pXjrbY+;lhbSaChnR!ab!HNI2VtUnFs6} zRk_B-<_$f!+;64Kbdf<699yiwQEbOL6^pT+Cu8^4h&s;Bt8kXkBVQVzvXBaFk?X;@)NdlS2XW!V#Nv-<}l$2LQ zwGPSU;#TIGZh}KP{awz0M1l3rb~MvQWQ?)osPr<;OWtk`jg|GUEA}IGFHgT>toE<& zUbMK*G5Jon>uv%Gr!)VQdua#u8(MM^-F}R8W%Kk0Nrgk6;`Ws@TwgITJ8o${>v{$R zio5hrKUS_c#>6v3|-6p3~g%YpR_ z4fU1O73n~xlj6khjT_u1vWJn8C%SL=e<$l7Ut;!T`BF8~gR1+!@tmt#4?9l&ZQY)+ zYW~ay;~gO4zRcl0G^5)F3oH6rXTeXzD}GILPUB}?8=BC2KLwbH^63L5fU5>S5Qz8F91QaAv~|Wy?szvSYBS9Uy}p)axc|32V&Em z0B-IF{@EEp8$~*1=EOm&sl&$bjQXp&%jIa_XQpL5f}bhlv(n5xJq0nk^9HBC+%~uh zT5txGE#3VCNH~KbI{^mroh%K`+?S2TXUkDVLJMyq!r!kS0I4XRo*SGI$3BU#1x_mx z^I-?NL>mK};Ui~S-j-7#Q%(42B36nF0WjV+PPK&0zN+>d0h(`H%<chChz7t;A*5=t zv8og2?r(6q(G1~QQ=~W8bP<~9*$H@bD)XId`ncw{NZaNf9vB(;mGw_@72{~mDk1lo)jS` zLxh(&^IU-!FXsYF$;vELVIA3gccl4E_Cu22RwOR#M`7s=(deH5rOqRo)hS?}8~whd z1gnQj)DCg??e%h501vdlE3!^+pqrZ|>fJ#JmWAREKPOEz9DyUdg8O45b(wDW*qxaa z#L*D0Huul{v9@v1(Q=owsAYQ**`a4bTjOs|xjODjXtmFDQ&&=F5AM?-Qts>J^jA`n zYz0kxMbK5gctgGG!~Nun1{r8cxBTL43irid7Amt<2Iq%U>X%c8X0#3Ouyi$17RSGX z(-^M$o8;(2uR7-%<@A{1j|aM;?ozWUQ&=}m?_Sq+mlTn%19>W>*k`AXnb*um!aj}x z#5dbzgVn`)on@6BjQ97;>#qU+{-0t13W|!t=H}FJaB!{Xe?Es1@})dnfv+BqDr+DQ zJAGj|-K}m{LE+(I(`@jcKSu-LATHE3*X{5j5bb5gN{wdW+WxZ}^Zkcvg%(%q9kVqp zpP#H?c?ZWQmW;E5u&|7XQ0J;wI3&k?vzupCVFALQ2jxn(7W81~E6gz08y;suo-SKF z0fW89#6?VuQU9CB$<~65f-49+V-*o#H2+A_{5$3XFVnL<5MQOQ`c1ESC23xNk2wfS z;3fEDHaG}DVDMLF;`tvz7aPRRs45%Q;09GFhwBx$F@p6`dFj2W@lo`s9~sTR2Npup z9$Y)atpUfBKGH_VVq=Urzp3?j^tq#}vEX#8qqucXcRd{erby-BHoJCLM4gdH7-2X8 zkGPH9tNl%7y9xJ{>*V37@F%!00>j0=srLq(Od4zV6dE=xzU){z!TdKQXfUk4XRx~gK^*y)HZzL~JH zU6Lq6w>~~MTYRD2ih&I5DQ|TA$(i2DrN05HX;6X?S5M?M`viYvvS<#>?-q7pTfRGy z%oK5=)!hNUEeIw&Tw*+%9p%4&pX+OKS?SbrzePfsr}4-j@x39v5605PeoKoAX0E+U zWZX-{$cha`H|$T&*iEIi01M);$cxR$jvrjt_|f19kmZelE8E2Trhs+^eSp0tQ?(*3 zg*;r=X)H=3%!TD(VW9)H{%Oh-n$+n_lDYbW_E>0-#5Vm-Urgp zSd1pGH8t3HGx!bwtw-Ko){m=9*4!b9I_WZaN|f&WuQbyiOLj)4+*qFdKZRYEy!b|p zS+X&z$h1w?89WcmPc%5GQ3%}8RKA?B?~SK-=?N^i$RsN>qZ^^RoRZ=U(VxLSQm|Qm z7)(T?YPq1LXUu4%`<%k=BKQFiy61@VLgfHvG?}DjyqKvKsx3KcF#fx_JMZ}8`m2nU zGFk(%w`6NOT7>m1vc_0b5-+$g&! zb@X6r-f>PR)0C;da`XqNOTBz)jMRbCY#MbIy;(xOG`Dr+K4)?Zp`{j_# zkoDpcsk61VgO#SMmoITMqVsXZM6IL9!vC(uxu2+ZTI^`=1a!Yegon5X%fj7T-=~nm?OI0rVYZ_ zM)c+b;^KYYf5J!luplJS?dWx({A~LfIGevplLS3#dZ5%Emv8jVVm7RE(Dve$OozQEm>l2zy60vF zCC7%R4xX7(WKh9a3&^N{S6O<$v-*jb-3j|757zm)HF~3Jgocbk2G(R_8y+x^udr}!84SBB&@`Xn;Ppg zY2-Y9Kn<~u1}N#`muz*M#aYYOO7RDa8%-H_N8dOu`_!=QlP10+g191U0niZ_zgBw? z+8SUG>d3uP#@p8_W8VQ~@*cU;#W%)RALCBG=z;7)2 z+?;SWf4mbThD8y67UB!F$4{2TKa@Iku3h%_3d2Dl*;_r{Epxi(7v|>_s33D}A^Du) zRHjuO@saxW8e0HXmo+M_h1pw+2Nr3sAA@!gJ0@_s^WxfuA^Ku@o^IVG9@MCp;CYS~ zeaCgPckAm+VHPEz?x9v?wJ7CGsWI(#E%F-|RemyHYPd7{EHZM9I z(a2Vdg06dUkM3aE)b2pwEIJpV*9+p9Ws?FJ-Y`{AU@A5GU~dZ8ibTIa0+}2f0pWo1 zbkUK>O1tthB-ucjzT=9sCKig5wXWPyvWF)yYKsMfdGV+n&I>Asll-6imWNE(xbHAa z8@EOdY#6ACI@y2<@9sR~rq<-gyR*prd~(PmnPkj&kP#d?Ir(hDQ@O-W36)e`yfq}{ z6XJXnjsZaOTOqO8>!W$3EV*E^LM3b_chX#OI@0@;5UcfHav7-ef9~w5>YuZ> z+-JOhi`b@9C?wh&IcCHru{Og^-H5dEd9$GbBaOz!sR!z`=qmVH3>_|E6A!s9G)9NX zGdeexbL zlm#O@>)GTJR#1j5GiTAtP1gjhff2?l15IU;s!@)(;5WYH?Cn10JzEc|2}iC+*X<$) zqk-+%iy7k@>kEVo^=F!KIiG~H?Jo09e3RNXw^Q<%?^pMF0kYWrRfVdtc@p29VLRnz zcNQX;rj!LGH+$-Rdv>I`H23UW`#JdCym2E(y%H#~UM0dv_4jst1GNN?Z9QPeCx04j zA=;7wHs{{yLjp8Xsk+Nu&m`U0oZe8=0--%r$(vXc0X&7jK@>%aSf+@X+qYGDz_kn) zx92({F6`7$x6q2Pq+4{xj)u6i27JKI{F`C(8+~+XlC7KT0u)6!?%>Lo;G*V&Fz+HH zj=eV5q98vZ{|tQkAIy^PF;bKo?!GB{5Q4!$!IeT&h4+D+a|>H?4CZ|=_2(?A)c zFuMN z2;8w;WhReJ{4AKE7Yh#46jZ}4Br;iU|A9KK7%b4{tKC06xx7fM{xF>%E^}YmmDcDO ze_g78zf_%Br1<%Gw9K4f2G?-k*_#_fTFRXU4txjV1H=}jFKyFEf9F}q9hTk@K2~92 zjR{(42>!Ax7n}Xt>gZ>Ag4l}xaGptCEOLEpIG!_V@_f}^5m9p4gdQR=KRPy6YUkq0 zXy3Yux$>Y8GoI=8Isa8?PZjnY|q^k}kuSdidzL)wyTj*kB>}LEUL*woRZ<{oj;k_fb(XPb=Tk2;N zHH8VPp(bE8v`I`@nWZEi5i4Nv_rGv6 z@7f=XUE@ID)ZOMwRA{TN6Or@Ued8f)v9#uj&}e^Yf29cC2*CfU#RWwT`o}NXu4vxb zdap0}gA0qxq?T}69rQSnz{LZ--^23n4qEC_AY5DD!qdhP9oll|Bd4D#>Yu~PNMv1v z{g#!svq$&X`7yZklF0@;i*yG)WP}4#LVuDQWmixO z!Y{W-u7h5aInTWMh^KQ6ew@Vctbc}CleZkl6QD1!Y(v_!IZ?ftw}_u#Z6ivqJre7T zCwpE`XStS2sKw!q&yd|b+^l;>jTaS_Kt&=UtF}n;@85ZZm z!9Xv6ew(!v|Ng4L1gDa!Ho7ew0e|Xb>8m;G*}D6pE!K>7X2 zjqP-;-L?Klp~{;%0=6q8J;WNo%NgxU%TM0wkF=ISf4J?{nJlu6KTrGfCfArM(2@T@1NuLX^+VR~p6Q~yV6*yN2f6Qr+;eIWZ%DjQ!6`gEE8Sfm>0Y&oG; z{Z-@URLU8WDR6cjmWNY)#u567k#Q%zciPNYL4gwu_sQLW(l;et$a&W{9FhZ0c3}-A z!Qn`-O2nBiWn;u;Cy5n2s1!ZoWJI{!S0J0Z*vbfZm9Td2nv{#qAH;kI+%ni=vT)Zx z!WMMk(_4z!>|S0YzbuQ1P6(#p**bPSvR3<(cO?EhQ`&Ow6m~#qxxs%lYy3L+Ek)8@PGrrKeaP|l9?JR8QN1V@%hz;2MMXvKI+3XZ zn>jIXSm<`Rb?pZbi*%Kp!}cBmF^BEa54Ue z-0}LDPy17hHFH9(C0N1GXZFY`-cS~Rhn0rMq_Wq}S`ZaSGg&5fHH6O<6Awcu*9)N3 z7Kzn1TlRVPifJwyNMLfeH|yDU2>UR~vjM)REb90hH+)v#_p(EP6%Jh%6QVA{QHWEA z)>CJH-`q>QQoz_XcyKG9W*-W-ka|Yqwo0lf|15L5TZ0|uH2BT*msmfR-Ax>I^F5h) z%bvr*GPtqerZ+o(`-P|dX@`*~rzgquofz#npTWccQj5+5=TXEQggZe*1u)9zWeu(4 z<`}@O&&3%Xpv*Q#RK5d&d7Zw*GPFhHUG){4G zvEYkC+`uTs<;5PqGC^Rr_IkfZO#|D;K3fQU$V8MK(d&oGo;RYmCEYj0xPkC89@g~v zW`teeQM}}6{Xic}HKQVZ<=_mCwvUqh&0#AVna0LieXZkrSq4ni`NiUBRx*H@6*ABI zW8_a%J!QIM71Tt`t!J86D=cviiwutNmt9h_WcS0Fjd|G7bCslymzGfBZ%LzsltHs~ zIpX59c}aCvaJ0MbV{TgrZRWFXBm$q1$kd(tON)2g#L(*So!_d0xxENY7M_mOC(qF5 zQ*MnU<|*RR?xRL=Xt-vxlR(1oUvSyrccL3iVxP|F^YSor>Rn8ppGto;6>Xr+z=^{F z(3E5Vg7i66-#!gS^UHVi&E)u0S8EhInR6Do*pR2g@U`G@zXI*QfZOA?*L_BV^(&{^ z`f07Wd3}=_5yCi5@7=!|l$h9Uq<#^lnYi!61ii7Atxjg{KljyMxQHpR7p6 zX&4tO$wJ1$1#-liEw2%*P7bRq0YzAmw(4ZN!XlN<&A{PgZuqax1mJL7#hs7)iB1ZZ z>+}?_{dJ;YZW*^|37|GvDC)hnS_`ckYq4#2>zhg^0r*elt=K#symHgf(|3AQgCU;@ z9yx%jEZl^szBwGt<&~8JY6y^$aA&3c$lf6}H$;ulf(pv8g)O6c`?Yr81Jfu;TChjT z>@JSv1S*Bc$|JL}!~Z;g$h%F4y_DIsk|SI34VYe3Xpe4GBVj&fTa3KXcU%`n)ENNU zp!Rek765Ezx}p;0%w*(&NJNsi6=(qTa;#hEY zgQjd@Qv7`vS12&BBBgrS&4zOWJ(QhEMd#dCKW@pDWVJEH?teMnfckZ~Z(o-6TL_3L z1zPXqK+H~xnqstXvQdo_;6?ZdbT6{&w7^+F>Z>bcOlb}~S~T8Y?Kd!v78AN{a z^;twA?@bJqya&8zGvnwdD81bcHlbVA*mO1oNM2rwUW%NxP0|*OrSZ5Rs;Drtvx~(5 z);zCdk55h@)p6pbVb#9GKsbVu5(v#=Wo701ax+h>)>lkbNdjP{)OdW_*4-_P*F~el zyUtJkvs=Cn;wHu8b!Ta#Ew9Z9`mz>_o4TP(m@O-V$l-Zjkdd`25f_-SwVDRg~#XfckoETy6I z@e&@B?n|4Kst%tyx}WF3F*u=yv+K7US>HsV8-sXYVI+MlR6y+u2%GC> zw&fXILRQW@I>6Nu?!$izrK>~raVSXZmQDVRC&}y?o5dN~yRW9q!xFQGA1*f_82(OX zbYYA(vVE*MR?=+r`G@QilF!rSjP{e1`hv}EzV!W<{m3W!>d7sy%*(Y?&z0v@;se0Z zdfK1^Ud+(b$;Si8Z1-4a7xn60Y03n>IMPs@wsRRAWr7=Gu9^t-C$X zo^fN_3YY%c#ugxb*Ali}=JkT}>POj0oCk4Ce6f_bLvBw6m(;Kz8rA*lr z(EETio-!IX*bV2{-dFvlbP-J#-c0NBYFCDIfny49AUM-2O+S1a7Oo??QB9KPkGz1t z;xYHe+hWVvIaSHcukH2Z)$28Cl58m9{Wd>L_h<%K#XM>Qv0h7|whqfPZ4~5ub;0O0 zkHj?+5<%j&JK5#q~nj;)AbAxtukw57e*+!$rYpO#(B%jo|_{47^BH zs2!F#sSHcgRuis-$5cI&G&ri zBBiQ!Ql;{l)sakYf0({L=7X*ho$?>TAyra>-h`y+TLYh^+JdEvIbTf=5nR#1IkP>_ zT=BKnQq4K4c`jA7Mmy}y7t>+@rS4zdup;!$m;)bQnzMo9#Ln_` za&6uPuX(n`gjN(zy?92N?x;+TGft}Q)lnUHpHLZ^ww%^K9LlDN3rg$Cq-Ibs?@?r$ ztpY2E_`_3t->EzDqb{?!^{>T3s5E;w_yMlO8vgp zSyubpVLEE}&(6j}u*c1_v+)c*53}WZdI-2_04e>1&u#Db`}cOe8lsbNn;Vg*Y|pWu0{^!Ho8Xmt^oE4N0B%;@#%m(gz{w1kBx zSDKX|;fc0>^9_VspA`ZQx!h5-Qd`Zc65$X^-}beEr_qK>RT5zc-NI;wnY3kz#odmRv9K zGF7npNV<{OP>ZB*^3L=QOmed>K1P0$d{>6Pek6qYZdEuNO7d@Y*$aWgj8O$eov~k5 z2)8sUeJYgZ`XAI=SZvK75`A$X#WD>05CXp7^+6(ET?5f%$y|1%(}nWl^GGO2BYA_mifXMjZx;fdqQmuEF_J|+@E-H zij~PEaJb|&ThnkgNvj*RJ{hTTi=3J&s;rFOOa0Jr5HKUL>|<;4x==DN_iDjC9;W~C z;rhDV+|ben6)QJ~c+nBY7Nk-#Q@`8uJgDjZ7d3 zsrIKnHD@xKQVkhQN9{t(`3fUQ47Y>^LO!THpX_Ey2nxQ#4E?03sR^p7Ihd~`^0+&t zr)4B2{%P^+=JcSL6Vm2C8tzF4DA~S}q!ggwuUBe|*FuVwS0Scv5G=WFC>Qdk$4-zoCDUoo2#D z08mmwJnW+FkB77=9M)0_3WFeE;z#n{-d^6j6^FktFrdfCwHJjZYrGaeVBTS;@#xI98@p_hBOhP?i|m@WJ?f)bLqT*|P_vh_fQA zETAw?k7alD_T`^lCVACdEx{*e$;m^N`i)HuQRL$dQApE-sAvz)*7i1}pOC_vB_-eK z5Ob=!$l*-o`a4Yz)0k;iNm=Pw;^2QaX_pu!C`q0yI3<|H2(kfqp*gYs7I17He$^GZiSXr5q?Wt*KAaFUlSQn%U ztfggWTiZuO#Cmj069cBDZSc|OJc+7*)`<6O{44>PY{1<2M7Fk$(wGkgR;{hA4_l$O zW-E;c7-!Q?10yp1U0uQZ`&0%7IfDZOS&a)2#24};zP+;(l$gi}?K{-*_JTS&Ioa4q zbMNW_sSHas{o|j;62v3(fgIz9gM-LyC^ZLGmN{-kMaBK2s{Ya3s1@W`GRVyX6&<}t zNnUn4I}n(2^tH}nHrr%Wwi1(vt*+1S|R z6`HObtp2O;^y@!X4TiC?{W&YgHUL&|Nr|Yqt^uS74V(E?+*;d{Vx0whnd%qqdAIYf zYRC+OtnXu!xq$;?W64{#c;fp`9>15oZ*%j^Lrdk?8CvXMt9;Kxsd)|AUd@H9d9b4M zE`^&)jpJ&)*3UY9Hc|Roy$Q3>zD*_R-(bMQsY*FM5zMOy=3>PpB)u~U`C&FRP@D}I zP3^rk!qSNJO6vogvbxy1tCu`nE?=GAGsS*Dx|)p?zdsub+Y`N-l%FPbM_fOXYm?Gc z-Wj(xmhRr1-cRGtRM$j%owTGU@oQ9KC zR5)2xsI{uylJcac{dFJl@@1a&BexF~PT-g{TnS6$xo(1f(_(tlN5D5lIMq@NF@=f0 zOR^w@^tSDI`I^X}omW-$lR`G-;;^*xW0lJp;ar7wI3#syX(^$?Q%Mo1lM!5^Qfk14 zTdh&97Uy%uJdw4IeOM^dWQ{c&pWS%7oojuDT2Kit##BkjKxWhtJCRPWrhU5P5!Xd+RqaG#P-4q7?PxAPnm;Qm# zSfp6UKEHH=F{b_5c&euC=NIUPfX2EMna+J^Y;V`F>#!jx{^xW2X85}+5iTF%der3+ zlDBK&TBGUpiq~|&uNb^MDtOFLe@LN1X1gt;61B>R&`?@`fB%JLp7YnI%&jvbLD3#_ zh`)4Tm^J%o4ktS;}9t|bF=_79XWEQRj1D(vBlh_prk}gFy;9jwDvF=8FO^c6k|ot zz1f)Vkp4~{p`)$$Zz4pKRm~@zw9aa4IP$mle#XCBqZMd(l3sIDDhb*zK>&ho$4%^| z{ZGm#E1}1`M>gG5kuoxY0)+q>d)69?Q^Yr=U#JcBr^+5ZnT@svgM4XT-9A3t5{l&- z>>oa!nZCVxK~P;JNI9ft=M$if4FeiQSbb7^Nah@9 zlA06ZB;X%EUqX8^$Iv6TdD5Eu{^n_H)XC^bI-P+FObnR)0m^gO?EDIGqDrfZ?Ir5G zTT&`Li*04f7qGT$$;}tDGdpSUyuNW~*g4YE4fU4q=|IU|+)Unjr zJC!|XG5bxJ;P$HP(@A9Z!}4q71>VusOy1J5m-~krhQ`UmY_OY+BPXs3bBx0Q72cjt zpJ_)rgPTT$`;^7XXBoU%|A1L-g)1d47d?#i)+cg2L+QR^E@-6fd^O6c8c7T~@EogU z;tNrKw>4GGUAuG=y&zFLZ@XXQDPHfH@hHYX7N;(_>6cTryg+xKy6wd+qv=3P>&ek< z`EToWP`#>_8PhTM>&-Xp@DFX(v@)cg?l4a zlmb4taG#M1vfqa&G5Hqo5BIc}7oC{*pu5DJba;RmofbrWQI$RY0Txa_VWj9MOXz%{WUzM^ewvgIu#sZJDiVcF)Tt z7^|Q<<%&@)+#hfMi@CQ7h@;!Ka3KT;p5RUhn&82m;O_43?(VL^-QC^Y-QA&qMjChL z_TK00f1j88bYJdsbywG#wR+X6Ip;Tq!+gp@G1)sZSm9`6N+#QM*#GV&rGk8}FOAKJ?*Ry7^(bR40RS{!RY$&23He!j1AsID0S9E&K8*Zr-pwxj16r z_hG3E1Co1quiLimwnX~RsvIIsiNUum(Z#8FH@g#hj-?wxnHWIN$*Sk;qI+IB*F;|9 z^n8OY)O4VfkrndJN!Aee{$Lmg&r+S+)jyL(6TP{qy56YsUb zFuL+F^!UQ&hjzeNChNJcKBJH_fyns5P4u^E7(GWDm26y|siRgnLBf^3C4nn4NX)@h z_1w-c7ILt>SQ-tlOnEUjMsN2H`$uFi$E!Jb+yVk6yH+xC_H5nR%2nQU7c&X5)5t$Y zASB&WeWe&xRoyQxWb>i=Uz&=W+Qqfy**I0I4+PD{V3qNqOgB3bCe2j_`kdp&jeDxZ8T-IWL*7mMom;GB2q@$&5lB4 z*xR{=b8!;T@&?q`zgK?wViLiw_PI=ho8#_Ow4YBWAv zN6<+|177|Idy5&#i@^<*6Ezjr5;C*tdOhAGNO4eV6gW5ANm_c~EbUY%9YW}6sfebd z0`>IskiE@|@r4$cltE%$QwbKn&oX8gSeu1=e-~nm@kX9Ji+QK-q2H66Fc3!6i^)c0b-_) z4qAi~Bs*LE)uEG}rxgmtI8QSXRp%b!YxlI|Y00;pP<}0hZ(ncsHb&ts5?;Yd6|>y{ zfhPa+!Y?N$7m3Rpxmw>1+#k&!7!Wk5Rkd|=G(25_Td2|s$>eOlIW8um_jfUeBpe++R66i!V}`p3n4~7F3#CHQ6^y>W~!mOd9|nzeiHKKKNwXXyZyc#2`jnZx8j7!t%RI0`pyjWwoFe zKOlKNcBKd{@qCGoKOJv*x!%=AuW0iSK82*bEL97tLX5+CyPi=s6>4|U$irdZ&{jLw zl5h`3#`{PFza4rJ!ZGj#IhDJvHy2g*lARiLd4c|{sW97F-nct!)#dhjDWozO{O(n0 z|JDqrH+IS{NwkJKU4i!MO^vjSDUkZUDM=2C*DD(@r$e{@2B){cFkt8UfphJtVfTTs z4Mwq8*pl1N86WTT#zM^tv_~WCQya6^mqrs?5_NHyc+q{B3Pkorv(io(t|<1Ho#_`;j7%It;hlzR z{N?ie>Tz}|xQ^9r09^29aufxOT$qeJSWKrh1FuIzPkb_Zlw6)Wu8CzH%tEs>-38uq zI>KML*_u84+!(WqWcX-9mMRQ<+K4vEefnaEQ2JP{NkG!7J*7x`mEu#FEKvexXBGPF z?XgH$@9t3LORm1tz|PSW8>3%l^ahAfe)OqXtuBtpgOk5?p=DV{X$QQ?-hkqF~Vrg(a zVps#s3Lau#nXP22cx(4(h&la?l<5axiXOEzyR}xsiWg4deQ*V2axx37qZSC+y;`zx z?8r6~5=po(NLcdiHF*BEZ<}bPslxCxtaRrf zq8UEi#3DEvF<*S`f;y8oStxy4Z{CuUq#afts$|Owhvf(Q!{a9SNQ}{oed(&lXKv7n zUoK2$$X$Mu87~#)gkEoWzN-JrYuK1dljOcBGKZ)45T}fKtKJ?SscP?u z_lYB!m5~=SbBC~LmUy1#@HkeB~#?KxRw!v zXgpk7Wy6g9_D|aqy`AM93bkrodV?=vAdKdg0^VzTG~m5Rc*^P}5&8zJ_O64QMkMmb z#5Rz8P5cY-xf6%_HO_=N&FLmm4$Lm6BMeW~u&6e(-%WRiSkY3tZ)`QdoJe4-k7r;Q z{cm$43Po~-pn||9!6T7ogb&Gmv7X>NTPC7|N!cB4^rllcqoO5lx3sPWoqaRUR4a5$ ze1a;}HsmzG{6Xf-SX`JV%50zFq=dc+)djP-_12?#^hbq7|Gn&|^ws z(vy~EQ7#V#qq&{#ce`N}MfxTtA%pt8=8Jar_JG$X(JikBBqpgn$kfe)Fa8@p~~E{-O4i7g4E znZT~cQgY{xcZe8TF|a<~z%pnn*W%%iqdm%SeGo6qHT(T|NJr}qQ|i{Ho4vS7M}4N9 zLT=9q|9Z3&TiZ38@y<1K>4};CIAcIcBVDv*CAUqaMW9!&wKfi8s*ZvaVk4p6Rhn5t z8dofGgf_|G&vY#d75nq$=9E@eSb?`<)wWYHf6xNijP8KR0x3tUz*rg<&TlKEpH0ME zUA7FSjUuA%PHcB~pYN+TqzhL0;^&@KWxPf&ueR;}AnfmEFJ0{V+t`^0O_HrWViv|AtOj{{vxmFFHN4g;2x9j4Gnat?=Qhf2S?Pz241)Ek|y z$LLv-z*fj@Gq{bw;{U8E9ChfvnkTu#Uf~UN^9a5)xrilt-k)sYv~=daw$0bm)&RC8BL>^+W4OOQGYLXeT)pUL{Dc!{3%ZYf-%DrH?|ejy zLG}sgWT$QLGGe$HUZh>m23yvyB(WsG@^j43UU<+RG?F)z(zxHlES-%Go9x^@e<4ju z%Kti!5-)t&JL`F>A!9PxXZOfz^`~Z+wT{ZP{-XKzU{{v4f`V5cAh2yOr*MhG5;1w< zXuOf%nfj16)=zSxYrr$8l0S4kc>Kfj*|Vo^8fHf&Z2AnT@h1CS`Rb_makr5klV~qr z&LD-rsJC@Iy&}tww-WZQ;MwLy;w2xUDVB&Hn46d(HSV`GPjw30UGUDXYEd92^rn54 zw_LEh@VlV?wUD8sh79X8?FKML$#|7*fU{y+m(MHx`segx}2>8+SbrYv@xm;op5iVUm^QLi+l#%s>|IaseMxT00uB ze1_#fq@nOE9aXShWWRlTw!fIw7q>V)C)zpiuorxD1s8;DQ*qc1>s6vqjaz1}bLF;| z+270!ZZS0dKCg}EI(XqI@hb##dQVmPShTqbsRmztBrV{I46ZglUiIK;I)5^K$3+*9 zeKqz}HFc!6zLe}vig)a3hFsCE78>z$qj~uJpww&4li;WswH=uS_bblgt+07of9WJ1 zJ5Dtu&Z{?5lCqViog#6!FqQg~aQ#`iRmK#o=nuCU3sa-jABb zewf5xQX0G2_1SI~$cg*E#YFXkW@CPGYsi~;P`lBr$&$HY)i-C5D!|4q)%6;EiQ3eX zv*fyTWQq5r&5#g-=EJqXEZ()~l8a2?EX*hv&-Vk+2H2G5C zhM*(=;5pvbUz%__1fNmPD>+==sbe?LKOf!k_~nN|97Fs>6^7e@8@lqZjH-Y!*yuOf5yoMl`~cpeb#-`AL2b|$WkPPuh{t248ku@9Lm9M3ZO5`-;@ zHzZ<22=(jNZaB$s1$_k=9~qS^T6zlH9^$BVC~UILgu>$g^p#3^D1lTqN^A1ki*B}5 zLbAp8K^LM4+1hp@7+&g?*2(hC-_TteK{WXJS{B~Nv##zAHj8q$o)wc{>) zu5xA0(1sM626t|EaC_y&5&j$OL!HMkIVSD4w*BnUjK6eL-|F33B4jauq?_JiPfgyD zexXvPraWB#bb`}^#I^VpZ!@3-1j~upIm}urfV1&I1LVOI?A%Fn#p6>^R9s9ZpDas` zq*m=JH7w)nRa36u7>$?GP|9npAjMxBCS&CLkqii*(O3az4(T6NWSf5}F{m`0Y0NFb z)4q69So70-dg+`@)22i70YZ2bz-a{V?pla!g3A)zhWxZ+_-c7cDPx9l4dbp^f&AtdkY%C0?=?(_0xKcSOE36jP_H#CNP?sWh<365hId7Hv(M zy^*%tQ5=mDmAe%JFqP=l^p+?t7{hOoWEYk}MoMLw7!2H~G_+tU#;g%fF*(cD;OmTE zU&irdw;8do|2{V;5E*^h2%a*$c+rv&(N-0Ea{IB_)!ypd1qfZX=JWD^1_f=eVrbdF zSgu{+>LKY+c?pP`2`Q|CHFoeL8xw8=gpsI@V37+X*QZWjHpK;+ge+a1>0y7JJOr;X zRak`Nar`y}5(jI_t)DC)*pu4LyGL_x$0ZSP&X3S$0naS*)ce%BcTsVD;1ejTK8Gj{ z=U~}69`e*V@kJP)ml#h#_e5(8oaieeE~GVtD#VX{)JkW%H_9oWA3QV=-`%3>eH5TD9!Ba*N*pW*}+l)H1s9ah~N)tWFcvwfbSFOXOLhb;)xRjZ8 z=04l%!#XvoDUdJ-MTbJ7?T6-C`-TCFSV##)Aw*_OQ4e8dSnBB z_4@f1b8NDtm}emAc-b{ZbflKSjW3q|j&iMf7a^z>NzU;rZbZUN5#L_^sss?h@Wys! zlnl+$YO+-Sh^Ra7`)kV>&|Lf~heea+CE*R(Q#yDTQy4ILr^Bl5bwxj1*QiOZ%}tSZ z#FVMaXLr}vzm)omM4Q^_Mvzv)!rX|&^vhuYgxnvP18@KcqVr$~TB^;FR0p^FMSL(Na0V#@J(fMwm__F zT$%jyedK*@F5jQ-tn66Px?Kn=eXn4V?-nMP-j6H3941lwlMACQ8nvZnQq?>&_0{RJ zvOF_fTH30Pt0527O5ELD6R@El87nK65TuF<>x4052nx}FIZdX6M}SI~GYqt2r>1wj zK0G{40&gUl(22^_>Fiw`5e9VOX|SZoeAT^^4R8P}|G2)Ls?)=fjJg7ayTDr39w4vNKul(dp~; zSqUAb>QLHOr%x$&Ftm`FYuy8Yiy?V#J;v>cM}M{9h@w^<$^dyvIjY?C@%3GUC)1~W+H@vfo1fXM$`Law}1=e4k zOOe#X6WBKB$-Kla6@hsxs;ks`{Y6Nc=)3GGO4h~7g2;q?A=9vQHIIEa_2zTYKdrb7VpVyz%a7oXgYuPNJVR%{eu2;d9YlgrVAnwj%#FHz^3=ogG^mV6vM zpm^VCd`3*n_mfAnxfhiFX2*})@>`G26@O=_WP Ui&CVlC(j8<`)QFuTH;}?l<(Y zP#X*)(U|X>Wjg~(BEBXoId;)bB(K{y98*#$+d8WGuoEVO#_|dlx5DF0vj>Y z@u^-eNK~yR#X4;bwq%CVB%14OT6a!|z$&sosL=Y|VNdISP;)E`afRfn9#Oz`=fBCx z!_C`M-@&yb_dEaN^=4j!p-6*;k`QcqU3zerxS&#Eoy8FzQi5Qy86`_4wrw`AaL>+s z=F1xUn_aJA*uG*5&gP0Q&l^Y|Hs^W9Ko?dMJt~25w*Xzl?%p3J_-h|W;zYMn2kiA? z*IowKCCEVHF9G>3Eq*fvTBs{CTK6Gxc+XPPPOCU%jhD1i5I&}K+;5E zil^gZqBK-Hpz;$73P@azkS}tkn?DFdB*a-|l0tBA%7> zrd)$mG`&Ag;2o_l2{!&!nR3+0Vu#&t(*Y6f9z&%|v*lEBWQZmvH)r-?83Ln&1>HEA zY+@+|yE8Ul`sui=-LG@mJiRcEH)_(E?N3I$7_c&BO4Y*5L|Bp7aObGPzEO!0<=w#a zLQeKnEIz1@Iw;Z8K~?HDaow2r275UOSpMg`f~{2u{Nb%u6#SW3AQH+RHM?5HW&>b7rfuLvTTZ|K7|G@hdA zDqA+uVfV1+2tGRKP_5e$MIg7(y8v(5mr(96K(Z`+n&sUI*kxI#^!|2;KoR? z)-+>mjF(u=MA{$uI^o0~zIV}sa{0o|FGKf;ebRS@dN;U+r_ZT_AZPm{!OvLn@UpZn z=oXlzW|$pNP-6%E3_A+>xXtG9b?oe6lEt{6T@zovvMm)q2=tpCI2hTW?#I%$vZ++*d_KdC(KIG53>z zxQ6eo^$(`&Q|cMMZ5JVxx<_45E|$Z*AE3hGlrIfzlV|QP#MOuD)9X_<)7bD%!$Xm| zd*hZqS%3cQ{(7Z7i1dMLtpez?Y%eRM{2-0&M z{-q!Tbd4&f5aleA=*g_!>LhXRff!ON*XSFEtRDgw{w7xc${96}fICPB;(y8`$ejybnKOZ;b5=US;Tk)-Cq z@|c23i*u!{tSr6`lWnTPG8JOQJ|9DAy>h00QKfrw=Xwduv5EEGd%4OebCxIk%1ykI z84@IG%(uZ6n-l7`Df&_-Pp>qQA2qCc)gOJ;UeXG(^#qmNR-%taKie4*aAvKU;?uBk zHd?XTrwn&hi@BvNb9as|@L8Z|$}@w!st2_`est`(V7_BddC4^H4H5z8^=4h~`B75aLWH^;5n(0fDi=o?+K0Z#o* z@pl}J*3)lBsFr%WFbqCyey9=A_Ra|Eu}$4BO%UH^oj0G;$dEjc!J$d+qPr z+=0@6U@F*QR~-8M0qhAfES=IN?gB9jOzdh8TkO$gkD-68w!;$MP z7uDeNu;@KLygG0>{>msX80)z5Kvt?N%r~V(AQ;zjtinUGE}V_A-Frqv8I^B&ikb7H z8gu)$?eQ8|cSf)=TZ{D1@dakm6xe7?UKHI2E_#RBEH?lg%Q%UV%HNr1jCjV{CgCCp zw|$4Il;x8xb$uXn6cmxs5 zvD5%3(D~Dt^iHPBk1_S~#PfAM67otp+IJTdu)k(e-t$ni*}3DRZj!H64`*-POulW05Rm4p!@4(?itoNZs4BZBXV3qlie`T|-+ufC1yxD_C8y#vvk0={+-or)!&g@n#~nynSWmnDsLb;hb} zRw;)Wlx032w7mg}vZz#m9d?WdTkznWuVqa$D@0HRuVV&nzB?nEfZp^fQ$m^5(PklT zDV2&s0UpDFnjK1o#NFlm{>r0lvlU8OYk2`DyFq?|?d}NNwTP36?XrGTw)C)UCo^|i zd!Ck;CI3&249(0M2?nJV%#9BopxZpcxlt9{v#pN@x7hT)8Bs5xw2l-l4%mKWiPR7@Y55`Z zILl67HfcE75)FIn#bnxGQ&*e%1FH^{@+uKsU0-$+vwH?BG*i7v0$W@t@3A}m?PQr{ zBTHzJ>@2vUZ5~KSE4CElOsJip&Fu0^*FKmV2)g4`b^*(MP1nMRl+ftyEW#cnnv4V`I9%?4_AH5`g?31HH2`01Z(rJf%oSAco4y7`!3S+&wFi^a3plE=mtf zP73~a9vS(#kVJc5kxox*PtRgU{UUDMxkT*JOtu@G znK|9GxtSr!t~*g>$(rPx?^61PeU@#V&aE;{? zRRgJeZ*dm_bjKOI-@D-?A78&bxs!voyb__ihvkp7BigdyUT+sb71N7yntj+SlR+Dm z7ndvDST%Qss4S33CDv$kMN*q2GBE6cJv+W*?Pr!lqI zoYaA$GZVh*3qAif_qGaDkgq!Gmf^>bA+cR z^aJmK^d_HahW5H2ZO_&1U7EJ6(QLaG?)(_c-0vC=b+uP(cKLRXi=P5mAE#|Z#O6Fp zREA^cb$BO5UczUP7LQqTpv|xCi+Jw|BV0uO9xH}?y&zTgZ1`*7GpeXIapjaAIuBvN zvA#?|$=-LVt8axgK!gI>yy-?hv2qGq)4L{BZY(Kew4{2(m68Cmkw_=VW~T$KbVV zR7Rx@!|{QeWw)uo+j#)aTs8Fh&zHyJ#h)FGd$_DjiR}M+^#l>MT9kk{f5^3laj})C zD+S-q;s4)or(4VDDxH3{c9uxjTRU<@q(M$+8i@zey+t$XP7pS|&e{Y3qj=`z!RWT& z3emjCL3fa_3O66BhSdJOBj|6g%w+x#~Z^xE9Rjk%}09Xu2a_0z9t^_0~XY&{k>0JlvIo?Vb9nyio2{Y=$ z)h=^hwf9b(hXuu=NXsC%k1k%}@Kn0GgTsSBDx@QBcT^N4b3ds8`=rzGVSj00ZXxwn zYOu{qZe;r7cda+Zm8WrBY2XRC6b!F3y+2Yncq)0H_f`WZ=~;ufsJ~oC+)<;cL2W$h z$7SwJF!~rqa;#1gzZsG2K&nt}sDX0{cg$CwMw|Gu^^HHYJxCtmg1B4W1|WYD^^wW} zML5`M%FMtkO-9pso2MB=;&re`S_rNcn-V*TbcL5csnF7-3;1Eo%#XI)A__#XFI+4m zF}uUmdg$Ao+%eu!cuCypGL}+peP5r8_sz|(s=tm3dhMk9YdeZj{emfL1K-r;!(((Meh)jBAWMVg4gadM=|4k?<- znwpW#&8%QE1f!FM;jpNvUNASNwOs2|HWXA6;nUvrueSZi3xgVm9@C#WN7B>p^ecP1 z^a{4Dv~pm#aez-22u-`d6Z@@sywyY`K{LsNG~{u$mJ;R?XFo#UH?Y;NnN-x?;I?}o zzx6rv8-Z4L#=0Pk4Lq1K$bIRj573MMW9-CNv5l#nCrf2HPLT^+UlmutG< zqe&14;>*dRcT!jXLxT{X6Fyk}UgjO&NEHK4t)qQ!&G5Xd8xK>Z`#8Vu#t@=WYm#Zz zo)HAUcGhly%#D&Q{^@-mup)Cg!T>uU1kB!t=Y%er3oSb-Dh=w=CqQ%5)%^x56c^>> zKtv46C*=#uy?@C9H}m?ywP8m#nXYZpXDR;&)Og&!*I~@tcOGRn%ykPJSEKnbtoeee-EioT8MRRd+L4?5&dNNQ9 z%j1Z(D6^H*_y@h&Fh->;1^3~zfQ7@ctq5cz;QuA7Kbx1$=ED;=Fz^|i0A*oeadmf} z&G|#p<^&o|_$d>cZ1RJLCmJkw4?YS8Ah57ha9Jz`!6yNCESHppS7xFq4}!Uvea_DE z|F9SX9~~avZaY5y_4R1pU0p|mYO3J~Bh%BxEiKp@8tL{5=1k!clVC(-Z*Naxv+b@# zi}Q<&j0~7so@GK{XJ-c!34q(<_<{mb0|QW~{+_hGI6S$+R_2z}e})3caw<=PP5MTb zD2qBE+6x9q@D?FiH8>s1!9war^L9Cg?79(y{QCM_3=FfDYYmWE&GzhMVP(Ho)>iZ` zjshz?vA|c24j1>IxPNs)0<#NoqX{D584eD33tPKh4YOwB6{4>ew2@hwaMCDv%)c=s zNbUY%Db6ubDO({iBZS@EVUCY+T-tv9CjfyhNQQ2@|G8NHQ2GA@Nd!`o`Jw)QJ~X5u zC;Ijmra2G$zo$x%j#v#?!)1HVCb6S{|LiaDy`^2_QeM48R@|)BDgSxv>J|FWx6JcP|S%Xa6F$;6Wv2nOAfIys#M!_+saXGJX*}s`= ziF-Vrf~@xmJ!_)>uco^}z<<9-v)3e=V4CT;f^6G2w%nvVOvO=iUEe=89Sz9&1b=U2 zFp)$?{?D)QN(IP(y-k~jtpg&3Pwq zUKpH`N!ev){ob&#PD9w2s~`}3x}qh&)8>}>Fj)HJqPZ92ykaNvY4S6Hh%CbH)oT}b z>KAiC&+zT~sG$)khY~&!AnG$<%N9>5hzEl?pY3jUc2I-Ooi@B=grtK}GzP#Cp;Lt& z55lDw41@f5Cc5ysg#7SoAwm}CW-e0CRh)?y2ehBJs_7gkD+qLUF)g&vku5X-YdVLn zS=5A~hSWyYoFB_pJE=Z6_UKvWyH^T?`moSgnRE}{>~R3}iVG`h{{C(xER(Y|;{;e6 zPo)iRlC0}$p(tr4KZjo^+cf_VszF5Q~=?SPNtI z4X~6y*lxt=2%i$?V_@B#yrKag4qD9a&;q`n%_0BKKhjzSjOK+onY_5`h}kP;aud{? zTk5<)NgfB{fBF8s=kGC6Cd@ zp;}G{9Sm&-A^VEZhmd1Z=Ak7Y0#hpF z=CRnP=1rA;cXl=qn#gGVBKvZsh8*j+eR86mjYu$0eW;Yk^fyAMXcCRC!>3p&+Uc#I z$dA%KK{l+Xhl;A`-*PEdu7)Wlh&Hgjf}&cSKa@7%l!$aPo%zp(3x>P`$+r55_5zI7 z+41qg#z^!cP1QU36v8tbWw=&V&O|EJS?0slps5bJuKil?B_KV_^&^(fwJI;(FZbU? zb*(x^ubA(i1l}pl&xy^#ygORa!f%uDXAO;n6oYOY+aZH!$zR8}d|X&#LNM}q5J*{w z{aTKAQ_hrlEF8V*YPiY$S{}c6XB`9s$D9?vZ4X9Q?rv0#+tYO5JU>#rc7ZnY3bw;PF3Q4ec%POQT}V0yh54JQU3QoGRn5~r_*|rZ7x$&N!E67 ze(v!^5cQRHa3)NT-_qN~Zlz8KB>{W75nNF?O8t4S!+6q7OEm`kuQGTxk3H6FNDd@I z(eW3G>!B^psyD#^XegcctSegS10OC$kDC6p!>?O%hCqA$%5iFTxQEYbU9i~>5V?u9`1PmxLH(>iBEIz^;Iy6 zJTdtbU$uF;fKQ=4pH}7Vd~n0R?ng^N^S$&lSa-mz3L4R5wViJ7gn?Cyqzl2Ep&3-B zX~lT{YL0XfbH`U)f<1<~!gMxL1G{CsQZtpjA&Gd%DH}Oj;bH!rz|-@H?NZM>f<8yJ0dO8Nw4=EQObo{qdEt8Ze&HEH%I?Y6!6%(B z$azTZ?LKsA2WW^BY`%u%v)4u!ZhQ6*?dWR`@$@VhyVDESSial3JD!SWhkcNJR}q?R zHk?r1rmC}7-^iR(>(V7h8BHKF z;t@*bH<)u{d>!F%HLs;@Z1r6IJWRz@X3PmPlq{PS!M#ZBc)I?o*(mF3je|h`P^{7o z$5nqz{bXu;{_x3W^HAYf zy+g|FcMx<^Py+A1KGy@|H19Pz9;##`5rx9o`jU?{Sh~FSj%a4i{WZgWI?~S4koih{4ci8TQvjr6*TVm~ar9LGdU~opt!*t}S9n_eMgnU* zr|a)5wDa!pK6N9`SlHS*2@7ywcs&$1HudsDqQi1Tw#ARKYpiiOC`P1iAYZ9v9{NTs8}7r_H^tINdLB#+KLe%MBJSlpkTt5El< zD6~U52`hJgUDRfg4?7vE=94BPTX4^a6ofe?*=?16Tyf6z@E}+}YX_9G#yY=#e#-RN znIA-f)xyzw&6w3C)!WwQx9I$S=6>jGHBO`N`(dleq|5e3m0cNDF!MR6IQepQj2+Rt zfZaNud}bL)_x`KJnR~gbE8}~C^fV1#@b=%954dHae?Pf>vS8xD>h0dd|;Rzwj@P5@aNQZJbyxdwLJ_-Ll1vbXTU_xna4b*j&g{b}6~?ip$IeJjvvxl$fQ;GKXQ zKAz~j&?fY$C$aL{sGa|bw}MOl%!e^8jsSZEVzhAYW7o6#dm+yNUX83 z%{v|k7cFYB)|#1O=QkHQrD2usOeaQirMuW9VC9&KUCGlhyY1UKTGD@X0gCnkxkkPC z9BDzNnRsi!%BxxVKY0ahUKJ*MNGGi+l%>DkK}v7z@~dmDARLPRRc`!|l#mz?g*Im^ zNd!*bj09BYs>djQWp2~0T0DYG@#8sl*dj^fQ+6Sog=@T5Q096gwaR*mGEIdXgx7NN zD$dLGqL3=<@UQ?v>~h;?)yeZG5zzIzJzge$tIe%dgT)X z%%bA1c-okLJI^vl*YjCLIYVij+lb<#qtN~@vcbX<-@IAH=SZ!_mq!jzDP3ba36|C} zTOz)}#jfXhmTrtq%=>MVS-{lSu`m+YpNK5AX6M7!Fuqu@w!%c9TLmXkSnnh80a0{*pNc>eE7Z({Z}3^fAmzF=h& zNAXl zTZ?@qXtN!))rEP=X68*Txg72H#x?WFs8()3$yG#Yljl7l6jWB+rRWoVI#qu32qGx< zL_TMi4*QU`Y}+13q(WGjP{@ds%z2bV9pX8|@rc35Qcs*i^I%I^hOkkV8XcfW%6&ee z7kce2zwstLel`D`$=tNWfsTlNpu&6!-riV(?l;nKvpuB(WlLyX9{#XLdX9wy| zP_#nC2!_8t&p{3~;WfOKC*oh<$wYWbqJl&zBu4}`Z!8e#J!OP&E zKw2GRsukN=smDA^NItq?4~C#^_vrrqn~>~2zgOCddjHXp&=V%Jd&rgWvF_@x5=E{U zg&Fm!Bq-zQ%ES3UMJrB!Hbcy&T6ZoeyCgyQbqr4jzc|6daB)Es`HV|=_ws#(17)vK zioz9%9-|;G)~0VvyLYJ{N(0*7V&=ENjQ-ZE9lividCYL=@}t*)1y(sV(qXfU_xBc_ z?Iq7b@FwqksNh#oijM19=Pb346$CB8#ooNsgVwV~9g6n=l|&!aW<@P{Oym;9u7tec)W~mS98zYGYm)3)hX~JJahE*yEG^sQSz%V{=ynr96?Ag z2zE<%I74ET!WeF5JLBLyY8z*c?YJi3NnOiSd1>A~x4Eb9Y{8BX!ob9ZRyb$qa=o&c zQr_e00Nx2Hojh**mUAQl2>JGN?$X1Df3q?AM`^aN_XjwiP91T6!L#%q(w7@0>TGL& zQ$U8a*&K~b?lM_V{cJBGJ}MKnzYty6;k@jqy`AV4klj)3{qGG}Vc^bSed=S6$1CAi zesOI$qe8cW;B%9M7l%~Gi{y+>Wue9My?;$lMb3A9XRmAcZv05%c0r+XY)YE)0EgkP zeOq2X55&GIkQd7c8+?vrC}xjW>Ixi4Y0CEfb)iWWeet~br?s$`+FKVshHj%LEX8t+ ziQInAf8}O8eS**fi->5l{F$rb`dJ?3a0w~wMg9Hf|3019WArah?X-LpTIo!sPhc6G zFl2%@vdpv(JrW{K8AGejU5VmHM@Xo!YACMx8e^1@{@#ndHL3lR7FkqLc?tvHjsl)I zQXOdz6eT5#4ARmV*MIUzb{3{z!{XAgH=QYS=?HZFFG}EjcijGPaDyAs4@SO0{|~@ zxY6L>VlZB|gd6TU>gQ?%DuFMEb)I);)DNZhXbNT4;;y6R>I@_vw?Iz#Pg0Eu5s5@D zPF{Y`;2n8&l+i**pxo5Weuk?dJSmy=J!H{@Wi#Rb3?F0x9{yb58W}kYjLpFZ%Lh0_ zZf6${k9)Vz>A3&B7;7sefKYQ`Y*0z?T4$2W;3#tiTWisjz8uoQ)j!b&+|PekPAGw) zgq!MK)q^;&1KXqhC(@zo3IU=C5a7&F`zkR+vo!vHazTM59!TVJ`l7UjY$ZEIL$dga zzQnD!MML1M`u}_+^m$$vl?(#-hp>G#`q_3v6E=u9^Wxf*kZa9ADJh!2k%I^?m* z=wj4x0&b#Llo;fjlI#fIy#BE%n22?l#m)k*%#;U&Z zmiszG|9b*5@@{X?x8%9`wvX?)LT{rgbgIj8Nu=)l*03&O%sc~L;r4l4 zN2qXiW~aR<7(J@!q*Y5HF$>{;Z;@Rq%$55N)xp^Wf9-acb?Los_riTuW z@O5)`4b5n9mV4^8#q%JfyDT&s%gz#t8mkGs2uLI&Y*t1przdzR$W6ob3wn=pg1EJd2zihwj% z@5=|^LZB91ODPLxnK*`_+wBXdI46Bh6rOUCWn5o$jny7u9)coYZ=IctaLl=khK-74 zq}o8n*5I~=^k4ZhWqNXtRRiv@tICF%b9QZ;`Xm|53zFqr8UgaiLK8e0%SBIlew13L zJ?TCNsT{T$kMmD!nXgF-QsfL}R=+Kk6dYedGBy7Gi{}&B{k{=Ww4E7w zyo6mIr#NM$(R)HNiVe{q-GK8uD_2V$a9oi8{byjl1}o~<*a$dTOV6W2-lQz6w4DM& zG&e(5v1Rdi^HVcs(;p-r&4c5!s0M%F<%U`6t=HM@rE+dEMb^1#uwJ@t?UT z=C%AE_07J$(@p1OBD>xac(Qnn06J4uS&2qk?4V4yS0JV2aOH-Z>&#x%ek0}Fr4A^} z)3AH?#lH8i>6-ldPIhUdY&AY;b7JVuTHl|7Bi(ay@Oc1T$HSX!ozeE)A@XX8KLX3G zbI|_1`BPCgFBYm)j!W^4sSnIE%l}i|TL#7T1^c2TBqR_dxC9A4xVr{-cZb0V?mEH4 z;O-vW-6d#n26r3W-R1F@^Uis%&aGQ@>wdU*zU(#Cd)KmFYjv;wb$9=CeOOp!_xJhA z^h$j6tloK73}~=xL0~W+w=n$x@xy$p4GJVMo>Gg}>twsh0>Mf^Lrg*PZtT)J|Ll@) z^ze4~L&Os>5gY#RT@+Hu+K5TMvt=_{coR=9D(*P2oMulz{wms}5qB+M>sMS@NwKo- zuZ22ZYIcRv-xc=x``+;V5$wmgF4gGCA)2w9m`mm%N)zJwa&dD2W?AmP66UH;JDGt~ z%C{blnnxLzb`3(3Za0}Pr(JkJ$+gH^BsV|9<@ zX(13e?^;`M+rypU^Mo|y+V^^8O?-^ItG6dI+wf)db1-B5<0F#5vwITn31rDrU0(-7 zAr_xW~frb8&RP(eA99CwXD|-uA%UI4q)z9cD+2CzQ_75+7^p`GqD3`kH_ebYJC|m zTOJCIIKNozKvhZU%c#&Zfvh0Li9Izw_(I(Bj!b)VDsJ7C8L{}x>|f>*G}2Dq?+{+z zyavYuV#l(eX&z z7)mh{UJP+eo@ftzdZfL`&7&4aYGsFsX25wADG14 zxW~qjl!s~tUx3t3H`X8_@7RH?A(pY3#(VycpdsQ9`Klh5me|1ZNjT_D3o1Rpc@Aw_ zp`B8)jIg1px^%+sZFd24buRYe29(JH537C|6ECI_y5_+ISUgylG$U!b zbpDG_I|BZ^^jemCBj>(g)$aJ93K z$MFpax2mio0{hp%AxMV^G=hI-0Y!F-1rg>cp*YNgs66kPr>HlIPP54Zm!KsYH3E3B zpEFOtXhrVe%of0EcaiL}R&h?BhkAgD$HDX z%qbBg$ki6=AB!j?c80D%D*w2?JTU1_UhOcDm$+L<+1%XPO=%yV%b=T6T}WG@ zL*RUTI(a9O_@;`iyfWvECAPBMI0G>6`~8K~7y6AV?#(YE3S)B=yhZJ-xR+mWtZvu4 z;x%b7SY=#-IjBG7*HFBH8F;IS?AoMU9}K|621hnsH7XJU5(Zkf&Ehsj2lFC`oxgJ- zh2K$yuC~#n73z17Q z1lx8&1s2Z#P5=)D&$Ez>NMgflXnNz_#-t1;rBrQ(TT?t*2iu*IR5y-vXNC00izMl0 z<4|1 zFh8ZVpfEZWNK>4!?qbjf`iRZqpcoOHwLtf zpWume;At*EonN$_T1q|MEC1wEBg|l>4mY%V?0pxU@KT|;C!gM`TUCpl*F1;Uk%-pb(gY%rM@BiG~#MafjoQhTU zU2#-J)qx4scio)wc0z|pMrLTVb~>n%_*Z2{gY~~Q;Hf;gOn~os!}_MpYf4NFdUoBT zszU?yTuh4MH&T8VRJ_T-uNUF_u$CfuoR5g8%3{!irtpn~_Qqa9@>;<*W84lcAJR0; z&X~wqbY+MUf-X=&nyT${hF=au=jp1xSPz;n(`#O6{GcH!{2J!UB(tKHR@7fiQITf! zbJkqF*#_y&csAf?tZ%dkp+NWVKg@*fv?2E4Zqy-}G#e~M)JZBb#lFn`(Wh)LCVSC- z2aQZ@*~M9aEskySf`p3>Okb5Gc9x1qKv`d$V#RGH0s!1~#~!a*5(U=I!$XWqMw`3t z#Xgj4^ANoBX)jcd$kP(j%xe;)JAT?>c8w#2r4cfefcl5?S9ejf#laQ`|BVyVoRZgmb_o2BecK z?Z^jj*PA~sX}5 zW?RAlGooMrf+MzLYS_as$AEv9(-cu8(P}pA{74RW$BX#d3HU-#cwMJW^YRAZjTag^ z`>!^l0QKofr>EYR_7&zyGMoANO502CDlR;evVP}DqL*4-41QL|E3&}jO?A`t5{@(h zrZlQCiy~V5GS!0gEFNu;O|S!dI`j-Yu)ol1-Q<^C;m&wfRW>VxU{(0mB=fWrB*e!RT5JllJcKIl;WhLv zGRIEUqA8)rLfjgD?UBn*YSiL)!#awMmuiJ@#xG4_NTsDr4xbqj;F%G%9(|IO9;k?o zhqII;83-)Dy`9rP)b9->WO?qLDvwB-i1|}#E*IK|?XUJdb#kdJrfS-4qRG{f+2yfw z%&Niu^j4Du?UUlAdRT^<>%d~#WurTL@ur`Wkd_pC!U}(*KBwUC{A85!J{6@J>esMb z{?FXoiBxpSPQ9N_D6l`MLAifCWM&FyigCBL-1O0T@CiKDXDok^x1*;HM3ZG(VSZmo zp8ZZ#YodP9uDM=KF!Ymn$q0~;#v*TqdYjPFU0arp!zuEXlt%IztE8KjM2%@l@v0WbA z%(m!UcsLzTnVbq~<}bNm#^!A$^F86z8A7tY#PzvW%5L#`u@ah|QK?X+l59ouDKqX^ z<{5{tGe5LRfb0oifmu;Nr1EFpW{Y=`hNq87|Q;7LJS>HC(75Nt;yHD{H*Hus7$3vDZCPL>ocW)WKuUGKx77#MLIK_FMN zr2Bo4IeTbGd@>*DsI?28p*Fi}^<$q)wb+-fAK`LMrKKegP*R6V&{_+lP z%nO5uEpzIpFzt&orKA`&A(b#bkIy5n^WeMu(JvuXcZ{j^nSIuN8P#|UstN-6i0LT1$~<)9H7hw!$M=vz-ZKaC*N%bi-#d5V^zn0k)Mks^jB0yqwx=8r zvrHH+j*H%0mcEls{<8}~8K3PFwAtWtd1{T$O{R_-h{hB!wUN%@QOK2HM~$MPBYrDJz%3x&ACKUn@Ak z)1RRG8_~j%{}-o_5Mo$n6HI|DQI#4TN5JM?4Iv$f&u;T~0FeS(kTGX(Q5e zOw8F96#g$1a7h8&R@3iX_Ux)Zo;HPI+0dKY$Pd~QRilHVGS7tCK^0uU04e1{(o48oz^=fv}GBO5)9}9aVwmK5< zJW|y#4m^UyLtwrCAR^}Y@$r>VBkIPs%d`Vo33SRJ+T0L_KLBp+05lv$dC!KQi$B0s zA~9rJtmaqsx0fg47ip7e;6%B9V1zluG2*s(jCG7r;T_yi0Us>+h+1*&v~QP1@uhR= z3Ihk06}=+o$EzA9Y{}6m9bJf4=4|1Yr6Ld@Xx@cm!-?w9}hc9+cm#l8QZAUMNJfHe3m zT!v6!=;(0*lJZaOn5rHLdk3aF;yaAA!CqtL<-@3dg~Ob0ja$s0!8vQZm8f!0X;ic{ zB>hn#Dz3FKFhn$xBHvVknO5Cuy3sb?y7H(iT};}#;LQF~WZmU+;$*Y$D~}~e)}Ex2 zq;H=${C9!&(48cBUiP^gkRIN1MybXL8(N@>#ga(|6jH@n=JY5?$=TEAi&5LK@K#;c zHvfMWMGxIhLbH{_;kJlj zaw!RFPB~^gX0GEbaItsYH26PIuMM@@)QUz>pma#~rD-s6k9E4IvF~A+VqL4XU?B zBK}`XQ6rO0IM2C9_S(WRrcWw=aJeYYofGseq27}|KUb>YP{zqwe_RNws*OvS&||kb z&)HHwUn`s7D;&fki&f)NN2-7d*tmb5XA!xx)aKSu49By4 z?dSR=P{0QJ2+p;!x#C|rGDzD_SCl}pHEGWCog7o9-E4{~SULv`pEJg%>ni}iUAUlBWsNaO> z%L~}HG=Y9gjMw-tF)#Ss-zZ>aYEK?<7_RU{D9t_q!k}*BuB~-uC})YrIJNa{ZVq!b zON&dNbQQ@wBfW+Xg5_g%?S<`6Q-*8&%@~_QIOvxLsSXox+Yy@QK^vS%HQYUY>6T&4jMQW*GQ99M?8*pc2*i2CmPh5bg{iZIwmM z@Um2**lePIsfpp>DC?QJQq?1VIpaOySsi8qxmMg8HL+;WIj50PW-m?bswL%3O13(Fl9=;Cf2|2TCn z+T_(??Ie&oE7yML!QKAU7_NoO=+7tM*g$V9XqS=7p+H3*gO=7QE+Ikn1p1l@oTIdS zHL5%n4g-C0YoxU_f+_b#*Xh-Vv$h zO3PnjS!cQ?d``Q2BU-zAj!}W)nI6~Ea%=1Leu$d(;q07%fsyt?onOQAhud^`#`5;X z?Ju*RkJ~0QnDHQ)0|&~A8}8qcse~rBKBQtunw`MdjI||>ikiHhpcP?(tM=Ha%92wI-_KNNuA9#%SnP*ot z1H95-LmUXGCv|h3Nxd9;jvbrv{svYxJ0`ru5NOt7g3juHWQ&gs@gC}TZ`sA5sqYy5 z9l-2lb91&ua05iI{s}sX>~Ei!!IHV|4uN9#K~{DoeOiIMqGlb?ExxK<-oCTjA1HT_ zd-(h#UQxxm{`0=S48Ixpx8rOAHFhUPSa4ph{v_OEK`xUSJ9%8)yGu(U8%eN_XAsb> zkva*S^%p|zJFW^Rq3{_T%A{i%*2CK;2FY1m!|J1p&}FqA{G%w3EbV z!IG;63WV~Ix>@Vvf_24@J?!7yARUM(m(cpv)2=VS&_TmYN%XBVpbL!WJ{ zFM=i0eCz>{2KD(|hK zp60wxaMy&g<8cAeu7|>C$V9za`;`P$5flljfb(V|Y;}U5DI_u*t-$(|FK)HhudcJd zthEF!h2BV|-kcuA3VZziytL@??Y9LpdxvOw|0L_tGz1DUAU66dt11yK-A71+3^TI<5mW_=h#d1XPj_w`Qu!+a+PCMZ&YPM z*rlaTJsZK9$EjoOV&7*XX!ygEzM=g#hm&{4>kLU~?$axAbclI4pk{UFOnc#>oc7O5 zzh`^+nC!VqPENbNpB4^%s?AtTzZ}b?&=yEfKtcIxD~=j2Cf6f0kWaRajmAz-(jg%_ zX^x*kO8C1dghNZ1_pU&I#AnRs(mw&x3M6ISM{%n3j9=rksdcOTqIr;GhsTEwwE z++)4RzLNWzadA;^3wmnQd;{{|x_H?he-!m%b0K8WuIoH|6g~idh`qmurks+al->EV z8G33qoV>U>fdG9eyz}K20lM?>J`30M`tJ!g><)dNm!1ex&+$*~T?GAuD8irYXyyjg zJvpi`uJJHZ4SA>P z;G;iaF@k12>H};IA@}dSQhq5xT-y^rZM7g}o}0;h6^HHE3NBeSVY@h&}75)B5pM!`<*1RNe0+j3Nc4gg+kM9MUU*Yb!|@R+ubD%@Mo-mJ9Q z;**|u#V!-R7(rerpc$?|mJ9fH7YnGgYHRnGkwV|X>fSs3e*69*!`p(WBbX1OPRlo5 z+iSh=$(cY)Y8oP7_=O|K=_QGRK&Nw*=!A#xH_0(6NyiuYx$SgwPf$=pX2G2SotkyC zaiG$xyYjG;XaFp`YzS~&G9@zx2T z-(4|wp_m#8xUpYrbs7Qm&c{|cjbL%J3hcPHRS z7dPOn|50gM@Yx$c&=0S_Hs6TX!g&z1F~je7dvb4o-J(9=nii}+iv^GZ>WFoOcwY;V zV1=(O4aSde3AoiyC^ni=F!|!f;k!qz$?^=M6F2iWI@6Lk&vR}F6>Gq5aE6;snr6fM z_*cHh;NnjSg$B5q8#hHK@AH0Sqz9r7B*wrZQyp=J>%k*p50c@ zIO}&V=TMYVaOwwGky~@(Lmx8+390jM2fsz2v|%}&^}ka#uTHEV58JBv@MU|I>AN5m|(}C zoq&KZRi_c2Eg#i4Aw##@ZC&wz~Xc z;P+>#*frdY{u?x7=s-X!d+&tzu!0WexLbHvhw|5wSy8=uj))EJn3iDj)9x>!@IR$U z>8OA7o;*t_y}%>743ZhRek{Ac>C-%T#eZCXe{m-jU49ssGy9gE{xmzJcR&m?~%t~*5#a+rH z(?fDivk{4(MMfFnr+gBFk)j>Sq)Ns8438_hmjurJ!|#e8TgSpJ8XUgL>O@Qwi{q9s z;YWr5yGqK{g;aptu1aY<LXtJ;teEKIu29R5x|)u{F~-hqAJ(Azd)1oSa0RH zCAca$-EIa0HB;Yho@h6}%W1-@wp1(9by3{S`w!VYM|D1c!T0Bs@gV&j93R6WDdAGw zh%(>Frhuz}UmjJeBWlU5@gkbbW|A_RC33PT+n;ej`Q^zCKEn;ZAP)`f-aI70xClcJ zwWWwPF-;0Qtubd*U>c=mxA#G%+Hn%lcbqa^LIG9%(p^j;+?(11jY z>(#eer_q(EkXjx&#dZCKgHrM>;JUF%E6R!cc;gw_lAQ^{#p>WHER0ZSLPnXdS{>^j z{ioxZ#NikxsIh$ebk4q(tJ;qeT{bb#lub|etorZ|I_TK!IWHJHqkHsC%-ieFPv#(x zrme}6Y0Y}NxlorrUXO>{jfw8gb`IF4eM44H2B$!#xP**MDvCj)*{DFQlI`*+0@;DU ztnfI~-fWjk!2+1Vq(Sf)2aYP6kmO2WQ)#h!ZGGwI$vaCG^tde`tM2*BCvUSrlf?o% z+f`J8K%Da!#Y-Ku7BDfxosoV@*tUDQsM$#vG3kEHMNw2tt=nl~CA8wVpr*|}Ia7MY z2;L`gbtnFl$?T&%vobJMPy4>nw2IUPL%k{+?ohu*I3;jW+WcWGXV04YqIIys6MRQxKF-q!90d2;ts6y28b;j_OZH14j5p>q3}mrB#0*LUsRkPQ z7iQj0>{rtLhtgw>POdd}_q`QF#sg|gy?M=co33;L&-cJ4;EdPZv8=VOpe8{))3~6O ztKhOOrUN|kt9e_5nSh*SVCH;gk5e{;$e+TBbTrVAd@l_{ENYUq_b2qC8S1*8#QRnr zG}v7r&^=iHepw3<`s|Bx`s$NO@mTK{a6zob^DTF6VNbLjXKiGrV~2$Ukfh=d?z`R2%{WlpHm$f&~}?X1yNzw10Q~I1yA7RJGNMc_%nR z%9o2cW-2=#D+vE%Lqgg1L-nX>idg`7K;`ZibKiGU;HzR1vFm%#n185Sn;u5PUv6p^ zqw5tV3M5<~qrG4#<1?R~i1}qsDmJ3`cH+e7PpQyjPQf_RI+mwS$% zJ-tl7`Q*Dc!a>+s1BfLuJyy#A;17(rEiDE>?M>}&dP)`sVSnMw>N)q(OM~kPL~+KI zuwP(ICq^>q^ILd$Sa^7nH3w_}-tR4}Uf7u{hnZ^gCMPRjk|d#k8A66&n}*7IE7i+G zEMu$eTGjm=^Sqye?+u%R?V_cBz`(D^JdhL}~?PdTodN(gnh2P}NXI658+mR)ndR ztwnZGPk~iJ`fd`d3Ek=klqENP&D?k+bh#p<-W%sGvuL(6n>xH_nmQsk1j~_A4|KrC0NeG;8Hid2Q0{RJ_o#z%=jnlBAf`!bgYSqJ=uX_X%JNREGE&*2mEm zwW^UNgel#vrw#BG{CRWqma5*leG~(lDBqORjEPa04 zq$92xIg^= zIAyEV{s||`9lJ>zIRk1GN)XSd;nCPU%@;ZMCORl>j=A_lN{( z%GYWYk%LN_Zi)NV_PYuASoh)!raHW($@5(%(Q8(wUu4>~k%g7)GN0?|3qNG|2wtT& z6tZf(#->LuzS*UWud)bd*U#ap&^)iAUWSclcMI4^3#ra~j$POci|we*0kilJ}f zyN)^5n-jM}dZ{t0yK}Gi1_ox$>ZCqh(a2+FH@r@}&U_dD3DP7IdUK>!MOmm}CXTev zClxuzvA-p|xW$o&!d^Bx7ahxXvd-C7b?_sOSxEWbRYAgRCTn)zKr?y6GfZ7?mby)y zuq1>u<0X4R*-m5z%lm2!He*JxMYLB^Q>xw;^84JwLbFLR-wH{dqaV!ePhiq6VZ+Q``*E~m+sAxB}YhCfkKA$P2 zBTB~Tzqgtxc`o|B&}`A+hfw4OrAT*jf~-%!P~PNauxn&Z2y_5|b1Z|8vxz2G1WF_I zd))OG0meSRfFK*H*{sY$;0s2POQt2etm;4nv^m)}@b{B#A*WS>4zxp|`6kNJWmt8u z4_S)nd{v5uATN~km*X&c2V~vk%rccvI~|S(llD1Rs$C=Z^779#HuZfTa>Z~w&4a8n zVRV+l7M5yhQqDw-*zhpI(Rh*ks^Vdk`c#qrrvc^C=BJ9iKh0jr%$t1GiXL|D&3&?y z<*L|(_zKJ0^VHJmMWvGH#&v3gZ9d6M)>_gr!{?&ZS_?Ae51?#0;TU5zYsKDv7UyPK zz`V;J?|a<3^h%8x*Z#DI$#{8ZtAhPPlOQdDW|bqO<#Q}hKJcWYT&u&WSL7DFW271! ze8F0Bl39Ck!B^Z2u?~Ob>(8oT0l%iimZh!JXZ5BXhe(uU`f<~pC%m~yfNXYvi&4dt zIS%ogygFrvu$R_a;48Q;i=S+1iE0>msTK^y{=Lj=`!n{criI)(DPRfKyf4<$6uFKbv{{xqXas z6TFz@d6+@@91jIn2L9*MAPM6H&^>9c?)|?CbEPTD1As1 z=kI}RH%@K?p<6a>oM&t%`f}`v_8aZG3~g*~B72?h@Z{ zE!RDC-2F3DB4}i*+Leyz9|d2Uwow*#ktM+p;YgqS`Rv3g*S%s#K4xRZlCXlSiKu?o zuk+axU3d3;&&M8Sjk*kv9{K#&(n@LE&Z$|rH=Bj}O0BKO{#yEs`i?UdlT{ymd@YV- zB2HKL$|e_*OJQqn*qw1Z>zMGX`vNd7efSb5|Ju1`ET6m8xbKKW!wGlTQKayGXXG}G z$*QYz0=W8%Z#fHO(|kfTrsy35s>|-~SG!2pPkVAoJ-011BkArF-5ukJeydTd z3oE>RJ$PXtluF+QtQT`Px8aM+Gom)tYVyOz<;AxxbQX=dXse_w9vU-F919{2b@P@R z7|C9n00XnMVzWp{9=%pVH&JykUyRKmDIuy2%HKgZHSa5oWL>uTbD24nkp5CmrGGr5 zs>SqoMRp?}rRe)#lJnc@NMcf|&BhIM6i-?o1{ z2SMtJq+jdmloeEP(4-)iWM%d)E-$Der4MH)|Em#rp@igrJz-!B-~Bg*PL6=yO297C z>lps^&gbbIY5;sAY_(RYtS+3LcD1(kd7MW|GvZ&wpO%W4aZ?24s-<-KMME$)(#Wa8 z$8E>?^ss+XbF@_6ldCRLW76PFL{fK zTLSqOa#ATiL#s_@`8RXHG}_G2rg{bQ{Y{hf_SGJNBcnDBm2vocHel?^gzA5!AW5ir z`|^-L*|-7lY3C(2R&^V}e>s`I{(fjv8h--#xaI8EtI^mE5=AqSP7K%iScQ9uURm;U zYxWTY2_<|*`b!Rgff4LU&M%502SAa9N5my7tTI z-~p#UDdw@XVj@9se|-iMMHVRY+WvEb>%#WVWyV-CA>t5>fvk6b6W6TD&!lbtF2dNL l|99N^5ADBY-2Xr61!eKJpoqLzBI_@Tq^O)oxsZY1{{u2_Wheju literal 0 HcmV?d00001 diff --git a/docs/how-to/llm-fine-tuning-optimization/model-quantization.rst b/docs/how-to/llm-fine-tuning-optimization/model-quantization.rst index b37fbab7e..479f37b49 100644 --- a/docs/how-to/llm-fine-tuning-optimization/model-quantization.rst +++ b/docs/how-to/llm-fine-tuning-optimization/model-quantization.rst @@ -181,7 +181,7 @@ Installing bitsandbytes # Clone the github repo git clone --recurse https://github.com/ROCm/bitsandbytes.git cd bitsandbytes - git checkout rocm_enabled + git checkout rocm_enabled_multi_backend # Install dependencies pip install -r requirements-dev.txt diff --git a/docs/how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst b/docs/how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst index f96dd3bae..6b1946ea3 100644 --- a/docs/how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst +++ b/docs/how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst @@ -91,7 +91,7 @@ Setting up the base implementation environment # Use -DBNB_ROCM_ARCH to target a specific GPU architecture. git clone --recurse https://github.com/ROCm/bitsandbytes.git cd bitsandbytes - git checkout rocm_enabled + git checkout rocm_enabled_multi_backend pip install -r requirements-dev.txt cmake -DBNB_ROCM_ARCH="gfx942" -DCOMPUTE_BACKEND=hip -S . python setup.py install diff --git a/docs/how-to/rocm-for-ai/install.rst b/docs/how-to/rocm-for-ai/install.rst index 4a7833ac8..50f6f4244 100644 --- a/docs/how-to/rocm-for-ai/install.rst +++ b/docs/how-to/rocm-for-ai/install.rst @@ -22,7 +22,7 @@ If you’re new to ROCm, refer to the :doc:`ROCm quick start install guide for L `. If you’re using a Radeon GPU for graphics-accelerated applications, refer to the -:doc:`Radeon installation instructions `. +`Radeon installation instructions `_. ROCm supports multiple :doc:`installation methods `: diff --git a/docs/how-to/system-optimization/mi100.md b/docs/how-to/system-optimization/mi100.md index c2e2e2c04..98bdd148a 100644 --- a/docs/how-to/system-optimization/mi100.md +++ b/docs/how-to/system-optimization/mi100.md @@ -342,8 +342,8 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to (logical) cores of the system: * In the server BIOS, set IOMMU to "Enabled". -* When configuring the Grub boot loader, add the following arguments for the - Linux kernel: `amd_iommu=on iommu=pt` +* When configuring the Grub boot loader, add the following argument for the + Linux kernel: `iommu=pt` * Update Grub to use the modified configuration: ```shell @@ -355,7 +355,7 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to ```none [...] - [ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt + [ 0.000000] Kernel command line: [...] iommu=pt [...] ``` diff --git a/docs/how-to/system-optimization/mi200.md b/docs/how-to/system-optimization/mi200.md index ae2ca8122..96d8a8f7b 100644 --- a/docs/how-to/system-optimization/mi200.md +++ b/docs/how-to/system-optimization/mi200.md @@ -327,8 +327,8 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to (logical) cores of the system: * In the server BIOS, set IOMMU to "Enabled". -* When configuring the Grub boot loader, add the following arguments for the - Linux kernel: `amd_iommu=on iommu=pt` +* When configuring the Grub boot loader, add the following argument for the + Linux kernel: `iommu=pt` * Update Grub to use the modified configuration: ```shell @@ -340,7 +340,7 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to ```none [...] - [ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt + [ 0.000000] Kernel command line: [...] iommu=pt [...] ``` diff --git a/docs/how-to/system-optimization/mi300x.rst b/docs/how-to/system-optimization/mi300x.rst index 1ae61a10f..e1819a88c 100644 --- a/docs/how-to/system-optimization/mi300x.rst +++ b/docs/how-to/system-optimization/mi300x.rst @@ -299,7 +299,7 @@ For a system that has AMD host CPUs add this to ``GRUB_CMDLINE_LINUX``: .. code-block:: text - amd_iommu=on iommu=pt + iommu=pt Otherwise, if the system has Intel host CPUs add this instead to ``GRUB_CMDLINE_LINUX``: @@ -500,7 +500,7 @@ If SMT is enabled by setting ``CCD/Core/Thread Enablement > SMT Control`` to #. In the server BIOS, set IOMMU to ``Enabled``. -#. When configuring the GRUB boot loader, add the following arguments for the Linux kernel: ``amd_iommu=on iommu=pt``. +#. When configuring the GRUB boot loader, add the following argument for the Linux kernel: ``iommu=pt``. #. Update GRUB. @@ -515,7 +515,7 @@ If SMT is enabled by setting ``CCD/Core/Thread Enablement > SMT Control`` to .. code-block:: shell [...] - [ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt + [ 0.000000] Kernel command line: [...] iommu=pt [...] Once the system is properly configured, ROCm software can be diff --git a/docs/how-to/system-optimization/w6000-v620.md b/docs/how-to/system-optimization/w6000-v620.md index 2a6d22b61..9c195720d 100644 --- a/docs/how-to/system-optimization/w6000-v620.md +++ b/docs/how-to/system-optimization/w6000-v620.md @@ -111,7 +111,7 @@ sudo virsh net-start default /*to enable Virtual network by default Enable input-output memory management unit (IOMMU) in GRUB settings by adding the following line to `/etc/default/grub`: ```none -GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on" for AMD CPU +GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" for AMD CPU ``` Update grub and reboot diff --git a/docs/index.md b/docs/index.md index 49977c59a..8513180ab 100644 --- a/docs/index.md +++ b/docs/index.md @@ -46,6 +46,7 @@ ROCm documentation is organized into the following categories: * [Fine-tuning LLMs and inference optimization](./how-to/llm-fine-tuning-optimization/index.rst) * [System optimization](./how-to/system-optimization/index.rst) * [AMD Instinct MI300X tuning guides](./how-to/tuning-guides/mi300x/index.rst) +* [GPU cluster networking](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html) * [System debugging](./how-to/system-debugging.md) * [Using MPI](./how-to/gpu-enabled-mpi.rst) * [Using advanced compiler features](./conceptual/compiler-topics.md) @@ -63,7 +64,7 @@ ROCm documentation is organized into the following categories: * [Using CMake](./conceptual/cmake-packages.rst) * [ROCm & PCIe atomics](./conceptual/More-about-how-ROCm-uses-PCIe-Atomics.rst) * [Inception v3 with PyTorch](./conceptual/ai-pytorch-inception.md) -* [Inference optimization with MIGraphX](./conceptual/ai-migraphx-optimization.md) +* [Oversubscription of hardware resources](./conceptual/oversubscription.rst) ::: diff --git a/docs/reference/gpu-arch-specs.rst b/docs/reference/gpu-arch-specs.rst index 90fe32a13..18d3b1a6b 100644 --- a/docs/reference/gpu-arch-specs.rst +++ b/docs/reference/gpu-arch-specs.rst @@ -7,6 +7,8 @@ Accelerator and GPU hardware specifications The following tables provide an overview of the hardware specifications for AMD Instinct™ accelerators, and AMD Radeon™ PRO and Radeon™ GPUs. +For more information about ROCm hardware compatibility, see the ROCm `Compatibility matrix `_. + .. tab-set:: .. tab-item:: AMD Instinct accelerators diff --git a/docs/reference/precision-support.rst b/docs/reference/precision-support.rst index 6155cc837..fadd4b7d1 100644 --- a/docs/reference/precision-support.rst +++ b/docs/reference/precision-support.rst @@ -41,6 +41,8 @@ together with their corresponding HIP type and a short description. - ``int64_t``, ``uint64_t`` - A signed or unsigned 64-bit integer +.. _precision_support_floating_point_types: + Floating-point types ========================================== diff --git a/docs/sphinx/_toc.yml.in b/docs/sphinx/_toc.yml.in index 2fa739e0c..9dd8af346 100644 --- a/docs/sphinx/_toc.yml.in +++ b/docs/sphinx/_toc.yml.in @@ -77,9 +77,11 @@ subtrees: title: System tuning - file: how-to/tuning-guides/mi300x/workload.rst title: Workload tuning - - file: how-to/system-debugging.md + - url: https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/${branch}/index.html + title: GPU cluster networking - file: how-to/gpu-enabled-mpi.rst title: Using MPI + - file: how-to/system-debugging.md - file: conceptual/compiler-topics.md title: Using advanced compiler features subtrees: @@ -156,8 +158,8 @@ subtrees: title: ROCm & PCIe atomics - file: conceptual/ai-pytorch-inception.md title: Inception v3 with PyTorch - - file: conceptual/ai-migraphx-optimization.md - title: Inference optimization with MIGraphX + - file: conceptual/oversubscription.rst + title: Oversubscription of hardware resources - caption: Reference entries: @@ -173,16 +175,13 @@ subtrees: - caption: Contribute entries: - file: contribute/contributing.md - title: Contribute to ROCm docs + title: Contributing to the ROCm docmentation subtrees: - entries: - - file: contribute/doc-structure.md - title: Documentation structure - file: contribute/toolchain.md - title: Documentation toolchain + title: ROCm documentation toolchain - file: contribute/building.md - title: Build our documentation - file: contribute/feedback.md - title: Provide feedback + title: Providing feedback about the ROCm documentation - file: about/license.md - title: ROCm license + title: ROCm licenses diff --git a/docs/sphinx/requirements.in b/docs/sphinx/requirements.in index 10d473385..ed4e04035 100644 --- a/docs/sphinx/requirements.in +++ b/docs/sphinx/requirements.in @@ -1,2 +1,2 @@ -rocm-docs-core==1.8.1 +rocm-docs-core==1.8.3 sphinx-reredirects diff --git a/docs/sphinx/requirements.txt b/docs/sphinx/requirements.txt index 39b60b7db..94f42069b 100644 --- a/docs/sphinx/requirements.txt +++ b/docs/sphinx/requirements.txt @@ -92,7 +92,7 @@ requests==2.32.3 # via # pygithub # sphinx -rocm-docs-core==1.8.1 +rocm-docs-core==1.8.3 # via -r requirements.in smmap==5.0.1 # via gitdb diff --git a/tools/autotag/components.xml b/tools/autotag/components.xml index 89e049b89..a151b8628 100644 --- a/tools/autotag/components.xml +++ b/tools/autotag/components.xml @@ -1,7 +1,7 @@ - diff --git a/tools/autotag/templates/highlights/5.3.0.md b/tools/autotag/templates/highlights/5.3.0.md index f193dba24..564c00395 100644 --- a/tools/autotag/templates/highlights/5.3.0.md +++ b/tools/autotag/templates/highlights/5.3.0.md @@ -184,5 +184,5 @@ clinfo, and HelloWord.cl and cause a system crash. * IRQ remapping does not support X2APIC mode * NMI error -Workaround: To avoid the system crash, add `amd_iommu=on iommu=pt` as the kernel bootparam, as +Workaround: To avoid the system crash, add `iommu=pt` as the kernel bootparam, as indicated in the warning message. From 34c0266496a5485b077a23483fab49f574bc7ae8 Mon Sep 17 00:00:00 2001 From: Peter Park Date: Wed, 6 Nov 2024 12:42:55 -0500 Subject: [PATCH 05/74] 6.2.4 release notes: add known/fixed issues (#193) * add "for compute workloads" wording for clarity * add AMDSMI resolved issue * add dlm known issue intro text wording * update wording rm bullet point update wording * fix spellcheck due to spacing * rm s * rm gfx1151 * remove dlm known issue * update list of updated docs; note for Radeon users fmt * update GA date for 6.2.4 * fix rdc version --- RELEASE.md | 73 ++++++++----------- docs/conf.py | 2 +- .../templates/extra_components/6.2.4.md | 9 +++ tools/autotag/templates/highlights/6.2.4.md | 27 ++++--- 4 files changed, 60 insertions(+), 51 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 935d6997b..726a599d4 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -15,8 +15,8 @@ The release notes provide a summary of notable changes since the previous ROCm r - [ROCm upcoming changes](#rocm-upcoming-changes) ```{note} -If you’re using Radeon™ PRO or Radeon GPUs for graphics workloads, -continue to use ROCm 6.2.3. See the [Use ROCm on Radeon +If you’re using Radeon™ PRO or Radeon GPUs in a workstation setting with a +display connected, continue to use ROCm 6.2.3. See the [Use ROCm on Radeon GPUs](https://rocm.docs.amd.com/projects/radeon/en/latest/index.html) documentation to verify compatibility and system requirements. ``` @@ -33,7 +33,6 @@ a wider variety of user needs and use cases. * Added a new GPU cluster networking guide. See [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html). - This documentation provides guidelines on validating network configurations in single-node and multi-node environments to attain optimal speed and bandwidth in AMD Instinct-powered clusters. @@ -46,9 +45,17 @@ a wider variety of user needs and use cases. * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hip_porting_driver_api.html) section. +* Updated the [Post-installation instructions](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.4/install/post-install.html) + with guidance on using the `update-alternatives` utility and environment modules to help you manage multiple ROCm + versions and streamline PATH configuration. + +* Updated the [LLM inference performance validation on AMD Instinct + MI300X](https://rocm.docs.amd.com/en/docs-6.2.4/how-to/performance-validation/mi300x/vllm-benchmark.html) + documentation with more detailed guidance, new models, and the `float8` data type. + ## Operating system and hardware support changes -ROCm 6.2.4 adds support for the [AMD Radeon PRO V710](https://www.amd.com/en/products/accelerators/radeon-pro/amd-radeon-pro-v710.html) GPU. See +ROCm 6.2.4 adds support for the [AMD Radeon PRO V710](https://www.amd.com/en/products/accelerators/radeon-pro/amd-radeon-pro-v710.html) GPU for compute workloads. See [Supported GPUs](https://advanced-micro-devices-demo--287.com.readthedocs.build/projects/install-on-linux-internal/en/287/reference/system-requirements.html) for more information. @@ -154,7 +161,7 @@ Click the component's updated version to go to a detailed list of its changes. C hipFFT - 1.0.15 ⇒ 1.0.16 + 1.0.15 ⇒ 1.0.16 @@ -166,7 +173,7 @@ Click the component's updated version to go to a detailed list of its changes. C hipRAND - 2.11.0 ⇒ 2.11.1 + 2.11.0 ⇒ 2.11.1 @@ -190,13 +197,13 @@ Click the component's updated version to go to a detailed list of its changes. C rocALUTION - 3.2.0 ⇒ 3.2.1 + 3.2.0 ⇒ 3.2.1 rocBLAS - 4.2.1 ⇒ 4.2.4 + 4.2.1 ⇒ 4.2.4 @@ -208,7 +215,7 @@ Click the component's updated version to go to a detailed list of its changes. C rocRAND - 3.1.0 ⇒ 3.1.1 + 3.1.0 ⇒ 3.1.1 @@ -220,7 +227,7 @@ Click the component's updated version to go to a detailed list of its changes. C rocSPARSE - 3.2.0 ⇒ 3.2.1 + 3.2.0 ⇒ 3.2.1 @@ -242,7 +249,7 @@ Click the component's updated version to go to a detailed list of its changes. C Primitives hipCUB - 3.2.0 ⇒ 3.2.1 + 3.2.0 ⇒ 3.2.1 @@ -254,13 +261,13 @@ Click the component's updated version to go to a detailed list of its changes. C rocPRIM - 3.2.1 ⇒ 3.2.2 + 3.2.1 ⇒ 3.2.2 rocThrust - 3.1.0 ⇒ 3.1.1 + 3.1.0 ⇒ 3.1.1 @@ -270,7 +277,7 @@ Click the component's updated version to go to a detailed list of its changes. C Tools System management AMD SMI - 24.6.3 + 24.6.3 ⇒ 24.6.3 @@ -282,7 +289,7 @@ Click the component's updated version to go to a detailed list of its changes. C ROCm Data Center Tool - 1.0.0 + 0.3.0 @@ -413,34 +420,16 @@ Click the component's updated version to go to a detailed list of its changes. C The following sections describe key changes to ROCm components. -### Hardware architecture support updates +### **AMD SMI** (24.6.3) -Updated the following math and primitives libraries to pre-enable support for -an upcoming hardware architecture. +#### Resolved issues -* hipCUB (3.2.1) +* Fixed support for the API calls `amdsmi_get_gpu_process_isolation` and + `amdsmi_clean_gpu_local_data`, along with the `amd-smi set + --process-isolation <0 or 1>` command. See issue + [#3500](https://github.com/ROCm/ROCm/issues/3500) on GitHub. -* hipFFT (1.0.16) - -* hipRAND (2.11.1) - -* rocALUTION (3.2.1) - -* rocBLAS (4.2.4) - -* rocFFT (1.0.30) - -* rocPRIM (3.2.2) - -* rocRAND (3.1.1) - -* rocSOLVER (3.26.2) - -* rocSPARSE (3.2.1) - -* rocThrust (3.1.1) - -### **rocFFT** (1.0.30)[*](#hardware-architecture-support-updates) +### **rocFFT** (1.0.30) #### Optimized @@ -450,7 +439,7 @@ an upcoming hardware architecture. * Fixed plan creation failure on some even-length real-complex transforms that use Bluestein's algorithm. -### **rocSOLVER** (3.26.2)[*](#hardware-architecture-support-updates) +### **rocSOLVER** (3.26.2) #### Resolved issues @@ -459,6 +448,8 @@ an upcoming hardware architecture. ## ROCm known issues ROCm known issues are tracked on [GitHub](https://github.com/ROCm/ROCm/labels/Verified%20Issue). +Known issues related to individual components are listed in the [Detailed component changes](#detailed-component-changes) +section. ## ROCm upcoming changes diff --git a/docs/conf.py b/docs/conf.py index cca8f0ad3..fe3f22199 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -38,7 +38,7 @@ all_article_info_author = "" # pages with specific settings article_pages = [ - {"file": "about/release-notes", "os": ["linux", "windows"], "date": "2024-10-18"}, + {"file": "about/release-notes", "os": ["linux", "windows"], "date": "2024-11-06"}, {"file": "how-to/deep-learning-rocm", "os": ["linux"]}, {"file": "how-to/rocm-for-ai/index", "os": ["linux"]}, {"file": "how-to/rocm-for-ai/install", "os": ["linux"]}, diff --git a/tools/autotag/templates/extra_components/6.2.4.md b/tools/autotag/templates/extra_components/6.2.4.md index 4dee22b54..37f6d2962 100644 --- a/tools/autotag/templates/extra_components/6.2.4.md +++ b/tools/autotag/templates/extra_components/6.2.4.md @@ -24,3 +24,12 @@ an upcoming hardware architecture. * rocSPARSE (3.2.1) * rocThrust (3.1.1) + +### **AMD SMI** (24.6.3) + +#### Resolved issues + +* Fixed support for the API calls `amdsmi_get_gpu_process_isolation` and + `amdsmi_clean_gpu_local_data`, along with the + `amd-smi set --process-isolation <0 or 1>` command. See issue + [#3500](https://github.com/ROCm/ROCm/issues/3500) on GitHub. diff --git a/tools/autotag/templates/highlights/6.2.4.md b/tools/autotag/templates/highlights/6.2.4.md index da45a17d3..4d8af7762 100644 --- a/tools/autotag/templates/highlights/6.2.4.md +++ b/tools/autotag/templates/highlights/6.2.4.md @@ -1,4 +1,6 @@ -These release notes provide a summary of notable changes since the previous ROCm release. +# ROCm 6.2.4 release notes + +The release notes provide a summary of notable changes since the previous ROCm release. - [Release highlights](#release-highlights) @@ -13,9 +15,8 @@ These release notes provide a summary of notable changes since the previous ROCm - [ROCm upcoming changes](#rocm-upcoming-changes) ```{note} -ROCm 6.2.3 is supported on systems using AMD Radeon™ or Radeon PRO workstation -GPUs for graphics workloads. If you’re using ROCm in this context, use ROCm -version 6.2.3. See the [Use ROCm on Radeon +If you’re using Radeon™ PRO or Radeon GPUs in a workstation setting with a +display connected, continue to use ROCm 6.2.3. See the [Use ROCm on Radeon GPUs](https://rocm.docs.amd.com/projects/radeon/en/latest/index.html) documentation to verify compatibility and system requirements. ``` @@ -30,8 +31,8 @@ The following are notable new features and improvements in ROCm 6.2.4. For chang ROCm documentation continues to be updated to provide clearer and more comprehensive guidance for a wider variety of user needs and use cases. -* Added a new GPU cluster networking topic. See - [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/docs-6.2.4/index.html). +* Added a new GPU cluster networking guide. See + [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html). This documentation provides guidelines on validating network configurations in single-node and multi-node environments to attain optimal speed and bandwidth @@ -39,8 +40,16 @@ a wider variety of user needs and use cases. * Updated the HIP runtime documentation. - * Added a new topic on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hipgraph.html). + * Added a new section on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hipgraph.html). - * Added a new topic about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/stream_ordered_allocator.html). + * Added a new section about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/stream_ordered_allocator.html). - * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hip_porting_driver_api.html) topic. + * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hip_porting_driver_api.html) section. + +* Updated the [Post-installation instructions](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.4/install/post-install.html) + with guidance on using the `update-alternatives` utility and environment modules to help you manage multiple ROCm + versions and streamline PATH configuration. + +* Updated [LLM inference performance validation on AMD Instinct + MI300X](https://rocm.docs.amd.com/en/docs-6.2.4/how-to/performance-validation/mi300x/vllm-benchmark.html) + documentation with more detailed guidance, new models, and the `float8` data type. From 47cda1bc705828048ae54ea1cc48da37947c644d Mon Sep 17 00:00:00 2001 From: Peter Park Date: Wed, 6 Nov 2024 13:51:36 -0500 Subject: [PATCH 06/74] fix RDC version strings (#196) --- docs/compatibility/compatibility-matrix-historical-6.0.csv | 2 +- docs/compatibility/compatibility-matrix.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/compatibility/compatibility-matrix-historical-6.0.csv b/docs/compatibility/compatibility-matrix-historical-6.0.csv index 942e173b6..1af1eca26 100644 --- a/docs/compatibility/compatibility-matrix-historical-6.0.csv +++ b/docs/compatibility/compatibility-matrix-historical-6.0.csv @@ -82,7 +82,7 @@ ROCm Version,6.2.4,6.2.2,6.2.1,6.2.0, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0 ,,,,,,,,, SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix-past-60:,,,,,,,, :doc:`AMD SMI `,24.6.3,24.6.3,24.6.3,24.6.2,24.5.1,24.5.1,24.4.1,23.4.2,23.4.2 - :doc:`ROCm Data Center Tool `,1.0.0,1.0.0,1.0.0,1.0.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0 + :doc:`ROCm Data Center Tool `,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0 :doc:`rocminfo `,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0 :doc:`ROCm SMI `,7.3.0,7.3.0,7.3.0,7.3.0,7.2.0,7.0.0,7.0.0,6.0.2,6.0.0 :doc:`ROCm Validation Suite `,rocm-6.2.4,rocm-6.2.2,rocm-6.2.1,rocm-6.2.0,rocm-6.1.2,rocm-6.1.1,rocm-6.1.0,rocm-6.0.2,rocm-6.0.0 diff --git a/docs/compatibility/compatibility-matrix.rst b/docs/compatibility/compatibility-matrix.rst index 873d47192..40a301437 100644 --- a/docs/compatibility/compatibility-matrix.rst +++ b/docs/compatibility/compatibility-matrix.rst @@ -105,7 +105,7 @@ Accelerators and GPUs listed in the following table support compute workloads (n ,,, SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix:,, :doc:`AMD SMI `,24.6.3,24.6.3,24.4.1 - :doc:`ROCm Data Center Tool `,1.0.0,1.0.0,0.3.0 + :doc:`ROCm Data Center Tool `,0.3.0,0.3.0,0.3.0 :doc:`rocminfo `,1.0.0,1.0.0,1.0.0 :doc:`ROCm SMI `,7.3.0,7.3.0,7.0.0 :doc:`ROCm Validation Suite `,rocm-6.2.4,rocm-6.2.2,rocm-6.1.0 From 48d2d1656312515c3034afea06bf815c820bfa23 Mon Sep 17 00:00:00 2001 From: Jeffrey Novotny Date: Thu, 7 Nov 2024 12:33:28 -0500 Subject: [PATCH 07/74] Remove deprecated architectures from LLVM targets in GPU architecture specs (#198) --- docs/reference/gpu-arch-specs.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/gpu-arch-specs.rst b/docs/reference/gpu-arch-specs.rst index 18d3b1a6b..536f83f2f 100644 --- a/docs/reference/gpu-arch-specs.rst +++ b/docs/reference/gpu-arch-specs.rst @@ -35,7 +35,7 @@ For more information about ROCm hardware compatibility, see the ROCm `Compatibil * - MI300X - CDNA3 - - gfx941 or gfx942 + - gfx942 - 192 - 304 - 64 @@ -50,7 +50,7 @@ For more information about ROCm hardware compatibility, see the ROCm `Compatibil * - MI300A - CDNA3 - - gfx940 or gfx942 + - gfx942 - 128 - 228 - 64 From b7ecf6d5529aac1ec500296540c567f1ca0bfbef Mon Sep 17 00:00:00 2001 From: Peter Park Date: Thu, 7 Nov 2024 18:01:26 -0500 Subject: [PATCH 08/74] Rename Omnitools to ROCm Compute/Systems Profiler (#183) * rename Omniperf and Omnitrace * rename labels rename more labels * update licenses and rocm-tools.md * fix rocprof-sys ref --- docs/about/license.md | 4 +- docs/compatibility/compatibility-matrix.rst | 6 +-- ...lysis.png => rocprof-compute-analysis.png} | Bin ...eline.png => rocprof-systems-timeline.png} | Bin .../profiling-and-debugging.rst | 4 +- docs/how-to/tuning-guides/mi300x/workload.rst | 48 +++++++++--------- docs/reference/rocm-tools.md | 4 +- docs/what-is-rocm.rst | 4 +- 8 files changed, 35 insertions(+), 35 deletions(-) rename docs/data/how-to/tuning-guides/{omniperf-analysis.png => rocprof-compute-analysis.png} (100%) rename docs/data/how-to/tuning-guides/{omnitrace-timeline.png => rocprof-systems-timeline.png} (100%) diff --git a/docs/about/license.md b/docs/about/license.md index cfa637b73..e78f0f963 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -52,8 +52,6 @@ additional licenses. Please review individual repositories for more information. | [MIGraphX](https://github.com/ROCm/AMDMIGraphX/) | [MIT](https://github.com/ROCm/AMDMIGraphX/blob/develop/LICENSE) | | [MIOpen](https://github.com/ROCm/MIOpen/) | [MIT](https://github.com/ROCm/MIOpen/blob/develop/LICENSE.txt) | | [MIVisionX](https://github.com/ROCm/MIVisionX/) | [MIT](https://github.com/ROCm/MIVisionX/blob/develop/LICENSE.txt) | -| [Omniperf](https://github.com/ROCm/omniperf) | [MIT](https://github.com/ROCm/omniperf/blob/main/LICENSE) | -| [Omnitrace](https://github.com/ROCm/omnitrace) | [MIT](https://github.com/ROCm/omnitrace/blob/main/LICENSE) | | [rocAL](https://github.com/ROCm/rocAL) | [MIT](https://github.com/ROCm/rocAL/blob/develop/LICENSE.txt) | | [rocALUTION](https://github.com/ROCm/rocALUTION/) | [MIT](https://github.com/ROCm/rocALUTION/blob/develop/LICENSE.md) | | [rocBLAS](https://github.com/ROCm/rocBLAS/) | [MIT](https://github.com/ROCm/rocBLAS/blob/develop/LICENSE.md) | @@ -67,11 +65,13 @@ additional licenses. Please review individual repositories for more information. | [ROCm CMake](https://github.com/ROCm/rocm-cmake/) | [MIT](https://github.com/ROCm/rocm-cmake/blob/develop/LICENSE) | | [ROCm Communication Collectives Library (RCCL)](https://github.com/ROCm/rccl/) | [Custom](https://github.com/ROCm/rccl/blob/develop/LICENSE.txt) | | [ROCm-Core](https://github.com/ROCm/rocm-core) | [MIT](https://github.com/ROCm/rocm-core/blob/master/copyright) | +| [ROCm Compute Profiler](https://github.com/ROCm/rocprofiler-compute) | [MIT](https://github.com/ROCm/rocprofiler-compute/blob/amd-staging/LICENSE) | | [ROCm Data Center (RDC)](https://github.com/ROCm/rdc/) | [MIT](https://github.com/ROCm/rdc/blob/develop/LICENSE) | | [ROCm-Device-Libs](https://github.com/ROCm/llvm-project/tree/amd-staging/amd/device-libs) | [The University of Illinois/NCSA](https://github.com/ROCm/llvm-project/blob/amd-staging/amd/device-libs/LICENSE.TXT) | | [ROCm-OpenCL-Runtime](https://github.com/ROCm/clr/tree/develop/opencl) | [MIT](https://github.com/ROCm/clr/blob/develop/opencl/LICENSE.txt) | | [ROCm Performance Primitives (RPP)](https://github.com/ROCm/rpp) | [MIT](https://github.com/ROCm/rpp/blob/develop/LICENSE) | | [ROCm SMI Lib](https://github.com/ROCm/rocm_smi_lib/) | [MIT](https://github.com/ROCm/rocm_smi_lib/blob/develop/License.txt) | +| [ROCm Systems Profiler](https://github.com/ROCm/rocprofiler-systems) | [MIT](https://github.com/ROCm/rocprofiler-systems/blob/amd-staging/LICENSE) | | [ROCm Validation Suite](https://github.com/ROCm/ROCmValidationSuite/) | [MIT](https://github.com/ROCm/ROCmValidationSuite/blob/master/LICENSE) | | [rocPRIM](https://github.com/ROCm/rocPRIM/) | [MIT](https://github.com/ROCm/rocPRIM/blob/develop/LICENSE.txt) | | [ROCProfiler](https://github.com/ROCm/rocprofiler/) | [MIT](https://github.com/ROCm/rocprofiler/blob/amd-master/LICENSE) | diff --git a/docs/compatibility/compatibility-matrix.rst b/docs/compatibility/compatibility-matrix.rst index 40a301437..28ef57674 100644 --- a/docs/compatibility/compatibility-matrix.rst +++ b/docs/compatibility/compatibility-matrix.rst @@ -111,10 +111,10 @@ Accelerators and GPUs listed in the following table support compute workloads (n :doc:`ROCm Validation Suite `,rocm-6.2.4,rocm-6.2.2,rocm-6.1.0 ,,, PERFORMANCE TOOLS,,, - :doc:`Omniperf `,2.0.1,2.0.1,N/A - :doc:`Omnitrace `,1.11.2,1.11.2,N/A :doc:`ROCm Bandwidth Test `,1.4.0,1.4.0,1.4.0 - :doc:`ROCProfiler `,2.0.60204,2.0.60202,2.0.60100 + :doc:`ROCm Compute Profiler `,2.0.1,2.0.1,N/A + :doc:`ROCm Systems Profiler `,1.11.2,1.11.2,N/A + :doc:`ROCProfiler `,2.0.60202,2.0.60201,2.0.60100 :doc:`ROCprofiler-SDK `,0.4.0,0.4.0,N/A :doc:`ROCTracer `,4.1.60204,4.1.60202,4.1.60100 ,,, diff --git a/docs/data/how-to/tuning-guides/omniperf-analysis.png b/docs/data/how-to/tuning-guides/rocprof-compute-analysis.png similarity index 100% rename from docs/data/how-to/tuning-guides/omniperf-analysis.png rename to docs/data/how-to/tuning-guides/rocprof-compute-analysis.png diff --git a/docs/data/how-to/tuning-guides/omnitrace-timeline.png b/docs/data/how-to/tuning-guides/rocprof-systems-timeline.png similarity index 100% rename from docs/data/how-to/tuning-guides/omnitrace-timeline.png rename to docs/data/how-to/tuning-guides/rocprof-systems-timeline.png diff --git a/docs/how-to/llm-fine-tuning-optimization/profiling-and-debugging.rst b/docs/how-to/llm-fine-tuning-optimization/profiling-and-debugging.rst index 6ee209fc8..cc3c9ebb5 100644 --- a/docs/how-to/llm-fine-tuning-optimization/profiling-and-debugging.rst +++ b/docs/how-to/llm-fine-tuning-optimization/profiling-and-debugging.rst @@ -22,8 +22,8 @@ as well as other profiling and debugging suggestions. * :ref:`ROCProfiler ` - * :ref:`Omniperf ` + * :ref:`ROCm Compute Profiler ` - * :ref:`Omnitrace ` + * :ref:`ROCm Systems Profiler ` * :ref:`ROCr Debug Agent ` diff --git a/docs/how-to/tuning-guides/mi300x/workload.rst b/docs/how-to/tuning-guides/mi300x/workload.rst index 6857eae1b..9401fa0a6 100644 --- a/docs/how-to/tuning-guides/mi300x/workload.rst +++ b/docs/how-to/tuning-guides/mi300x/workload.rst @@ -67,7 +67,7 @@ When profiling indicates that GPUs are a performance bottleneck, delve deeper into kernel-level profiling. Tools such as the :ref:`ROCr Debug Agent `, :ref:`ROCProfiler `, and -:ref:`Omniperf ` offer detailed insights +:ref:`ROCm Compute Profiler ` offer detailed insights into GPU kernel execution. These tools can help isolate problematic GPU operations and provide data needed for targeted optimizations. @@ -171,9 +171,9 @@ tools available depending on their specific profiling needs. :doc:`ROCProfiler ` documentation. -* Omniperf builds upon ROCProfiler but provides more guided analysis. +* ROCm Compute Profiler builds upon ROCProfiler but provides more guided analysis. For more information, see - :doc:`Omniperf documentation `. + :doc:`ROCm Compute Profiler documentation `. Refer to :doc:`/how-to/llm-fine-tuning-optimization/profiling-and-debugging` to explore commonly used profiling tools and their usage patterns. @@ -244,9 +244,9 @@ working with AMD Instinct accelerators have multiple tools depending on their sp * :ref:`ROCProfiler ` -* :ref:`Omniperf ` +* :ref:`ROCm Compute Profiler ` -* :ref:`Omnitrace ` +* :ref:`ROCm Systems Profiler ` .. _mi300x-rocprof: @@ -271,61 +271,61 @@ ability to collect timeline traces of the accelerator software stack as well as gives the user full access and control of raw performance profiling data, but requires extra effort to analyze the collected data. -.. _mi300x-omniperf: +.. _mi300x-rocprof-compute: -Omniperf +ROCm Compute Profiler ^^^^^^^^ -:doc:`Omniperf ` is a system performance profiler for high-performance computing (HPC) and -machine learning (ML) workloads using Instinct accelerators. Under the hood, Omniperf uses -:ref:`ROCProfiler ` to collect hardware performance counters. The Omniperf tool performs +:doc:`ROCm Compute Profiler ` is a system performance profiler for high-performance computing (HPC) and +machine learning (ML) workloads using Instinct accelerators. Under the hood, ROCm Compute Profiler uses +:ref:`ROCProfiler ` to collect hardware performance counters. The ROCm Compute Profiler tool performs system profiling based on all approved hardware counters for Instinct accelerator architectures. It provides high level performance analysis features including System Speed-of-Light, IP block Speed-of-Light, Memory Chart Analysis, Roofline Analysis, Baseline Comparisons, and more. -Omniperf takes the guesswork out of profiling by removing the need to provide text input files with lists of counters -to collect and analyze raw CSV output files as is the case with ROC-profiler. Instead, Omniperf automates the collection +ROCm Compute Profiler takes the guesswork out of profiling by removing the need to provide text input files with lists of counters +to collect and analyze raw CSV output files as is the case with ROC-profiler. Instead, ROCm Compute Profiler automates the collection of all available hardware counters in one command and provides a graphical interface to help users understand and analyze bottlenecks and stressors for their computational workloads on AMD Instinct accelerators. .. note:: - Omniperf collects hardware counters in multiple passes, and will therefore re-run the application during each pass + ROCm Compute Profiler collects hardware counters in multiple passes, and will therefore re-run the application during each pass to collect different sets of metrics. -.. figure:: ../../../data/how-to/tuning-guides/omniperf-analysis.png +.. figure:: ../../../data/how-to/tuning-guides/rocprof-compute-analysis.png - Omniperf memory chat analysis panel. + ROCm Compute Profiler memory chat analysis panel. -In brief, Omniperf provides details about hardware activity for a particular GPU kernel. It also supports both +In brief, ROCm Compute Profiler provides details about hardware activity for a particular GPU kernel. It also supports both a web-based GUI or command-line analyzer, depending on your preference. -.. _mi300x-omnitrace: +.. _mi300x-rocprof-systems: -Omnitrace +ROCm Systems Profiler ^^^^^^^^^ -:doc:`Omnitrace ` is a comprehensive profiling and tracing tool for parallel applications, +:doc:`ROCm Systems Profiler ` is a comprehensive profiling and tracing tool for parallel applications, including HPC and ML packages, written in C, C++, Fortran, HIP, OpenCL, and Python which execute on the CPU or CPU and GPU. It is capable of gathering the performance information of functions through any combination of binary instrumentation, call-stack sampling, user-defined regions, and Python interpreter hooks. -Omnitrace supports interactive visualization of comprehensive traces in the web browser in addition to high-level +ROCm Systems Profiler supports interactive visualization of comprehensive traces in the web browser in addition to high-level summary profiles with ``mean/min/max/stddev`` statistics. Beyond runtime -information, Omnitrace supports the collection of system-level metrics such as CPU frequency, GPU temperature, and GPU +information, ROCm Systems Profiler supports the collection of system-level metrics such as CPU frequency, GPU temperature, and GPU utilization. Process and thread level metrics such as memory usage, page faults, context switches, and numerous other hardware counters are also included. .. tip:: When analyzing the performance of an application, it is best not to assume you know where the performance - bottlenecks are and why they are happening. Omnitrace is the ideal tool for characterizing where optimization would + bottlenecks are and why they are happening. ROCm Systems Profiler is the ideal tool for characterizing where optimization would have the greatest impact on the end-to-end execution of the application and to discover what else is happening on the system during a performance bottleneck. -.. figure:: ../../../data/how-to/tuning-guides/omnitrace-timeline.png +.. figure:: ../../../data/how-to/tuning-guides/rocprof-systems-timeline.png - Omnitrace timeline trace example. + ROCm Systems Profiler timeline trace example. For details usage and examples of using these tools, refer to the `Introduction to profiling tools for AMD hardware `_ diff --git a/docs/reference/rocm-tools.md b/docs/reference/rocm-tools.md index fca21b892..3b0dfa6d8 100644 --- a/docs/reference/rocm-tools.md +++ b/docs/reference/rocm-tools.md @@ -29,9 +29,9 @@ :::{grid-item-card} Performance :class-body: rocm-card-banner rocm-hue-6 -* {doc}`Omniperf ` -* {doc}`Omnitrace ` * {doc}`ROCm Bandwidth Test ` +* {doc}`ROCm Compute Profiler ` +* {doc}`ROCm Systems Profiler ` * {doc}`ROCProfiler ` * {doc}`ROCprofiler-SDK ` * {doc}`ROCTracer ` diff --git a/docs/what-is-rocm.rst b/docs/what-is-rocm.rst index 4fca8fd16..e421a5a5c 100644 --- a/docs/what-is-rocm.rst +++ b/docs/what-is-rocm.rst @@ -113,9 +113,9 @@ Performance .. csv-table:: :header: "Component", "Description" - ":doc:`Omniperf `", "System performance profiling tool for machine learning and HPC workloads" - ":doc:`Omnitrace `", "Comprehensive profiling and tracing tool for HIP applications" ":doc:`ROCm Bandwidth Test `", "Captures the performance characteristics of buffer copying and kernel read/write operations" + ":doc:`ROCm Compute Profiler `", "System performance profiling tool for machine learning and HPC workloads" + ":doc:`ROCm Systems Profiler `", "Comprehensive profiling and tracing tool for HIP applications" ":doc:`ROCProfiler `", "Profiling tool for HIP applications" ":doc:`ROCprofiler-SDK `", "Toolkit for developing analysis tools for profiling and tracing GPU compute applications. This toolkit is in beta and subject to change" ":doc:`ROCTracer `", "Intercepts runtime API calls and traces asynchronous activity" From c9754cb9d8c1217b0776053e7b5cb7bf95339c6d Mon Sep 17 00:00:00 2001 From: Sam Wu <22262939+samjwu@users.noreply.github.com> Date: Fri, 8 Nov 2024 15:46:04 -0700 Subject: [PATCH 09/74] Test Sphinx Sitemap (#162) * Use sphinx-sitemap * Update baseurl for sphinx-sitemap * Regenerate doc reqs --- docs/conf.py | 4 +-- docs/sphinx/requirements.in | 1 + docs/sphinx/requirements.txt | 47 +++++++++++++++++++----------------- 3 files changed, 28 insertions(+), 24 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index fe3f22199..89c5ab964 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -95,11 +95,11 @@ article_pages = [ external_toc_path = "./sphinx/_toc.yml" -extensions = ["rocm_docs", "sphinx_reredirects"] +extensions = ["rocm_docs", "sphinx_reredirects", "sphinx_sitemap"] external_projects_current_project = "rocm" -html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "rocm-stg.amd.com") +html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "https://rocm-stg.amd.com/") html_context = {} if os.environ.get("READTHEDOCS", "") == "True": html_context["READTHEDOCS"] = True diff --git a/docs/sphinx/requirements.in b/docs/sphinx/requirements.in index ed4e04035..9ad44c267 100644 --- a/docs/sphinx/requirements.in +++ b/docs/sphinx/requirements.in @@ -1,2 +1,3 @@ rocm-docs-core==1.8.3 sphinx-reredirects +sphinx-sitemap diff --git a/docs/sphinx/requirements.txt b/docs/sphinx/requirements.txt index 94f42069b..05e0ea517 100644 --- a/docs/sphinx/requirements.txt +++ b/docs/sphinx/requirements.txt @@ -6,9 +6,9 @@ # accessible-pygments==0.0.5 # via pydata-sphinx-theme -alabaster==0.7.16 +alabaster==1.0.0 # via sphinx -babel==2.15.0 +babel==2.16.0 # via # pydata-sphinx-theme # sphinx @@ -16,9 +16,9 @@ beautifulsoup4==4.12.3 # via pydata-sphinx-theme breathe==4.35.0 # via rocm-docs-core -certifi==2024.7.4 +certifi==2024.8.30 # via requests -cffi==1.16.0 +cffi==1.17.1 # via # cryptography # pynacl @@ -42,7 +42,7 @@ gitdb==4.0.11 # via gitpython gitpython==3.1.43 # via rocm-docs-core -idna==3.7 +idna==3.10 # via requests imagesize==1.4.1 # via sphinx @@ -54,13 +54,13 @@ markdown-it-py==3.0.0 # via # mdit-py-plugins # myst-parser -markupsafe==2.1.5 +markupsafe==3.0.0 # via jinja2 -mdit-py-plugins==0.4.1 +mdit-py-plugins==0.4.2 # via myst-parser mdurl==0.1.2 # via markdown-it-py -myst-parser==3.0.1 +myst-parser==4.0.0 # via rocm-docs-core packaging==24.1 # via @@ -72,18 +72,18 @@ pydata-sphinx-theme==0.15.4 # via # rocm-docs-core # sphinx-book-theme -pygithub==2.3.0 +pygithub==2.4.0 # via rocm-docs-core pygments==2.18.0 # via # accessible-pygments # pydata-sphinx-theme # sphinx -pyjwt[crypto]==2.8.0 +pyjwt[crypto]==2.9.0 # via pygithub pynacl==1.5.0 # via pygithub -pyyaml==6.0.1 +pyyaml==6.0.2 # via # myst-parser # rocm-docs-core @@ -98,9 +98,9 @@ smmap==5.0.1 # via gitdb snowballstemmer==2.2.0 # via sphinx -soupsieve==2.5 +soupsieve==2.6 # via beautifulsoup4 -sphinx==7.3.7 +sphinx==8.0.2 # via # breathe # myst-parser @@ -112,37 +112,40 @@ sphinx==7.3.7 # sphinx-external-toc # sphinx-notfound-page # sphinx-reredirects + # sphinx-sitemap sphinx-book-theme==1.1.3 # via rocm-docs-core sphinx-copybutton==0.5.2 # via rocm-docs-core -sphinx-design==0.6.0 +sphinx-design==0.6.1 # via rocm-docs-core sphinx-external-toc==1.0.1 # via rocm-docs-core -sphinx-notfound-page==1.0.2 +sphinx-notfound-page==1.0.4 # via rocm-docs-core sphinx-reredirects==0.1.5 # via -r requirements.in -sphinxcontrib-applehelp==1.0.8 +sphinx-sitemap==2.6.0 + # via -r requirements.in +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.6 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.5 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.7 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.10 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx -tomli==2.0.1 +tomli==2.0.2 # via sphinx typing-extensions==4.12.2 # via # pydata-sphinx-theme # pygithub -urllib3==2.2.2 +urllib3==2.2.3 # via # pygithub # requests From c1f0a8d9d976e6864d82af0bffbe2dcad9fce973 Mon Sep 17 00:00:00 2001 From: srawat <120587655+SwRaw@users.noreply.github.com> Date: Sat, 9 Nov 2024 04:16:59 +0530 Subject: [PATCH 10/74] Adding TransferBench into list of components (#185) --- docs/reference/rocm-tools.md | 1 + docs/what-is-rocm.rst | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/reference/rocm-tools.md b/docs/reference/rocm-tools.md index 3b0dfa6d8..0b2f92b53 100644 --- a/docs/reference/rocm-tools.md +++ b/docs/reference/rocm-tools.md @@ -35,6 +35,7 @@ * {doc}`ROCProfiler ` * {doc}`ROCprofiler-SDK ` * {doc}`ROCTracer ` +* {doc}`TransferBench ` ::: (development-tools)= diff --git a/docs/what-is-rocm.rst b/docs/what-is-rocm.rst index e421a5a5c..b95f3b153 100644 --- a/docs/what-is-rocm.rst +++ b/docs/what-is-rocm.rst @@ -119,6 +119,7 @@ Performance ":doc:`ROCProfiler `", "Profiling tool for HIP applications" ":doc:`ROCprofiler-SDK `", "Toolkit for developing analysis tools for profiling and tracing GPU compute applications. This toolkit is in beta and subject to change" ":doc:`ROCTracer `", "Intercepts runtime API calls and traces asynchronous activity" + ":doc:`TransferBench `", "Utility to benchmark simultaneous transfers between user-specified devices (CPUs or GPUs)" Development ^^^^^^^^^^^ From 0a237dfd42c8927e7e46a8ef1ec5ca1d606117ca Mon Sep 17 00:00:00 2001 From: alexxu-amd <159800977+alexxu-amd@users.noreply.github.com> Date: Thu, 14 Nov 2024 13:14:37 -0500 Subject: [PATCH 11/74] Sync develop from external repo (#205) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update version list with 6.2.0 (#3505) (#3506) * Fix link to meta-llama finetuning recipes * Spellcheck fixes in release notes templates (#3526) (#3548) * fix spelling in 5.4.x templates * add to wordlist * update templates update wordlist * remove extra_components rm extra_components * fix spelling Co-authored-by: Peter Park * Fix link to rocr debug agent (#3533) Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> * Fix intersphinx links (#3546) * update fw install links * fix more intersphinx links * fix more links * add rocPyDecode repo to ROCm6.2 manifest file (#3541) (#3553) Co-authored-by: Yanyao Wang Co-authored-by: Wang, Yanyao * Fix typo for TFLOPs metric in MI250 architecture page * Add rocm-examples to default.xml (#3583) * Add rocm 6.2.0 manifest file for rocm-build scripts (#3538) * Add rocm 6.2.0 manifest file for rocm-build scripts Signed-off-by: David Galiffi * Add "rocm-examples" --------- Signed-off-by: David Galiffi * Add a section on increasing memory allocation to the MI300A system op… (#3587) * Add a section on increasing memory allocation to the MI300A system optimization guide * Addition to wordlist * Change GB to GiB for consistency * Standardize GiB/KiB spacing * Minor wording changes * Update build scripts for ROCm6.2 release * fix README.md for Ubuntu24 docker * Correct ttm to amdttm (#3648) * Expand the section on changing thread affinity (#3653) * Expand the section on changing thread affinity * Clarify the methods for configuring allocatable memory settings * Small correction * Update model-quantization.rst to import `BitsAndBytesConfig` from transformers library (#3638) * remove unneeded file (#3663) * Fix intersphinx links (#3668) * fix links in install.rst * fix links in sys opt guides * Add introduction and links to the new guide to the vLLM optimized Doc… (#3637) * Add introduction and links to the new guide to the vLLM optimized Docker image on AMD Infinity Hub * Update target link for the Docker vLLM guide * Change target URL * Change link target URL again * Fixed broken link to RISC-V documentation * Add FBGEMM/FBGEMM_GPU to the Model acceleration libraries page (#3659) * Add FBGEMM/FBGEMM_GPU to the Model acceleration libraries page * Add words to wordlist and fix a typo * Add new sections for Docker and testing * Incorporate comments from the external review * Some minor edits and clarifications * Incorporate further review coments and fix test section * Add comment to test section * Change git clone command for FBGEMM repo * Change Docker command * Changes from internal review * Fix linting issue * Fixed broken links for tensile, rocprofiler, roctracer, hipify, rocm-cmake * add missing make command to bitsandbytes install commands (#3722) * Update link to rocRAND data type support (#3736) * Fix Radeon link and point at R6.1.3 as absolute link (#3757) * Fix Radeon link and point at R6.1.3 as absolute link (#3757) * Include rocal version change in the highlights (#177) * Include rocal version change in the highlights * Reworded rocal known issues and added link to rocal in highlights * Update ROCm manifest to 6.2.1 * Update ROCm branch name * Add 6.2.1 to version list (#3770) * Add links to GH issues in 6.2.1 release notes (#3769) * add MAD page * link to GitHub issues in release notes known issues * update templates for 6.2.1 * Revert "add MAD page" This reverts commit 9cce72bba306286c7eb317d592645d4e0e1b27aa. * update wordlist for spellcheck linter * add rccl note * update rocal version change heading to be more obvious * make rocal note more specific * fix missing space * fix capitalization * Update RCCL known issue wording (#3775) * add MAD page * fix wording in RCCL known issue * Revert "add MAD page" This reverts commit c81d0f3b0a3620305b11de8745686c86b060b006. * update llvm version for 6.2.1 (#3779) * Fix broken links in 6.2.1 release notes (#3782) * External CI: Replace libomp dependencies with aomp (#3781) Add roctracer dependency for hipBLAS and rocWMMA testing * External CI: Add rocprofiler v1 and v2 smoke tests (#3784) * External CI: ROCgdb smoke tests (#3785) - Since this is an autotools project and not cmake, build and test on gfx942 system instead of separating into two jobs. Pipeline time is short anyway. - Follow build instructions to update build flags and to incorporate the ROCdbgapi. - Results are not parsed and graphed, but the log contents are printed at the end. This was helpful for debugging and will be kept in the pipeline, as the make check-gdb command's output was not helpful on its own. * External CI: rocPyDecode Smoke Test (#3786) * External CI: omniperf pipeline (#3788) - Referred to public documentation, source, and iterative attempts to create and improve build and test pipeline. - ctest failures are due to the test node not having expected marketing name string and override not working. - The fix should be on the omniperf repo side of things, so this pull request should be fine as is. * External CI: create omniperf pipeline IDs, update nightly build (#3790) * Fixed greater than to be less than in rocFFT changes * fix footnote for 6.1.0 (#3791) * fix footnote for 6.1.0 * fix empty columns in historical KFD title * External CI: Publish wheel as artifact for rocPyDecode (#3796) * fix build rocal for ROCm6.2.1 * Add ROCm6.2.1 manifest file * External CI: fix hip-tests symlink creation (#3799) * Docs: Add Ubuntu 24.04.1 (#3801) * add ubuntu 24.04.1 * add 24.04.1 to bottom os section * fix heading and template * Update compatibility-matrix.rst for OpenMP version * Update compatibility-matrix-historical-6.0.csv for OpenMP version * rm ubuntu 24.04.1 from 6.2.0 * Update docs/compatibility/compatibility-matrix.rst Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * rm duplicate ubuntu in historical --------- Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * Docs: Add Ubuntu 24.04.1 (#3801) * add ubuntu 24.04.1 * add 24.04.1 to bottom os section * fix heading and template * Update compatibility-matrix.rst for OpenMP version * Update compatibility-matrix-historical-6.0.csv for OpenMP version * rm ubuntu 24.04.1 from 6.2.0 * Update docs/compatibility/compatibility-matrix.rst Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * rm duplicate ubuntu in historical --------- Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * External CI: fixes for rocMLIR and nightly build (#3800) * External CI: fix symlinks for rocMLIR and nightly build * add pipeline IDs for hip-tests * fix hip-test ID typo * remove llvm-alt license (#3727) * remove llvm-alt license * fix linting error * External CI: enable ROCR-Runtime tests (#3809) * External CI: default branches for hip-tests, omniperf (#3811) * External CI: torch and torchvision smoke tests (#3810) * External CI: torch and torchvision smoke tests - Fixed issues with package name and version for the vision wheel that prevented it from installing. A patch is used until my pull request in vision repo is merged. - Referred to rocAutomation scripts to pick which test scripts to run out of the many in the torch and vision repo, and iteratively tested suggested scripts to see which ones completed in a timely manner. - Leveraging pytest-azurepipelines module to automatically parse and graph results from these tests. * External CI: omnitrace build pipeline (#3812) * External CI: omnitrace build pipeline starter - Adding initial set of dependencies and build flags. * External CI: omnitrace build pipeline - Add bison, rccl, texinfo dependencies based on build failures. - Add AMDGPU_TARGETS flag - Add ROCm binaries to PATH for clang-format and other tools used. * Fix indentation --------- Co-authored-by: Daniel Su * External CI: AMDMIGraphX Build Fix (#3814) - Swap to default gcc on OS to resolve build errors from recent commits. - Added libdnnl-dev dependency from iterative attempts with compiler change. - Referred to the passing GitHub checks to observe the compilers that was used. - Build CK jit lib and include in AMDMIGraphX build. * External CI: test fixes w/ roctracer, list omniperf as partially succeeding (#3815) * External CI: rpp tests (#3816) * External CI: Build pipeline for rocprofiler-sdk (#3819) * External CI: Pipeline for rocprofiler-sdk * Add rocprofiler dependency * External CI: rocprofiler-sdk build pipeline --------- Co-authored-by: Daniel Su * External CI: Fix/add missing pipeline IDs (#3818) * Update default.xml - Change 6.2.1 to 6.2.2 * Add ROCm6.2.1 manifest file * External CI: omnitrace tests (#3822) * Update tags to 6.2.2 (#3827) * Update tags to 6.2.2 (#3827) * External CI: add roctracer to roc/hipSOLVER test deps (#3825) * External CI: add rocprofiler-sdk pipeline IDs (#3824) * External CI: AMDMIGraphX Smoke Tests (#3830) Co-authored-by: Daniel Su * External CI: MIOpen tests (#3837) * Point to release history instead of deprecated changelog (#3836) * External CI: filter out hipTensor extended tests (#3838) * added revised note re. radeon gpus (#3839) * Restructured the contributions section. (#3715) * testing if this file is editable * changed 'kebob-case' to 'dash-case' * Restructured the page to be more straightforward and provide additional repo information * forgot to save * Moved the topic sentence * Wrong accent on the a in diataxis * Removed the feedback info from contributing and moved it to Feedback * fixed spelling errors * fixed some wording and removed second person text * consolidated Build and Structure into Contribute; edited toolchai to (hopefully) conform to style guide; updated toc * updated the titles in the toc * made changes based on feedback * it's better when you save * removed structure and build; fixed something for the linter * added rst to wordlist * added customizations to wordlist * Add links to gpu cluster network guides (#3763) * Add links to gpu cluster network guides * Add newline character to eof * Make link absolute * add dynamic branch in toc * remove unnecessary page clean up * clean up index/toc * make multi-node topics adjacent --------- Co-authored-by: Peter Park * Point to release history instead of deprecated changelog (#3836) * Restructured the contributions section. (#3715) * testing if this file is editable * changed 'kebob-case' to 'dash-case' * Restructured the page to be more straightforward and provide additional repo information * forgot to save * Moved the topic sentence * Wrong accent on the a in diataxis * Removed the feedback info from contributing and moved it to Feedback * fixed spelling errors * fixed some wording and removed second person text * consolidated Build and Structure into Contribute; edited toolchai to (hopefully) conform to style guide; updated toc * updated the titles in the toc * made changes based on feedback * it's better when you save * removed structure and build; fixed something for the linter * added rst to wordlist * added customizations to wordlist * Add links to gpu cluster network guides (#3763) * Add links to gpu cluster network guides * Add newline character to eof * Make link absolute * add dynamic branch in toc * remove unnecessary page clean up * clean up index/toc * make multi-node topics adjacent --------- Co-authored-by: Peter Park * updated the radeon note (#3850) * External CI: Fix rocPyDecode wheel creation (#3852) - Set values for expected environment variables. - Accompanying changes required in rocPyDecode repo. Pull request will be made. * External CI: pytorch vision patch removal (#3855) My pull request applying this patch was merged upstream, so this is no longer needed and will break the pipeline since it can no longer be applied. * Build(deps): Bump rocm-docs-core from 1.8.1 to 1.8.2 in /docs/sphinx (#3807) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.1 to 1.8.2. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/v1.8.2/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.1...v1.8.2) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated the radeon note, as it were (#3857) * updated the radeon note, as it were * updated the note again * Set devops team as codeowners for rocm-build (#3860) * Set ext CI as codeowners for rocm-build * Update CODEOWNERS to rocm-devops * External CI: Add option to pull mainline branch for dependencies (#3689) * External CI: Add option to pull mainline branch for dependencies * Missing parameter for mainline branch dependencies. * External CI: mainline branch definitions * Removed MIGraphX optimization page (#3848) * External CI: add a global variable to control gfx942 tests (#3864) * External CI: update component default/mainline branches (#3871) * External CI: Stop building gfx90a (#3872) Save on VM resources until infrastructure has test targets. * External CI: add libstdc++-12 to rocMLIR (#3874) * Add building doc section (#3873) * External CI: programmatically get latest aqlprofile (#3876) * External CI: use ctest for rocm-examples (#3877) * External CI: Tensile pipeline (#3884) * add oversubscription conceptual doc (#3885) add mitigiation steps add to toc move page for build move doc fix spelling update doc update oversubscription update order fix spelling add oversubscription to wordlist move oversubscription topic to bottom of toc and index * add oversubscription conceptual doc (#3885) add mitigiation steps add to toc move page for build move doc fix spelling update doc update oversubscription update order fix spelling add oversubscription to wordlist move oversubscription topic to bottom of toc and index (cherry picked from commit d0ecf51b0c9202475e2abe90a45b50df0de6d7ae) * add oversubscription conceptual doc (#3885) (cherry picked from commit d0ecf51b0c9202475e2abe90a45b50df0de6d7ae) * Add building doc section (#3873) (cherry picked from commit abc0e6a08781e4ec98c1566b2e82ed3806db3af5) * External CI: Add pipeline to build upstream boost (#3896) * Update bitsandbytes branch in docs (#3898) * Update bitsandbytes branch in docs (#3898) (cherry picked from commit b541be7bcb6541b4a00633972ed5d0a546ad4e85) * Documentation: Add reference to precision-support floating-point types (#3899) * External CI: use Boost template for MIOpen (#3903) * External CI: create rocprofiler-systems pipeline (#3906) * External CI: omnitrace/rocprof-sys pipeline IDs (#3908) * External CI: MIOpen parse test results (#3913) * External CI: Use pip to install latest cmake on test system (#3915) * added a link to the compatibility matrix (#3904) * added a link to the compatibility matrix * removed quotes * docs: Remove invalid amd_iommu=on parameter Per kernel-parameters.txt, there is no "on" option for amd_iommu. While intel_iommu has it, amd_iommu is automatically on unless specified otherwise. For more info, see these 2 links: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt https://github.com/torvalds/linux/blob/75aa74d52f43e75d0beb20572f98529071b700e5/drivers/iommu/amd/init.c#L3481 Signed-off-by: Kent Russell * docs: Remove invalid amd_iommu=on parameter Per kernel-parameters.txt, there is no "on" option for amd_iommu. While intel_iommu has it, amd_iommu is automatically on unless specified otherwise. For more info, see these 2 links: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt https://github.com/torvalds/linux/blob/75aa74d52f43e75d0beb20572f98529071b700e5/drivers/iommu/amd/init.c#L3481 Signed-off-by: Kent Russell (cherry picked from commit 74333b667d34ce9d90726572533b28059f3fe5b6) * External CI: hipBLASLt build now requires python packaging module (#3926) https://github.com/ROCm/hipBLASLt/pull/1250/files#diff-fee2e6f068b33fca3a1dc49392de8848dbf05c3f4632b680abb1052523e5a30fR35 * External CI: Moved location of upstream pytorch build scripts (#3930) https://github.com/pytorch/pytorch/pull/138103 * External CI: disable rocMLIR tests (#3931) * External CI: disable rocMLIR tests * roctracer AMDGPU_TARGETS flag * External CI: create a GPU diagnostics template (#3932) * External CI: Add CK into pytorch build environment (#3934) * Update rocm-6.2.2.xml (#3927) vim typo removed * External CI: add support to disable individual component tests (#3938) * External CI: AMDMIGraphX greater-equal pip dependencies (#3939) * Build(deps): Bump rocm-docs-core from 1.8.2 to 1.8.3 in /docs/sphinx (#3933) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.2 to 1.8.3. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.2...v1.8.3) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * External CI: rocDecode add libva-amdgpu-dev dependency (#3940) * External CI: enumerate GPUs in gpu-diagnostics (#3942) * External CI: move gpu-diag directly before tests (#3943) * External CI: fix HIP_PIPELINE_ID (#3944) * External CI: pytorch pipeline updates (#3948) To support recent upstream changes and issues observed. * External CI: rocpydecode dependency installation change (#3954) - Install pybind11 through pip instead of apt - Add pip-installed pybind11 path to CMAKE_PREFIX_PATH - Tested against source of PR 122 * External CI: do not assume python is python3 for rocpydecode (#3955) * Improve consistency of the gpu-arch-specs table. (#3936) * Improve consistency of the gpu-arch-specs table. * Add XCD to the glossary. * External CI: Always force rocPyDecode cleanup step * External CI: Add aqlprofile to Tensile test dependencies (#3961) * add vllm performance validation doc (#3964) * External CI: various fixes (#3963) * add suggestions to vllm perf validation doc (#3968) * External CI: move allowPartiallySucceededBuilds to library variable (#3970) * External CI: suppress GPU diag warnings (#3972) * External CI: rocprofiler-compute pipeline files (#3973) * External CI: disable reload AMDGPU (#3974) * Update links to vllm perf validation doc (#3971) * update links to vllm perf validation doc * add PagedAttention to wordlist * External CI: Change test setup for rocPyDecode (#3978) - Use multiple potential locations for pybind11 to be found by cmake. * External CI: add roctracer to rocBLAS deps (#3982) * External CI: decode test changes (#3983) - Only target container with access to first device - Ensure pybind11-dev is uninstalled before the package manager install steps * Changed the introductory text linked to Radeon (#3988) Co-authored-by: prbasyal * External CI: finish rocprofiler-compute enablement (#3995) * External CI: add aomp as rocprofiler-systems dependency (#3996) * External CI: remove omniperf from nightly (#4000) * Sync from internal develop 6.2.4 (#4002) * add radeon pro v710 to gpu arch specs (#192) * Add V710 specs gpg: using RSA key 22223038B47B3ED4B3355AB11B54779B4780494E gpg: Good signature from "Peter Park (MKMPETEPARK01) " [ultimate] add some specs add cols clean up extra line * fix graphics l1 cache description * update SGPR for RDNA2 and RDNA3 archs * update VGPR * Apply suggestions from code review * change l2 cache to 4 * Update docs/reference/gpu-arch-specs.rst * ROCm 6.2.4 compatibility matrix (#186) * prep compat column (historical) and mi300x column * update historical compat matrix for 6.2.4 * update compat matrix for 6.2.4 * fix compat * fix thunk version * fix hipify ver * ROCm 6.2.4 release notes (#184) * prep 6.2.4 release notes * add mathlibs * add detail component changes * rm non-updated linnks * fix sentence * fix rocthrust v * rm offline installer * condense * add leo/ram fdback words * update documentation section * add rocm on radeon note * update os support note wording * update release * update version and GA date to 10-17 * update 6.2.4 rn * update wording * add link to v710 * update wording * update templ * simplify note * words os note words * change URLs to latest * update link to supported GPUs * Update versions.md 6.2.4 date to Oct 18 * Update conf.py release note date to Oct 18 --------- Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> * Sync change from ROCm to ROCm-internal (#194) * Fix Radeon link and point at R6.1.3 as absolute link (#3757) * Update ROCm manifest to 6.2.1 * Update ROCm branch name * Add 6.2.1 to version list (#3770) * Add links to GH issues in 6.2.1 release notes (#3769) * add MAD page * link to GitHub issues in release notes known issues * update templates for 6.2.1 * Revert "add MAD page" This reverts commit 9cce72bba306286c7eb317d592645d4e0e1b27aa. * update wordlist for spellcheck linter * add rccl note * update rocal version change heading to be more obvious * make rocal note more specific * fix missing space * fix capitalization * Update RCCL known issue wording (#3775) * add MAD page * fix wording in RCCL known issue * Revert "add MAD page" This reverts commit c81d0f3b0a3620305b11de8745686c86b060b006. * update llvm version for 6.2.1 (#3779) * Fix broken links in 6.2.1 release notes (#3782) * External CI: Replace libomp dependencies with aomp (#3781) Add roctracer dependency for hipBLAS and rocWMMA testing * External CI: Add rocprofiler v1 and v2 smoke tests (#3784) * External CI: ROCgdb smoke tests (#3785) - Since this is an autotools project and not cmake, build and test on gfx942 system instead of separating into two jobs. Pipeline time is short anyway. - Follow build instructions to update build flags and to incorporate the ROCdbgapi. - Results are not parsed and graphed, but the log contents are printed at the end. This was helpful for debugging and will be kept in the pipeline, as the make check-gdb command's output was not helpful on its own. * External CI: rocPyDecode Smoke Test (#3786) * External CI: omniperf pipeline (#3788) - Referred to public documentation, source, and iterative attempts to create and improve build and test pipeline. - ctest failures are due to the test node not having expected marketing name string and override not working. - The fix should be on the omniperf repo side of things, so this pull request should be fine as is. * External CI: create omniperf pipeline IDs, update nightly build (#3790) * Fixed greater than to be less than in rocFFT changes * fix footnote for 6.1.0 (#3791) * fix footnote for 6.1.0 * fix empty columns in historical KFD title * External CI: Publish wheel as artifact for rocPyDecode (#3796) * External CI: fix hip-tests symlink creation (#3799) * Docs: Add Ubuntu 24.04.1 (#3801) * add ubuntu 24.04.1 * add 24.04.1 to bottom os section * fix heading and template * Update compatibility-matrix.rst for OpenMP version * Update compatibility-matrix-historical-6.0.csv for OpenMP version * rm ubuntu 24.04.1 from 6.2.0 * Update docs/compatibility/compatibility-matrix.rst Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * rm duplicate ubuntu in historical --------- Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> * External CI: fixes for rocMLIR and nightly build (#3800) * External CI: fix symlinks for rocMLIR and nightly build * add pipeline IDs for hip-tests * fix hip-test ID typo * remove llvm-alt license (#3727) * remove llvm-alt license * fix linting error * External CI: enable ROCR-Runtime tests (#3809) * External CI: default branches for hip-tests, omniperf (#3811) * External CI: torch and torchvision smoke tests (#3810) * External CI: torch and torchvision smoke tests - Fixed issues with package name and version for the vision wheel that prevented it from installing. A patch is used until my pull request in vision repo is merged. - Referred to rocAutomation scripts to pick which test scripts to run out of the many in the torch and vision repo, and iteratively tested suggested scripts to see which ones completed in a timely manner. - Leveraging pytest-azurepipelines module to automatically parse and graph results from these tests. * External CI: omnitrace build pipeline (#3812) * External CI: omnitrace build pipeline starter - Adding initial set of dependencies and build flags. * External CI: omnitrace build pipeline - Add bison, rccl, texinfo dependencies based on build failures. - Add AMDGPU_TARGETS flag - Add ROCm binaries to PATH for clang-format and other tools used. * Fix indentation --------- Co-authored-by: Daniel Su * External CI: AMDMIGraphX Build Fix (#3814) - Swap to default gcc on OS to resolve build errors from recent commits. - Added libdnnl-dev dependency from iterative attempts with compiler change. - Referred to the passing GitHub checks to observe the compilers that was used. - Build CK jit lib and include in AMDMIGraphX build. * External CI: test fixes w/ roctracer, list omniperf as partially succeeding (#3815) * External CI: rpp tests (#3816) * External CI: Build pipeline for rocprofiler-sdk (#3819) * External CI: Pipeline for rocprofiler-sdk * Add rocprofiler dependency * External CI: rocprofiler-sdk build pipeline --------- Co-authored-by: Daniel Su * External CI: Fix/add missing pipeline IDs (#3818) * External CI: omnitrace tests (#3822) * Update tags to 6.2.2 (#3827) * External CI: add roctracer to roc/hipSOLVER test deps (#3825) * External CI: add rocprofiler-sdk pipeline IDs (#3824) * External CI: AMDMIGraphX Smoke Tests (#3830) Co-authored-by: Daniel Su * External CI: MIOpen tests (#3837) * Point to release history instead of deprecated changelog (#3836) * External CI: filter out hipTensor extended tests (#3838) * added revised note re. radeon gpus (#3839) * Restructured the contributions section. (#3715) * testing if this file is editable * changed 'kebob-case' to 'dash-case' * Restructured the page to be more straightforward and provide additional repo information * forgot to save * Moved the topic sentence * Wrong accent on the a in diataxis * Removed the feedback info from contributing and moved it to Feedback * fixed spelling errors * fixed some wording and removed second person text * consolidated Build and Structure into Contribute; edited toolchai to (hopefully) conform to style guide; updated toc * updated the titles in the toc * made changes based on feedback * it's better when you save * removed structure and build; fixed something for the linter * added rst to wordlist * added customizations to wordlist * Add links to gpu cluster network guides (#3763) * Add links to gpu cluster network guides * Add newline character to eof * Make link absolute * add dynamic branch in toc * remove unnecessary page clean up * clean up index/toc * make multi-node topics adjacent --------- Co-authored-by: Peter Park * updated the radeon note (#3850) * External CI: Fix rocPyDecode wheel creation (#3852) - Set values for expected environment variables. - Accompanying changes required in rocPyDecode repo. Pull request will be made. * External CI: pytorch vision patch removal (#3855) My pull request applying this patch was merged upstream, so this is no longer needed and will break the pipeline since it can no longer be applied. * Build(deps): Bump rocm-docs-core from 1.8.1 to 1.8.2 in /docs/sphinx (#3807) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.1 to 1.8.2. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/v1.8.2/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.1...v1.8.2) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated the radeon note, as it were (#3857) * updated the radeon note, as it were * updated the note again * Set devops team as codeowners for rocm-build (#3860) * Set ext CI as codeowners for rocm-build * Update CODEOWNERS to rocm-devops * External CI: Add option to pull mainline branch for dependencies (#3689) * External CI: Add option to pull mainline branch for dependencies * Missing parameter for mainline branch dependencies. * External CI: mainline branch definitions * Removed MIGraphX optimization page (#3848) * External CI: add a global variable to control gfx942 tests (#3864) * External CI: update component default/mainline branches (#3871) * External CI: Stop building gfx90a (#3872) Save on VM resources until infrastructure has test targets. * External CI: add libstdc++-12 to rocMLIR (#3874) * Add building doc section (#3873) * External CI: programmatically get latest aqlprofile (#3876) * External CI: use ctest for rocm-examples (#3877) * External CI: Tensile pipeline (#3884) * add oversubscription conceptual doc (#3885) add mitigiation steps add to toc move page for build move doc fix spelling update doc update oversubscription update order fix spelling add oversubscription to wordlist move oversubscription topic to bottom of toc and index * add oversubscription conceptual doc (#3885) (cherry picked from commit d0ecf51b0c9202475e2abe90a45b50df0de6d7ae) * External CI: Add pipeline to build upstream boost (#3896) * Update bitsandbytes branch in docs (#3898) * Documentation: Add reference to precision-support floating-point types (#3899) * External CI: use Boost template for MIOpen (#3903) * External CI: create rocprofiler-systems pipeline (#3906) * External CI: omnitrace/rocprof-sys pipeline IDs (#3908) * External CI: MIOpen parse test results (#3913) * External CI: Use pip to install latest cmake on test system (#3915) * added a link to the compatibility matrix (#3904) * added a link to the compatibility matrix * removed quotes * docs: Remove invalid amd_iommu=on parameter Per kernel-parameters.txt, there is no "on" option for amd_iommu. While intel_iommu has it, amd_iommu is automatically on unless specified otherwise. For more info, see these 2 links: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt https://github.com/torvalds/linux/blob/75aa74d52f43e75d0beb20572f98529071b700e5/drivers/iommu/amd/init.c#L3481 Signed-off-by: Kent Russell * External CI: hipBLASLt build now requires python packaging module (#3926) https://github.com/ROCm/hipBLASLt/pull/1250/files#diff-fee2e6f068b33fca3a1dc49392de8848dbf05c3f4632b680abb1052523e5a30fR35 * External CI: Moved location of upstream pytorch build scripts (#3930) https://github.com/pytorch/pytorch/pull/138103 * External CI: disable rocMLIR tests (#3931) * External CI: disable rocMLIR tests * roctracer AMDGPU_TARGETS flag * External CI: create a GPU diagnostics template (#3932) * External CI: Add CK into pytorch build environment (#3934) * External CI: add support to disable individual component tests (#3938) * External CI: AMDMIGraphX greater-equal pip dependencies (#3939) * Build(deps): Bump rocm-docs-core from 1.8.2 to 1.8.3 in /docs/sphinx (#3933) Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.2 to 1.8.3. - [Release notes](https://github.com/ROCm/rocm-docs-core/releases) - [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md) - [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.2...v1.8.3) --- updated-dependencies: - dependency-name: rocm-docs-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * External CI: rocDecode add libva-amdgpu-dev dependency (#3940) * External CI: enumerate GPUs in gpu-diagnostics (#3942) * External CI: move gpu-diag directly before tests (#3943) * External CI: fix HIP_PIPELINE_ID (#3944) --------- Signed-off-by: dependabot[bot] Signed-off-by: Kent Russell Co-authored-by: Jeffrey Novotny Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> Co-authored-by: Wang, Yanyao Co-authored-by: Yanyao Wang Co-authored-by: Peter Park Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> Co-authored-by: Joseph Macaranas <145489236+amd-jmacaran@users.noreply.github.com> Co-authored-by: Daniel Su Co-authored-by: Sandra Polifroni Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com> Co-authored-by: Michael Benavidez Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: MKKnorr Co-authored-by: Kent Russell Co-authored-by: Joseph Greathouse * 6.2.4 release notes: add known/fixed issues (#193) * add "for compute workloads" wording for clarity * add AMDSMI resolved issue * add dlm known issue intro text wording * update wording rm bullet point update wording * fix spellcheck due to spacing * rm s * rm gfx1151 * remove dlm known issue * update list of updated docs; note for Radeon users fmt * update GA date for 6.2.4 * fix rdc version * fix RDC version strings (#196) * revert outdataed change for .azuredevops * Fix 6.2.4 date in versions.md Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] Signed-off-by: Kent Russell Co-authored-by: Peter Park Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> Co-authored-by: Jeffrey Novotny Co-authored-by: Wang, Yanyao Co-authored-by: Yanyao Wang Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> Co-authored-by: Joseph Macaranas <145489236+amd-jmacaran@users.noreply.github.com> Co-authored-by: Daniel Su Co-authored-by: Sandra Polifroni Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com> Co-authored-by: Michael Benavidez Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: MKKnorr Co-authored-by: Kent Russell Co-authored-by: Joseph Greathouse * fix links in release notes 6.2.4 (#4008) * Remove extra line * Update xml files for 6.2.4 (#4012) * Update xml files for 6.2.4 * Update README with 6.2.4 * Increase visibility of programming guide * Docs: Update what is rocm description * Apply suggestions from code review Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com> * Update docs/how-to/hip_programming_guide.rst Co-authored-by: MKKnorr * WIP * Update docs/index.md * Update docs/how-to/hip_programming_guide.rst Co-authored-by: MKKnorr * Update docs/how-to/programming_guide.rst * Update docs/what-is-rocm.rst * Apply suggestions from code review Co-authored-by: Leo Paoletti <164940351+lpaoletti@users.noreply.github.com> * Update docs/how-to/programming_guide.rst Co-authored-by: Leo Paoletti <164940351+lpaoletti@users.noreply.github.com> * Remove tip * External CI: allow test failures to present as failures on Github (#3993) * External CI: disable rdmatest and rocrtstFunc.Memory_Max_Mem (#4016) * Added 6.2.4 manifest.xml * External CI: fix comgr build (#4025) * External CI: increase Tensile test timeout to 90 mins (#4027) --------- Signed-off-by: David Galiffi Signed-off-by: dependabot[bot] Signed-off-by: Kent Russell Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com> Co-authored-by: Jeffrey Novotny Co-authored-by: Peter Park Co-authored-by: Yanyao Wang Co-authored-by: Wang, Yanyao Co-authored-by: David Galiffi Co-authored-by: Chris Kime Co-authored-by: ozziemoreno <109979778+ozziemoreno@users.noreply.github.com> Co-authored-by: Sandra Polifroni Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com> Co-authored-by: Joseph Macaranas <145489236+amd-jmacaran@users.noreply.github.com> Co-authored-by: Daniel Su Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com> Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com> Co-authored-by: Michael Benavidez Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: MKKnorr Co-authored-by: Kent Russell Co-authored-by: Joseph Greathouse Co-authored-by: Johannes Maria Frank Co-authored-by: Brian Cornille Co-authored-by: Joseph Macaranas Co-authored-by: Pratik Basyal Co-authored-by: prbasyal Co-authored-by: Istvan Kiss Co-authored-by: Leo Paoletti <164940351+lpaoletti@users.noreply.github.com> Co-authored-by: Ameya Keshava Mallya --- .wordlist.txt | 8 + README.md | 2 +- RELEASE.md | 10 +- default.xml | 2 +- docs/compatibility/compatibility-matrix.rst | 4 +- docs/conf.py | 1 + .../llm-inference-frameworks.rst | 6 +- .../mi300x/vllm-benchmark.rst | 407 ++++++++++++++++++ docs/how-to/programming_guide.rst | 37 ++ docs/how-to/tuning-guides/mi300x/index.rst | 2 + docs/how-to/tuning-guides/mi300x/workload.rst | 13 +- docs/index.md | 17 +- docs/reference/gpu-arch-specs.rst | 14 +- docs/release/versions.md | 2 +- docs/sphinx/_toc.yml.in | 6 +- docs/what-is-rocm.rst | 21 +- tools/autotag/components.xml | 2 +- tools/rocm-build/ROCm.mk | 16 +- tools/rocm-build/build_amdmigraphx.sh | 3 +- tools/rocm-build/build_composable_kernel.sh | 12 +- tools/rocm-build/build_hipblas.sh | 3 +- tools/rocm-build/build_hipblaslt.sh | 3 +- tools/rocm-build/build_hipcub.sh | 3 +- tools/rocm-build/build_hipfft.sh | 3 +- tools/rocm-build/build_hiprand.sh | 1 - tools/rocm-build/build_hipsolver.sh | 3 +- tools/rocm-build/build_hipsparse.sh | 3 +- tools/rocm-build/build_hipsparselt.sh | 3 +- tools/rocm-build/build_hiptensor.sh | 4 +- tools/rocm-build/build_lightning.sh | 120 ++++-- tools/rocm-build/build_miopen-deps.sh | 2 +- tools/rocm-build/build_miopen-hip.sh | 5 +- tools/rocm-build/build_mivisionx.sh | 5 +- tools/rocm-build/build_omniperf.sh | 171 ++++++++ tools/rocm-build/build_omnitrace.sh | 191 ++++++++ tools/rocm-build/build_opencl_icd_loader.sh | 141 ++++++ tools/rocm-build/build_rccl.sh | 4 +- tools/rocm-build/build_rocal.sh | 71 +++ tools/rocm-build/build_rocalution.sh | 3 +- tools/rocm-build/build_rocblas.sh | 3 +- tools/rocm-build/build_rocdecode.sh | 5 +- tools/rocm-build/build_rocfft.sh | 3 +- tools/rocm-build/build_rocm-cmake.sh | 31 +- tools/rocm-build/build_rocprim.sh | 5 +- tools/rocm-build/build_rocprofiler-sdk.sh | 222 ++++++++++ tools/rocm-build/build_rocrand.sh | 4 +- tools/rocm-build/build_rocsolver.sh | 4 +- tools/rocm-build/build_rocsparse.sh | 8 +- tools/rocm-build/build_rocthrust.sh | 4 +- tools/rocm-build/build_rocwmma.sh | 4 +- tools/rocm-build/build_rpp.sh | 8 +- tools/rocm-build/compute_helper.sh | 59 +++ .../docker/ubuntu20/install-prerequisites.sh | 12 +- tools/rocm-build/docker/ubuntu20/packages | 4 + .../docker/ubuntu22/install-prerequisities.sh | 89 +++- tools/rocm-build/docker/ubuntu22/packages | 3 + tools/rocm-build/docker/ubuntu24/Dockerfile | 11 + tools/rocm-build/docker/ubuntu24/README.md | 27 ++ .../docker/ubuntu24/install-prerequisites.sh | 237 ++++++++++ tools/rocm-build/docker/ubuntu24/local-pin-60 | 3 + tools/rocm-build/docker/ubuntu24/packages | 140 ++++++ tools/rocm-build/envsetup.sh | 6 +- tools/rocm-build/rocm-6.2.0.xml | 28 +- .../{rocm-6.1.1.xml => rocm-6.2.1.xml} | 67 +-- .../{rocm-6.1.0.xml => rocm-6.2.2.xml} | 65 +-- .../{rocm-6.1.2.xml => rocm-6.2.4.xml} | 146 ++++--- 66 files changed, 2214 insertions(+), 308 deletions(-) create mode 100644 docs/how-to/performance-validation/mi300x/vllm-benchmark.rst create mode 100644 docs/how-to/programming_guide.rst create mode 100755 tools/rocm-build/build_omniperf.sh create mode 100755 tools/rocm-build/build_omnitrace.sh create mode 100755 tools/rocm-build/build_opencl_icd_loader.sh create mode 100755 tools/rocm-build/build_rocal.sh create mode 100755 tools/rocm-build/build_rocprofiler-sdk.sh create mode 100644 tools/rocm-build/docker/ubuntu24/Dockerfile create mode 100644 tools/rocm-build/docker/ubuntu24/README.md create mode 100644 tools/rocm-build/docker/ubuntu24/install-prerequisites.sh create mode 100644 tools/rocm-build/docker/ubuntu24/local-pin-60 create mode 100644 tools/rocm-build/docker/ubuntu24/packages rename tools/rocm-build/{rocm-6.1.1.xml => rocm-6.2.1.xml} (89%) rename tools/rocm-build/{rocm-6.1.0.xml => rocm-6.2.2.xml} (90%) rename tools/rocm-build/{rocm-6.1.2.xml => rocm-6.2.4.xml} (85%) diff --git a/.wordlist.txt b/.wordlist.txt index 580890845..2b7b7eb70 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -36,6 +36,7 @@ Bluefield Bootloader CCD CDNA +CHTML CIFAR CLI CLion @@ -70,6 +71,7 @@ Concretized Conda ConnectX CuPy +Dashboarding DDR DF DGEMM @@ -227,6 +229,7 @@ Mellanox's Meta's Miniconda MirroredStrategy +Mixtral Multicore Multithreaded MyEnvironment @@ -273,6 +276,7 @@ OpenSSL OpenVX OpenXLA Oversubscription +PagedAttention PCC PCI PCIe @@ -294,6 +298,7 @@ PowerShell PyPi PyTorch Qcycles +Qwen RAII RAS RCCL @@ -563,6 +568,7 @@ hipfort hipify hipsolver hipsparse +hlist hotspotting hpc hpp @@ -586,6 +592,7 @@ intra invariants invocating ipo +jax kdb kfd latencies @@ -606,6 +613,7 @@ migraphx miopen miopengemm mivisionx +mjx mkdir mlirmiopen mtypes diff --git a/README.md b/README.md index 2f42b747b..3d34f00fe 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ The Build time will reduce significantly if we limit the GPU Architecture/s agai mkdir -p ~/WORKSPACE/ # Or any folder name other than WORKSPACE cd ~/WORKSPACE/ -export ROCM_VERSION=6.2.2 # Or 6.2.0 or 6.2.1 +export ROCM_VERSION=6.2.4 # Or 6.2.0, 6.2.1, 6.2.2 ~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.2.x -m tools/rocm-build/rocm-${ROCM_VERSION}.xml ~/bin/repo sync diff --git a/RELEASE.md b/RELEASE.md index 726a599d4..ea9f62b5a 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -32,18 +32,18 @@ ROCm documentation continues to be updated to provide clearer and more comprehen a wider variety of user needs and use cases. * Added a new GPU cluster networking guide. See - [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html). + [Cluster network performance validation for AMD Instinct accelerators](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/docs-6.2.4/index.html). This documentation provides guidelines on validating network configurations in single-node and multi-node environments to attain optimal speed and bandwidth in AMD Instinct-powered clusters. * Updated the HIP runtime documentation. - * Added a new section on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hipgraph.html). + * Added a new section on how to use [HIP graphs](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hipgraph.html). - * Added a new section about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/stream_ordered_allocator.html). + * Added a new section about the [Stream ordered memory allocator (SOMA)](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/stream_ordered_allocator.html). - * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hip_porting_driver_api.html) section. + * Updated the [Porting CUDA driver API](https://rocm.docs.amd.com/projects/HIP/en/docs-6.2.4/how-to/hip_porting_driver_api.html) section. * Updated the [Post-installation instructions](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.4/install/post-install.html) with guidance on using the `update-alternatives` utility and environment modules to help you manage multiple ROCm @@ -56,7 +56,7 @@ a wider variety of user needs and use cases. ## Operating system and hardware support changes ROCm 6.2.4 adds support for the [AMD Radeon PRO V710](https://www.amd.com/en/products/accelerators/radeon-pro/amd-radeon-pro-v710.html) GPU for compute workloads. See -[Supported GPUs](https://advanced-micro-devices-demo--287.com.readthedocs.build/projects/install-on-linux-internal/en/287/reference/system-requirements.html) +[Supported GPUs](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.4/reference/system-requirements.html#supported-gpus) for more information. This release maintains the same operating system support as 6.2.2. diff --git a/default.xml b/default.xml index d3088683d..419b90d33 100644 --- a/default.xml +++ b/default.xml @@ -1,7 +1,7 @@ - diff --git a/docs/compatibility/compatibility-matrix.rst b/docs/compatibility/compatibility-matrix.rst index 28ef57674..55ea7e990 100644 --- a/docs/compatibility/compatibility-matrix.rst +++ b/docs/compatibility/compatibility-matrix.rst @@ -112,9 +112,7 @@ Accelerators and GPUs listed in the following table support compute workloads (n ,,, PERFORMANCE TOOLS,,, :doc:`ROCm Bandwidth Test `,1.4.0,1.4.0,1.4.0 - :doc:`ROCm Compute Profiler `,2.0.1,2.0.1,N/A - :doc:`ROCm Systems Profiler `,1.11.2,1.11.2,N/A - :doc:`ROCProfiler `,2.0.60202,2.0.60201,2.0.60100 + :doc:`ROCProfiler `,2.0.60204,2.0.60202,2.0.60100 :doc:`ROCprofiler-SDK `,0.4.0,0.4.0,N/A :doc:`ROCTracer `,4.1.60204,4.1.60202,4.1.60100 ,,, diff --git a/docs/conf.py b/docs/conf.py index 89c5ab964..727a5dee6 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -81,6 +81,7 @@ article_pages = [ "file": "how-to/llm-fine-tuning-optimization/profiling-and-debugging", "os": ["linux"], }, + {"file": "how-to/performance-validation/mi300x/vllm-benchmark", "os": ["linux"]}, {"file": "how-to/system-optimization/index", "os": ["linux"]}, {"file": "how-to/system-optimization/mi300x", "os": ["linux"]}, {"file": "how-to/system-optimization/mi200", "os": ["linux"]}, diff --git a/docs/how-to/llm-fine-tuning-optimization/llm-inference-frameworks.rst b/docs/how-to/llm-fine-tuning-optimization/llm-inference-frameworks.rst index 3ee672353..84e839391 100644 --- a/docs/how-to/llm-fine-tuning-optimization/llm-inference-frameworks.rst +++ b/docs/how-to/llm-fine-tuning-optimization/llm-inference-frameworks.rst @@ -16,7 +16,7 @@ This section discusses how to implement `vLLM `_ vLLM inference ============== -vLLM is renowned for its paged attention algorithm that can reduce memory consumption and increase throughput thanks to +vLLM is renowned for its PagedAttention algorithm that can reduce memory consumption and increase throughput thanks to its paging scheme. Instead of allocating GPU high-bandwidth memory (HBM) for the maximum output token lengths of the models, the paged attention of vLLM allocates GPU HBM dynamically for its actual decoding lengths. This paged attention is also effective when multiple requests share the same key and value contents for a large value of beam search or @@ -139,9 +139,7 @@ Refer to :ref:`mi300x-vllm-optimization` for performance optimization tips. ROCm provides a prebuilt optimized Docker image for validating the performance of LLM inference with vLLM on the MI300X accelerator. The Docker image includes ROCm, vLLM, PyTorch, and tuning files in the CSV -format. For more information, see the guide to -`LLM inference performance validation with vLLM on the AMD Instinct™ MI300X accelerator `_ -on the ROCm GitHub repository. +format. For more information, see :doc:`/how-to/performance-validation/mi300x/vllm-benchmark`. .. _fine-tuning-llms-tgi: diff --git a/docs/how-to/performance-validation/mi300x/vllm-benchmark.rst b/docs/how-to/performance-validation/mi300x/vllm-benchmark.rst new file mode 100644 index 000000000..90883ea84 --- /dev/null +++ b/docs/how-to/performance-validation/mi300x/vllm-benchmark.rst @@ -0,0 +1,407 @@ +.. meta:: + :description: Learn how to validate LLM inference performance on MI300X accelerators using AMD MAD and the unified + ROCm Docker image. + :keywords: model, MAD, automation, dashboarding, validate + +*********************************************************** +LLM inference performance validation on AMD Instinct MI300X +*********************************************************** + +.. _vllm-benchmark-unified-docker: + +The `ROCm vLLM Docker `_ image offers +a prebuilt, optimized environment designed for validating large language model +(LLM) inference performance on the AMD Instinct™ MI300X accelerator. This +ROCm vLLM Docker image integrates vLLM and PyTorch tailored specifically for the +MI300X accelerator and includes the following components: + +* `ROCm 6.2.1 `_ + +* `vLLM 0.6.4 `_ + +* `PyTorch 2.5.0 `_ + +* Tuning files (in CSV format) + +With this Docker image, you can quickly validate the expected inference +performance numbers on the MI300X accelerator. This topic also provides tips on +optimizing performance with popular AI models. + +.. hlist:: + :columns: 6 + + * Llama 3.1 8B + + * Llama 3.1 70B + + * Llama 3.1 405B + + * Llama 2 7B + + * Llama 2 70B + + * Mixtral 8x7B + + * Mixtral 8x22B + + * Mixtral 7B + + * Qwen2 7B + + * Qwen2 72B + + * JAIS 13B + + * JAIS 30B + +.. _vllm-benchmark-vllm: + +.. note:: + + vLLM is a toolkit and library for LLM inference and serving. AMD implements + high-performance custom kernels and modules in vLLM to enhance performance. + See :ref:`fine-tuning-llms-vllm` and :ref:`mi300x-vllm-optimization` for + more information. + +Getting started +=============== + +Use the following procedures to reproduce the benchmark results on an +MI300X accelerator with the prebuilt vLLM Docker image. + +.. _vllm-benchmark-get-started: + +1. Disable NUMA auto-balancing. + + To optimize performance, disable automatic NUMA balancing. Otherwise, the GPU + might hang until the periodic balancing is finalized. For more information, + see :ref:`AMD Instinct MI300X system optimization `. + + .. code-block:: shell + + # disable automatic NUMA balancing + sh -c 'echo 0 > /proc/sys/kernel/numa_balancing' + # check if NUMA balancing is disabled (returns 0 if disabled) + cat /proc/sys/kernel/numa_balancing + 0 + +2. Download the :ref:`ROCm vLLM Docker image `. + + Use the following command to pull the Docker image from Docker Hub. + + .. code-block:: shell + + docker pull rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4 + +Once setup is complete, you can choose between two options to reproduce the +benchmark results: + +- :ref:`MAD-integrated benchmarking ` + +- :ref:`Standalone benchmarking ` + +.. _vllm-benchmark-mad: + +MAD-integrated benchmarking +=========================== + +Clone the ROCm Model Automation and Dashboarding (``__) repository to a local +directory and install the required packages on the host machine. + +.. code-block:: shell + + git clone https://github.com/ROCm/MAD + cd MAD + pip install -r requirements.txt + +Use this command to run a performance benchmark test of the Llama 3.1 8B model +on one GPU with ``float16`` data type in the host machine. + +.. code-block:: shell + + export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models" + python3 tools/run_models.py --tags pyt_vllm_llama-3.1-8b --keep-model-dir --live-output --timeout 28800 + +ROCm MAD launches a Docker container with the name +``container_ci-pyt_vllm_llama-3.1-8b``. The latency and throughput reports of the +model are collected in the following path: ``~/MAD/reports_float16/``. + +Although the following models are preconfigured to collect latency and +throughput performance data, you can also change the benchmarking parameters. +Refer to the :ref:`Standalone benchmarking ` section. + +Available models +---------------- + +.. hlist:: + :columns: 3 + + * ``pyt_vllm_llama-3.1-8b`` + + * ``pyt_vllm_llama-3.1-70b`` + + * ``pyt_vllm_llama-3.1-405b`` + + * ``pyt_vllm_llama-2-7b`` + + * ``pyt_vllm_llama-2-70b`` + + * ``pyt_vllm_mixtral-8x7b`` + + * ``pyt_vllm_mixtral-8x22b`` + + * ``pyt_vllm_mistral-7b`` + + * ``pyt_vllm_qwen2-7b`` + + * ``pyt_vllm_qwen2-72b`` + + * ``pyt_vllm_jais-13b`` + + * ``pyt_vllm_jais-30b`` + + * ``pyt_vllm_llama-3.1-8b_fp8`` + + * ``pyt_vllm_llama-3.1-70b_fp8`` + + * ``pyt_vllm_llama-3.1-405b_fp8`` + + * ``pyt_vllm_mixtral-8x7b_fp8`` + + * ``pyt_vllm_mixtral-8x22b_fp8`` + +.. _vllm-benchmark-standalone: + +Standalone benchmarking +======================= + +You can run the vLLM benchmark tool independently by starting the +:ref:`Docker container ` as shown in the following +snippet. + +.. code-block:: + + docker pull rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4 + docker run -it --device=/dev/kfd --device=/dev/dri --group-add video --shm-size 128G --security-opt seccomp=unconfined --security-opt apparmor=unconfined --cap-add=SYS_PTRACE -v $(pwd):/workspace --env HUGGINGFACE_HUB_CACHE=/workspace --name vllm_v0.6.4 rocm/vllm:rocm6.2_mi300_ubuntu20.04_py3.9_vllm_0.6.4 + +In the Docker container, clone the ROCm MAD repository and navigate to the +benchmark scripts directory at ``~/MAD/scripts/vllm``. + +.. code-block:: + + git clone https://github.com/ROCm/MAD + cd MAD/scripts/vllm + +Command +------- + +To start the benchmark, use the following command with the appropriate options. +See :ref:`Options ` for the list of +options and their descriptions. + +.. code-block:: shell + + ./vllm_benchmark_report.sh -s $test_option -m $model_repo -g $num_gpu -d $datatype + +See the :ref:`examples ` for more information. + +.. note:: + + The input sequence length, output sequence length, and tensor parallel (TP) are + already configured. You don't need to specify them with this script. + +.. note:: + + If you encounter the following error, pass your access-authorized Hugging + Face token to the gated models. + + .. code-block:: shell + + OSError: You are trying to access a gated repo. + + # pass your HF_TOKEN + export HF_TOKEN=$your_personal_hf_token + +.. _vllm-benchmark-standalone-options: + +Options +------- + +.. list-table:: + :header-rows: 1 + :align: center + + * - Name + - Options + - Description + + * - ``$test_option`` + - latency + - Measure decoding token latency + + * - + - throughput + - Measure token generation throughput + + * - + - all + - Measure both throughput and latency + + * - ``$model_repo`` + - ``meta-llama/Meta-Llama-3.1-8B-Instruct`` + - Llama 3.1 8B + + * - (``float16``) + - ``meta-llama/Meta-Llama-3.1-70B-Instruct`` + - Llama 3.1 70B + + * - + - ``meta-llama/Meta-Llama-3.1-405B-Instruct`` + - Llama 3.1 405B + + * - + - ``meta-llama/Llama-2-7b-chat-hf`` + - Llama 2 7B + + * - + - ``meta-llama/Llama-2-70b-chat-hf`` + - Llama 2 70B + + * - + - ``mistralai/Mixtral-8x7B-Instruct-v0.1`` + - Mixtral 8x7B + + * - + - ``mistralai/Mixtral-8x22B-Instruct-v0.1`` + - Mixtral 8x22B + + * - + - ``mistralai/Mistral-7B-Instruct-v0.3`` + - Mixtral 7B + + * - + - ``Qwen/Qwen2-7B-Instruct`` + - Qwen2 7B + + * - + - ``Qwen/Qwen2-72B-Instruct`` + - Qwen2 72B + + * - + - ``core42/jais-13b-chat`` + - JAIS 13B + + * - + - ``core42/jais-30b-chat-v3`` + - JAIS 30B + + * - ``$model_repo`` + - ``amd/Meta-Llama-3.1-8B-Instruct-FP8-KV`` + - Llama 3.1 8B + + * - (``float8``) + - ``amd/Meta-Llama-3.1-70B-Instruct-FP8-KV`` + - Llama 3.1 70B + + * - + - ``amd/Meta-Llama-3.1-405B-Instruct-FP8-KV`` + - Llama 3.1 405B + + * - + - ``amd/Mixtral-8x7B-Instruct-v0.1-FP8-KV`` + - Mixtral 8x7B + + * - + - ``amd/Mixtral-8x22B-Instruct-v0.1-FP8-KV`` + - Mixtral 8x22B + + * - ``$num_gpu`` + - 1 or 8 + - Number of GPUs + + * - ``$datatype`` + - ``float16`` or ``float8`` + - Data type + +.. _vllm-benchmark-run-benchmark: + +Running the benchmark on the MI300X accelerator +----------------------------------------------- + +Here are some examples of running the benchmark with various options. +See :ref:`Options ` for the list of +options and their descriptions. + +Example 1: latency benchmark +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use this command to benchmark the latency of the Llama 3.1 8B model on one GPU with the ``float16`` and ``float8`` data types. + +.. code-block:: + + ./vllm_benchmark_report.sh -s latency -m meta-llama/Meta-Llama-3.1-8B-Instruct -g 1 -d float16 + ./vllm_benchmark_report.sh -s latency -m amd/Meta-Llama-3.1-8B-Instruct-FP8-KV -g 1 -d float8 + +Find the latency reports at: + +- ``./reports_float16/summary/Meta-Llama-3.1-8B-Instruct_latency_report.csv`` + +- ``./reports_float8/summary/Meta-Llama-3.1-8B-Instruct-FP8-KV_latency_report.csv`` + +Example 2: throughput benchmark +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use this command to benchmark the throughput of the Llama 3.1 8B model on one GPU with the ``float16`` and ``float8`` data types. + +.. code-block:: shell + + ./vllm_benchmark_report.sh -s throughput -m meta-llama/Meta-Llama-3.1-8B-Instruct -g 1 -d float16 + ./vllm_benchmark_report.sh -s throughput -m amd/Meta-Llama-3.1-8B-Instruct-FP8-KV -g 1 -d float8 + +Find the throughput reports at: + +- ``./reports_float16/summary/Meta-Llama-3.1-8B-Instruct_throughput_report.csv`` + +- ``./reports_float8/summary/Meta-Llama-3.1-8B-Instruct-FP8-KV_throughput_report.csv`` + +.. raw:: html + + + +.. note:: + + Throughput is calculated as: + + - .. math:: throughput\_tot = requests \times (\mathsf{\text{input lengths}} + \mathsf{\text{output lengths}}) / elapsed\_time + + - .. math:: throughput\_gen = requests \times \mathsf{\text{output lengths}} / elapsed\_time + +Further reading +=============== + +- For application performance optimization strategies for HPC and AI workloads, + including inference with vLLM, see :doc:`/how-to/tuning-guides/mi300x/workload`. + +- To learn more about the options for latency and throughput benchmark scripts, + see ``_. + +- To learn more about system settings and management practices to configure your system for + MI300X accelerators, see :doc:`/how-to/system-optimization/mi300x`. + +- To learn how to run LLM models from Hugging Face or your own model, see + :doc:`Using ROCm for AI `. + +- To learn how to optimize inference on LLMs, see + :doc:`Fine-tuning LLMs and inference optimization `. + +- For a list of other ready-made Docker images for ROCm, see the + :doc:`Docker image support matrix `. + +- To compare with the previous version of the ROCm vLLM Docker image for performance validation, refer to + `LLM inference performance validation on AMD Instinct MI300X (ROCm 6.2.0) `_. + diff --git a/docs/how-to/programming_guide.rst b/docs/how-to/programming_guide.rst new file mode 100644 index 000000000..8a489b20d --- /dev/null +++ b/docs/how-to/programming_guide.rst @@ -0,0 +1,37 @@ +.. meta:: + :description: Programming guide + :keywords: HIP, programming guide, heterogeneous programming, AMD GPU programming + +.. _hip-programming-guide: + +******************************************************************************** +Programming guide +******************************************************************************** + +ROCm provides a robust environment for heterogeneous programs running on CPUs +and AMD GPUs. ROCm supports various programming languages and frameworks to +help developers access the power of AMD GPUs. The natively supported programming +languages are HIP (Heterogeneous-Compute Interface for Portability) and +OpenCL, but HIP bindings are available for Python and Fortran. + +HIP is an API based on C++ that provides a runtime and kernel language for GPU +programming and is the essential ROCm programming language. HIP is also designed +to be a marshalling language, allowing code written for NVIDIA CUDA to be +easily ported to run on AMD GPUs. Developers can use HIP to write kernels that +execute on AMD GPUs while maintaining compatibility with CUDA-based systems. + +OpenCL (Open Computing Language) is an open standard for cross-platform, +parallel programming of diverse processors. ROCm supports OpenCL for developers +who want to use standard frameworks across different hardware platforms, +including CPUs, GPUs, and other accelerators. For more information, see +`OpenCL `_. + +Python bindings can be found at https://github.com/ROCm/hip-python. +Python is popular in AI and machine learning applications due to available +frameworks like TensorFlow and PyTorch. + +Fortran bindings can be found at https://github.com/ROCm/hipfort. +It enables scientific, academic, and legacy applications, particularly those in +high-performance computing, to run on AMD GPUs via HIP. + +For a complete description of the HIP programming language, see the :doc:`HIP programming guide`. diff --git a/docs/how-to/tuning-guides/mi300x/index.rst b/docs/how-to/tuning-guides/mi300x/index.rst index 1947a28d1..28389f40a 100644 --- a/docs/how-to/tuning-guides/mi300x/index.rst +++ b/docs/how-to/tuning-guides/mi300x/index.rst @@ -8,6 +8,8 @@ accelerators. They include detailed instructions on system settings and application tuning suggestions to help you fully leverage the capabilities of these accelerators, thereby achieving optimal performance. +* :doc:`/how-to/performance-validation/mi300x/vllm-benchmark` + * :doc:`/how-to/tuning-guides/mi300x/system` * :doc:`/how-to/tuning-guides/mi300x/workload` diff --git a/docs/how-to/tuning-guides/mi300x/workload.rst b/docs/how-to/tuning-guides/mi300x/workload.rst index 9401fa0a6..66e1dfa8c 100644 --- a/docs/how-to/tuning-guides/mi300x/workload.rst +++ b/docs/how-to/tuning-guides/mi300x/workload.rst @@ -152,9 +152,7 @@ address any new bottlenecks that may emerge. ROCm provides a prebuilt optimized Docker image that has everything required to implement the tips in this section. It includes ROCm, vLLM, PyTorch, and tuning files in the CSV -format. For more information, see the guide to -`LLM inference performance validation with vLLM on the AMD Instinct™ MI300X accelerator `_ -on the ROCm GitHub repository. +format. For more information, see :doc:`/how-to/performance-validation/mi300x/vllm-benchmark`. .. _mi300x-profiling-tools: @@ -378,11 +376,10 @@ Refer to `vLLM documentation `_ -on the ROCm GitHub repository. +ROCm provides a prebuilt optimized Docker image for validating the performance +of LLM inference with vLLM on the MI300X accelerator. The Docker image includes +ROCm, vLLM, PyTorch, and tuning files in the CSV format. For more information, +see :doc:`/how-to/performance-validation/mi300x/vllm-benchmark`. Maximize throughput ------------------- diff --git a/docs/index.md b/docs/index.md index 8513180ab..e1677054c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,16 +9,14 @@ ROCm is an open-source software platform optimized to extract HPC and AI workload performance from AMD Instinct accelerators and AMD Radeon GPUs while maintaining -compatibility with industry software frameworks. For more information, see [What is ROCm?](./what-is-rocm.rst) +compatibility with industry software frameworks. For more information, see +[What is ROCm?](./what-is-rocm.rst) -If you're using Radeon GPUs, consider reviewing {doc}`Radeon-specific ROCm documentation`. +ROCm supports multiple programming languages and programming interfaces such as +{doc}`HIP (Heterogeneous-Compute Interface for Portability)`, OpenCL, +and OpenMP, as explained in the [Programming guide](./how-to/programming_guide.rst). -Installation instructions are available from: - -* {doc}`ROCm installation for Linux` -* {doc}`HIP SDK installation for Windows` -* [Deep learning frameworks installation](./how-to/deep-learning-rocm.rst) -* [Build ROCm from source](./how-to/build-rocm.rst) +If you're using AMD Radeon™ PRO or Radeon GPUs in a workstation setting with a display connected, review {doc}`Radeon-specific ROCm documentation`. ROCm documentation is organized into the following categories: @@ -41,11 +39,12 @@ ROCm documentation is organized into the following categories: :::{grid-item-card} How to :class-body: rocm-card-banner rocm-hue-12 +* [Programming guide](./how-to/hip_programming_guide.rst) * [Using ROCm for AI](./how-to/rocm-for-ai/index.rst) * [Using ROCm for HPC](./how-to/rocm-for-hpc/index.rst) * [Fine-tuning LLMs and inference optimization](./how-to/llm-fine-tuning-optimization/index.rst) * [System optimization](./how-to/system-optimization/index.rst) -* [AMD Instinct MI300X tuning guides](./how-to/tuning-guides/mi300x/index.rst) +* [AMD Instinct MI300X performance validation and tuning](./how-to/tuning-guides/mi300x/index.rst) * [GPU cluster networking](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html) * [System debugging](./how-to/system-debugging.md) * [Using MPI](./how-to/gpu-enabled-mpi.rst) diff --git a/docs/reference/gpu-arch-specs.rst b/docs/reference/gpu-arch-specs.rst index 536f83f2f..dde0a2eb8 100644 --- a/docs/reference/gpu-arch-specs.rst +++ b/docs/reference/gpu-arch-specs.rst @@ -37,11 +37,11 @@ For more information about ROCm hardware compatibility, see the ROCm `Compatibil - CDNA3 - gfx942 - 192 - - 304 + - 304 (38 per XCD) - 64 - 64 - 256 - - 32 + - 32 (4 per XCD) - 32 - 16 per 2 CUs - 64 per 2 CUs @@ -52,11 +52,11 @@ For more information about ROCm hardware compatibility, see the ROCm `Compatibil - CDNA3 - gfx942 - 128 - - 228 + - 228 (38 per XCD) - 64 - 64 - 256 - - 24 + - 24 (4 per XCD) - 32 - 16 per 2 CUs - 64 per 2 CUs @@ -82,7 +82,7 @@ For more information about ROCm hardware compatibility, see the ROCm `Compatibil - CDNA2 - gfx90a - 128 - - 208 + - 208 (104 per GCD) - 64 - 64 - @@ -788,3 +788,7 @@ scalar instructions. **GCD** Graphics Compute Die. + +**XCD** + +Accelerator Complex Die. diff --git a/docs/release/versions.md b/docs/release/versions.md index 3c01f5d72..e7d346057 100644 --- a/docs/release/versions.md +++ b/docs/release/versions.md @@ -8,7 +8,7 @@ | Version | Release date | | ------- | ------------ | -| [6.2.4](https://rocm.docs.amd.com/en/docs-6.2.4/) | October 18, 2024 | +| [6.2.4](https://rocm.docs.amd.com/en/docs-6.2.4/) | November 6, 2024 | | [6.2.2](https://rocm.docs.amd.com/en/docs-6.2.2/) | September 27, 2024 | | [6.2.1](https://rocm.docs.amd.com/en/docs-6.2.1/) | September 20, 2024 | | [6.2.0](https://rocm.docs.amd.com/en/docs-6.2.0/) | August 2, 2024 | diff --git a/docs/sphinx/_toc.yml.in b/docs/sphinx/_toc.yml.in index 9dd8af346..5180ceb17 100644 --- a/docs/sphinx/_toc.yml.in +++ b/docs/sphinx/_toc.yml.in @@ -23,6 +23,8 @@ subtrees: - caption: How to entries: + - file: how-to/programming_guide.rst + title: Programming guide - file: how-to/rocm-for-ai/index.rst title: Using ROCm for AI subtrees: @@ -70,9 +72,11 @@ subtrees: - file: how-to/system-optimization/w6000-v620.md title: AMD RDNA 2 - file: how-to/tuning-guides/mi300x/index.rst - title: AMD MI300X tuning guides + title: AMD MI300X performance validation and tuning subtrees: - entries: + - file: how-to/performance-validation/mi300x/vllm-benchmark.rst + title: Performance validation - file: how-to/tuning-guides/mi300x/system.rst title: System tuning - file: how-to/tuning-guides/mi300x/workload.rst diff --git a/docs/what-is-rocm.rst b/docs/what-is-rocm.rst index b95f3b153..1ee3988a3 100644 --- a/docs/what-is-rocm.rst +++ b/docs/what-is-rocm.rst @@ -6,24 +6,19 @@ What is ROCm? *********************************************************** -ROCm is an open-source stack, composed primarily of open-source software, designed for -graphics processing unit (GPU) computation. ROCm consists of a collection of drivers, development -tools, and APIs that enable GPU programming from low-level kernel to end-user applications. +ROCm is a software stack, composed primarily of open-source software, that +provides the tools for programming AMD Graphics Processing Units (GPUs), from +low-level kernels to high-level end-user applications. .. image:: data/rocm-software-stack-6_2_0.jpg :width: 800 :alt: AMD's ROCm software stack and neighboring technologies. :align: center -ROCm is powered by -:doc:`Heterogeneous-computing Interface for Portability (HIP) `; -it supports programming models, such as OpenMP and OpenCL, and includes all necessary open -source software compilers, debuggers, and libraries. It's fully integrated into machine learning (ML) -frameworks, such as PyTorch and TensorFlow. - -.. tip:: - If you're using Radeon GPUs, refer to the - :doc:`Radeon-specific ROCm documentation `. +Specifically, ROCm provides the tools for +:doc:`HIP (Heterogeneous-computing Interface for Portability) `, +OpenCL and OpenMP. These include compilers, libraries for high-level +functions, debuggers, profilers and runtimes. ROCm components =============================================== @@ -150,5 +145,5 @@ Runtimes :header: "Component", "Description" ":doc:`AMD Common Language Runtime (CLR) `", "Contains source code for AMD's common language runtimes: HIP and OpenCL" - ":doc:`HIP `", "AMD's GPU programming language extension and the GPU runtime" + ":doc:`HIP `", "C++ runtime API and kernel language that lets developers create portable applications for AMD and NVIDIA GPUs from single source code." ":doc:`ROCR-Runtime `", "User-mode API interfaces and libraries necessary for host applications to launch compute kernels on available HSA ROCm kernel agents" diff --git a/tools/autotag/components.xml b/tools/autotag/components.xml index a151b8628..c0eadced7 100644 --- a/tools/autotag/components.xml +++ b/tools/autotag/components.xml @@ -1,7 +1,7 @@ - diff --git a/tools/rocm-build/ROCm.mk b/tools/rocm-build/ROCm.mk index faca1661d..092d659b6 100644 --- a/tools/rocm-build/ROCm.mk +++ b/tools/rocm-build/ROCm.mk @@ -67,16 +67,18 @@ endef $(call adddep,amd_smi_lib,${ASAN_DEP}) $(call adddep,aqlprofile,${ASAN_DEP} hsa) -$(call adddep,clang-ocl,lightning rocm-cmake) $(call adddep,comgr,lightning devicelibs) $(call adddep,dbgapi,hsa comgr) $(call adddep,devicelibs,lightning) -$(call adddep,hip_on_rocclr,${ASAN_DEP} rocclr rocprofiler-register) +$(call adddep,hip_on_rocclr,${ASAN_DEP} hsa comgr hipcc rocprofiler-register) $(call adddep,hipcc,) $(call adddep,hipify_clang,hip_on_rocclr lightning) $(call adddep,hsa,${ASAN_DEP} thunk lightning devicelibs rocprofiler-register) $(call adddep,lightning,) -$(call adddep,opencl_on_rocclr,${ASAN_DEP} rocclr) +$(call adddep,omniperf,${ASAN_DEP}) +$(call adddep,omnitrace,hipcc hsa hip_on_rocclr rocm_smi_lib rocprofiler roctracer) +$(call adddep,opencl_icd_loader,) +$(call adddep,opencl_on_rocclr,${ASAN_DEP} hsa comgr opencl_icd_loader) $(call adddep,openmp_extras,thunk lightning devicelibs hsa) $(call adddep,rdc,${ASAN_DEP} rocm_smi_lib hsa rocprofiler) $(call adddep,rocclr,${ASAN_DEP} hsa comgr hipcc rocprofiler-register) @@ -87,14 +89,15 @@ $(call adddep,rocm-core,${ASAN_DEP}) $(call adddep,rocm-gdb,dbgapi) $(call adddep,rocminfo,${ASAN_DEP} hsa) $(call adddep,rocprofiler-register,${ASAN_DEP}) -$(call adddep,rocprofiler,${ASAN_DEP} hsa roctracer aqlprofile opencl_on_rocclr hip_on_rocclr comgr dbgapi rocm_smi_lib) +$(call adddep,rocprofiler-sdk,${ASAN_DEP} hsa aqlprofile opencl_on_rocclr hip_on_rocclr comgr) +$(call adddep,rocprofiler,${ASAN_DEP} hsa roctracer aqlprofile opencl_on_rocclr hip_on_rocclr comgr) $(call adddep,rocr_debug_agent,${ASAN_DEP} hip_on_rocclr hsa dbgapi) $(call adddep,roctracer,${ASAN_DEP} hsa hip_on_rocclr) $(call adddep,thunk,${ASAN_DEP}) # rocm-dev points to all possible last finish components of Stage1 build. rocm-dev-components :=rdc hipify_clang openmp_extras \ - rocm-core amd_smi_lib hipcc clang-ocl \ + omniperf omnitrace rocm-core amd_smi_lib hipcc \ rocm_bandwidth_test rocr_debug_agent rocm-gdb $(call adddep,rocm-dev,$(filter-out ${NOBUILD},${rocm-dev-components})) @@ -117,6 +120,7 @@ $(call adddep,mivisionx,amdmigraphx miopen-hip rpp lightning hipcc) $(call adddep,rccl,hip_on_rocclr hsa lightning hipcc rocm_smi_lib hipify_clang) $(call adddep,rocalution,rocblas rocsparse rocrand lightning hipcc) $(call adddep,rocblas,hip_on_rocclr openmp_extras lightning hipcc) +$(call adddep,rocal,mivisionx) $(call adddep,rocdecode,hip_on_rocclr lightning hipcc) $(call adddep,rocfft,hip_on_rocclr rocrand hiprand lightning hipcc openmp_extras) $(call adddep,rocmvalidationsuite,hip_on_rocclr hsa rocblas rocm-core lightning hipcc rocm_smi_lib) @@ -221,7 +225,7 @@ rocm-dev: T_rocm-dev ${OUT_DIR}/logs: sudo mkdir -p -m 775 "${ROCM_INSTALL_PATH}" && \ - sudo chown -R "$(shell id -u):$(shell id -g)" "${ROCM_INSTALL_PATH}" + sudo chown -R "$(shell id -u):$(shell id -g)" "/opt" sudo chown -R "$(shell id -u):$(shell id -g)" "/home/$(shell id -un)" mkdir -p "${@}" mkdir -p ${HOME}/.ccache diff --git a/tools/rocm-build/build_amdmigraphx.sh b/tools/rocm-build/build_amdmigraphx.sh index 9ddad2151..05a7528f5 100755 --- a/tools/rocm-build/build_amdmigraphx.sh +++ b/tools/rocm-build/build_amdmigraphx.sh @@ -22,12 +22,13 @@ build_amdmigraphx() { else GPU_TARGETS="gfx908;gfx90a;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params mkdir -p ${BUILD_DIR} && rm -rf ${BUILD_DIR}/* && mkdir -p ${HOME}/amdmigraphx && rm -rf ${HOME}/amdmigraphx/* rbuild package -d "${HOME}/amdmigraphx" -B "${BUILD_DIR}" \ --cxx="${ROCM_PATH}/llvm/bin/clang++" \ --cc="${ROCM_PATH}/llvm/bin/clang" \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DCMAKE_MODULE_LINKER_FLAGS="-Wl,--enable-new-dtags -Wl,--rpath,$ROCM_LIB_RPATH" \ -DGPU_TARGETS="${GPU_TARGETS}" \ -DCMAKE_INSTALL_RPATH="" diff --git a/tools/rocm-build/build_composable_kernel.sh b/tools/rocm-build/build_composable_kernel.sh index 577c2c6b2..7d1b38abc 100755 --- a/tools/rocm-build/build_composable_kernel.sh +++ b/tools/rocm-build/build_composable_kernel.sh @@ -17,9 +17,7 @@ build_miopen_ck() { mkdir "$BUILD_DIR" && cd "$BUILD_DIR" if [ -n "$GPU_ARCHS" ]; then - GPU_TARGETS="$GPU_ARCHS" - else - GPU_TARGETS="gfx908;gfx90a;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" + GPU_TARGETS="-DAMDGPU_TARGETS=${GPU_ARCHS}" fi if [ "${ASAN_CMAKE_PARAMS}" == "true" ] ; then @@ -43,7 +41,7 @@ build_miopen_ck() { ${LAUNCHER_FLAGS} \ -DINSTANCES_ONLY=ON \ -DENABLE_ASAN_PACKAGING=true \ - -DAMDGPU_TARGETS=${GPU_TARGETS} \ + "${GPU_TARGETS}" \ "$COMPONENT_SRC" else cmake -DBUILD_DEV=OFF \ @@ -63,9 +61,11 @@ build_miopen_ck() { -DROCM_DISABLE_LDCONFIG=ON \ -DROCM_PATH=${ROCM_PATH} \ -DCPACK_GENERATOR="${PKGTYPE^^}" \ + -DCMAKE_CXX_COMPILER="${ROCM_PATH}/llvm/bin/clang++" \ + -DCMAKE_C_COMPILER="${ROCM_PATH}/llvm/bin/clang" \ ${LAUNCHER_FLAGS} \ -DINSTANCES_ONLY=ON \ - -DAMDGPU_TARGETS=${GPU_TARGETS} \ + "${GPU_TARGETS}" \ "$COMPONENT_SRC" fi @@ -106,8 +106,6 @@ build_miopen_ckProf() { architectures='gfx10 gfx11 gfx90 gfx94' if [ -n "$GPU_ARCHS" ]; then architectures=$(echo ${GPU_ARCHS} | awk -F';' '{for(i=1;i<=NF;i++) a[substr($i,1,5)]} END{for(i in a) printf i" "}') - else - architectures='gfx10 gfx11 gfx90 gfx94' fi for arch in ${architectures} diff --git a/tools/rocm-build/build_hipblas.sh b/tools/rocm-build/build_hipblas.sh index 8a2b1d8bf..e09f61eb6 100755 --- a/tools/rocm-build/build_hipblas.sh +++ b/tools/rocm-build/build_hipblas.sh @@ -28,9 +28,10 @@ build_hipblas() { rebuild_lapack fi + init_rocm_common_cmake_params cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DUSE_CUDA=OFF \ -DBUILD_CLIENTS_TESTS=ON \ -DBUILD_CLIENTS_BENCHMARKS=ON \ diff --git a/tools/rocm-build/build_hipblaslt.sh b/tools/rocm-build/build_hipblaslt.sh index 4895adde2..f99eb7772 100755 --- a/tools/rocm-build/build_hipblaslt.sh +++ b/tools/rocm-build/build_hipblaslt.sh @@ -27,11 +27,12 @@ build_hipblaslt() { GPU_TARGETS=all fi + init_rocm_common_cmake_params CXX=$(set_build_variables CXX)\ cmake \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DTensile_LOGIC= \ -DTensile_CODE_OBJECT_VERSION=default \ -DTensile_CPU_THREADS= \ diff --git a/tools/rocm-build/build_hipcub.sh b/tools/rocm-build/build_hipcub.sh index c559fe102..592c81364 100755 --- a/tools/rocm-build/build_hipcub.sh +++ b/tools/rocm-build/build_hipcub.sh @@ -17,6 +17,7 @@ build_hipcub() { fi mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -27,7 +28,7 @@ build_hipcub() { CXX=$(set_build_variables CXX)\ cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DCMAKE_MODULE_PATH="${ROCM_PATH}/lib/cmake/hip;${ROCM_PATH}/hip/cmake" \ -Drocprim_DIR="${ROCM_PATH}/rocprim" \ -DBUILD_TEST=ON \ diff --git a/tools/rocm-build/build_hipfft.sh b/tools/rocm-build/build_hipfft.sh index d8d077f1f..17e8c6c9e 100755 --- a/tools/rocm-build/build_hipfft.sh +++ b/tools/rocm-build/build_hipfft.sh @@ -16,6 +16,7 @@ build_hipfft() { cd $COMPONENT_SRC mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -26,7 +27,7 @@ build_hipfft() { cmake \ -DCMAKE_CXX_COMPILER=$(set_build_variables CXX) \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DCMAKE_MODULE_PATH="${ROCM_PATH}/lib/cmake/hip" \ -DCMAKE_SKIP_BUILD_RPATH=TRUE \ diff --git a/tools/rocm-build/build_hiprand.sh b/tools/rocm-build/build_hiprand.sh index 293b05b6c..a9cdcbb53 100755 --- a/tools/rocm-build/build_hiprand.sh +++ b/tools/rocm-build/build_hiprand.sh @@ -61,7 +61,6 @@ build_hiprand() { rm -rf _CPack_Packages/ && find -name '*.o' -delete mkdir -p $PACKAGE_DIR && cp ${BUILD_DIR}/*.${PKGTYPE} $PACKAGE_DIR - $SCCACHE_BIN -s || echo "Unable to display sccache stats" } clean_hiprand() { diff --git a/tools/rocm-build/build_hipsolver.sh b/tools/rocm-build/build_hipsolver.sh index ded0279df..22c778832 100755 --- a/tools/rocm-build/build_hipsolver.sh +++ b/tools/rocm-build/build_hipsolver.sh @@ -27,10 +27,11 @@ build_hipsolver() { rebuild_lapack fi + init_rocm_common_cmake_params cmake \ -DUSE_CUDA=OFF \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DBUILD_CLIENTS_TESTS=ON \ -DBUILD_CLIENTS_BENCHMARKS=ON \ -DBUILD_CLIENTS_SAMPLES=ON \ diff --git a/tools/rocm-build/build_hipsparse.sh b/tools/rocm-build/build_hipsparse.sh index 727421635..10171905c 100755 --- a/tools/rocm-build/build_hipsparse.sh +++ b/tools/rocm-build/build_hipsparse.sh @@ -22,11 +22,12 @@ build_hipsparse() { echo "CXX compiler: $CXX" mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params cmake \ -DCPACK_SET_DESTDIR=OFF \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DUSE_CUDA=OFF \ -DBUILD_CLIENTS_SAMPLES=ON \ -DBUILD_CLIENTS_TESTS=ON \ diff --git a/tools/rocm-build/build_hipsparselt.sh b/tools/rocm-build/build_hipsparselt.sh index 99b70edd0..dddff1437 100755 --- a/tools/rocm-build/build_hipsparselt.sh +++ b/tools/rocm-build/build_hipsparselt.sh @@ -28,6 +28,7 @@ build_hipsparselt() { cd $COMPONENT_SRC mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -41,7 +42,7 @@ build_hipsparselt() { cmake \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DTensile_LOGIC= \ -DTensile_CODE_OBJECT_VERSION=default \ -DTensile_CPU_THREADS= \ diff --git a/tools/rocm-build/build_hiptensor.sh b/tools/rocm-build/build_hiptensor.sh index c239caab9..04346797b 100755 --- a/tools/rocm-build/build_hiptensor.sh +++ b/tools/rocm-build/build_hiptensor.sh @@ -16,6 +16,8 @@ build_hiptensor() { cd "$COMPONENT_SRC" mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params + if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -25,7 +27,7 @@ build_hiptensor() { cmake \ -B "${BUILD_DIR}" \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ $(set_build_variables CMAKE_C_CXX) \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ ${LAUNCHER_FLAGS} \ diff --git a/tools/rocm-build/build_lightning.sh b/tools/rocm-build/build_lightning.sh index dbf9fae22..ed4ba6bc2 100755 --- a/tools/rocm-build/build_lightning.sh +++ b/tools/rocm-build/build_lightning.sh @@ -17,7 +17,8 @@ printUsage() { echo " -r, --release Build a release version of the package" echo " -a, --address_sanitizer Enable address sanitizer (enabled by default)" echo " -A --no_address_sanitizer Disable address sanitizer" - echo " -s, --static Supports static CI by accepting this param & not bailing out. No effect of the param though" + echo " -s, --static Build static lib (.a). build instead of dynamic/shared(.so) " + echo " -w, --wheel Creates python wheel package of rocm-llvm. It needs to be used along with -r option" echo " -l, --build_llvm_static Build LLVM libraries statically linked. Default is to build dynamic linked libs" echo " -o, --outdir Print path of output directory containing packages of type referred to by pkg_type" @@ -42,6 +43,7 @@ DEB_PATH="$(getDebPath lightning)" RPM_PATH="$(getRpmPath lightning)" INSTALL_PATH="${ROCM_INSTALL_PATH}/lib/llvm" LLVM_ROOT_LCL="${LLVM_ROOT}" +ROCM_WHEEL_DIR="${BUILD_PATH}/_wheel" TARGET="all" MAKEOPTS="$DASH_JAY" @@ -69,14 +71,29 @@ ASSERT_LLVM_VERSION_MINOR="" SKIP_LIT_TESTS=0 BUILD_MANPAGES="ON" +STATIC_FLAG= SANITIZER_AMDGPU=1 HSA_INC_PATH="$WORK_ROOT/ROCR-Runtime/src/inc" COMGR_INC_PATH="$WORK_ROOT/llvm-project/amd/comgr/include" -VALID_STR=`getopt -o htcV:v:draAslo:BPNM --long help,alt,clean,assert_llvm_ver_major:,assert_llvm_ver_minor:,debug,release,address_sanitizer,no_address_sanitizer,static,build_llvm_static,build,package,skip_lit_tests,skip_man_pages,outdir: -- "$@"` +VALID_STR=`getopt -o htcV:v:draAswlo:BPNM --long help,alt,clean,assert_llvm_ver_major:,assert_llvm_ver_minor:,debug,release,address_sanitizer,no_address_sanitizer,static,build_llvm_static,wheel,build,package,skip_lit_tests,skip_man_pages,outdir: -- "$@"` eval set -- "$VALID_STR" +set_dwarf_version(){ + case "$DISTRO_ID" in + (sles*|rhel*) + SET_DWARF_VERSION_4="-gdwarf-4" + ;; + (*) + SET_DWARF_VERSION_4="" + ;; + esac + export CFLAGS="$CFLAGS $SET_DWARF_VERSION_4 " + export CXXFLAGS="$CXXFLAGS $SET_DWARF_VERSION_4 " + export ASMFLAGS="$ASMFLAGS $SET_DWARF_VERSION_4 " +} + while true ; do case "$1" in @@ -95,6 +112,7 @@ do (-r | --release) BUILD_TYPE="Release" ; shift ;; (-a | --address_sanitizer) + set_dwarf_version SANITIZER_AMDGPU=1 ; HSA_INC_PATH="$WORK_ROOT/hsa/runtime/opensrc/hsa-runtime/inc" ; COMGR_INC_PATH="$WORK_ROOT/external/llvm-project/amd/comgr/include" ; shift ;; @@ -103,9 +121,12 @@ do unset HSA_INC_PATH ; unset COMGR_INC_PATH ; shift ;; (-s | --static) - SHARED_LIBS="OFF" ; shift ;; + SHARED_LIBS="OFF" ; + STATIC_FLAG="-DBUILD_SHARED_LIBS=$SHARED_LIBS" ; shift ;; (-l | --build_llvm_static) BUILD_LLVM_DYLIB="OFF"; shift ;; + (-w | --wheel) + WHEEL_PACKAGE=true ; shift ;; (-o | --outdir) TARGET="outdir"; PKGTYPE=$2 ; OUT_DIR_SPECIFIED=1 ; ((CLEAN_OR_OUT|=2)) ; shift 2 ;; (-B | --build) @@ -151,6 +172,7 @@ else fi clean_lightning() { + rm -rf "$ROCM_WHEEL_DIR" rm -rf "$BUILD_PATH" rm -rf "$DEB_PATH" rm -rf "$RPM_PATH" @@ -196,7 +218,10 @@ LLVM_VERSION_MINOR="" LLVM_VERSION_PATCH="" LLVM_VERSION_SUFFIX="" get_llvm_version() { - local LLVM_VERSIONS=($(awk '/set\(LLVM_VERSION/ {print substr($2,1,length($2)-1)}' ${LLVM_ROOT_LCL}/CMakeLists.txt)) + local LLVM_VERSIONS=($(awk '/set\(LLVM_VERSION/ {print substr($2,1,length($2)-1)}' ${LLVM_ROOT_LCL}/../cmake/Modules/LLVMVersion.cmake)) + if [ ${#LLVM_VERSIONS[@]} -eq 0 ]; then + LLVM_VERSIONS=($(awk '/set\(LLVM_VERSION/ {print substr($2,1,length($2)-1)}' ${LLVM_ROOT_LCL}/CMakeLists.txt)) + fi LLVM_VERSION_MAJOR=${LLVM_VERSIONS[0]} LLVM_VERSION_MINOR=${LLVM_VERSIONS[1]} LLVM_VERSION_PATCH=${LLVM_VERSIONS[2]} @@ -260,15 +285,22 @@ build_lightning() { if [ ! -e Makefile ]; then echo "Building LLVM CMake environment" - if [ -e "$LLVM_ROOT_LCL/../flang/docs/AssumedRank.md" ]; then - FLANG_NEW=1 - LLVM_PROJECTS="$LLVM_PROJECTS;flang;mlir" - else - echo "NOT building project flang" - fi + if [ -e "$LLVM_ROOT_LCL/../flang/AFARrelease" ]; then + FLANG_NEW=1 + LLVM_PROJECTS="$LLVM_PROJECTS;flang;mlir" + ENABLE_RUNTIMES="$ENABLE_RUNTIMES;openmp"; + else + if [[ "${JOB_NAME}" != *afar* ]] && [ -e "$LLVM_ROOT_LCL/../flang/DoROCmRelease" ]; then + FLANG_NEW=1 + LLVM_PROJECTS="$LLVM_PROJECTS;flang;mlir" + else + echo "NOT building project flang" + fi + fi set -x cmake $(rocm_cmake_params) ${GEN_NINJA} \ + ${STATIC_FLAG} \ -DCMAKE_INSTALL_PREFIX="$INSTALL_PATH" \ -DLLVM_TARGETS_TO_BUILD="AMDGPU;X86" \ -DLLVM_ENABLE_PROJECTS="$LLVM_PROJECTS" \ @@ -307,9 +339,9 @@ build_lightning() { -DCMAKE_SHARED_LINKER_FLAGS=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_LLVM_LIB_RPATH \ -DROCM_LLVM_BACKWARD_COMPAT_LINK="$ROCM_INSTALL_PATH/llvm" \ -DROCM_LLVM_BACKWARD_COMPAT_LINK_TARGET="./lib/llvm" \ - -DCLANG_LINK_FLANG_LEGACY=ON \ - -DCMAKE_CXX_STANDARD=17 \ - -DFLANG_INCLUDE_DOCS=OFF \ + -DCLANG_LINK_FLANG_LEGACY=ON \ + -DCMAKE_CXX_STANDARD=17 \ + -DFLANG_INCLUDE_DOCS=OFF \ "$LLVM_ROOT_LCL" set +x echo "CMake complete" @@ -326,8 +358,23 @@ build_lightning() { echo "End Workaround for race condition" cmake --build . -- $MAKEOPTS + case "$DISTRO_ID" in + (rhel*|centos*) + RHEL_BUILD=1 + ;; + (*) + RHEL_BUILD=0 + ;; + esac + if [ $SKIP_LIT_TESTS -eq 0 ]; then - if [ "$DISTRO_NAME" != "sles" ] && [ $BUILD_ALT != 1 ]; then + if [ $RHEL_BUILD -eq 1 ] && [ $BUILD_ALT != 1 ]; then + if [ $FLANG_NEW -eq 1 ]; then + cmake --build . -- $MAKEOPTS check-lld check-mlir + else + cmake --build . -- $MAKEOPTS check-lld + fi + elif [ "$DISTRO_NAME" != "sles" ] && [ $BUILD_ALT != 1 ]; then if [ $FLANG_NEW -eq 1 ]; then cmake --build . -- $MAKEOPTS check-llvm check-clang check-lld check-mlir else @@ -733,7 +780,7 @@ package_lightning_static() { local amd_compiler_commands=("amdclang" "amdclang++" "amdclang-cl" "amdclang-cpp" "amdflang" "amdlld" "offload-arch") local amd_man_pages=("amdclang.1.gz" "flang.1.gz" "amdflang.1.gz") local core_bin=("amdgpu-arch" "amdgpu-offload-arch" "amdlld" "amdllvm" "clang" "clang++" "clang-${LLVM_VERSION_MAJOR}" "clang-cl" - "clang-cpp" "clang-build-select-link" "clang-offload-bundler" "clang-offload-packager" "clang-offload-wrapper" "flang" "flang-new" + "clang-cpp" "clang-build-select-link" "clang-offload-bundler" "clang-offload-packager" "clang-offload-wrapper" "clang-linker-wrapper" "clang-nvlink-wrapper" "flang" "flang-new" "ld64.lld" "ld.lld" "llc" "lld" "lld-link" "llvm-ar" "llvm-bitcode-strip" "llvm-dwarfdump" "llvm-install-name-tool" "llvm-link" "llvm-mc" "llvm-objcopy" "llvm-objdump" "llvm-otool" "llvm-ranlib" "llvm-readelf" "llvm-readobj" "llvm-strip" "nvidia-arch" "nvptx-arch" "offload-arch" "opt" "wasm-ld" "amdclang" "amdclang++" "amdclang-${LLVM_VERSION_MAJOR}" "amdclang-cl" @@ -934,7 +981,7 @@ package_lightning_static() { if [ $BUILD_ALT -eq 0 ]; then echo "cp -R $LLVM_ROOT_LCL/LICENSE.TXT \$RPM_BUILD_ROOT/$licenseDir" >> $specFile - echo "cp -P $backwardsCompatibleSymlink \$RPM_BUILD_ROOT/$ROCM_INSTALL_PATH" >> $specFile + echo "cp -P $backwardsCompatibleSymlink \$RPM_BUILD_ROOT/$ROCM_INSTALL_PATH" >> $specFile else echo "cp -R $LLVM_PROJECT_ALT_ROOT/EULA \$RPM_BUILD_ROOT/$licenseDir" >> $specFile echo "cp -R $LLVM_PROJECT_ALT_ROOT/DISCLAIMER.txt \$RPM_BUILD_ROOT/$licenseDir" >> $specFile @@ -948,7 +995,6 @@ package_lightning_static() { echo "cp -d \"$distBin/flang\" \$RPM_BUILD_ROOT/$installPath/bin/" >> $specFile - # Copy the config files if [ $BUILD_ALT -eq 0 ]; then echo "cp -d \"$distBin\"/*.cfg \$RPM_BUILD_ROOT/$installPath/bin/" >> $specFile fi @@ -970,14 +1016,12 @@ package_lightning_static() { if [ "$BUILD_MANPAGES" == "ON" ]; then if [ $BUILD_ALT -eq 0 ]; then echo "mkdir -p \$RPM_BUILD_ROOT/$installPath/share/man/man1" >> $specFile - for i in "${core_man_pages[@]}"; do if [ -f "$distMan/man1/$i" ]; then echo "gzip -f $distMan/man1/$i" >> $specFile echo "cp -d $distMan/man1/${i}.gz \$RPM_BUILD_ROOT/$installPath/share/man/man1/" >> $specFile fi done - if [ -f "$distMan/man1/clang.1.gz" ]; then for i in "${amd_man_pages[@]}"; do echo "ln -sf clang.1.gz \"$distMan/man1/$i\"" >> $specFile @@ -1064,7 +1108,6 @@ package_lightning_static() { contains "$bin" "${core_bin[@]}" "${amd_compiler_commands[@]}" && continue echo "cp -d \"$i\" \$RPM_BUILD_ROOT/$installPath/bin/" >> $specFileExtra done - for i in "$distLib"/*; do lib=$(basename "$i") contains "$lib" "${core_lib[@]}" && continue @@ -1072,18 +1115,15 @@ package_lightning_static() { done echo "cp -R $distInc \$RPM_BUILD_ROOT/$installPath" >> $specFileExtra - echo "rm -rf \$RPM_BUILD_ROOT/$installPath/lib/clang" >> $specFileExtra if [ $FLANG_NEW -eq 1 ]; then - echo "rm -rf \$RPM_BUILD_ROOT/$installPath/include/flang" >> $specFileExtra fi if [ "$BUILD_MANPAGES" == "ON" ]; then if [ $BUILD_ALT -eq 0 ]; then echo "mkdir -p \$RPM_BUILD_ROOT/$installPath/share/man/man1" >> $specFileExtra - for i in "${extra_man_pages[@]}"; do if [ -f "$distMan/man1/$i" ]; then echo "gzip -f $distMan/man1/$i" >> $specFileExtra @@ -1125,34 +1165,34 @@ package_docs() { local packageName="rocm-llvm-docs" local packageSummary="ROCm LLVM compiler documentation" local packageSummaryLong="Documenation for LLVM $llvmParsedVersion" - local installPath="$ROCM_INSTALL_PATH/lib/llvm/share" local packageArch="amd64" local packageVersion="${llvmParsedVersion}.${LLVM_COMMIT_GITDATE}" local packageMaintainer="ROCm Compiler Support " - local distDoc="$INSTALL_PATH/share/doc" + local distDoc="$INSTALL_PATH/share/doc/LLVM" local licenseDir="$ROCM_INSTALL_PATH/share/doc/$packageName" local packageDir="$BUILD_PATH/package" local packageDeb="$packageDir/deb" local controlFile="$packageDeb/DEBIAN/control" + local debDependencies="rocm-core" local packageRpm="$packageDir/rpm" local specFile="$packageDir/$packageName.spec" + local rpmRequires="rocm-core" rm -rf "$packageDir" echo "rm -rf $packageDir" if [ "$PACKAGEEXT" = "deb" ]; then - mkdir -p "$packageDeb/$installPath" - mkdir "${controlFile%/*}" mkdir -p "$packageDeb/$licenseDir" + mkdir "${controlFile%/*}" cp -r "$LLVM_ROOT_LCL/LICENSE.TXT" "$packageDeb/$licenseDir" - cp -r "$distDoc" "$packageDeb/$installPath/doc" + cp -r "$distDoc" "$packageDeb/$licenseDir" { echo "Package: $packageName" @@ -1162,6 +1202,7 @@ package_docs() { echo "Maintainer: $packageMaintainer" echo "Version: ${packageVersion}.${ROCM_LIBPATCH_VERSION}-${JOB_DESIGNATOR}${BUILD_ID}~${DISTRO_RELEASE}" echo "Release: ${JOB_DESIGNATOR}${BUILD_ID}~${DISTRO_RELEASE}" + echo "Depends: $debDependencies" echo "Recommends: $debRecommends" echo "Description: $packageSummary" echo " $packageSummaryLong" @@ -1182,6 +1223,7 @@ package_docs() { echo "Summary: $packageSummary" >> $specFile echo "Group: System Environment/Libraries" >> $specFile echo "License: ASL 2.0 with exceptions" >> $specFile + echo "Requires: $rpmRequires" >> $specFile echo "%description" >> $specFile echo "$packageSummaryLong" >> $specFile @@ -1190,16 +1232,13 @@ package_docs() { echo "%setup -T -D -c -n $packageName" >> $specFile echo "%install" >> $specFile - echo "rm -rf \$RPM_BUILD_ROOT/$installPath" >> $specFile - echo "mkdir -p \$RPM_BUILD_ROOT/$installPath/doc" >> $specFile echo "mkdir -p \$RPM_BUILD_ROOT/$licenseDir" >> $specFile echo "cp -R $LLVM_ROOT_LCL/LICENSE.TXT \$RPM_BUILD_ROOT/$licenseDir" >> $specFile - echo "cp -R \"$distDoc\" \$RPM_BUILD_ROOT/$installPath" >> $specFile + echo "cp -R \"$distDoc\" \$RPM_BUILD_ROOT/$licenseDir" >> $specFile echo "%clean" >> $specFile - echo "rm -rf \$RPM_BUILD_ROOT/$installPath" >> $specFile echo "%files " >> $specFile echo "%defattr(-,root,root,-)" >> $specFile @@ -1232,6 +1271,18 @@ build() { fi } +create_wheel_package() { + echo "Creating rocm-llvm wheel package" + mkdir -p "$ROCM_WHEEL_DIR" + cp -f $SCRIPT_ROOT/generate_setup_py.py $ROCM_WHEEL_DIR + cp -f $SCRIPT_ROOT/repackage_wheel.sh $ROCM_WHEEL_DIR + cd $ROCM_WHEEL_DIR + # Currently only supports python3.6 + ./repackage_wheel.sh $RPM_PATH/rocm-llvm*.rpm python3.6 + # Copy the wheel created to RPM folder which will be uploaded to artifactory + mv "$ROCM_WHEEL_DIR"/dist/*.whl "$RPM_PATH" +} + case $TARGET in (clean) clean_lightning ;; (all) @@ -1250,4 +1301,9 @@ case $TARGET in (*) die "Invalid target $TARGET" ;; esac +if [[ $WHEEL_PACKAGE == true ]]; then + echo "Wheel Package build started !!!!" + create_wheel_package +fi + echo "Operation complete" diff --git a/tools/rocm-build/build_miopen-deps.sh b/tools/rocm-build/build_miopen-deps.sh index 48910fac0..51bd23f1a 100755 --- a/tools/rocm-build/build_miopen-deps.sh +++ b/tools/rocm-build/build_miopen-deps.sh @@ -44,7 +44,7 @@ build_miopen_deps() { cd "$COMPONENT_SRC" # Commenting the rocMLIR & composable_kernel from requirements.txt - sed -i '/ROCmSoftwarePlatform\/rocMLIR@\|ROCmSoftwarePlatform\/composable_kernel@/s/^/#/' requirements.txt + sed -i '/ROCm\/rocMLIR@\|ROCm\/composable_kernel@/s/^/#/' requirements.txt # Extract MLIR commit from requirements.txt MLIR_COMMIT="$(awk '/rocMLIR/ {split($1, s, "@"); print s[2]}' requirements.txt)" diff --git a/tools/rocm-build/build_miopen-hip.sh b/tools/rocm-build/build_miopen-hip.sh index e4267b96d..b07c0f484 100755 --- a/tools/rocm-build/build_miopen-hip.sh +++ b/tools/rocm-build/build_miopen-hip.sh @@ -13,7 +13,7 @@ build_miopen_hip() { echo "Start build" cd $COMPONENT_SRC - + git config --global --add safe.directory "$COMPONENT_SRC" checkout_lfs if [ "${ENABLE_ADDRESS_SANITIZER}" == "true" ]; then @@ -22,8 +22,9 @@ build_miopen_hip() { fi mkdir "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params cmake \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DMIOPEN_BACKEND=HIP \ -DCMAKE_CXX_COMPILER="${ROCM_PATH}/llvm/bin/clang++" \ -DCMAKE_C_COMPILER="${ROCM_PATH}/llvm/bin/clang" \ diff --git a/tools/rocm-build/build_mivisionx.sh b/tools/rocm-build/build_mivisionx.sh index 7bfe40dc3..ae37444b5 100755 --- a/tools/rocm-build/build_mivisionx.sh +++ b/tools/rocm-build/build_mivisionx.sh @@ -16,6 +16,8 @@ build_mivisionx() { BUILD_DEV=OFF fi + init_rocm_common_cmake_params + if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" else @@ -23,7 +25,7 @@ build_mivisionx() { fi cmake \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DROCM_PATH="$ROCM_PATH" \ -DBUILD_DEV=$BUILD_DEV \ -DCMAKE_INSTALL_LIBDIR=$(getInstallLibDir) \ @@ -34,6 +36,7 @@ build_mivisionx() { "$COMPONENT_SRC" cmake --build "$BUILD_DIR" -- -j${PROC} + cmake --build "$BUILD_DIR" -- install cpack -G ${PKGTYPE^^} rm -rf _CPack_Packages/ && find -name '*.o' -delete diff --git a/tools/rocm-build/build_omniperf.sh b/tools/rocm-build/build_omniperf.sh new file mode 100755 index 000000000..78ac7ab9a --- /dev/null +++ b/tools/rocm-build/build_omniperf.sh @@ -0,0 +1,171 @@ +#!/bin/bash + +source "$(dirname "${BASH_SOURCE}")/compute_utils.sh" + +printUsage() { + echo + echo "Usage: ${BASH_SOURCE##*/} [options ...]" + echo + echo "Options:" + echo " -c, --clean Clean output and delete all intermediate work" + echo " -s, --static Build static lib (.a). build instead of dynamic/shared(.so) " + echo " -p, --package Specify packaging format" + echo " -r, --release Make a release build instead of a debug build" + echo " -a, --address_sanitizer Enable address sanitizer" + echo " -o, --outdir Print path of output directory containing packages of + type referred to by pkg_type" + echo " -w, --wheel Creates python wheel package of omniperf. + It needs to be used along with -r option" + echo " -h, --help Prints this help" + echo + echo "Possible values for :" + echo " deb -> Debian format (default)" + echo " rpm -> RPM format" + echo + + return 0 +} + +API_NAME="omniperf" +PROJ_NAME="$API_NAME" +LIB_NAME="lib${API_NAME}" +TARGET="build" +MAKETARGET="deb" +PACKAGE_ROOT="$(getPackageRoot)" +PACKAGE_LIB="$(getLibPath)" +BUILD_DIR="$(getBuildPath $API_NAME)" +PACKAGE_DEB="$(getPackageRoot)/deb/$API_NAME" +PACKAGE_RPM="$(getPackageRoot)/rpm/$API_NAME" +ROCM_WHEEL_DIR="${BUILD_DIR}/_wheel" +BUILD_TYPE="Debug" +MAKE_OPTS="$DASH_JAY -C $BUILD_DIR" +SHARED_LIBS="ON" +CLEAN_OR_OUT=0; +MAKETARGET="deb" +PKGTYPE="deb" +WHEEL_PACKAGE=false + + +#parse the arguments +VALID_STR=$(getopt -o hcraso:p:w --long help,clean,release,static,address_sanitizer,outdir:,package:,wheel -- "$@") +eval set -- "$VALID_STR" + +while true ; +do + case "$1" in + -h | --help) + printUsage ; exit 0;; + -c | --clean) + TARGET="clean" ; ((CLEAN_OR_OUT|=1)) ; shift ;; + -r | --release) + BUILD_TYPE="Release" ; shift ;; + -a | --address_sanitizer) + set_asan_env_vars + set_address_sanitizer_on ; shift ;; + -s | --static) + SHARED_LIBS="OFF" ; shift ;; + -o | --outdir) + TARGET="outdir"; PKGTYPE=$2 ; OUT_DIR_SPECIFIED=1 ; ((CLEAN_OR_OUT|=2)) ; shift 2 ;; + -p | --package) + MAKETARGET="$2" ; shift 2 ;; + -w | --wheel) + WHEEL_PACKAGE=true ; shift ;; + --) shift; break;; # end delimiter + *) + echo " This should never come but just incase : UNEXPECTED ERROR Parm : [$1] ">&2 ; exit 20;; + esac + +done + +RET_CONFLICT=1 +check_conflicting_options "$CLEAN_OR_OUT" "$PKGTYPE" "$MAKETARGET" +if [ $RET_CONFLICT -ge 30 ]; then + print_vars "$API_NAME" "$TARGET" "$BUILD_TYPE" "$SHARED_LIBS" "$CLEAN_OR_OUT" "$PKGTYPE" "$MAKETARGET" + exit $RET_CONFLICT +fi + +clean() { + echo "Cleaning $PROJ_NAME" + rm -rf "$ROCM_WHEEL_DIR" + rm -rf "$BUILD_DIR" + rm -rf "$PACKAGE_DEB" + rm -rf "$PACKAGE_RPM" + rm -rf "$PACKAGE_ROOT/${PROJ_NAME:?}" + rm -rf "$PACKAGE_LIB/${LIB_NAME:?}"* +} + +build() { + echo "Building $PROJ_NAME" + if [ "$DISTRO_ID" = centos-7 ]; then + echo "Skip make and uploading packages for Omniperf on Centos7 distro, due to python dependency" + exit 0 + fi + + if [ ! -d "$BUILD_DIR" ]; then + mkdir -p "$BUILD_DIR" + pushd "$BUILD_DIR" || exit + + echo "ROCm CMake Params: $(rocm_cmake_params)" + echo "ROCm Common CMake Params: $(rocm_common_cmake_params)" + + print_lib_type $SHARED_LIBS + cmake \ + $(rocm_cmake_params) \ + $(rocm_common_cmake_params) \ + -DCHECK_PYTHON_DEPS=NO \ + -DPYTHON_DEPS=${BUILD_DIR}/python-libs \ + -DMOD_INSTALL_PATH=${BUILD_DIR}/modulefiles \ + "$OMNIPERF_ROOT" + fi + + make $MAKE_OPTS + make $MAKE_OPTS install + make $MAKE_OPTS package + + copy_if DEB "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_DEB" "$BUILD_DIR/${API_NAME}"*.deb + copy_if RPM "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_RPM" "$BUILD_DIR/${API_NAME}"*.rpm +} + +create_wheel_package() { + echo "Creating Omniperf wheel package" + + # Copy the setup.py generator to build folder + mkdir -p "$ROCM_WHEEL_DIR" + cp -f "$SCRIPT_ROOT"/generate_setup_py.py "$ROCM_WHEEL_DIR" + cp -f "$SCRIPT_ROOT"/repackage_wheel.sh "$ROCM_WHEEL_DIR" + cd "$ROCM_WHEEL_DIR" || exit + + # Currently only supports python3.6 + ./repackage_wheel.sh "$BUILD_DIR"/*.rpm python3.6 + + # Copy the wheel created to RPM folder which will be uploaded to artifactory + copy_if WHL "WHL" "$PACKAGE_RPM" "$ROCM_WHEEL_DIR"/dist/*.whl +} + +print_output_directory() { + case ${PKGTYPE} in + ("deb") + echo "${PACKAGE_DEB}";; + ("rpm") + echo "${PACKAGE_RPM}";; + (*) + echo "Invalid package type \"${PKGTYPE}\" provided for -o" >&2; exit 1;; + esac + exit +} + +verifyEnvSetup + +case "$TARGET" in + (clean) clean ;; + (build) build ;; + (outdir) print_output_directory ;; + (*) die "Invalid target $TARGET" ;; +esac + +if [[ $WHEEL_PACKAGE == true ]]; then + echo "Wheel Package build started !!!!" + create_wheel_package +fi + +echo "Operation complete" \ No newline at end of file diff --git a/tools/rocm-build/build_omnitrace.sh b/tools/rocm-build/build_omnitrace.sh new file mode 100755 index 000000000..c604351de --- /dev/null +++ b/tools/rocm-build/build_omnitrace.sh @@ -0,0 +1,191 @@ +#!/bin/bash + +source "$(dirname "${BASH_SOURCE}")/compute_utils.sh" + +printUsage() { + echo + echo "Usage: ${BASH_SOURCE##*/} [options ...]" + echo + echo "Options:" + echo " -c, --clean Clean output and delete all intermediate work" + echo " -s, --static Build static lib (.a). build instead of dynamic/shared(.so) " + echo " -p, --package Specify packaging format" + echo " -r, --release Make a release build instead of a debug build" + echo " -a, --address_sanitizer Enable address sanitizer" + echo " -o, --outdir Print path of output directory containing packages of + type referred to by pkg_type" + echo " -w, --wheel Creates python wheel package of omnitrace. + It needs to be used along with -r option" + echo " -h, --help Prints this help" + echo + echo "Possible values for :" + echo " deb -> Debian format (default)" + echo " rpm -> RPM format" + echo + + return 0 +} + +API_NAME="omnitrace" +PROJ_NAME="$API_NAME" +LIB_NAME="lib${API_NAME}" +TARGET="build" +MAKETARGET="deb" +PACKAGE_ROOT="$(getPackageRoot)" +PACKAGE_LIB="$(getLibPath)" +BUILD_DIR="$(getBuildPath $API_NAME)" +PACKAGE_DEB="$(getPackageRoot)/deb/$API_NAME" +PACKAGE_RPM="$(getPackageRoot)/rpm/$API_NAME" +BUILD_TYPE="Debug" +MAKE_OPTS="-j 8" +SHARED_LIBS="ON" +CLEAN_OR_OUT=0 +MAKETARGET="deb" +PKGTYPE="deb" +ASAN=0 + +#parse the arguments +VALID_STR=$(getopt -o hcraso:p:w --long help,clean,release,address_sanitizer,static,outdir:,package:,wheel -- "$@") +eval set -- "$VALID_STR" + +while true; do + case "$1" in + -h | --help) + printUsage + exit 0 + ;; + -c | --clean) + TARGET="clean" + ((CLEAN_OR_OUT |= 1)) + shift + ;; + -r | --release) + BUILD_TYPE="RelWithDebInfo" + shift + ;; + -a | --address_sanitizer) + ack_and_ignore_asan + + ASAN=1 + shift + ;; + -s | --static) + SHARED_LIBS="OFF" + shift + ;; + -o | --outdir) + TARGET="outdir" + PKGTYPE=$2 + ((CLEAN_OR_OUT |= 2)) + shift 2 + ;; + -p | --package) + MAKETARGET="$2" + shift 2 + ;; + -w | --wheel) + echo "omnitrace: wheel build option accepted and ignored" + shift + ;; + --) + shift + break + ;; + *) + echo " This should never come but just incase : UNEXPECTED ERROR Parm : [$1] " >&2 + exit 20 + ;; + esac + +done + +RET_CONFLICT=1 +check_conflicting_options $CLEAN_OR_OUT $PKGTYPE $MAKETARGET +if [ $RET_CONFLICT -ge 30 ]; then + print_vars $API_NAME $TARGET $BUILD_TYPE $SHARED_LIBS $CLEAN_OR_OUT $PKGTYPE $MAKETARGET + exit $RET_CONFLICT +fi + +clean() { + echo "Cleaning $PROJ_NAME" + rm -rf "$BUILD_DIR" + rm -rf "$PACKAGE_DEB" + rm -rf "$PACKAGE_RPM" + rm -rf "$PACKAGE_ROOT/${PROJ_NAME:?}" + rm -rf "$PACKAGE_LIB/${LIB_NAME:?}"* +} + +build_omnitrace() { + echo "Building $PROJ_NAME" + if [ "$DISTRO_ID" = "mariner-2.0" ] || [ "$DISTRO_ID" = "ubuntu-24.04" ] || [ "$DISTRO_ID" = "azurelinux-3.0" ]; then + echo "Skip make and uploading packages for Omnitrace on \"${DISTRO_ID}\" distro" + exit 0 + fi + + if [ $ASAN == 1 ]; then + echo "Skip make and uploading packages for Omnitrace on ASAN build" + exit 0 + fi + if [ ! -d "$BUILD_DIR" ]; then + mkdir -p "$BUILD_DIR" + echo "Created build directory: $BUILD_DIR" + fi + + echo "Build directory: $BUILD_DIR" + pushd "$BUILD_DIR" || exit + print_lib_type $SHARED_LIBS + + echo "ROCm CMake Params: $(rocm_cmake_params)" + echo "ROCm Common CMake Params: $(rocm_common_cmake_params)" + + + if [ $ASAN == 1 ]; then + echo "Address Sanitizer path" + + else + cmake \ + $(rocm_cmake_params) \ + $(rocm_common_cmake_params) \ + -DOMNITRACE_BUILD_{LIBUNWIND,DYNINST}=ON \ + -DDYNINST_BUILD_{TBB,BOOST,ELFUTILS,LIBIBERTY}=ON \ + "$OMNITRACE_ROOT" + fi + + + popd || exit + + echo "Make Options: $MAKE_OPTS" + cmake --build "$BUILD_DIR" --target all -- $MAKE_OPTS + cmake --build "$BUILD_DIR" --target install -- $MAKE_OPTS + cmake --build "$BUILD_DIR" --target package -- $MAKE_OPTS + + copy_if DEB "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_DEB" "$BUILD_DIR/${API_NAME}"*.deb + copy_if RPM "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_RPM" "$BUILD_DIR/${API_NAME}"*.rpm +} + +print_output_directory() { + case ${PKGTYPE} in + "deb") + echo "${PACKAGE_DEB}" + ;; + "rpm") + echo "${PACKAGE_RPM}" + ;; + *) + echo "Invalid package type \"${PKGTYPE}\" provided for -o" >&2 + exit 1 + ;; + esac + exit +} + +verifyEnvSetup + +case "$TARGET" in +clean) clean ;; +build) build_omnitrace ;; +outdir) print_output_directory ;; +*) die "Invalid target $TARGET" ;; +esac + +echo "Operation complete" diff --git a/tools/rocm-build/build_opencl_icd_loader.sh b/tools/rocm-build/build_opencl_icd_loader.sh new file mode 100755 index 000000000..1e74d1cb9 --- /dev/null +++ b/tools/rocm-build/build_opencl_icd_loader.sh @@ -0,0 +1,141 @@ +#!/bin/bash + +source "$(dirname "${BASH_SOURCE}")/compute_utils.sh" +PROJ_NAME=OpenCL-ICD-Loader +TARGET="build" +MAKEOPTS="$DASH_JAY" +BUILD_TYPE="Debug" +PACKAGE_ROOT="$(getPackageRoot)" +PACKAGE_DEB="$PACKAGE_ROOT/deb/${PROJ_NAME,,}" +PACKAGE_RPM="$PACKAGE_ROOT/rpm/${PROJ_NAME,,}" +CLEAN_OR_OUT=0; +PKGTYPE="deb" +MAKETARGET="deb" +API_NAME="rocm-opencl-icd-loader" + +printUsage() { + echo + echo "Usage: $(basename "${BASH_SOURCE}") [options ...]" + echo + echo "Options:" + echo " -c, --clean Clean output and delete all intermediate work" + echo " -p, --package Specify packaging format" + echo " -r, --release Make a release build instead of a debug build" + echo " -h, --help Prints this help" + echo " -o, --outdir Print path of output directory containing packages" + echo " -s, --static Component/Build does not support static builds just accepting this param & ignore. No effect of the param on this build" + echo + echo "Possible values for :" + echo " deb -> Debian format (default)" + echo " rpm -> RPM format" + echo + return 0 +} + +RET_CONFLICT=1 +check_conflicting_options $CLEAN_OR_OUT $PKGTYPE $MAKETARGET +if [ $RET_CONFLICT -ge 30 ]; then + print_vars $TARGET $BUILD_TYPE $CLEAN_OR_OUT $PKGTYPE $MAKETARGET + exit $RET_CONFLICT +fi + +clean_opencl_icd_loader() { + echo "Cleaning $PROJ_NAME" + rm -rf "$PACKAGE_DEB" + rm -rf "$PACKAGE_RPM" + rm -rf "$PACKAGE_ROOT/${PROJ_NAME,,}" +} + +copy_pkg_files_to_rocm() { + local comp_folder=$1 + local comp_pkg_name=$2 + + cd "${OUT_DIR}/${PKGTYPE}/${comp_folder}"|| exit 2 + if [ "${PKGTYPE}" = 'deb' ]; then + dpkg-deb -x ${comp_pkg_name}_*.deb pkg/ + else + mkdir pkg && pushd pkg/ || exit 2 + if [[ "${comp_pkg_name}" != *-dev* ]]; then + rpm2cpio ../${comp_pkg_name}-*.rpm | cpio -idmv + else + rpm2cpio ../${comp_pkg_name}el-*.rpm | cpio -idmv + fi + popd || exit 2 + fi + ls ./pkg -alt + cp -r ./pkg/*/rocm*/* "${ROCM_PATH}" || exit 2 + rm -rf pkg/ +} + +build_opencl_icd_loader() { + echo "Downloading $PROJ_NAME" package + if [ "$DISTRO_NAME" = ubuntu ]; then + mkdir -p "$PACKAGE_DEB" + local rocm_ver=${ROCM_VERSION} + if [ ${ROCM_VERSION##*.} = 0 ]; then + rocm_ver=${ROCM_VERSION%.*} + fi + local url="https://repo.radeon.com/rocm/apt/${rocm_ver}/pool/main/r/${API_NAME}/" + local package + package=$(curl -s "$url" | grep -Po 'href="\K[^"]*' | grep "${DISTRO_RELEASE}" | head -n 1) + + if [ -z "$package" ]; then + echo "No package found for Ubuntu version $DISTRO_RELEASE" + exit 1 + fi + + wget -t3 -P "$PACKAGE_DEB" "${url}${package}" + copy_pkg_files_to_rocm ${PROJ_NAME,,} ${API_NAME} + else + echo "$DISTRO_ID is not supported..." + exit 2 + fi + + echo "Installing $PROJ_NAME" package +} + +print_output_directory() { + case ${PKGTYPE} in + ("deb") + echo ${PACKAGE_DEB};; + ("rpm") + echo ${PACKAGE_RPM};; + (*) + echo "Invalid package type \"${PKGTYPE}\" provided for -o" >&2; exit 1;; + esac + exit +} + +VALID_STR=`getopt -o hcraswlo:p: --long help,clean,release,outdir:,package: -- "$@"` +eval set -- "$VALID_STR" +while true ; +do + case "$1" in + (-c | --clean ) + TARGET="clean" ; ((CLEAN_OR_OUT|=1)) ; shift ;; + (-r | --release ) + BUILD_TYPE="RelWithDebInfo" ; shift ;; + (-h | --help ) + printUsage ; exit 0 ;; + (-a | --address_sanitizer) + ack_and_ignore_asan ; shift ;; + (-o | --outdir) + TARGET="outdir"; PKGTYPE=$2 ; OUT_DIR_SPECIFIED=1 ; ((CLEAN_OR_OUT|=2)) ; shift 2 ;; + (-p | --package) + MAKETARGET="$2" ; shift 2;; + (-s | --static) + echo "-s parameter accepted but ignored" ; shift ;; + --) shift; break;; + (*) + echo " This should never come but just incase : UNEXPECTED ERROR Parm : [$1] ">&2 ; exit 20;; + esac +done + +case $TARGET in + (clean) clean_opencl_icd_loader ;; + (build) build_opencl_icd_loader ;; + (outdir) print_output_directory ;; + (*) die "Invalid target $TARGET" ;; +esac + +echo "Operation complete" diff --git a/tools/rocm-build/build_rccl.sh b/tools/rocm-build/build_rccl.sh index f6c420474..3dd3d3bd5 100755 --- a/tools/rocm-build/build_rccl.sh +++ b/tools/rocm-build/build_rccl.sh @@ -26,14 +26,16 @@ build_rccl() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params CC=${ROCM_PATH}/bin/amdclang \ CXX=$(set_build_variables CXX) \ cmake \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DHIP_COMPILER=clang \ -DCMAKE_PREFIX_PATH="${ROCM_PATH};${ROCM_PATH}/share/rocm/cmake/" \ + ${LAUNCHER_FLAGS} \ -DCPACK_GENERATOR="${PKGTYPE^^}" \ -DROCM_PATCH_VERSION=$ROCM_LIBPATCH_VERSION \ -DBUILD_ADDRESS_SANITIZER="${ADDRESS_SANITIZER}" \ diff --git a/tools/rocm-build/build_rocal.sh b/tools/rocm-build/build_rocal.sh new file mode 100755 index 000000000..8bf5e3a97 --- /dev/null +++ b/tools/rocm-build/build_rocal.sh @@ -0,0 +1,71 @@ +#!/bin/bash + +set -ex +source "$(dirname "${BASH_SOURCE[0]}")/compute_helper.sh" + +set_component_src rocAL + +build_rocal() { + + if [ "$DISTRO_ID" = "mariner-2.0" ] ; then + echo "Not building rocal for ${DISTRO_ID}. Exiting..." + return 0 + fi + + echo "Start build" + + # Enable ASAN + if [ "${ENABLE_ADDRESS_SANITIZER}" == "true" ]; then + set_asan_env_vars + set_address_sanitizer_on + fi + +# python3 ${COMPONENT_SRC}/rocAL-setup.py + pushd /tmp + # PyBind11 + git clone -b v2.11.1 https://github.com/pybind/pybind11 + cd pybind11 && mkdir build && cd build + cmake -DDOWNLOAD_CATCH=ON -DDOWNLOAD_EIGEN=ON ../ + make -j$(nproc) && sudo make install + cd ../.. + # Turbo JPEG + git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git + cd libjpeg-turbo && mkdir build && cd build + cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=RELEASE -DENABLE_STATIC=FALSE -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib -DWITH_JPEG8=TRUE .. + make -j$(nproc) && sudo make install + cd ../.. + # RapidJSON + git clone https://github.com/Tencent/rapidjson.git + cd rapidjson && mkdir build && cd build + cmake .. && make -j$(nproc) && sudo make install + popd + + mkdir -p $BUILD_DIR && cd $BUILD_DIR + + cmake -DAMDRPP_PATH=$ROCM_PATH ${COMPONENT_SRC} + make -j${PROC} + cmake --build . --target PyPackageInstall + sudo make install + sudo make package + sudo chown -R $(id -u):$(id -g) ${BUILD_DIR} + + rm -rf _CPack_Packages/ && find -name '*.o' -delete + mkdir -p $PACKAGE_DIR + cp ${BUILD_DIR}/*.${PKGTYPE} $PACKAGE_DIR + show_build_cache_stats +} + +clean_rocal() { + echo "Cleaning rocAL build directory: ${BUILD_DIR} ${PACKAGE_DIR}" + rm -rf "$BUILD_DIR" "$PACKAGE_DIR" + echo "Done!" +} + +stage2_command_args "$@" + +case $TARGET in + build) build_rocal ;; + outdir) print_output_directory ;; + clean) clean_rocal ;; + *) die "Invalid target $TARGET" ;; +esac diff --git a/tools/rocm-build/build_rocalution.sh b/tools/rocm-build/build_rocalution.sh index d60b6c055..ba4215c07 100755 --- a/tools/rocm-build/build_rocalution.sh +++ b/tools/rocm-build/build_rocalution.sh @@ -22,6 +22,7 @@ build_rocalution() { echo "CXX compiler: $CXX" mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -32,7 +33,7 @@ build_rocalution() { cmake \ -DSUPPORT_HIP=ON \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DCPACK_SET_DESTDIR=OFF \ -DBUILD_CLIENTS_SAMPLES=ON \ diff --git a/tools/rocm-build/build_rocblas.sh b/tools/rocm-build/build_rocblas.sh index 28593c648..980d199cc 100755 --- a/tools/rocm-build/build_rocblas.sh +++ b/tools/rocm-build/build_rocblas.sh @@ -28,11 +28,12 @@ build_rocblas() { else GPU_TARGETS="gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params cmake \ -DCMAKE_TOOLCHAIN_FILE=toolchain-linux.cmake \ -DBUILD_DIR="${BUILD_DIR}" \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DROCM_DIR="${ROCM_PATH}" \ ${LAUNCHER_FLAGS} \ -DCMAKE_PREFIX_PATH="${DEPS_DIR};${ROCM_PATH}" \ diff --git a/tools/rocm-build/build_rocdecode.sh b/tools/rocm-build/build_rocdecode.sh index bc768c8ee..35bb9b5ed 100755 --- a/tools/rocm-build/build_rocdecode.sh +++ b/tools/rocm-build/build_rocdecode.sh @@ -4,15 +4,16 @@ source "$(dirname "${BASH_SOURCE[0]}")/compute_helper.sh" set_component_src rocDecode BUILD_DEV=ON build_rocdecode() { - if [ "$DISTRO_ID" = "centos-7" ] ; then + if [ "$DISTRO_ID" = "centos-7" ] || [ "$DISTRO_ID" = "sles-15.4" ] ; then echo "Not building rocDecode for ${DISTRO_ID}. Exiting..." return 0 fi echo "Start build" mkdir -p $BUILD_DIR && cd $BUILD_DIR + python3 ${COMPONENT_SRC}/rocDecode-setup.py --developer OFF - cmake ${COMPONENT_SRC} + cmake -DROCM_DEP_ROCMCORE=ON ${COMPONENT_SRC} make -j8 make install make package diff --git a/tools/rocm-build/build_rocfft.sh b/tools/rocm-build/build_rocfft.sh index 180406fa1..27d6009c3 100755 --- a/tools/rocm-build/build_rocfft.sh +++ b/tools/rocm-build/build_rocfft.sh @@ -16,6 +16,7 @@ build_rocfft() { set_address_sanitizer_on fi mkdir -p "$BUILD_DIR" && cd "$BUILD_DIR" + init_rocm_common_cmake_params if [ -n "$GPU_ARCHS" ]; then GPU_TARGETS="$GPU_ARCHS" @@ -26,7 +27,7 @@ build_rocfft() { CXX="${ROCM_PATH}/bin/hipcc" \ cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DUSE_HIP_CLANG=ON \ -DHIP_COMPILER=clang \ diff --git a/tools/rocm-build/build_rocm-cmake.sh b/tools/rocm-build/build_rocm-cmake.sh index 267313c13..f700f7be7 100755 --- a/tools/rocm-build/build_rocm-cmake.sh +++ b/tools/rocm-build/build_rocm-cmake.sh @@ -10,11 +10,13 @@ printUsage() { echo " -c, --clean Clean output and delete all intermediate work" echo " -r, --release Build a release version of the package" echo " -a, --address_sanitizer Enable address sanitizer" - echo " -s, --static Supports static CI by accepting this param & not bailing out. No effect of the param though" + echo " -s, --static Build static lib (.a). build instead of dynamic/shared(.so) " + echo " -w, --wheel Creates python wheel package of rocm-cmake. + It needs to be used along with -r option" echo " -o, --outdir Print path of output directory containing packages of type referred to by pkg_type" echo " -p, --package Specify packaging format" - echo " -h, --help Prints this help" + echo " -h, --help Prints this help" echo echo @@ -30,6 +32,7 @@ ROCM_CMAKE_BUILD_DIR="$(getBuildPath rocm-cmake)" ROCM_CMAKE_BUILD_DIR="$(getBuildPath rocm-cmake)" ROCM_CMAKE_PACKAGE_DEB="$(getPackageRoot)/deb/rocm-cmake" ROCM_CMAKE_PACKAGE_RPM="$(getPackageRoot)/rpm/rocm-cmake" +ROCM_WHEEL_DIR="${ROCM_CMAKE_BUILD_DIR}/_wheel" ROCM_CMAKE_BUILD_TYPE="debug" BUILD_TYPE="Debug" SHARED_LIBS="ON" @@ -37,7 +40,7 @@ CLEAN_OR_OUT=0; PKGTYPE="deb" MAKETARGET="deb" -VALID_STR=`getopt -o hcraso:p: --long help,clean,release,static,address_sanitizer,outdir:,package: -- "$@"` +VALID_STR=`getopt -o hcraswo:p: --long help,clean,release,static,wheel,address_sanitizer,outdir:,package: -- "$@"` eval set -- "$VALID_STR" while true ; @@ -53,6 +56,8 @@ do ack_and_ignore_asan ; shift ;; (-s | --static) SHARED_LIBS="OFF" ; shift ;; + (-w | --wheel) + WHEEL_PACKAGE=true ; shift ;; (-o | --outdir) TARGET="outdir"; PKGTYPE=$2 ; OUT_DIR_SPECIFIED=1 ; ((CLEAN_OR_OUT|=2)) ; shift 2 ;; (-p | --package) @@ -73,6 +78,7 @@ fi clean_rocm_cmake() { + rm -rf "$ROCM_WHEEL_DIR" rm -rf $ROCM_CMAKE_BUILD_DIR rm -rf $ROCM_CMAKE_PACKAGE_DEB rm -rf $ROCM_CMAKE_PACKAGE_RPM @@ -87,6 +93,7 @@ build_rocm_cmake() { cmake \ $(rocm_cmake_params) \ + -DBUILD_SHARED_LIBS=$SHARED_LIBS \ -DCPACK_SET_DESTDIR="OFF" \ -DROCM_DISABLE_LDCONFIG=ON \ $ROCM_CMAKE_ROOT @@ -99,6 +106,19 @@ build_rocm_cmake() { copy_if RPM "${CPACKGEN:-"DEB;RPM"}" "$ROCM_CMAKE_PACKAGE_RPM" $ROCM_CMAKE_BUILD_DIR/rocm-cmake*.rpm } +create_wheel_package() { + echo "Creating rocm-cmake wheel package" + # Copy the setup.py generator to build folder + mkdir -p $ROCM_WHEEL_DIR + cp -f $SCRIPT_ROOT/generate_setup_py.py $ROCM_WHEEL_DIR + cp -f $SCRIPT_ROOT/repackage_wheel.sh $ROCM_WHEEL_DIR + cd $ROCM_WHEEL_DIR + # Currently only supports python3.6 + ./repackage_wheel.sh $ROCM_CMAKE_BUILD_DIR/rocm-cmake*.rpm python3.6 + # Copy the wheel created to RPM folder which will be uploaded to artifactory + copy_if WHL "WHL" "$ROCM_CMAKE_PACKAGE_RPM" "$ROCM_WHEEL_DIR"/dist/*.whl +} + print_output_directory() { case ${PKGTYPE} in ("deb") @@ -118,4 +138,9 @@ case $TARGET in (*) die "Invalid target $TARGET" ;; esac +if [[ $WHEEL_PACKAGE == true ]]; then + echo "Wheel Package build started !!!!" + create_wheel_package +fi + echo "Operation complete" diff --git a/tools/rocm-build/build_rocprim.sh b/tools/rocm-build/build_rocprim.sh index e00cd8038..b0cb796f4 100755 --- a/tools/rocm-build/build_rocprim.sh +++ b/tools/rocm-build/build_rocprim.sh @@ -24,13 +24,14 @@ build_rocprim() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params CXX="${ROCM_PATH}/bin/hipcc" \ cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DBUILD_BENCHMARK=OFF \ - -DBUILD_BENCHMARK=OFF \ + -DBUILD_SHARED_LIBS=ON \ -DBUILD_TEST=ON \ -DCMAKE_MODULE_PATH="${ROCM_PATH}/lib/cmake/hip;${ROCM_PATH}/hip/cmake" \ "$COMPONENT_SRC" diff --git a/tools/rocm-build/build_rocprofiler-sdk.sh b/tools/rocm-build/build_rocprofiler-sdk.sh new file mode 100755 index 000000000..abfba538b --- /dev/null +++ b/tools/rocm-build/build_rocprofiler-sdk.sh @@ -0,0 +1,222 @@ +#!/bin/bash + +source "$(dirname "${BASH_SOURCE}")/compute_utils.sh" + +printUsage() { + echo + echo "Usage: ${BASH_SOURCE##*/} [options ...]" + echo + echo "Options:" + echo " -c, --clean Clean output and delete all intermediate work" + echo " -s, --static Build static lib (.a). build instead of dynamic/shared(.so) " + echo " -w, --wheel Creates python wheel package of rocprofiler-sdk. + It needs to be used along with -r option" + echo " -p, --package Specify packaging format" + echo " -r, --release Make a release build instead of a debug build" + echo " -a, --address_sanitizer Enable address sanitizer" + echo " -o, --outdir Print path of output directory containing packages of + type referred to by pkg_type" + echo " -h, --help Prints this help" + echo + echo "Possible values for :" + echo " deb -> Debian format (default)" + echo " rpm -> RPM format" + echo + + return 0 +} + +API_NAME="rocprofiler-sdk" +PROJ_NAME="$API_NAME" +LIB_NAME="lib${API_NAME}" +TARGET="build" +MAKETARGET="deb" +PACKAGE_ROOT="$(getPackageRoot)" +PACKAGE_LIB="$(getLibPath)" +PACKAGE_INCLUDE="$(getIncludePath)" +BUILD_DIR="$(getBuildPath $API_NAME)" +PACKAGE_DEB="$(getPackageRoot)/deb/$API_NAME" +PACKAGE_RPM="$(getPackageRoot)/rpm/$API_NAME" +ROCM_WHEEL_DIR="${BUILD_DIR}/_wheel" +PACKAGE_PREFIX="$ROCM_INSTALL_PATH" +BUILD_TYPE="Debug" +MAKE_OPTS="$DASH_JAY" +SHARED_LIBS="ON" +CLEAN_OR_OUT=0 +MAKETARGET="deb" +PKGTYPE="deb" + +GPU_LIST="gfx900;gfx906;gfx908;gfx90a;gfx940;gfx941;gfx942;gfx1030;gfx1031;gfx1100;gfx1101;gfx1102" +ASAN=0 + +VALID_STR=$(getopt -o hcrawso:p: --long help,clean,release,static,address_sanitizer,wheel,outdir:,package: -- "$@") +eval set -- "$VALID_STR" + +while true; do + case "$1" in + -h | --help) + printUsage + exit 0 + ;; + -c | --clean) + TARGET="clean" + ((CLEAN_OR_OUT |= 1)) + shift + ;; + -r | --release) + BUILD_TYPE="RelWithDebInfo" + shift + ;; + -a | --address_sanitizer) + set_address_sanitizer_on + set_asan_env_vars + ASAN=1 + shift + ;; + -s | --static) + SHARED_LIBS="OFF" + shift + ;; + -w | --wheel) + WHEEL_PACKAGE=true + shift + ;; + -o | --outdir) + TARGET="outdir" + PKGTYPE=$2 + OUT_DIR_SPECIFIED=1 + ((CLEAN_OR_OUT |= 2)) + shift 2 + ;; + -p | --package) + MAKETARGET="$2" + shift 2 + ;; + --) + shift + break + ;; # end delimiter + *) + echo " This should never come but just incase : UNEXPECTED ERROR Parm : [$1] " >&2 + exit 20 + ;; + esac + +done + +RET_CONFLICT=1 +check_conflicting_options $CLEAN_OR_OUT $PKGTYPE $MAKETARGET +if [ $RET_CONFLICT -ge 30 ]; then + print_vars $API_NAME $TARGET $BUILD_TYPE $SHARED_LIBS $CLEAN_OR_OUT $PKGTYPE $MAKETARGET + exit $RET_CONFLICT +fi + +clean() { + echo "Cleaning $PROJ_NAME" + rm -rf "$ROCM_WHEEL_DIR" + rm -rf "$BUILD_DIR" + rm -rf "$PACKAGE_DEB" + rm -rf "$PACKAGE_RPM" + rm -rf "$PACKAGE_ROOT/${PROJ_NAME}" + rm -rf "$PACKAGE_ROOT/libexec/${PROJ_NAME}" + rm -rf "$PACKAGE_INCLUDE/${PROJ_NAME}" + rm -rf "$PACKAGE_LIB/${LIB_NAME}"* + rm -rf "$PACKAGE_LIB/${PROJ_NAME}" +} + +build_rocprofiler-sdk() { + if [ ! -f "${ROCPROFILER_SDK_ROOT}/CMakeLists.txt" ]; then + echo "Warning: $ROCPROFILER_SDK_ROOT not found" + else + echo "Building $PROJ_NAME" + PACKAGE_CMAKE="$(getCmakePath)" + if [ ! -d "$BUILD_DIR" ]; then + mkdir -p "$BUILD_DIR" + pushd "$BUILD_DIR" + print_lib_type $SHARED_LIBS + + if [ $ASAN == 1 ]; then + cmake \ + $(rocm_cmake_params) \ + $(rocm_common_cmake_params) \ + -DAMDDeviceLibs_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/AMDDeviceLibs" \ + -Dhip_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/hip" \ + -Dhip-lang_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/hip-lang" \ + -Damd_comgr_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/amd_comgr" \ + -Dhsa-runtime64_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/hsa-runtime64" \ + -Dhsakmt_DIR="${ROCM_INSTALL_PATH}/lib/asan/cmake/hsakmt" \ + -DCMAKE_HIP_COMPILER_ROCM_ROOT="${ROCM_INSTALL_PATH}" \ + -DCMAKE_PREFIX_PATH="${ROCM_INSTALL_PATH};${ROCM_INSTALL_PATH}/lib/asan" \ + -DBUILD_SHARED_LIBS=$SHARED_LIBS \ + -DGPU_TARGETS="$GPU_LIST" \ + -DCPACK_DEBIAN_PACKAGE_SHLIBDEPS=OFF \ + -DPython3_EXECUTABLE=$(which python3) \ + "$ROCPROFILER_SDK_ROOT" + else + cmake \ + $(rocm_cmake_params) \ + $(rocm_common_cmake_params) \ + -DCMAKE_PREFIX_PATH="${ROCM_INSTALL_PATH}" \ + -DBUILD_SHARED_LIBS=$SHARED_LIBS \ + -DGPU_TARGETS="$GPU_LIST" \ + -DROCPROFILER_BUILD_SAMPLES=ON \ + -DROCPROFILER_BUILD_TESTS=ON \ + -DCPACK_DEBIAN_PACKAGE_SHLIBDEPS=OFF \ + -DPython3_EXECUTABLE=$(which python3) \ + "$ROCPROFILER_SDK_ROOT" + fi + + popd + fi + cmake --build "$BUILD_DIR" --target all -- $MAKE_OPTS + cmake --build "$BUILD_DIR" --target install -- $MAKE_OPTS + cmake --build "$BUILD_DIR" --target package -- $MAKE_OPTS + + copy_if DEB "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_DEB" "$BUILD_DIR/${API_NAME}"*.deb + copy_if RPM "${CPACKGEN:-"DEB;RPM"}" "$PACKAGE_RPM" "$BUILD_DIR/${API_NAME}"*.rpm + fi +} + +create_wheel_package() { + echo "Creating rocprofiler sdk wheel package" + mkdir -p "$ROCM_WHEEL_DIR" + cp -f "$SCRIPT_ROOT"/generate_setup_py.py "$ROCM_WHEEL_DIR" + cp -f "$SCRIPT_ROOT"/repackage_wheel.sh "$ROCM_WHEEL_DIR" + cd "$ROCM_WHEEL_DIR" + # Currently only supports python3.6 + ./repackage_wheel.sh "$BUILD_DIR"/*.rpm python3.6 + # Copy the wheel created to RPM folder which will be uploaded to artifactory + copy_if WHL "WHL" "$PACKAGE_RPM" "$ROCM_WHEEL_DIR"/dist/*.whl +} + +print_output_directory() { + case ${PKGTYPE} in + "deb") + echo ${PACKAGE_DEB} + ;; + "rpm") + echo ${PACKAGE_RPM} + ;; + *) + echo "Invalid package type \"${PKGTYPE}\" provided for -o" >&2 + exit 1 + ;; + esac + exit +} + +verifyEnvSetup + +case "$TARGET" in + clean) clean ;; + build) build_rocprofiler-sdk ;; + outdir) print_output_directory ;; + *) die "Invalid target $TARGET" ;; +esac + +if [[ $WHEEL_PACKAGE == true ]]; then + echo "Wheel Package build started !!!!" + create_wheel_package +fi + +echo "Operation complete" diff --git a/tools/rocm-build/build_rocrand.sh b/tools/rocm-build/build_rocrand.sh index 8e7aa13af..691e64a63 100755 --- a/tools/rocm-build/build_rocrand.sh +++ b/tools/rocm-build/build_rocrand.sh @@ -25,10 +25,12 @@ build_rocrand() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params + CXX=$(set_build_variables CXX)\ cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DBUILD_TEST=ON \ -DBUILD_BENCHMARK=ON \ diff --git a/tools/rocm-build/build_rocsolver.sh b/tools/rocm-build/build_rocsolver.sh index fbffc5dfd..e310671eb 100755 --- a/tools/rocm-build/build_rocsolver.sh +++ b/tools/rocm-build/build_rocsolver.sh @@ -28,11 +28,13 @@ build_rocsolver() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params + CXX="${ROCM_PATH}/bin/hipcc" \ cmake \ -DCPACK_SET_DESTDIR=OFF \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -Drocblas_DIR="${ROCM_PATH}/rocblas/lib/cmake/rocblas" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DBUILD_CLIENTS_TESTS=ON \ diff --git a/tools/rocm-build/build_rocsparse.sh b/tools/rocm-build/build_rocsparse.sh index 2426b98c9..8cda3d59f 100755 --- a/tools/rocm-build/build_rocsparse.sh +++ b/tools/rocm-build/build_rocsparse.sh @@ -27,12 +27,14 @@ build_rocsparse() { fi ROCSPARSE_TEST_MIRROR=$MIRROR \ - CXX=$(set_build_variables CXX)\ - CC=$(set_build_variables CC)\ + export CXX=$(set_build_variables CXX)\ + export CC=$(set_build_variables CC)\ + + init_rocm_common_cmake_params cmake \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}"\ -DBUILD_CLIENTS_SAMPLES=ON \ -DBUILD_CLIENTS_TESTS=ON \ -DBUILD_CLIENTS_BENCHMARKS=ON \ diff --git a/tools/rocm-build/build_rocthrust.sh b/tools/rocm-build/build_rocthrust.sh index cd7cc16e4..84e058b4d 100755 --- a/tools/rocm-build/build_rocthrust.sh +++ b/tools/rocm-build/build_rocthrust.sh @@ -27,10 +27,12 @@ build_rocthrust() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1030;gfx1100;gfx1101" fi + init_rocm_common_cmake_params + CXX=$(set_build_variables CXX)\ cmake \ ${LAUNCHER_FLAGS} \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DCMAKE_MODULE_PATH="${ROCM_PATH}/lib/cmake/hip;${ROCM_PATH}/hip/cmake" \ -DROCPRIM_ROOT="${ROCM_PATH}/rocprim" \ diff --git a/tools/rocm-build/build_rocwmma.sh b/tools/rocm-build/build_rocwmma.sh index 54eca1a11..694723c07 100755 --- a/tools/rocm-build/build_rocwmma.sh +++ b/tools/rocm-build/build_rocwmma.sh @@ -27,9 +27,11 @@ build_rocwmma() { GPU_TARGETS="gfx908:xnack-;gfx90a:xnack-;gfx90a:xnack+;gfx940;gfx941;gfx942;gfx1100;gfx1101" fi + init_rocm_common_cmake_params + CXX=$(set_build_variables CXX)\ cmake \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ ${LAUNCHER_FLAGS} \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ -DROCWMMA_BUILD_VALIDATION_TESTS=ON \ diff --git a/tools/rocm-build/build_rpp.sh b/tools/rocm-build/build_rpp.sh index b1051a9d8..d20a62ab7 100755 --- a/tools/rocm-build/build_rpp.sh +++ b/tools/rocm-build/build_rpp.sh @@ -12,7 +12,7 @@ ROCM_LLVM_LIB_RPATH="\$ORIGIN/llvm/lib" rpp_specific_cmake_params() { local rpp_cmake_params if [ "${ASAN_CMAKE_PARAMS}" == "true" ] ; then - rpp_cmake_params="-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--rpath,$ROCM_ASAN_EXE_RPATH:$LLVM_LIBDIR" + rpp_cmake_params="-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_ASAN_EXE_RPATH:$LLVM_LIBDIR" else rpp_cmake_params="" fi @@ -41,14 +41,16 @@ build_rpp() { GPU_TARGETS="gfx908;gfx90a;gfx940;gfx941;gfx942;gfx1030;gfx1100" fi + init_rocm_common_cmake_params + cmake \ - $(rocm_common_cmake_params) \ + "${rocm_math_common_cmake_params[@]}" \ ${LAUNCHER_FLAGS} \ -DBACKEND=HIP \ -DCMAKE_INSTALL_LIBDIR=$(getInstallLibDir) \ $(rpp_specific_cmake_params) \ -DAMDGPU_TARGETS=${GPU_TARGETS} \ - -DCMAKE_SHARED_LINKER_FLAGS_INIT="-fno-openmp-implicit-rpath -Wl,--enable-new-dtags,--rpath,${ROCM_LIB_RPATH}:${DEPS_DIR}/lib:${ROCM_LLVM_LIB_RPATH}" \ + -DCMAKE_SHARED_LINKER_FLAGS_INIT="-fno-openmp-implicit-rpath -Wl,--enable-new-dtags,--build-id=sha1,--rpath,${ROCM_LIB_RPATH}:${DEPS_DIR}/lib:${ROCM_LLVM_LIB_RPATH}" \ -DCMAKE_PREFIX_PATH="${DEPS_DIR};${ROCM_PATH}" \ "$COMPONENT_SRC" diff --git a/tools/rocm-build/compute_helper.sh b/tools/rocm-build/compute_helper.sh index 08412ad4c..5fbf1b916 100755 --- a/tools/rocm-build/compute_helper.sh +++ b/tools/rocm-build/compute_helper.sh @@ -189,6 +189,65 @@ echo " PKGTYPE= $6 " echo " MAKETARGET = $7 " } +rocm_math_common_cmake_params=() +init_rocm_common_cmake_params(){ + local retCmakeParams=${1:-rocm_math_common_cmake_params} + local SET_BUILD_TYPE=${BUILD_TYPE:-'RelWithDebInfo'} + local ASAN_LIBDIR="lib/asan" + local CMAKE_PATH=$(getCmakePath) +# Common cmake parameters can be set +# component build scripts can use this function + local cmake_params + if [ "${ASAN_CMAKE_PARAMS}" == "true" ] ; then + cmake_params=( + "-DCMAKE_PREFIX_PATH=$CMAKE_PATH;${ROCM_PATH}/$ASAN_LIBDIR;$ROCM_PATH/llvm;$ROCM_PATH" + "-DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_ASAN_LIB_RPATH" + "-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_ASAN_EXE_RPATH" + "-DENABLE_ASAN_PACKAGING=true" + ) + else + cmake_params=( + "-DCMAKE_PREFIX_PATH=${ROCM_PATH}/llvm;${ROCM_PATH}" + "-DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_LIB_RPATH" + "-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ROCM_EXE_RPATH" + ) + fi + + cmake_params+=( + "-DCMAKE_VERBOSE_MAKEFILE=1" + "-DCMAKE_BUILD_TYPE=${SET_BUILD_TYPE}" + "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=FALSE" + "-DCMAKE_INSTALL_PREFIX=${ROCM_PATH}" + "-DCMAKE_PACKAGING_INSTALL_PREFIX=${ROCM_PATH}" + "-DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF" + "-DROCM_SYMLINK_LIBS=OFF" + "-DCPACK_PACKAGING_INSTALL_PREFIX=${ROCM_PATH}" + "-DROCM_DISABLE_LDCONFIG=ON" + "-DROCM_PATH=${ROCM_PATH}" + ) + + #TODO :remove if clause once debug related issues are fixed + if [ "${DISABLE_DEBUG_PACKAGE}" == "true" ] ; then + SET_BUILD_TYPE=${BUILD_TYPE:-'Release'} + cmake_params+=( + "-DCPACK_DEBIAN_DEBUGINFO_PACKAGE=FALSE" + "-DCPACK_RPM_DEBUGINFO_PACKAGE=FALSE" + "-DCPACK_RPM_INSTALL_WITH_EXEC=FALSE" + "-DCMAKE_BUILD_TYPE=${SET_BUILD_TYPE}" + ) + elif [ "$SET_BUILD_TYPE" == "RelWithDebInfo" ] || [ "$SET_BUILD_TYPE" == "Debug" ]; then + # RelWithDebinfo optimization level -O2 is having performance impact + # So overriding the same to -O3 + cmake_params+=( + "-DCPACK_DEBIAN_DEBUGINFO_PACKAGE=TRUE" + "-DCPACK_RPM_DEBUGINFO_PACKAGE=TRUE" + "-DCPACK_RPM_INSTALL_WITH_EXEC=TRUE" + "-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O3 -g -DNDEBUG" + ) + fi + eval "${retCmakeParams}=( \"\${cmake_params[@]}\" ) " +} + # Common cmake parameters can be set # component build scripts can use this function rocm_common_cmake_params() { diff --git a/tools/rocm-build/docker/ubuntu20/install-prerequisites.sh b/tools/rocm-build/docker/ubuntu20/install-prerequisites.sh index 0c73e06b1..7828292de 100755 --- a/tools/rocm-build/docker/ubuntu20/install-prerequisites.sh +++ b/tools/rocm-build/docker/ubuntu20/install-prerequisites.sh @@ -2,7 +2,6 @@ set -ex - apt-get update DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true apt-get install --no-install-recommends -y $(grep -v '^#' /tmp/packages) apt-get clean @@ -105,7 +104,7 @@ git clone --recurse-submodules -b v1.61.0 https://github.com/grpc/grpc cd grpc mkdir -p cmake/build cd cmake/build -cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/grpc -DCMAKE_CXX_STANDARD=14 ../.. +cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/grpc -DCMAKE_CXX_STANDARD=14 -DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,'$ORIGIN' ../.. make -j$(nproc) make install cd / @@ -120,11 +119,11 @@ mv amd-blis-mt /usr/blis cd / rm -rf /tmp/blis -## Download aocl-linux-aocc-4.0_1_amd64.deb +## Download aocl-linux-gcc-4.2.0_1_amd64.deb mkdir -p /tmp/aocl cd /tmp/aocl -wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-0/aocl-linux-aocc-4.0_1_amd64.deb -apt install ./aocl-linux-aocc-4.0_1_amd64.deb +wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-2/aocl-linux-gcc-4.2.0_1_amd64.deb +apt install ./aocl-linux-gcc-4.2.0_1_amd64.deb rm -rf /tmp/aocl ## lapack(3.9.1v) @@ -180,8 +179,7 @@ cd ninja-1.11.1.g95dee.kitware.jobserver-1 cp ninja /usr/local/bin/ rm -rf /tmp/ninja -# Install pre-built FFmpeg and dependencies -# See docker/build-deps for instructions on how to build these packages +# Install FFmpeg from source wget -qO- https://www.ffmpeg.org/releases/ffmpeg-4.4.2.tar.gz | tar -xzv -C /usr/local command -v lbzip2 diff --git a/tools/rocm-build/docker/ubuntu20/packages b/tools/rocm-build/docker/ubuntu20/packages index bdcc0d3e3..dc42f2d90 100644 --- a/tools/rocm-build/docker/ubuntu20/packages +++ b/tools/rocm-build/docker/ubuntu20/packages @@ -8,6 +8,7 @@ bison bridge-utils build-essential bzip2 +ccache check chrpath cifs-utils @@ -97,6 +98,7 @@ libva-dev libvirt-clients libvirt-daemon-system libyaml-cpp-dev +libzstd-dev llvm llvm-6.0-dev llvm-dev @@ -119,9 +121,11 @@ python3-yaml python3.8-dev re2c redis-tools +# Eventually we should be able to remove rpm for debian builds. rpm rsync ssh +# This makes life more pleasent inside the container strace sudo systemtap-sdt-dev diff --git a/tools/rocm-build/docker/ubuntu22/install-prerequisities.sh b/tools/rocm-build/docker/ubuntu22/install-prerequisities.sh index 3ea039bc4..16127d846 100644 --- a/tools/rocm-build/docker/ubuntu22/install-prerequisities.sh +++ b/tools/rocm-build/docker/ubuntu22/install-prerequisities.sh @@ -1,6 +1,6 @@ #! /usr/bin/bash -set -ex +set -x apt-get -y update DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true apt-get install --no-install-recommends -y $(sed 's/#.*//' /tmp/packages) @@ -60,7 +60,6 @@ apt install -y sharp apt clean rm -rf /var/cache/apt/ /var/lib/apt/lists/* mlnx /etc/apt/sources.list.d/sharp.list - apt update apt -y install libunwind-dev apt -y install libgoogle-glog-dev @@ -118,12 +117,12 @@ git clone --recurse-submodules -b v1.61.0 https://github.com/grpc/grpc cd grpc mkdir -p build cd build -cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/grpc -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD=14 .. +cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/grpc -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD=14 -DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,'$ORIGIN' .. make -j $(nproc) install rm -rf /tmp/grpc ## rocBLAS Pre-requisites -## Download prebuilt AMD multithreaded (2.0) +## Download prebuilt AMD multithreaded blis (2.0) ## Reference : https://github.com/ROCmSoftwarePlatform/rocBLAS/blob/develop/install.sh#L403 mkdir -p /tmp/blis cd /tmp/blis @@ -131,12 +130,13 @@ wget -O - https://github.com/amd/blis/releases/download/2.0/aocl-blis-mt-ubuntu- mv amd-blis-mt /usr/blis cd / rm -rf /tmp/blis + ## rocBLAS Pre-requisites(SWDEV-404612) -## Download aocl-linux-aocc-4.0_1_amd64 +## Download aocl-linux-gcc-4.2.0_1_amd64.deb mkdir -p /tmp/aocl cd /tmp/aocl -wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-0/aocl-linux-aocc-4.0_1_amd64.deb -apt install ./aocl-linux-aocc-4.0_1_amd64.deb +wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-2/aocl-linux-gcc-4.2.0_1_amd64.deb +apt install ./aocl-linux-gcc-4.2.0_1_amd64.deb rm -rf /tmp/aocl ## hipBLAS Pre-requisites @@ -194,9 +194,62 @@ cd ninja-1.11.1.g95dee.kitware.jobserver-1 cp ninja /usr/local/bin/ rm -rf /tmp/ninja -# Install pre-built FFmpeg and dependencies -# See docker/build-deps for instructions on how to build these packages -wget -qO- https://www.ffmpeg.org/releases/ffmpeg-4.4.2.tar.gz | tar -xzv -C /usr/local +# Install FFmpeg and dependencies +# Build NASM +mkdir -p /tmp/nasm-2.15.05 +cd /tmp +wget -qO- "https://distfiles.macports.org/nasm/nasm-2.15.05.tar.bz2" | tar -xvj +cd nasm-2.15.05 +./autogen.sh +./configure --prefix="/usr/local" +make -j$(nproc) install +rm -rf /tmp/nasm-2.15.05 + +# Build YASM +mkdir -p /tmp/yasm-1.3.0 +cd /tmp +wget -qO- "http://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz" | tar -xvz +cd yasm-1.3.0 +./configure --prefix="/usr/local" +make -j$(nproc) install +rm -rf /tmp/yasm-1.3.0 + +# Build x264 +mkdir -p /tmp/x264-snapshot-20191217-2245-stable +cd /tmp +wget -qO- "https://download.videolan.org/pub/videolan/x264/snapshots/x264-snapshot-20191217-2245-stable.tar.bz2" | tar -xvj +cd /tmp/x264-snapshot-20191217-2245-stable +PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" ./configure --prefix="/usr/local" --enable-shared +make -j$(nproc) install +rm -rf /tmp/x264-snapshot-20191217-2245-stable + +# Build x265 +mkdir -p /tmp/x265_2.7 +cd /tmp +wget -qO- "https://get.videolan.org/x265/x265_2.7.tar.gz" | tar -xvz +cd /tmp/x265_2.7/build/linux +cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on ../../source +make -j$(nproc) install +rm -rf /tmp/x265_2.7 + +# Build fdk-aac +mkdir -p /tmp/fdk-aac-2.0.2 +cd /tmp +wget -qO- "https://sourceforge.net/projects/opencore-amr/files/fdk-aac/fdk-aac-2.0.2.tar.gz" | tar -xvz +cd /tmp/fdk-aac-2.0.2 +autoreconf -fiv +./configure --prefix="/usr/local" --enable-shared --disable-static +make -j$(nproc) install +rm -rf /tmp/fdk-aac-2.0.2 + +# Build FFmpeg +cd /tmp +git clone -b release/4.4 https://git.ffmpeg.org/ffmpeg.git ffmpeg +cd ffmpeg +PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" +./configure --prefix="/usr/local" --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --extra-libs=-lpthread --extra-libs=-lm --enable-shared --disable-static --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-gpl --enable-nonfree +make -j$(nproc) install +rm -rf /tmp/ffmpeg cp /tmp/local-pin-600 /etc/apt/preferences.d @@ -212,21 +265,21 @@ make -j -C build cd /tmp/Gbenchmark/build make install -# Build boost-1.82.0 from source for RPP +# Build boost-1.85.0 from source for RPP # Installing in a non-standard location since the test packages of hipFFT and rocFFT pick up the version of # the installed Boost library and declare a package dependency on that specific version of Boost. -# For example, if this was installed in the standard location it would declare a dependency on libboost-dev(el)1.82.0 +# For example, if this was installed in the standard location it would declare a dependency on libboost-dev(el)1.85.0 # which is not available as a package in any distro. # Once this is fixed, we can remove the Boost package from the requirements list and install this # in the standard location -mkdir -p /tmp/boost-1.82.0 -cd /tmp/boost-1.82.0 -wget -nv https://sourceforge.net/projects/boost/files/boost/1.82.0/boost_1_82_0.tar.bz2 -O ./boost_1_82_0.tar.bz2 -tar -xf boost_1_82_0.tar.bz2 --use-compress-program="/usr/local/bin/compressor" -cd boost_1_82_0 +mkdir -p /tmp/boost-1.85.0 +cd /tmp/boost-1.85.0 +wget -nv https://sourceforge.net/projects/boost/files/boost/1.85.0/boost_1_85_0.tar.bz2 -O ./boost_1_85_0.tar.bz2 +tar -xf boost_1_85_0.tar.bz2 --use-compress-program="/usr/local/bin/compressor" +cd boost_1_85_0 ./bootstrap.sh --prefix=${RPP_DEPS_LOCATION} --with-python=python3 ./b2 stage -j$(nproc) threading=multi link=shared cxxflags="-std=c++11" ./b2 install threading=multi link=shared --with-system --with-filesystem ./b2 stage -j$(nproc) threading=multi link=static cxxflags="-std=c++11 -fpic" cflags="-fpic" ./b2 install threading=multi link=static --with-system --with-filesystem -rm -rf /tmp/boost-1.82.0 +rm -rf /tmp/boost-1.85.0 diff --git a/tools/rocm-build/docker/ubuntu22/packages b/tools/rocm-build/docker/ubuntu22/packages index 56a22631a..72483d363 100644 --- a/tools/rocm-build/docker/ubuntu22/packages +++ b/tools/rocm-build/docker/ubuntu22/packages @@ -8,6 +8,7 @@ bison bridge-utils build-essential bzip2 +ccache check chrpath cifs-utils @@ -99,6 +100,7 @@ libva-dev libvirt-clients libvirt-daemon-system libyaml-cpp-dev +libzstd-dev llvm llvm-dev llvm-runtime @@ -137,3 +139,4 @@ unzip vim wget xsltproc +zlib1g-dev diff --git a/tools/rocm-build/docker/ubuntu24/Dockerfile b/tools/rocm-build/docker/ubuntu24/Dockerfile new file mode 100644 index 000000000..d23d420a1 --- /dev/null +++ b/tools/rocm-build/docker/ubuntu24/Dockerfile @@ -0,0 +1,11 @@ +FROM ubuntu:noble as builder +COPY packages /tmp/packages +COPY local-pin-600 /tmp/local-pin-600 +COPY install-prerequisities.sh /tmp/install-prerequisities.sh +RUN chmod +x /tmp/install-prerequisities.sh +ENV KBUILD_PKG_ROOTCMD= +ENV RPP_DEPS_LOCATION=/usr/local/rpp-deps +ENV PATH="/opt/venv/bin:$PATH" +ENV PATH=$PATH:"/usr/local/bin" +RUN /tmp/install-prerequisities.sh +WORKDIR /src diff --git a/tools/rocm-build/docker/ubuntu24/README.md b/tools/rocm-build/docker/ubuntu24/README.md new file mode 100644 index 000000000..c7a709f8b --- /dev/null +++ b/tools/rocm-build/docker/ubuntu24/README.md @@ -0,0 +1,27 @@ +## Steps to build the Docker Image + +1. Clone this repositry + + ```bash + git clone https://github.com/ROCm/rocm-build.git + ``` + +2. Go into the OS specific docker directory in build-infra + + ```bash + cd rocm-build/build/docker/ubuntu24 + ``` + +3. Build the docker image + + ```bash + docker build -t . + ``` + + replace the `` with the new Docker image Name of your choice, + +4. After successful build, verify your \ in the list all available docker images. + + ```bash + docker images + ``` diff --git a/tools/rocm-build/docker/ubuntu24/install-prerequisites.sh b/tools/rocm-build/docker/ubuntu24/install-prerequisites.sh new file mode 100644 index 000000000..4898a7792 --- /dev/null +++ b/tools/rocm-build/docker/ubuntu24/install-prerequisites.sh @@ -0,0 +1,237 @@ + +#! /usr/bin/bash +set -ex + +# The following assumes that you have a cache, e.g. +# https://docs.docker.com/engine/examples/apt-cacher-ng/ +# Comment out if it breaks things +echo 'Acquire::http { Proxy "http://rocm-ci-services.amd.com:3142"; };' > /etc/apt/apt.conf.d/01proxy + +apt-get update +DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true apt-get install --no-install-recommends -y $(sed 's/#.*//' /tmp/packages) +update-ccache-symlinks +apt-get upgrade +apt-get clean +rm -rf /var/cache/apt/ /var/lib/apt/lists/* /etc/apt/apt.conf.d/01proxy + +#Install 2.17.1 version of git as we are seeing issues with 2.25 , where it was not allowing to add git submodules if the user is different for parent git directory +curl -o git.tar.gz https://cdn.kernel.org/pub/software/scm/git/git-2.17.1.tar.gz +tar -zxf git.tar.gz +cd git-* +make prefix=/usr/local all +make prefix=/usr/local install +git --version + +# venv for python to be able to run pip3 without --break-system-packages +python3 -m venv /opt/venv + +pip3 install --no-cache-dir setuptools wheel tox +pip3 install --no-cache-dir --pre CppHeaderParser argparse requests lxml barectf recommonmark jinja2==3.0.0 websockets matplotlib numpy scipy minimal msgpack pytest sphinx joblib PyYAML==5.3.1 rocm-docs-core cmake==3.25.2 pandas myst-parser + +# Allow sudo for everyone user +echo 'ALL ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/everyone + +# Install OCaml packages to build LLVM's OCaml bindings to be used in lightning compiler test pipeline +wget -nv https://sourceforge.net/projects/opam.mirror/files/2.1.4/opam-2.1.4-x86_64-linux -O /usr/local/bin/opam +chmod +x /usr/local/bin/opam +opam init --yes --disable-sandboxing +opam install ctypes --yes + +# Install and modify git-repo (#!/usr/bin/env python -> #!/usr/bin/env python3) +curl https://storage.googleapis.com/git-repo-downloads/repo > /usr/bin/repo +chmod a+x /usr/bin/repo + +# Build ccache from the source +cd /tmp +git clone https://github.com/ccache/ccache -b v4.7.5 +cd ccache +mkdir build +cd build +cmake -DCMAKE_BUILD_TYPE=Release .. +make +make install +cd /tmp +rm -rf ccache + +#Install older version of hwloc-devel package for rocrtst +curl -lO https://download.open-mpi.org/release/hwloc/v1.11/hwloc-1.11.13.tar.bz2 +tar -xvf hwloc-1.11.13.tar.bz2 +cd hwloc-1.11.13 +./configure +make +make install +cp /usr/local/lib/libhwloc.so.5 /usr/lib +hwloc-info --version + +# Install gtest +mkdir -p /tmp/gtest +cd /tmp/gtest +wget https://github.com/google/googletest/archive/refs/tags/v1.14.0.zip -O googletest.zip +unzip googletest.zip +cd googletest-1.14.0/ +mkdir build +cd build +cmake .. +make -j$(nproc) +make install +rm -rf /tmp/gtest + +## Install gRPC from source +## RDC Pre-requisites +GRPC_ARCHIVE=grpc-1.61.0.tar.gz +mkdir /tmp/grpc +mkdir /usr/grpc +cd /tmp +git clone --recurse-submodules -b v1.61.0 https://github.com/grpc/grpc +cd grpc +mkdir -p build +cd build +cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/grpc -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD=14 -DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,'$ORIGIN' .. +make -j $(nproc) install +rm -rf /tmp/grpc + +## rocBLAS Pre-requisites(ROCMOPS-3856) +## Download prebuilt AMD multithreaded blis (2.0) +## Reference : https://github.com/ROCmSoftwarePlatform/rocBLAS/blob/develop/install.sh#L403 +mkdir -p /tmp/blis +cd /tmp/blis +wget -O - https://github.com/amd/blis/releases/download/2.0/aocl-blis-mt-ubuntu-2.0.tar.gz | tar xfz - +mv amd-blis-mt /usr/blis +cd / +rm -rf /tmp/blis + +## rocBLAS Pre-requisites(SWDEV-404612) +## Download aocl-linux-gcc-4.2.0_1_amd64.deb +mkdir -p /tmp/aocl +cd /tmp/aocl +wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-2/aocl-linux-gcc-4.2.0_1_amd64.deb +apt install ./aocl-linux-gcc-4.2.0_1_amd64.deb +rm -rf /tmp/aocl + +## hipBLAS Pre-requisites +## lapack(3.9.1v) +## Reference https://github.com/ROCmSoftwarePlatform/rocSOLVER/blob/develop/install.sh#L174 +lapack_version=3.9.1 +lapack_srcdir=lapack-$lapack_version +lapack_blddir=lapack-$lapack_version-bld +mkdir -p /tmp/lapack +cd /tmp/lapack +rm -rf "$lapack_srcdir" "$lapack_blddir" +wget -O - https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.9.1.tar.gz | tar xzf - +cmake -H$lapack_srcdir -B$lapack_blddir -DCMAKE_BUILD_TYPE=Release -DCMAKE_Fortran_FLAGS=-fno-optimize-sibling-calls -DBUILD_TESTING=OFF -DCBLAS=ON -DLAPACKE=OFF +make -j$(nproc) -C "$lapack_blddir" +make -C "$lapack_blddir" install +cd $lapack_blddir +cp -r ./include/* /usr/local/include/ +cp -r ./lib/* /usr/local/lib +cd / +rm -rf /tmp/lapack + +## rocSOLVER Pre-requisites +## FMT(7.1.3v) +## Reference https://github.com/ROCmSoftwarePlatform/rocSOLVER/blob/develop/install.sh#L152 +fmt_version=7.1.3 +fmt_srcdir=fmt-$fmt_version +fmt_blddir=fmt-$fmt_version-bld +mkdir -p /tmp/fmt +cd /tmp/fmt +rm -rf "$fmt_srcdir" "$fmt_blddir" +wget -O - https://github.com/fmtlib/fmt/archive/refs/tags/7.1.3.tar.gz | tar xzf - +cmake -H$fmt_srcdir -B$fmt_blddir -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_EXTENSIONS=OFF -DCMAKE_CXX_STANDARD_REQUIRED=ON -DFMT_DOC=OFF -DFMT_TEST=OFF +make -j$(nproc) -C "$fmt_blddir" +make -C "$fmt_blddir" install + +# Build and install libjpeg-turbo +mkdir -p /tmp/libjpeg-turbo +cd /tmp/libjpeg-turbo +wget -nv https://github.com/rrawther/libjpeg-turbo/archive/refs/heads/2.0.6.2.zip -O libjpeg-turbo-2.0.6.2.zip +unzip libjpeg-turbo-2.0.6.2.zip +cd libjpeg-turbo-2.0.6.2 +mkdir build +cd build +cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=RELEASE -DENABLE_STATIC=FALSE -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib .. +make -j$(nproc) install +rm -rf /tmp/libjpeg-turbo + +# Get released ninja from source +mkdir -p /tmp/ninja +cd /tmp/ninja +wget -nv https://codeload.github.com/Kitware/ninja/zip/refs/tags/v1.11.1.g95dee.kitware.jobserver-1 -O ninja.zip +unzip ninja.zip +cd ninja-1.11.1.g95dee.kitware.jobserver-1 +./configure.py --bootstrap +cp ninja /usr/local/bin/ +rm -rf /tmp/ninja + +# Install FFmpeg and dependencies +# Build NASM +mkdir -p /tmp/nasm-2.15.05 +cd /tmp +wget -qO- "https://distfiles.macports.org/nasm/nasm-2.15.05.tar.bz2" | tar -xvj +cd nasm-2.15.05 +./autogen.sh +./configure --prefix="/usr/local" +make -j$(nproc) install +rm -rf /tmp/nasm-2.15.05 + +# Build YASM +mkdir -p /tmp/yasm-1.3.0 +cd /tmp +wget -qO- "http://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz" | tar -xvz +cd yasm-1.3.0 +./configure --prefix="/usr/local" +make -j$(nproc) install +rm -rf /tmp/yasm-1.3.0 + +# Build x264 +mkdir -p /tmp/x264-snapshot-20191217-2245-stable +cd /tmp +wget -qO- "https://download.videolan.org/pub/videolan/x264/snapshots/x264-snapshot-20191217-2245-stable.tar.bz2" | tar -xvj +cd /tmp/x264-snapshot-20191217-2245-stable +PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" ./configure --prefix="/usr/local" --enable-shared +make -j$(nproc) install +rm -rf /tmp/x264-snapshot-20191217-2245-stable + +# Build x265 +mkdir -p /tmp/x265_2.7 +cd /tmp +wget -qO- "https://get.videolan.org/x265/x265_2.7.tar.gz" | tar -xvz +cd /tmp/x265_2.7/build/linux +cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on ../../source +make -j$(nproc) install +rm -rf /tmp/x265_2.7 + + +# Build fdk-aac +mkdir -p /tmp/fdk-aac-2.0.2 +cd /tmp +wget -qO- "https://sourceforge.net/projects/opencore-amr/files/fdk-aac/fdk-aac-2.0.2.tar.gz" | tar -xvz +cd /tmp/fdk-aac-2.0.2 +autoreconf -fiv +./configure --prefix="/usr/local" --enable-shared --disable-static +make -j$(nproc) install +rm -rf /tmp/fdk-aac-2.0.2 + +# Build FFmpeg +cd /tmp +rm -rf ffmpeg +git clone -b release/4.4 https://git.ffmpeg.org/ffmpeg.git ffmpeg +cd ffmpeg +PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" +./configure --prefix="/usr/local" --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --extra-libs=-lpthread --extra-libs=-lm --enable-shared --disable-static --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-gpl --enable-nonfree +make -j$(nproc) install +rm -rf /tmp/ffmpeg + +cp /tmp/local-pin-600 /etc/apt/preferences.d + +command -v lbzip2 +ln -sf $(command -v lbzip2) /usr/local/bin/compressor || ln -sf $(command -v bzip2) /usr/local/bin/compressor + +# Install Google Benchmark (ROCMOPS-5283) +mkdir -p /tmp/Gbenchmark +cd /tmp/Gbenchmark +wget -qO- https://github.com/google/benchmark/archive/refs/tags/v1.6.1.tar.gz | tar xz +cmake -Sbenchmark-1.6.1 -Bbuild -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DBENCHMARK_ENABLE_TESTING=OFF -DCMAKE_CXX_STANDARD=14 +make -j -C build +cd /tmp/Gbenchmark/build +make install diff --git a/tools/rocm-build/docker/ubuntu24/local-pin-60 b/tools/rocm-build/docker/ubuntu24/local-pin-60 new file mode 100644 index 000000000..8cfa54d34 --- /dev/null +++ b/tools/rocm-build/docker/ubuntu24/local-pin-60 @@ -0,0 +1,3 @@ +Package: * +Pin: origin "" +Pin-Priority: 600 diff --git a/tools/rocm-build/docker/ubuntu24/packages b/tools/rocm-build/docker/ubuntu24/packages new file mode 100644 index 000000000..346335a15 --- /dev/null +++ b/tools/rocm-build/docker/ubuntu24/packages @@ -0,0 +1,140 @@ +# List of packages needed for stage1 build +apt-utils +aria2 +autoconf +automake +bc +bison +bridge-utils +build-essential +bzip2 +ccache +check +chrpath +cifs-utils +cmake +cpio +curl +devscripts +dos2unix +doxygen +fakeroot +flex +freeglut3-dev +g++ +g++-multilib +# gawk is needed for aomp +gawk +gcc +gcc-multilib +gfortran +git-lfs +gnupg +googletest +graphviz +kernel-wedge +# kmod for kernel build +kmod +lbzip2 +# less is needed by repo +less +libass-dev +libatlas-base-dev +libbabeltrace-dev +libboost-all-dev +libboost-dev +libboost-filesystem-dev +libboost-program-options-dev +libboost-system-dev +libbz2-dev +libc++-dev +libc++1 +libc++abi-dev +libc++abi1 +libc6-dev-i386 +libcap-dev +libcurl4-openssl-dev +libdrm-dev +libdw-dev +libdw1 +libdwarf-dev +libelf-dev +libelf1 +libexpat1-dev +libfftw3-dev +libfile-find-rule-perl +libgflags-dev +libglew-dev +libgmp-dev +libgoogle-glog-dev +libgtk2.0-dev +libhdf5-dev +libjpeg-dev +libleveldb-dev +liblmdb-dev +liblzma-dev +libmpfr-dev +libmpich-dev +libmsgpack-dev +libncurses-dev +libnuma-dev +libomp-dev +libopenblas-dev +libpci-dev +libpci3 +libpciaccess-dev +libpciaccess0 +libprotobuf-dev +libpython3-dev +libreadline-dev +libsnappy-dev +libssl-dev +libsuitesparse-dev +libsystemd-dev +libtool +liburi-encode-perl +libva-dev +libvirt-clients +libvirt-daemon-system +libyaml-cpp-dev +llvm +llvm-dev +llvm-runtime +mesa-common-dev +mpich +ocaml +ocaml-findlib +patchelf +pigz +pkg-config +protobuf-compiler +python-is-python3 +python3-barectf +python3-dev +python3-pip +python3-pip-whl +python3-requests +python3-venv +python3-yaml +python3-yaml +re2c +redis-tools +# hipclang needs rpm +rpm +rsync +ssh +# This makes life more pleasent inside the container +strace +sudo +systemtap-sdt-dev +texinfo +texlive +texlive-extra-utils +texlive-plain-generic +texlive-xetex +unzip +vim +wget +xsltproc +xxd +zlib1g-dev diff --git a/tools/rocm-build/envsetup.sh b/tools/rocm-build/envsetup.sh index 9cf30190d..2d1d27e4b 100755 --- a/tools/rocm-build/envsetup.sh +++ b/tools/rocm-build/envsetup.sh @@ -123,9 +123,12 @@ if [ -d "$HSA_OPENSOURCE_ROOT/ROCT-Thunk-Interface" ]; then export THUNK_ROOT=$HSA_OPENSOURCE_ROOT/ROCT-Thunk-Interface fi export AQLPROFILE_ROOT=$WORK_ROOT/hsa/aqlprofile +export OMNIPERF_ROOT=$WORK_ROOT/omniperf export ROCPROFILER_ROOT=$WORK_ROOT/rocprofiler export ROCTRACER_ROOT=$WORK_ROOT/roctracer export ROCPROFILER_REGISTER_ROOT=$WORK_ROOT/rocprofiler-register +export ROCPROFILER_SDK_ROOT=$WORK_ROOT/rocprofiler-sdk +export OMNITRACE_ROOT=$WORK_ROOT/omnitrace export RDC_ROOT=$WORK_ROOT/rdc export RDCTST_ROOT=$RDC_ROOT/tests/rdc_tests export UTILS_ROOT=$WORK_ROOT/rocm-utils @@ -147,7 +150,6 @@ export ROCM_CORE_ROOT=$WORK_ROOT/rocm-core export ROCM_CMAKE_ROOT=$WORK_ROOT/rocm-cmake export ROCM_BANDWIDTH_TEST_ROOT=$WORK_ROOT/rocm_bandwidth_test export ROCMINFO_ROOT=$WORK_ROOT/rocminfo -export CLANG_OCL_ROOT=$WORK_ROOT/clang-ocl export ROCR_DEBUG_AGENT_ROOT=$WORK_ROOT/rocr_debug_agent export COMGR_ROOT=$LLVM_PROJECT_ROOT/amd/comgr export COMGR_LIB_PATH=$OUT_DIR/build/amd_comgr @@ -179,7 +181,7 @@ export BUILD_ARTIFACTS=$OUT_DIR/$PACKAGEEXT export HIPCC_COMPILE_FLAGS_APPEND="-O3 -Wno-format-nonliteral -parallel-jobs=4" export HIPCC_LINK_FLAGS_APPEND="-O3 -parallel-jobs=4" -export PATH="${ROCM_PATH}/lib/llvm/bin:${PATH}" +export PATH="${ROCM_PATH}/bin:${ROCM_PATH}/lib/llvm/bin:${PATH}" export LC_ALL=C.UTF-8 export LANG=C.UTF-8 diff --git a/tools/rocm-build/rocm-6.2.0.xml b/tools/rocm-build/rocm-6.2.0.xml index 981598b67..d9bdc9629 100644 --- a/tools/rocm-build/rocm-6.2.0.xml +++ b/tools/rocm-build/rocm-6.2.0.xml @@ -6,30 +6,29 @@ sync-c="true" sync-j="4" /> + + + + - + - - - - - - - - - + + + + - - - + + + @@ -72,5 +71,4 @@ - - + \ No newline at end of file diff --git a/tools/rocm-build/rocm-6.1.1.xml b/tools/rocm-build/rocm-6.2.1.xml similarity index 89% rename from tools/rocm-build/rocm-6.1.1.xml rename to tools/rocm-build/rocm-6.2.1.xml index 868fabfd9..e70601529 100644 --- a/tools/rocm-build/rocm-6.1.1.xml +++ b/tools/rocm-build/rocm-6.2.1.xml @@ -1,71 +1,74 @@ - - + - + + + + + - - + - - + - - + - + - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - + + + + + + + + - + \ No newline at end of file diff --git a/tools/rocm-build/rocm-6.1.0.xml b/tools/rocm-build/rocm-6.2.2.xml similarity index 90% rename from tools/rocm-build/rocm-6.1.0.xml rename to tools/rocm-build/rocm-6.2.2.xml index 83f1b0e41..b63a304aa 100644 --- a/tools/rocm-build/rocm-6.1.0.xml +++ b/tools/rocm-build/rocm-6.2.2.xml @@ -1,68 +1,71 @@ - - + - + + + + + - - + - - + - - + - + - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - + + + + + + + + diff --git a/tools/rocm-build/rocm-6.1.2.xml b/tools/rocm-build/rocm-6.2.4.xml similarity index 85% rename from tools/rocm-build/rocm-6.1.2.xml rename to tools/rocm-build/rocm-6.2.4.xml index 94dedfa3c..7ed58a463 100644 --- a/tools/rocm-build/rocm-6.1.2.xml +++ b/tools/rocm-build/rocm-6.2.4.xml @@ -1,71 +1,75 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file From d83ed9d58a15121d86beb57b8dfa07e3acd87258 Mon Sep 17 00:00:00 2001 From: randyh62 <42045079+randyh62@users.noreply.github.com> Date: Mon, 18 Nov 2024 10:19:32 -0800 Subject: [PATCH 12/74] Update How to TOC and Index (#210) --- docs/index.md | 12 ++++++------ docs/sphinx/_toc.yml.in | 26 +++++++++++++++----------- 2 files changed, 21 insertions(+), 17 deletions(-) diff --git a/docs/index.md b/docs/index.md index e1677054c..fccddc4a2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -40,16 +40,16 @@ ROCm documentation is organized into the following categories: :class-body: rocm-card-banner rocm-hue-12 * [Programming guide](./how-to/hip_programming_guide.rst) -* [Using ROCm for AI](./how-to/rocm-for-ai/index.rst) -* [Using ROCm for HPC](./how-to/rocm-for-hpc/index.rst) -* [Fine-tuning LLMs and inference optimization](./how-to/llm-fine-tuning-optimization/index.rst) +* [Use ROCm for AI](./how-to/rocm-for-ai/index.rst) +* [Use ROCm for HPC](./how-to/rocm-for-hpc/index.rst) +* [Fine-tune LLMs and inference optimization](./how-to/llm-fine-tuning-optimization/index.rst) * [System optimization](./how-to/system-optimization/index.rst) * [AMD Instinct MI300X performance validation and tuning](./how-to/tuning-guides/mi300x/index.rst) * [GPU cluster networking](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html) * [System debugging](./how-to/system-debugging.md) -* [Using MPI](./how-to/gpu-enabled-mpi.rst) -* [Using advanced compiler features](./conceptual/compiler-topics.md) -* [Setting the number of CUs](./how-to/setting-cus) +* [Use MPI](./how-to/gpu-enabled-mpi.rst) +* [Use advanced compiler features](./conceptual/compiler-topics.md) +* [Set the number of CUs](./how-to/setting-cus) * [ROCm examples](https://github.com/amd/rocm-examples) ::: diff --git a/docs/sphinx/_toc.yml.in b/docs/sphinx/_toc.yml.in index 5180ceb17..8850b7f28 100644 --- a/docs/sphinx/_toc.yml.in +++ b/docs/sphinx/_toc.yml.in @@ -26,18 +26,21 @@ subtrees: - file: how-to/programming_guide.rst title: Programming guide - file: how-to/rocm-for-ai/index.rst - title: Using ROCm for AI + title: Use ROCm for AI subtrees: - entries: - file: how-to/rocm-for-ai/install.rst title: Installation - file: how-to/rocm-for-ai/train-a-model.rst + title: Train a model - file: how-to/rocm-for-ai/hugging-face-models.rst + title: Run models from Hugging Face - file: how-to/rocm-for-ai/deploy-your-model.rst + title: Deploy your model - file: how-to/rocm-for-hpc/index.rst - title: Using ROCm for HPC + title: Use ROCm for HPC - file: how-to/llm-fine-tuning-optimization/index.rst - title: Fine-tuning LLMs and inference optimization + title: Fine-tune LLMs and inference optimization subtrees: - entries: - file: how-to/llm-fine-tuning-optimization/overview.rst @@ -46,17 +49,18 @@ subtrees: subtrees: - entries: - file: how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst - title: Using a single accelerator + title: Use a single accelerator - file: how-to/llm-fine-tuning-optimization/multi-gpu-fine-tuning-and-inference.rst - title: Using multiple accelerators + title: Use multiple accelerators - file: how-to/llm-fine-tuning-optimization/model-quantization.rst - file: how-to/llm-fine-tuning-optimization/model-acceleration-libraries.rst - file: how-to/llm-fine-tuning-optimization/llm-inference-frameworks.rst - file: how-to/llm-fine-tuning-optimization/optimizing-with-composable-kernel.md - title: Optimizing with Composable Kernel + title: Optimize with Composable Kernel - file: how-to/llm-fine-tuning-optimization/optimizing-triton-kernel.rst - title: Optimizing Triton kernels + title: Optimize Triton kernels - file: how-to/llm-fine-tuning-optimization/profiling-and-debugging.rst + title: Profile and Debug - file: how-to/system-optimization/index.rst title: System optimization subtrees: @@ -84,20 +88,20 @@ subtrees: - url: https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/${branch}/index.html title: GPU cluster networking - file: how-to/gpu-enabled-mpi.rst - title: Using MPI + title: Use MPI - file: how-to/system-debugging.md - file: conceptual/compiler-topics.md - title: Using advanced compiler features + title: Use advanced compiler features subtrees: - entries: - url: https://rocm.docs.amd.com/projects/llvm-project/en/latest/index.html title: ROCm compiler infrastructure - url: https://rocm.docs.amd.com/projects/llvm-project/en/latest/conceptual/using-gpu-sanitizer.html - title: Using AddressSanitizer + title: Use AddressSanitizer - url: https://rocm.docs.amd.com/projects/llvm-project/en/latest/conceptual/openmp.html title: OpenMP support - file: how-to/setting-cus - title: Setting the number of CUs + title: Set the number of CUs - url: https://github.com/amd/rocm-examples title: ROCm examples From c18694b0fe49c3d1a0ffef258fcd29f05465c097 Mon Sep 17 00:00:00 2001 From: randyh62 <42045079+randyh62@users.noreply.github.com> Date: Fri, 22 Nov 2024 08:04:01 -0800 Subject: [PATCH 13/74] add azure image (#214) --- docs/data/rocm-software-stack-6_3_0-azure.jpg | Bin 0 -> 353032 bytes docs/what-is-rocm.rst | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 docs/data/rocm-software-stack-6_3_0-azure.jpg diff --git a/docs/data/rocm-software-stack-6_3_0-azure.jpg b/docs/data/rocm-software-stack-6_3_0-azure.jpg new file mode 100644 index 0000000000000000000000000000000000000000..663b136b842b0ce41f05e4c7f63131e49375eeab GIT binary patch literal 353032 zcma&OS@QeJu_mT=zx7iuL7|Rt*nyTXlYp$N4h+Oh0K`nZ+yFoT#5|9C@(TRmZTQh% zK!=xbT|e?%=}4`5uzA{q7_DkKp^yW%zf${S=w2 zfBGHs^VHu4ck{PgH>SU(|A78JK@jY>SNadpCA#4BW&Wq%{qE;MfBAen_DcTpoPHVW&r@CroYrZR zw(|rA5rzLzJup0fsrU~^{=ZX6l$-qDu9(9`lmGXgdB6YLm7C@t)|_T5{p*1K!}d_5TjsIBSuL^v0`5(2X z|COHnO}D@ePznEsqXF&#ef~#%___0~_EE~OuKekh{4wjsCYt|TG*Ofb)@)7-yr*`I(C|B)NhpDA#WdJC*$6|itPArWY8ZO*YI2YSB3t*I@q7Uh(-U6fPc`4`>)IXogC-A zG;tdh_4Ie+`!nc&JG{Rr%6B$CTTUO1Q~I-{{^@uBx~BfJM}15GQU?s$&z|tdt$5Av zKV#(O)4g9^`fIEHQtxNtJ|*61k-uj!($rsT{c`Q!s`iUEE9Oo7YrS7C{#*4v>i=h$ zE@6UU9(ENfR?e_Yeq(Hgl^KSIuMcK&#rOIW=`#}PP3G^*oneSaRqG6dFrN?0fbu_| z8}mQ^r=9uVn8#0L@OdyS3=s@p2~~_KH6uEo%S;Bxj0|q(j~mOlH#T<#Hm^kX@l?1i z7uh`5c&1aity4b&d6v2J%=Yu&gJ%fy#!v>&-W`E`IJbR^#eGVooyz|5-*%JOEzx-f zd4eZA#kXq-R` z0~Gqp=OHkgEP`Q?IHm&K1NU;7|NC-Zfe(fQy#?cv$lRmTKv51nfM@at2d;q;0fO2) zn5?@6#$S*vd4G5}@rK=sFx^m&cn9?hHkJCZ`M7P9S;IFZy$lE;UHn)3w7k9`zxqv zKg8aB)XY=tlj{?=&{z8+(xAU)5(;IwaeF@_sgN7BAP&9kAg&^9ukO=l6Mh5^ajRfy zOR8nB`b~c+`e9h6F}kds#V|7HzRa@^pe~Q+jROi1|BUVX#P}jTUCS+bTIQMs= z_FHNkshoC8v z`Y1c7)q;0=|5?=YoA?N<`wp(*l!wb}32z)5v%6*^&|{o3DswZKVYFweM3GrVR|%F- z3EmXYvfhag>jhe-*)1sZx%B)9#H;241o^c|F}afywTn@80Gi1mJ-vgfKfm!`{r06J z0Fq`kOOn_Fa>E4Pcpw*-J_1E#2&o1)VeJ7fiKwrW#sZ#Pch0rUXHQ=h`h*We0n&ed zslo{}_nG^Md*Zc7z+`lT2m%|LZwqUc_-WzWTnx2~C~7`2bXE-HnB?8fnJY0=UzPbq z7d$J%L2uR`hFIf4*=8!@+r6a_SosL7U@{ACulGpS3*@?C{(9J>*pnK@C|h0x+Usqw zZ~8If>x)>=qjnqBI8dA(-I@6qCw&7QB|(U zN1!BK4NO4~k81y;sL zpBb|Eo<6^j7OuPC+Q2`pjU%!Tz_K#`RqUA7KTGfD!ncK=hmA&)U15g(it)}^j) zO!Nb-7Mr|Ce3#oHVg!bhbe7#XF%ekunn?O;qTHr ztQ}HnQ<;bOMMJa_lo3zB8GYQA^Z@ow|?jXqZ)9Y|8DdWDfpZA@>iJpwfm0IwuzWU*; zt}+`3^%6PT8?A_H_^xG!l+WSSTqXI&cle`L%S?Bv9`deEaK834Lc@!tbGJLYlDBtJ zX-Sc~<F{Y`0(NJ04EYcjFjH@9V)t=2jm zfv3RZ1ne2(XWt>ORSa(}%3+PL$BN8~h|Lzy_IbQ}9kp{p$~JGJFHlDH-<4vgkdGqq zS$8wy+xMOhmrj=I^x+YYo>yLVAwp7t3rAWY`0j*PL?GX=mLkaLz}}qcMicD=lN9zj zea2AfLS{0+~@&optx(Wlw?~2hy!U075-m?&m*}VU zz8z4%hpfF^=BuY_W?ItG*A>ysgN|^1i(-;O8aRB8#ltp`;vs`#iY;x*ihx9k!U-Piv)V3<7d**MI5#gJflp+;c|N4XHYu9! zp10J!@VC2fSfK`d%1U$|zz4-dPNPo=9>B(t)be%hE3@Zc)heAzn9_3sQ}+A2meUt# zghph2YSNS0?GO}PM5i^1^yc%+8)x$C}5Z`4mglK4yRxlW^I{QZ!CZ}^&uR9dyWbp#6o_PaZO z);hVMT{N5V_6XR*S6%GTMTKj&sv;*cJoImuBYG+)+`7%zhgsuZrlGkuMSJ%^yK|?s zuzWdwpU1^Uj+X#C549;{S1{;DHOPWFKPVhHz?o04G5?LE610@!w0WR&ss{B9IAfP& zoEb&Dji>usGFctyRNj2xK-F|llGNMB(YFi!Tpzf}*xM0IQ^yURSC`*Osckz_$E0P; z${P??Q&nL$c~a{#VX=Sx^y_> z^L0GDF!~6Ll(r&qqvr58us>Lk-}ud33JvACF*^nuAnnrefZFtB%KG?XffG6Jf_~GY z3+-RqWs}>ah-|}f$8TuX+%1W%U#Z4dDue7ZsP5x%8ywopFEAWpuCL5?kZVfVRQ;~3 z=oUl8lnwFQAN5O_MUeWIgy2n8J@siJ8quJ8rbCQ@m8nIf=)7|H7lG80}HMe=Vktnd%{j<)EJy-kvPX%k{(yWd&l(zitN$XVN=R za&>@*vr-Zq86@QGy#2PZH<)~nBh1J(-_Tbc8iEbBsrN_VHGr*>CV}rFlSsS`|CaCa zbz4yKR+JUB$y33iv<|@1nw;sMRqpp03{i}8)l?r1(x!%Z>)#iVlz1g({^?|xOwsW z(Q#$gsj29la4@u%Tt< z`8&(45c{ZE77bQ71TjWfPbDsf`0;MAz@gTOp8|D5ImJ8+zxo4^qj)!QLWszjXs?v# z*B^m6+<>b?G2(Ff6EjoK{YQ=FA2srAfMA4dh3LdUpECj{7JMmy7L_E>q6TOYPeo~{ zn_n&Z?uV`s*1s27D~*W-XS$au6zoJe;$H~`8t@55#H zy_0rZ^=|1qA2V_BrjmEbDi|k#Tx zh;u&Jb$pw!sbm+z;DWRYf1FfsSBp=3PF)GrJq^Su=KUj(mz(=SH}?iiNHck;t)_;{ zMF9XGUKjTm9{S<|9Eg8B@|kSj>FbF{oL>93%hW=60eIt!@xgw6?A;ekBi^_?kO=Qz zy~ZaI?2?B9Zzyu;%3e;grp!MJ<%B1PkXXva3^L6$5cOSFDS^9!hQ?zTiK3yoApb5s z*=z3r0q9)LT5zqm>=ZKr0*b5p-ZdkJAe6B8=*gQ=#gRo-R9;scqGicFqoS6*jTmdF z;5n{Yx1TPbJROuvhL|brk%Y&nFO-mGQVny55Y1SW0>v$}eYs6X2GH@7+Hvxv~0y-PSyewj@zUGW;D0q$p}fsTtD0+gK96uG;#T~!Of zPrIyJcERxA=Lu+*`gDc>M=HB~-MJL2ax3I2>NbU(A%#JADGC4PR1&P1dE!m4Lx`R0 z9u(FnK21}gpKrG#v@I7_&vhY1m3NYlpTzk5M)b9fBY_t;f8LDaX8#au4dO{)3RNnB`z0{YR6dw+S&1~4OEyDYAjL-$1BrkBascZa%h zP!k-9k+#lwDGOcg2+|0myQ5N8){<2WnyjIJ6Lwh78dve!?%qH%rN_%mb2 z?mD%y9m^Ag0t7|sf=fn(B`{@K4pq|te=|NaowBHETG_pBI}f{JZyQ^M;)#QB@J^{a z=Y{Gl-G`WY@mJ<%hkJ;#*#!kFZMd~$c`;r=J`3+eRR*Wp;+5!{4;KM?3=d~n8CiQOabvgHL_!M|ek28uS$2 zTu8<#=X$6# z7r4*B-`_r3EU;as{;QJ~>+(nqkUB9dqo-_7hrUj`0k|X-W?l5eUt#q$wfcRZTGZa> z(Higj1DiB^8n)ATX3pcVMu!l=(s*F-ZGZ)3EKioaaE_0)Q1&(S>Y}-)<9ip1GFm0p zp8hDuQR>R`b}DKZC9JsfLJf51Ifj;Aj!WmKiMv)y&QC;Hfg4_(ri!L9Na4|%!w5|CPKsLIhZrge#ry@ z64D=mIx{Hz-V8mV8U`x75R5t^5J>@43c)@2^pMz{jhnVcr@i@!BU5YafE+-S;a z&#*2-C&KsSQGL~hCPs2VIP*dK5t6Sm{Q$Pi_UG&Ec}&HvwlmzVF3fP5`i>OQhuyRH zhP9u-VY_}DHh3?k9~1z&f4ls*P{4(mYZys2*bM*{5NZunZcrtQq+s3w+hybL`U{`| z`I*sq=9xc0fK87p00Jnu`(j@SiHzIvKlc59AWLha4FFj-R7I;d`BE>MzF70rDFCwk zfdW2bSh^d)1H@NG5Yz}#L0XhS761nr-fApjN@)}(0|n2L-cDbcridu7=Vi&XXDCwu@9u*e=-IUo` zNQ0M8Ef@ogtN7;q(v-HZ9ya=^M-711`DX32*p!DzpC9fOgeZK_{i zDjnXYIO$;>3LfPC$Qf=E;q z`Fi!dxG5iZ(L-Tx6y^%g)4YxZ|F^9e7I1~9IOPizX+C?=PxG$c^ zeT-+zP-#Krf9h7~m0YMYcwJ^vCr!7!ylGpng`@$N0t7HZxe5_p7Md;Oa}|eHt~~pBDE#wA--{xOxLX_-id)L%6=9668ln)k8^)D)A{YQ^=CV z6~L%Gi2YGq2|MJYhZk#UN7%cjh5;<;q3_(w!xd5jZEnmorURt(Uv{z3GNSAJ+h^te z8X4{}Dry}aa9DZ<0PJ=zMIHAeuxf7tTet-+29OI-^{wu<13p?dN^f)e)?hV>L2z&7 z4O|)DV1CtM`RAgFUuN@{uWQDXU(bPLUWL4I=5i=8vp-3pnhe%|e6rDN=VQ`e+@XR@ z4yb%UmGVltA6LlyxI!DJJi-;)%;D=}Qzo)1DY_z0IL#1cNG>7qvMZ47xQ%NDTm@m` zIa8M)YFZBv4&ETACXA;KS)HuVnQcUR=+gSRCF~a>aQV%!wh{%Ay>iprWokk6<@+yl zfTrmF99mRt@3$B*FEVbJRdGb6oyWV^JKkX=xDh_o(g?!oDAA zQx(OdcEe&ATwGBT4<8p58Q(V$&D=vCx@UU6bW!$Hdjdlqkgb};)4AQ{v&N@yasgco zT|qfj5TQ*l`Rynkcyi?&%h8UEG~d*b`NWFs;U#=+jS~C#DiIs__{FYukL_tbrx3^d8Rsne* zD1?1xbtxnfK$DX6o3N(Mi5wmDCsEZao{q+Sblj`@#wS5|k>%WXjWYp+C>twQ-Cxb4 zozNxprqzJ!BSm?O6YU71MXPSUE##P06l%||Q#a=)K}ramRL0ZBFjy<1DDn;GRJzwN zfCp^{E?hVJrS%JYw+M0V3{Hs<0#O8OEu-%SYo@K(z%_vH)|cTE8E{-Ubjf8$i%v^j zc~l13MMysL^^#P5X$p4sR(gzC51S?XI0aRCAqkin*1v1H0N%{=3{PA@FIsSqYa#M= z;ns1Ad;&<9UEfax+vkWvQ5f>Eat4S7+TpEc$BjsoAU*^Li?J`yM&>`0au*BVCjCOi zDOj#ZL=_XHSss~}kr^h|fwyg_Z35GY`v!}ZP5lV8I&<8S@t)H$z&G!#Upl8W zsaAgh$u!#_4TI*xDa&$`vJoT)eb^*SfFVE=W|Sa2e&OkKY>jw*2%aWY4rfhf-JDmh zVnUqE`j>KbiPQLE4gh?hISD3xhx9?11|7hp%JXzF_ih#8S{Y)v(aEQb)iq246APdP zSalu%7`sxd%}^oWk67WM9Pdj1t`xmAmgw5kBy)L8d$`n|d<^au$on&7uASt}3788~ zI3qi4?~KO_4{X0&bvNYhh2c~VUX5>$_DO~0|; z)gQV5Wu@0iWV>^$E(@PiQ^VATk^m0_z9r}~JhZ^}EGs=`zxb8_Qa-oK-?^2a7XD36 z{EI8+um;?FY`6?_pWH82Ek;wwL_>-5AumaVZg}k>-eMjpGO8GtTUm!ub5!z2pjVY% z_&hH;8x&L$%MtYk_-#Da`I}#pCrNei8=^H0sFZq~ljLc8e8H;M) zT}b)6V|Q)kvuOHAwu;8VL4f6U8#NJ zhlaVfZGCZY02SlhRzyDAhxYNcdPUML_O6RaAzX|bz}9uT=p+4ICzY;4*4m09foi>z5X*eljuBbATNp66_$fLlxLPK5Lu=^V$>>dOpRzxd1q_026>p)}AWw#xrNy zn;tSP;Bvb?ggA3KHKeR!u(F=?%YO8+kAQ%AMc++h-+frV<5v)s7=iG_zAmw-HH@{7 z2P8?MP=_&G^CVHN=5Pm|a}-P!+fLWiC#+`Sct$84#8GcAcuWEbEs87sDlN=EdJC>K z=+otAGX0?7{h%j-2OR(Q?W)Gj>5mGqc-UIlRyNJ#>oo?<#_+{LZbp{CA_9y`CJ+!e z40Lzm`eeGz{hI-IhMcJt2{=mPlZ|Nm{;Dv~M`2Yvdwoo+OCZMvR-yyuUnby|HJTrK zo!Anla#KVP&R?qV!dxmEsb6Bf1M^T6a@uy+&`~|hd4!0Et;;f=;+9*4ZdfYm812_t zR#lO_xlzju`>US1L88jlOrSjLK~%ie-@wOt?w5SBZF{~Sjyh`M^ehLY-6@-$4{Qv? zZLY})Men_==C}BElb}Vck}%S*8lgcZokT$i(pr~b`$hp2fV+`Z^w*KX^A!ihf3n(H zHUYH7DpA!l5s=BtpF>jAzp{!JT`M33A#Bgl6UIm7fe4HEOv+&4+C{lByU<%h0IPNc zn0)u2t!7tw9lipB&L@g>|GLxMNaXt8>@+`{%tzo~*f;;0!4jJH8>OY@^+afWKa#hF zP)?O7G>BU^hXn|DKyFYlwuqyI6wfj2MI4STr~I!TYO^R@n2O)V*-=U<@bY!H;-~~>AGJAgmlni{R*1-*kHUN1IBCo1gzh0l#tsK78U%LVpkE!6>bfd87 zN`~uLhq^L9XKmi4J;eoJF)COAu-@Uy&@FfANvFxVutKbK&cG*Syk)I?!|`8CzyU@v zSvmg42YwAwWrGOjMj@BXOxOFsJN=W}Z`ybO{&I9;_}xg=Dp_iq;~mC-NNI!OLqKZl z5PxSmG3sd2ys-1%2xQoXc-y);^qR=_F96w-ByliS!3hSl5HAs@PA?CZ2i3NX>!hd6 z)Q3VvE7JQhJbw|W%$J={`+Lv*XU=QSV^yDjyh$)Ca#0FaIC?_1b%Znc2@ya|^uOJt98 zu|CKBd@FsvXHkPR$}!RU><)WuDu|2ee4+cadu~IrR(D{SxI{(p%hVP3$LoDNf%`V3 zO89ti7BVmZg}0l8Ll)Y0Em6L2MVMbL6tDn0nUyad^ZY9IXak$d3sGR%fCAm?+|_cd zyHr+>+V^Vxaa3`h69Da#&3EoM1t1d@vzBTp;GH*scMSeiUJ6qQAc=ky`~LjL9;bzZ zbgy<=UmU#%LEu;bOi1wnd640%1|&?GyPjr0s8DByAa@UceQX*8Omj@%%%9LPPcARw zReLL`Q%j&X6$~kz9K;Be+y_hDXj=(+c!3k;BxXc^Y>d+JcJjWqO8G9nG=4fDpjgNZ zF~`!XxcN}OMSz>#@+n{!vO-a-*5pWv2B*EX@pc_OEfp%1$1R;#4gPDC>OxNT>2glK zLlV$hBN8R$u~3xm6iLEXMZc5sh?H(xg1gay|l0JUiFEH^c0q5VQlDk`Yd}8M> z!>*xUvK5R}2XjqtRLhMz3G1v4C|)U>29T&Q`W{>g+>227d36o5qpEDp0K;QS(DfkM zr|}#|8}NFIF0qb{`?OGBv=?vHY?rXNv`CF7Sxp=|sH%~0y6}jvDr2IdBgab!!Ue9$ zs>@Clc#?MfX$|`RbmT#DOp8yOuO7lTByU1lt812fKOV~-%^Z^eh*MhCIB4<=ShK3t z0m!&KX~XvwnE>Ph=Qu3EaFNn(RP5&Qp1>5@H#hpec%2TB0OJMzvh2ai;`n|~UYZ=q z&~me8bLm6a;%z>)o=8w_7RO(*Yv2$sJX-*$8unQY5Z?e?}^i!d< zWNa>oZe;N+%brg_O(~u;2P=@MQpK9xJyLmrgPEBd#!vmiUsB|evN(uqB*4yck(a&} z(Cr&Z>43Qf8qPU@Q|t{V$o)f6*yeY1mxO2LU2{YBKuoi$z$+Havi9%yZl-g526;jX zJs@X2(7G*)H+=sR$!sxo&@j~R%d;xq<&4+#gKCF8@`Qp@Iehnov@th_@+Q9kGZdYG zr*$QV_4SzdTLvfzPd};d@0vDVAGKNWTfMDPq1@gsi$bz_DE6E20Bc&9$pmcu2uLcH zC!p~qCsoAt%c#ruyr~1LNDy7srdpP{mPOiy6m-^PhA~7nt(@` za0CqwiJ)NvztRc^>uLr~00g21Rsl$ao%qB55k_c}tw7bjWj!ws{hDaDHi#q8A8}#T7 zdiv?I0249LWv--i`~rbL`}cvr zMUU{R^YEZS<1eiH`qVC9=pVJw!&I*TYV>CxW^Ud6!ebdUWIx=T_UeP>2N)IHQ!?6A z`o}!IPYZud))V;Soo$#gjftaPfrlu_s5S%SRkU`oAjLipADs!J=?Lm7ChH|{ed9q& zU@~PE$K|`0UpQt6MCw2ibsm^{UGKl|Rxim0I9E#%QlG+q< zb+qMd0I&8Hg|CO1U=vbo>01o!$hUGC_`1W$Bx|96vlRksTzm8EQU5w!{S9>vx@$WB zHB7E+A6R}b9=m!YEaRNy<51ZHT@Kjs)H&(&AsK`4x&>PSL%|= zgKZ-Vu=F3bITri`*-DVmLZ(>JedJBTgMRSIeeH4Em=0T}&cbTQ9O^Q>!S_4q9{Cj_#kY0MP*?-&MJ0=^f+&01MEDaG2#mpfqj+N^!dO1$Eg= zap!6OhZmtq@>IV!_288b9228uAK2TUy@XMJxLIG9?k5-E3?Q-n4_vy>82m z`dN2MS0cKnBg#jI3z=J0m10_hVcuK-w!5&hjr(f{{2jQ?g~tR<`eFH@E6mh4UQP?}PRRsRg17&>y{$Q7~2 zH~_XMl2N77R5_qLf4Zy(ej20h)N7liaOI8ysKxiekstM8Xfo%q!zMZGa&)qNMM|nQ z5iLBhbi#Wy99=!#&KfQKD>%1OJ0F4X(9Ra<6-kXVvB+~7mzi~PYRzgz$aFS=fLC;# zImFX2^{nS&xH;))v(=Kh>Qs5^(wx%hh{wzZ%SYgtWr62B_J_7>4 zw0`~-=uur(U6Xno%=4v%;IN}Bppqg|6gAr7lX^a$u?MMzs%L-=3dj6?0B5H__4ide zXvnERmgx?5lIJ*lU1A(8$3VHhg5p+XhUfbfI|FJ52CFF;{KPAj1Xl>0h}BzoH?0x(tpbTpn{qlU+tz85-_) z5N}&@W{YMTk}s!3PrCt{X%LHPEt^|s`6^BTGtgm-f1Tq2JfHdM7ADVqjTFdLyy=v> zX0(4GFe$ZA^HP$9B+mCvlaLMA%p}sU{XD{l3AQjtBwt4=Nq9@j;$?yLH%npyNQc~= zuRw-5l>0(fG6Axm9)PesKd*$7mBd*;MyjmcYtr6id~(M*zPhm;pd$!`YDVU}o$b=) z!N@Tw-rtt>CF4P=365rIF~F6>BM<;51m<`8K%fx5zc2W-!fY!QHQp0 zJov2{KL>U#Aag_FH%RY=<}dfl1Tf$j?f$+0RU3bsmg|Lxko3L{gAjpOwKJDZ07wFCK|#R?LarrsBb((&sG#lU!g)YPZTmi_Rl*AI6#F~wXs4JGfZ zM6*{mp4ilawCDA8FH&vg0i6^EwakLjsWZ@L;PDiOCyfa6ywO=KhTpa9yo;FQC~$zP z_nS*Mql4e>mpO!5T~b2ZP8;z)Ir7MEqFRg3yUGvu+$ZiM@G4Fe3MS-^)=}yKv+$)Q zT-E0my-L%>P2^5xKMEog3gm<=&b6Q z@pQ84O@dNdG64k>%r#^d68&uk1F3Y|9AHgHeQrj^4tkkwTjis7FlCB(sc-l^`=b`#e4 z?PmqfOqt|O_I0I<|en|ed$VybO*ivT{dW`jSY<9+^b zJySzP&_gP|-j@r$2fO%l3H)KBh}4Ku_?tf>+DHDUOouz_Q^F+xIJ?$Tv*3j`zv8}p zGmjh!9hz-(KX`1Uy=^E~!yhsU4r@KJ@icP=s6fbD~qF9vgVYLwSXT);bA5P&Xx_teWUn zAr&k~0$};f@g-zXA_wPZzd%sMtBxY@Tf*HapSE9~e2pRTK@hJq-on>dpY-)-nUVXp zB6(fkjCznE;7hsBHs> zgJ79IUjV>@kNQ78*&p)y{|wn0G#Wi4WeD*P*6|tt`ZK)%NKYBlt&kwj2-9l z>GyI$!18*+0@!!K!Zt&yn2g@S@4+{fnfiHFJR!?}ys<(Z>>QYMLi=eYOu939v1t&o z7dT-0vDZQc(P_0@6M&=|eP#6qzseo0WUfpk`SRVqL;?(u<*|^BHytLRnV2GFF>mj} zjzL~TNpvN#{mmAXr>g+e4(vY!mD=b9^t2mZJ>&#Ghdflrx4s)}^a3pAsYa!EO(c}$ zHL=_`MFb2zI71)9K0hj2FM921Gw!ySP?DKOfK z&Pxu^;`C zGXQDpcYrOMq$)MJRzVp+qg_y-kS;cxrhM++x5>C9vTA1}|s1P&;_d!VJ8z3jT7IdV-1|Q2;2vd0Nb= z^Dm5Y?xO_|AbYUO!FxHDQK~lZ+IIq_?y3cA*ID^*(is?5N2hI_w?1GA=sGFG3wDW~hMw`9XjXv_cBl&u6AI{FQ}X*rJmWXjMsR#8I{N}3vQ_9Rh?F2#C~#5NZNRsEQ#8kU%I2U5cG9QdPhLiXcU*ODfJ#C&CAbh67(E0wWSQ5s`gtnJgazq%X{Hd6I&WDKs0J zxg84OqV48mz~&VE%yZWU3v+*KU2u3~uce1{Mw)~0Ho74Za9wu`Ljz7@!2Gh*SeSm; z_Y-g(Knu`6;5D@b9~cD>(Sv)3!o$IVpiOWDWeEwM>Ix6!0s2~oAq;D}GuDrRh_L^~ZUw?9fwOxXHxR=B zs|&Zs0il^YyPA`H@ZkuWE4V|+unS)Hx6xuoYuh5h-K-FHsA0IZr54dUjO^F3S@PGoF4L!YS|5fH4sOd$ho4EVK={brA{X10=Zeq>Jzm zHUtJ)wk;JG9O386S{iM@5h@5#5kdhgg~aYoy?2KfhrJAj6PurLc^ z6kUrDMI}Y?dKbT(Z*Q2(gK!u^ll+RPWm{6Ba=Z4XCXpe zZG(U<2E+wL&@J51z#G;!isVWL&Nn_*Bpe~s0k{$c(ikKvkx7cM)u-BcV_i_T7=Lh2 z3=_4ihki^wTav54V;I8PokR?$Vv&A6u6DpD(#6J_ZSDl~x3IN#akF*8*hZrC!>nvj z_BuMSh)5DSi~)p1i*jMp?O66f;lWnk%Yy?e3I#X=yssi$Kp-9~3;ke!3oJNI0+DTi zy^H~Kw(+LYFyR>8FbkA-6iy#_FVHbgFduEYJp~)!;HGZ_9PYwhBSF7j~uv8x{1H2th zYuW4^>1i#9_bj08v8 zh7+(s@MvNnf(AT%!w`-R=DH|2409O|q=;zzdnmh;##%JFO6KT{s-6Wk+`iBt&`p*!xn0z-e=oz4@}*u!-(uKc7&G z$Ve=b$|jTOV5y>ujCNsLU_$h94#E1)FmNl|9^)KLrvzgyqRc~x6e|}bD%8)C9R+TT zQGqwHA(*&=fvtO~>#;yAHL5ET_!^=}egrrtp9KP~fvXCU2Eg17!qfOi`e=g)Y*Z46 zw16}xF~G!R>+A+r7@RBTe%b_JPX;z-60kiiuN<&I<8Ae{tccMe;Jgx~*--$wvy(O4 z72_MC9SRPSfYpf#E~OxSfX5RE>=uG!dWQ!C%LhXb4ji69NH-#wX5pleC4{bUe_9|1 z5ClB(LjdHCEMhnui@|3i7@A& zwIG8at09IAM|LQE+2v9X+{wlPZ%uFk#RBLD>Mtr8po?$^F-F{U?Z82xHq|hSj52W6 zvt;S&XaiBLeKTg5lJV02;wRG~5t_wl%cTVj_{@HV*bw64)SP zES=meETgp1(ZEdR;6iYx>gXGiscg$&Z3mVs!j-AxzC7yzPYbI+gcI5k8E&WNgar}l z{GMCd2=5Ax(L=2945BWH9%^l;qesPKd;%habgij0YZ4*c)rtt#tjI9! zFmuBt9(sowxJ5)qT3RtM?oRfuxS(kJs4zO((jgLwg_B(=XlpB?JMg=J5#b^DP_jOW z0%Hb%+mj6c0IXdE7&k2`k&a}aaLaImWt#)pC)7GD#LpGnNnq$XGZ-NVyQmQ0R%saG z=j;f462r6ts8NJ)Jru=Tn*_Z0fHlS+Ox*}@6+YO)(u(e*M)mgQZC?pnHK57vcm(vF0gkFrLuwj-h~YhAeggs48bWaf*Q$2 z*y^(V^?u{f{_geHm(L}Oen#L#K43BL&FjeqsepwedoX+7blXrzc&~G6t{|C`3mni(VAuE5MWE4q-ZTLy!mj#KQ1A3xASM=b>SdJ0BpI`d#TjWKg=P-03Jj{A*0|hR$!1W91G46 z4Qzm?Lilnv7-IvzXGd5@fmou;RuM2CSo(s+0{FkHol`i>3EWB4A%Mu2VQ_sb3%ETC z9)ch{fP)2aBRvX4yCi|-GjIXi66o|G)M*T%<7M$>-z*Y?o6WjqWwJC;}9}PN#pql`vxudl% z0$fQj|Dicc6ap~-1p(wZS_dMSJ|I>R2y*CT$Z-+_1;H+X@IVwe>G$EN4w#5x2Ft1g zuB%cUanXQrVKh3JIhP8S5ONfY3@8Xhc_X8NFhUUeoJ|eG0HlG|S#jDjNBISV6Cf~s zQXSBoPY3Wi-^tW-2R?9c?WKYxgrF2|jsc}CCpaPkn5{ujFMYBNNAch=aHd2G2HD^& zYbo3M_v}z`znKHQoc+D00N?%ZymEzcDAlG?QGi!qfe60Mp5y~J^xyh3sP%Hh>c7Uc z{{Isp4qPWgWvFMsI;3L`Rw)8$OqWS}A(_JUU_e&qcm`DeWR zWq@BaYl22Kf&XCLkBq-o`PT#G{~e z?~(j2g8$?}jsK&fTNdc=o$*J{{(I*&{^bc4Bj->u3&iflv4VW);DHn1Hf9X}hw`5+ z`VGtX=SG~vyb1sE#gX4}{ssCMoZzJvBR~o@j6!35?;$wwdQbq}n*{>b$!Z&Da^4yF z4eIyWkw0EOA<P4uJgCD?^Tz%_si=-OGDanE_k~Lyqg4HdKHQDLek$x zyFW*x|DB1S-}Ju|>tAsFw~@avIAOa9Rh&BX5rm%zUovjF*7a*}{OIbDm;-r86zI1~p zA;IslhBZ!B_GkWoloFu?vqHhEk~jdZO9f9Sg7iL+4u}X1=HyR;w75684grC1@o>^C zKp;r3<)nR=)9Onw%V{%Cn#v5M0vHa>!Bi%dlfDVkiQ!>1kmj}n=>y?(S_DXUg0wk2xP@d-(S++zobKHk$_GJ#4;#2 z3hbx-LS9ZkUgKY#yN_$SmqCpfZQhWLbEk{ML;{?ELhb${js6+s}#DZraUKl8kgLLl|~ zA&|Atehgg{pBgFxyY{_;JN9DVua9ZfooqRC_Gn%vTX1iDNR=6uS)zsuJ{XCztEu$9 z5Bzr${vTHRg$7P?S+7At?DRK?xR)km%|mer_q+QL9x)*Zx94$C0{v5Nb^;^_r{}qB z8eP_VkOt*T|Niz5S33BUi%s{HcZjkK|3d9e>gs>nHkQm5r$N@+Sf##W8t0nghHwjUOLA}K z*5F2RTXH*cyL0<+v$*59leu%bi@48mU*c}ze#||@{g!){hmS{$N0vu}XA6%t50Qt$ z6T%b6lgg9FQ_54vbCc&W&oIv<&o|yxyfVBnUL#&CuPd)FZv^iF-dx@@ytTZyc)NIC z@_ymtlSYDQR0SW_4) zOcjn7&KIr|?ht;zT41%}YV+0GS4XeTU0t*K;p(?*1lB06v0Ou56SpRRO~aZeYo!}Kzvd{R6-M@}!o!UD5x`=hh*WFt8cD?xeE$b=k zlh;?Te&Ptuj56``k{=j%9brp3C0GKFhw( zLD3=5q23YVh;__z?8hnNLU5Pye0Ur_AOFfp+bPDW#aYbR%el+d4P8+eLSNcar-V_bCswM~=r!PlV?I&+cu?+akBM zZeO=OV0)w2YA>=^%?{ojE<4J0%#$6-MdT@OtapL;B*l_)m@-Z^qh?dz(6-VtX`?<{ zeKLJs`)>8k@_plH>X+;HmTpPUqkmvnGfpt3{T=;F{1*dU0;-rRm^+vafue!_fj5IT z21N$72g8C71P_H6h2(@xvTRvr*iiO1_QgXXdN)lw?MT|p=lSJza%Q0hbMPZ0Ef$hpzT+V)xT0o==9JQlGx=3F!IoEd1GA@2)=H zzU2Nj{ka1&14VpZC6?yci#0kIcW^_iELv+|f;=XJ6~QzVgQE z&7(1ov6th)<8yBhyc2m>@LuKpg$c8X`;)GdFF&w8EPhP=B>k!6v)LP_Kfu0CcezvJY%8 z{^K9l&mZ@nzwfoVpioc>{`<4wKTRxNg@~-+Hs=xLhKfMAM4;Rv(8Zfjd9aRw@&e?X zRm<{^n+wXvze0eQN08Gd1kCk6OQBrcJiL4>1o#ElfJF`{;9nsuyjnz<2fAvh1OhU& zMM8Fqiir!DpTG7vTY@OL8Q-px+2Nc;=*;QLm6{}H$XPhCFYMRvc3F!ol-EU|X)~gDKDtu2W}@Eeo_>=@79OwH(r$#`x@?z zxEQrAQ3L|z1`8N|0e;~XD|vaixSQh7YelS%k#EGWRB-ex*SCCSj{`7&tibskzRky>jbHffMlBPT3>L5jgQ!lnCEKHVc657<*Oxmr6{{_65 z>$g%Hp0k)=DrXBgmHz@qO0=*#b=O_qcU^9 zbyJLV^Y!|PpvSh6dw}qNsbjD4T)-($=4hsY#@r73ZxYQ>@$(DR-Q}{$@zO`-Cw3dm zkbo$=IG@g`rL^4k(zp9q&kb>E?Bp;iJ@tOZ^ws=%7)K2_-~O{QE<~>C_D|pj6=XJ1 z*sfK^KklhmT-9v%pC8}Ep`2^^=Ji<{!BSe6#$*k8kF1 zw+J9L{Z1l>xrsCP?|^4W`bDj9-W@H@}%I2ijpRLh5)1X^W5t^%Jl~$cc(W z2x=MT7nYpmI;v8n4rE{&x{RvZOyZ6P&j}7suPyUb$is6_jv9As1EAVGf1cCZ?Bx?- z&Zy?!lrGz5qkZd^=1WUTe{T^|)d>lKT|N9uHz^%LW@9D412F+jpf_|uLNsLdq~HFj z&F9T;7NI}>98xZr5i-xl9LubS&ygMU<1TM%oUC}HB2b3@7-!SJ@MzRj$ZKH|bdVAR zlscc&Fjx2>{M%)aRq@D0fk6a7KL$v`ZoBMZoTOvxcFem3!x{HQ$i#j#e$mHHak?K} z28G#^wTqBJ!F_@i#VN`omA&7dE{(-5WBXT#dq^wJ`;IjK7_G%mx}CLY!nMXZzDqLd z{rWB*42=Ystgzy#%!JzqZ0DXkG;u~`{M8Wk(}OD1<|c*lhRI`X;lV1(z&(w#3j8? zLE}y46KPaPwXSWGbL(IXef0V)e-mPbgu9`lkm|_0!TxnoHB*DK)3+;m#TolQ9(R~~ z;b{V{%UW7K4mUE%bd zr5 z+c4O#6k9X3ghG&a{mXz;)2%0Bv@>?TaFZo&=gFW2oC3U6ik-ZTei>KVb+Tx$q7VRT zm^=NjY3-fH4;6*2@-IA87C!n8uB2|C=`t)UQm@R8fX`8v%S#KfGu?MhV~$SWkd|uF zBbr5dsXQCJWH%Qyej5##>XQ_|Zq{}p$8?4wH+?%n(J3M4cgt$eXC{^7w1AEd#0~ed%G2hm|H@ScWFm^<_xltxcl)wlWeF>?`C{5-Sb#h?U^~0CC=i z+*zaAtT%GU=p5AID7=J!23mG@Jb0>1CPV*^G61F4ecMK^j6>iK{ovW0!#-ul))iH9 z;DxQ5qU93nyrLHvt)FwSWh}gzhuX^SJ@%G)$=p2*H62x=xXs4Fte2gFlZu_JB#Yh~ z(S^>3a1|;Uv9&hsZB}#e#^~gg9YJ-8Y@FnSbq@tAv5n4|1!`Pl1R;BDDcj9fvUmQN zjZE$VY?!1Ix@+)lr;xAWx`VFRMn8?q3g-hFcldxI;Im=b$oYVxv^KRVt}S}!nM0-J zEZat58<~#%byIw{mI{~Ol~%KDntCT5jns)wbPb;bowA~`IE5zcFRxdoA(MHl=0LTi z72Tz<&2J=IyOu@?aUc;%UX2Wr>YOgNWjue+x@uHE1u@Vl9(iGa5ij!ZN>lcr> zYF*T&bFGBJDK&(H_@xAdQ$qb7pi}dU5ZMiA(v|V%55wIqlNFt&6PMvt_@=}OP42ud zqtfJ4dQNq{LC@+~c%zG5KV-%cwjICA-WoTSYB!NsnVO9zd@r*{eZ9+(-gm(R9Xp~w z8I`jk^K8yOG(JzQyK@Op1m#$FFUX@3MO;!imE_vGk;JlO^H1Hw;{-!FCK(8v-molq zpeT>t*0*+5oWP2YTXRr@I^~|ps{9W<5-M>cNwxd&K%Gc+I(P{@P%1HKO~U<(%Lu~j zE(w1QJPZA)T^wCK_hg*pc)HBqIWw872j_D^lp)iJ+jtL;&#);i!hM-Pp?T6}x8|vA z8_9w7U4jLB_LEU5@5)FA83sME_HrM@c!IbY}qlyX#sv zI5LJCxu9cPTAitRZV}>h(!)@tlXcyG+??5?m_i%pM(}Y2?gG-{fpnKrk)Gm|AYQab z=C;=?+^cMj`cfTtX~jV%IxCO(WiT#5@sGoCRjasK1c@}Jx`zteO}ZHGH z%E(g$3Rt}QEWRwbxYMvKd+QbPA$-K>EM!crc68e!Bwe_=dB0ELA|$paHahJy*fq&? zpY1{srHgXvdgHokGLyP)bNcp@)Rwxo(UlE(_c9!+%PIq?MQTOMW$p~+ZjStZh#rL( zzgEBZe(BpBy9znA)OJ$cGGMY?Y@J7RKSQpPGlYnsh?VsUXoAz~syL%BQvv9A9c71VAPE97FMHA{rX2ZCy2!zQL;p!#_N(m$4 zFhK<`rqcyBzUkh?_M*Ld`R|HkI(cgj)Lbuj3L=b@2=2O+!}Zt+%wA5n>J}l&P6>*I zncL9W8DSA~2(F%1t5dn6LGD6L{ui=XQ8F|FlY|eBZ z#Jin$U19VcVvj-!^P594N?K)mf}%_ox)>T3QEqG^AxuYX%PiySi4)unW?bB>3)v5~xmPD%o3`mG6~G4B3{x6y0BcV9KIIw$K$XH^<01m6!I# zD|$*?`t%@k&8w53+aH&3&x$TlL=t#AOC|Gw{#EC4ri*Mtjg^^W>Zeo6n|n$nAJ`yd z{>X}c50DDX3h~evDLc=y*`gP#uHBW?!AU5jT{COjA5}5b8ID;>mqvb)K@vN z^@D}20g8j=H_Xmo&vEmxmAos{ZAB*lT}km>MO9DjsIts!Z&d+`H>uJk)NOIDRly_Z zW0;z3tHozSm61FR6`}G*x8ccBHn!|-4o>pkBtdZJNLd2nbm0`zuUc14LNXIwzvrYI zP_)7}|J}>ACslNCO?R)$WU3W(;U({N;!d{{&cCf&0PCf&ZJB2+#|Vr9)iBO#D2Nv+ zzHRoKnNf0O)RXuZH!7-z-AdyJ!@ahjiWM2~ATl@F{Wou0HA(&N$m%rpAHCjiihgsie@RtGdC2N60Ht6z8oz+2*Iv zX;^x4Yi`jgxg0#I@ZGG*HnS?OZ5S1>fG{YNFys}yo+Q;vsFdjp0ux>7$u^}qzXSvn z^!drM&ipPz)$1oUi_I={0`<#2&zUR=Tm7hq@ic{P+rG7#f$6>U<-YvxJ-yYolJ~m# z7%MsAJ_nRQrLUt3CtqU=riv#`} zm)(8lS_f29*QTf)3|&b7%~YS9XqoPVSc~3kF1xKtw^wsE3cc}JXYpXfC~U zWa9=}ykMK0HKH)54f_3+29wV&!R^jt$lm4dNVOUk!`gf;cR9 zj7D^S+KM8gpFMnjG(C21V;pzk+?jKSJ&Xs}y?i7x@2ox1OnG=U)HJkn+&BM)J=AGI z3_tEZJ@Yo|qsG@3%V~Rux~L|-N8+Z*;il8$Q8Q~&-shHHksp}p?3u>LB_kKA8a{M< znpBkPS6qVZnRY_A<2MOUynFOv#A_-uTURz@t9MjAKUdYy=Zvf`sVuejb9caM-wldm`*+x@F|8~^S&zaL8`uNzV?HH0?-r&NkDWjfOoFdX#rDUCQf8QOUTMaK;vU(RGm$!eXv^>No zvp4f?zQ)G>3aW>1&z9~>ZolVGm4=4x(C7Qq@LkUv>Od~1K&2??TfI&9H1BPER)0$W4{L7NUC|@ z?tMFXFgOK${6$D|+un&gA0{=V`ZpLY zf4*%qx_njD&KTB~`IX;6O~Lgpvu#oU7rN zSlFjFnX@%A^oZB5Xtnimh;1eUr>DOji}@52_Z)oHrdyBhO?QUQsALU31E3sszAs*c zNKxQ}_K$~(9OG4=JTz!|uk`RzVcfwl{@cIIbg*aC4i7$SYLC!kEiqr}^(P-l<(yx` z;GG3V@of0yOW{f7frDOv!<|B>hL7m?&pc!|XMOFqo#_Ab(}X_9)^K>U^F&5a_a(2m zH%}fep<8PFlj)%0y`iPteh&2l5s!q>Kqfv7Kqmgh)9*_Mcy_kmd#?((?`4)I*))2a za`@W5rUl}cq*sv(93JzY4Bz!Wt|!o?e-UCOIydF~O!L(;Nta)%7Iim%(@`qzi{BmLbYcQSf=``HqMqt>^Aw#noE8Qc(g0uad1P+P)AT4BW(!NDAV*D zj28PpYd-cL7+m|n1ic9Hpf}ExT-e4T7ASe(~J>km3}} zH1ocN^Pa}X20t+kn#I5DekBapL`jqTx}E`qqwQ(3>df@>stb-T4?pjb}# z0xBKbncPJ-$Ic7&ntF|YTU|2!42U}x`)-Y5$eub zv}gGMuzO3Ov+-rMM(xkdZkE4pt6qeRg7IMQn}w;uOUBk7g#6cCoV>9KPF9J#vGrOm z^k)XA{i%qIk=Uw|>F#6H4{Gj$xX%H-m%n_y>^mN7dT#-7w%bvZuU*C6fLU7~6T7Yc zssO0&rdKSkj+H#PJ|=%==5im;+<-)ho3YJ-;y8Yd=Bir*L(0?LC9cLce%4I~^Mk&{ zX--2;n`-T^$t#8y3t^abjqhGOn+O)0io4r6&YxtzwoAd?n7L}&c;~kGKU5b!pXvLc zfmey%%B*b~c(QZPi9s9YL(w}-(W+D3{lHRx*tX8=Gp}h=cJFb;GKE1*Tm2%$^xbB~EwbZ-2hO z)_PuKeX|!V_PRWvZD%szk5yoQsuG( zTYgY&|IK1i%<0?XQ6^yb7L)!!$GECX?NXb{E<9rZophM*$Q5+5yMgGlxxrhmWT>5P z!??TD`uCY)4l4|`VHb_3rUNz%oO_8+PEaIsY`hwWfnm4kETxC}^-i1U*kxe1?JVs= zI*rM6*=!sT3|6`PX?|Cjg!?C8^Jz(N^SWMx1BU;%O^sGx<*!^6_9u)L0n3F=FSD7^ z(~25ZC7!v{PJEl)e1xv(}ph~H=Gr6x7w?eN>}iCM86b4X$a%4@5LLR|Fx z27kP$-B|D!l_c(-8&4ZWlz^c%u2w`YF#~KW0Y#ZxuWZO~7hSmoBLAs=X#81X7N@e@ zTu^gdqlm)J$Snp%3hBP9y+5ON&-PA+y+dJ(fZZk2bZeY&ZD|sB@3kHd7;>jnk-{Kn zXH=ws7lqf&O^>&W@{d+s>#h}1cu=z18`RhmC)`+?Bn2#XQ>97bz<>vs^cj`$Zs_em z;vrw>g!3ruren?g+dENtaUEO*uPv;8Fu1phA!jcV11iprPli>Zu)1e& zJS|dKy+mUETd=9`L?z3ifuU|8TFz{KYQ_kVbhHz2SQIlFbnOZF`v13?py|utpZ3gM z*u9Wi-QAm+ptv7~(fdESL2stkzR~tM}XBraN@C?#*;)$OVkribUK> zHrJ!kBP<`T&sGS=KIrW_;-zL@EaSC46}pd9`xf13lrDV8yC>P@%u|z$d6!H*>g#NT zCTV7_bs~l!n~QOX)xGHw7rbX-fh?N4a1FkupExCZo4sm zV8{_)8Wq%bv@%PNDs>~PjL4sAYjf$MUBR@~zQo7U$^uf-$wbyE1ydtom*8p*%Io3B z8@cXk+wILlD6iAOic6wzNBVD+6PhnF5i6;|ph^kc1y-A0w4BwFJ5=kDNIt;ByKYB? z(7|i9cdoqK@cPAud$lbEp|g4Nb(uG{J$EX?1f54j`NV`Z4){y$IB2@&<$CZ;3+GYQe5}Yf*w}z7~4w~u*=mm z$K~$j$z+QkP};ZOLTP&BfX7Lc96C6VC;8cJS0Mp}-w8+QBhjMA8875V5>oSewZz3_ zU%isBdqZ*K+VSZAtH%0G%9(W~4#4|g0D22`NM6Ju!-pO))*K+@M z_8$YoYt7b6h47>ZzEl?}w3A4y;VYpcO)!VazGEHu5*d%g9#-ddF)1=50kKu|462&; zP(jy#eXQHkll1XGsqy808zB2%Q3ejC#DQLQIF3B`1~beom-H=PuBseD?cGha8XuaaWHe!sCJ&g1nQyW5BF zYKOfZmfWx)xgm64e$LCNKtq)P{3idftwV?DMWGLWw+-ZI&*{-R!x!zwf+cAY4ki z`k7bL-CnPA-hYnxhR{0ENiTVe@4h_kCA#y)*g5@Z+t1#d>EKCrLnfvs;e52*!|0e> z+I`i-xfyr58|BpaOZr-J@_6nP3R1-HoPUd(zD;m&@$4HBJ4%G9T?0{}6h0gtkqq&k z9g};I$LLUZ(IeaKMP@X&=8e98;FNg$*r|XU0jf>@dM!`8=2Y(En=^;H&ok=UYl^JX zRN9Yco4Long&k$~N}>}#C_#fBTrBc7rqN*!3U{QpIov(p&RCzbIp4cJ%c*+fhgE^9 z7ZP^d8SWq^y?^Cac2Jgi_j<7Uw!0hlp?$|N8z^rvhbmJGb)?+xr8qzx^%p^`MqGgmfb zO1vrZPxjKulW{wOP-`iX9tn>uQVRX1Hd7g(>RrEf~7#7~zHAEM$_%o7k&p~*?L zsp^x-Uj!t&7a@gDw`f+&&$uQTpP%3xS!;zkPHDKyYmaIz@F;YB7W0*Dnf&6^Z5_$k zjVG=PgJ(hS*y_C5P5wwf5EXM&A}#-Oy!hTX6v5HAvGRwSOq(o+t7Cvuf>U=zX|hai zHL44ec|LdGL3i;=diyu=vA!*82>O+=o6m04?vL$&I}uOKQ;jbdTv&wg#1@>$J%t>U z9`Tgzn;G5Ynj){}m0aYetY{{wY%a|tg+^I`utn{ma-HvjkcAeYRn6iP0}hFTM7&|l~{P}*f7s#9xQM#sSzf5v(iX-Ua3oN zdmALc0w=^L?-4yFb*ezrSE|ItZd&oC26OG1ZCq&vB|+4dKYETj*k04zb+_~Qg)aD* zpu|06$VIO0LmI(}mU8!AU+6uJGR)$7G5T`d!N_#R*>oZ72a8v9P2Kb><{qrcA}CBd~09ZA z(sJR`=_51z@hAQee{Y)NRrJx&=;9+BqFI?m*Z74HP37xkh^E|A&<+JyuAeOc;5BIM&FUO zwd@pQZ8ufz>xSkhDNcKa{R6EV@6_zuEIci6bC-f^IHI`G%wAR};HdPikNnnli1*Ix za*cN}&+eCr-+!TI>m9cN_wFjYj{}W5s$5!!+>)*<-hEI_=&L@Icqk)ayTa?a*3r=z z`Ga8sn>^B=OFJ;=vCXSV>_2#cg?a`|4f?ZZvsW znzF6>TP8wmw_HU{KtpOlVMfVlK4wjo>%_WXWD+FbK0+hy&H1BmTD$e%v)%YMy8DWX z7LaKNhgZ$)>65z^+4I= z_Xk|lYSyE3pO$_$F1}LzZC1|e$k2ZC8zVwma&p}nei4LM4?IGuW$YTBdrpf*9K&>A zPCrUoPsr2Q%%+IlXvy(%T!_>%TVh3 z9G(-JHGAGaeRsaje098>6|(DMhxe|@YjSKV(?rkfH-Xf{Iy@^K*8gF^SMYo-7bhAiuFS1NZ+q^!; zB*e&lq+hQk=$+~j?V1p{SY}&z`n_Z8H53jaHqexHSdCvBHm|*!ZLS_6Ad%>h6((5R z480_lb~uf%bVH8T(5(a$ zH~zNMlKyq0qLc{SYQFDetiMc#2cfVVA8fq!I*I$2o!Wo0%{%mcx26<^+$jz=PD}eZ zYjFPURf)1L0>|Og6HmY{4{RkrqxKpVY)zK4o_{xE^rc9lUFspq8u-f$R)Mw$gxkP= zY-uAY8y7ymVJBYH51fc8q(Ai9Tgi|!`;|nFW5%!ByxqU;*3y^vacFRmy17s4m)C%`CXI+WlHp{%T zmc7>;c%|av4+$5QZx0)snH-1BkQke$`wu@U9YjSOhPE+|5r89%st2?F9qk3 z2Lcy_I~?x@CGPTTJrxb&<7#C~j!^I(jmUd{RDV1EQXLmFyZ2RfPXBo7jgQ~-vhNhk zmfdQ6@15Qzz3tc&w*JUipTsAt&j+Mi*WgQ}Z$00|CGpNbBZ#lKb~Zz&&gT%Sopv0Y zMeONuSDU{+_*{Oy*DPY)p_;}bmj`JF>Z$NCMU{O)N+AV958&eF5>>YvBwue0&7<_3 zVR!1ovS}@-bsl#n6#^ow)LO_Ll0AHShosZ<#hO(e`hZ>a#3lLg>~b&velq_Nr%ZwK zE*c_x*IL&$nU-%;8;aE=d#Yv1Y1rR!uNE3`%A3yImKB^I;1hm#2BFo`aa{O-K?64O zT^SZC@D^GEKjL}|dcGhr0DXv>SJe^nHT~i2*w(C-0g?gjx_xWDJhJ>KkzLs4VCK!# z)K}b`B<6p_8HK;HnZ>(nKt5o{zTlvS5RvGo&BfkO)w}CrbQKEm$*0bvSZ+$Jqe)^} zv$}8d)GbbIzzkyPiF$@6+qS;tzFT`x(JV!UzBhzNb>@lO&0v{NyPv#%!ZoZHJ#~U^ z6Owv5$}KbvwQ0jBe@?aJSxKv-ms*?*HFh17G9%0Rvg|Hw9!;)reQ=S7>;8&&-Z1A= zgG8g4t36Uqe_WZ$$+0=_S;m$+W}#!-huI7_VQSx&7V&?}V4X4U=uybgHO4wr^IzM4 z{kgPq5fpPoS7YbO6!|y&r@go=*VSroe!$u+2G_;8!SGL4ai<3Qd|{on`zo!RvT@*9 znaTb4{TurHJe9_wQD-x=ozEW@ep#WtE5%m(c<|%x^!6j4Pn|Wsta**)BQyV^AWtXL zF7jQDv_itA>Lvqqi}$S?3}Mg0UaeQ$*3$P}s^vW2({}e)S>778`#z`BqXi0%@`@bC zy6!-30-GUw*BeTAhaGJ^M~|8}%&o6TDe%Om9kB4Xac;j65TQ9@F2ECM4=q%eIjoL8 zv##_>wwH%HS0b9N>FFiiWwqf>?6i^6iq14+OH&;cJAbmymYkKT$pY(hb?Uclf5}Tv zOjM!gx*^SYhoGqc#ok*+*SYM7f=Q-N4Bd|#F(hXE z@+Ei7I`flF5O}iLDuMg&3mVcx6*+{ZYl`NZ z6f+;rM}(p9U2Jo(Ud91!{{heOa28fY6&@AqeCeXbDYl89?iHrGywa zWUu>hQ%z5u!NkT-itoy9Z&CQ$WORYcZM==A(N~m?jp*MX14XD$QLL}w+>3t4+X_Cu zY6Fu}evuMQ0d-ziDa?|04tETqIw^Uzk*JM@G0?^>(DeqTag^LY&_{?zh$Kjf53sEr zVQNNVJ_zpUuNgxit-9Seltfr5n~yX6baHXWQ}vU}l)5?ku^gAgYuGZuhX!&Ikcgg_ zs(@wCwnY$27gu5@kAkDFMH0?08$XcBK{^GePTVvt<%ke*Rpc-SbkG{x_8m;)=i%fFI};kpP77RYY`p%GCY^y1 zOCAtz__g_~A+ilG_5|p+Sc_Ho&bb#llTuYTW%Qi;j!3wH*x5g z2jAcvm1oa{6Qslw%R?_yS_zmiG-d`|`|MRtsxU_C9x`n-?Yv9j3X31hcfTfeByHu; z_M1;(PjK4#IjUzh_c>ugAd16+ULx(<-HN*b{8U)hb#fS`9ZHi>#Mkj}jYZ`_ zx#h5dq-Cwc7UmK83}LfL$;$^7DCg2Ttyx@^)?EGIY@?S?B(DBo=V%=oemvq-!2-D; z?>diOtIs1oYG+H-+5<~k0e&QVCM+UeRIlAZ)#;{EE zJhIu+ef{_ugC=yRBO0Ih+JiZLmvRyCNp3{_>{kZ9A#2-MR z`!bNWf*R>UzDi&K(Vga{S*2cQ9=G#SwOJg3!Fd(Q6RlX7nj0fVfpouHlETH29ZDnL zQVNG{hDJCF>PLlPG*c-@(CnnI@dLYMfd<}iG!9R_7H#9lm*!nzIQ8^wopoffPXH3d zT)5Nw-T}$kv(Q<{e)vkmjO3j!5$J|v&$!t)v66KafY)deqyz&ca@H2fwK$2i<(&vd zeYvomB9`Pfyf7RQn^m`gzJ8d!wfT>iz z-48&LM#PQ8U5iloG@7=LL!=fCn<_T5E0x{GJh@{&hKDQ6#*j_( z-N%Ed&<;e_QVhpJ#Cxq9jF@U+SdOZ_+J?k?IG%elu0&m6bUxwcR!`5k+CdMT&m!=Pav2by+-sudamiQgi0#7q0t8MTO|rQFk^iXYGmg6 z_8+M6B98+<4!GbF%M#OrxPL+8AQoNvm0Op2}d_Sc$m_r+P55j~s@oF!CnO&?ZL_Es#Q^V?~l zJ`9gc);=Z+1PD`nQH}!3`|7N;jOZFxbK zv~j8^rhHvR4QJ#hPmvjwIOfRL1|%&MUY3DSJNpj z11PC@MipwIRj*u9On%AR53^vqdP0U$-_s)06e{BvZEWLKiX4^(a8nNtNI!>ovfgP* zl`d@$R(TGUVrmihgHw_QVsfd4*SI%?){cQHH-i%hk29svmO{uuPqkvX>`09n^1 z58G?H%Q?$Zzm3A`dQIgf(r3%}O})j!FDN>P&IXEIp86?=c5#qn0n-4;VU00%hxWoT zYG93FojbxAV^ujq8KK|Xdp~k?Ap11^yGw}ZKe&}xffvdD@jLyQw2A+kvL*lBrS!Mc zF8+_RN%F6~)c^QK<-gpmLB6jm{uWW{KX1ek|JqMUWyurt1KoIk?X>>odHfH4vHzYC zbSLHz&syl7L7%Zr0Zn8GGBE1V7XQBk^}leSRyOJeutkZPF*ZVzWQc0}K{)N(49V9( z7l;ndm1s|;4GAIATZx9xg-%HK)`%l?H}}wd!BeIoc;Bf%p|o-}f%hffqYuFVUtwrc zjw@G8Y+~=Gl5Ccn03rGTZiP%IEVM{E3e}b_3E46LVsFGi`==Af@@4WxOryMC5B=j@ z{@*uo#35>V?#i$UZv16!k$&ksk6n(REH;C`a{%vs(0~+jd^dgy#@bwq=JwSMs8!r2!SCZRvs#HQ?5WK3 zHTbgt%6tfv3GiUL85*$v(gjS;!P(5p`N_>vw+1i|Fw(#nWZ#p>bFgy|^+3zl4KgxJ zEF|s=mxd0)(fr0dfD6UZaB4k`*^@h~z&kt1*c;8+jpH~FDUM@VWDso_jK#BN%xK(k z?!9r}aqhc14UAj+$qgJb(|uXWr%p6~y7S^@FzCmw%1*Zclul)$0UbCl zSNC4EH1ilH&*vz%A0OckVJG)Jcbo5ruk!8uX6 z@G|&!)7G;lfGf!aUReDZ1G2c^5OtHMpoCA>`UZ_rx$UN72Y1{7g6N7ndeLqC?iH9c z?pnqCcoM`9jto-H1ow}OSh!z+Pw)g>@2$EIb)J?xuB2`dmerq88Yqd~Z`0OL^dz`o z!{DhX)ne2~Qz0!MQjO|-!xRnXC`Ue&RF0gj8k=PPvBt`)w9(Ufw!im`DsF1_#iDjr zM6JpFkY%p$u3*3bT(9%6ymT&x(~0hsC%oy9el8VYJJ&o~>ffMY-|7Vv{=H@M{o*^M z42JfXCV!R#8yqOMM}p;=(i?M>q&F^(4~E-w(YsYqY72V_cqgBb4^ymzenBZ;hS zxnf80S5mL&t^}8*#rF=w_BXw1mlQ<4SOYuTmB6&SeXCVoUykoPw(u)SYh+iP%fsr= zwyJeH{anut;9G`S{W305Re3JN*X1O3vdG!Slg%eJ=b*K#oJCInJuxxM+UZ*4LRS&4lwmxSItmiajgx zTtHbu6k4Sh75{V+`R$-5;RaZ5J15TLpS{KFWZYj5-n)l8ntvprVs?J?8ywf`^lZs{ zX9G$u@zCuGeuQ=YbM)U(ksthz!1NVJqJh_vO=p4k*Xmul!vJi5^)i{`YekMLJ9XX% z=EHk2jS`+rE>sINhN>(z|`J>*(A8ExmEbw_lcXe~*ao4-N4^D!En+0m`Tjv7gVp5nc6%=9 z?daE=duEc)^X!e&GZcR68qHgg6&QLY(>w2%FL)n%a=B6%RafgT5eZ1ZE2UiO&zB-5 z5}lXS2J+yrN6&P223M)klc03>8}H*z+LkBGAm~o>j#pyMb>n#!_#F;|J^YbOr@!ah zzCpIT`QzC4u5@nPvb?XvSGlTutQ@!ykwd-zWZpSeeYgL+yhtbc~9%8I7-ft{;6r0 z{jDd;JH3398dqf?XimBlP|MduC8a^QVuNLDdxm=_ z&l+FzGR;3#AKKq;BD||=%8_S@iwiLO6?0q(ESC*bre}mYn7|Wwu zCDLigxx3pf4FC8Gw64fqYBX;%^PB;&>6mbDEVEI-!BnEu2BbZ*pX&hcA9bCi+r;4H z&eK0tqdiOWPa#%h+uV3!y8NGwY*k9l=75Ej&m6abzm$H{_4?t zxP?)Yui9wU?)vxQh4IrlCY2!V!U#3y(NhbnE8e{Equ(GeM>fYjNdsZ-^Ri`cVkAJmkS>H1#dFx&iDAo2e6ZjT=-Sqr{ z&?#=E^c1+{G2oEfP`yLgdX=8NW_m$$&+1x&dQOa?5|w^GUgACe_((2k$;b*L1jhbtye!;$kh4J?3&s1d2+1G3X zr15@yFJyk;zPxm&r+)Em+qjy`Xv8x;QPv{bG^Wt3ARodEiWx9dc_AjxjOeEg{O*vh zafdK}V(NF?&fKi;NkohhfSTVkVOkXR5Ey%YbUohGV*qOD(cb-&-QOA6E!^hQK1VFY z|1rAS2Z<$j`!_I)dp1nV(wdOe4Z%i+D$C3F8l>=iLw-cz`ryRWPxT@)wSN`z^)`r3 zDLSVXT*C5!nx21!msqhThjIkUh0%X|%FnHz(O$m8^fd%HWInRmURVjF{03psV(cBW zC(@1T=qMhPN#Wa2JHY7~Y%x)=%L9yL@do zhvvVpa{U6cQhQ@?|PQ?{O8dBu4ff^Bde?YFfT&j+4r z)eTAMYqkT)6IFK~4X8J-)p{QC_M2kqcimH5AC|f0ugclFvx+4>_+!7--8Ew|C z6?}bAfL&LVID4AcJr|Q`ev4iGc1#X7LLYz0M|EN!(eMR=ZPCUnh8iyQwJT3B8pR^2vJV@9hYf?l_lX)}gXjy~Ufx~8rmNnn`k z{w1=ng#0&%GIV$JmeC98F;UdX^eL4O%4dX_i}_WSAdue_886MR2vL}jIW$_fO9k>f zVw$J~xj|{bRLEG~{|k&}K&Mv6@){(_5ep!{H>HV|{@?h8L?}rpV$U2}-Um4o^3E=F z8}v}*mn~4ie;_2Gh=qvn{3VbGrT-vD6Y(eV2X6rN6ZEC@g>N4r=KS)c_=M}3DC+p( z=KBr%HRb;$HYy2_f9pcI+kNxp5B)z@Rr>GP5S@#-L$*thkU4xFFHXP6AQx!qQVCLx zQp{hVKuv~{j>Mio9ut-D!t@LN6iN{P=Q1&Q(Hv!lDaf&sGOmxlqE$MOX=>C;L#Rh& zUQ4_qWYSz8`ThXKkcW`}f)W1%QbPadmo&K2Kc$MqTrC8-3V?u_dFe!FCqBU(9mqmp z;<=3C+mF5$1j(V%b~$sef#NCd!L8i&-k|_*^CPow9a(m>7P1&qjnv}BaX zc*BG>s6f(qV7|2a5QZ<{yHZH7AcC;&#SS3w-*D|}8ug#Q!UL@BHl7>yb$uK!d<9!p zFa3L?jE=UrfKes?6DKK~d^jnCHrbmadW28wi+lQ=>Ao~_9Km}pQIo5}UHV<8ftJl} zhR^zEz&sXiO&YO}8jm?=8gp5C^G{i&cp)QQW4}S1(%VY^Tfd#2RfM+9ATNG{%xU`f z{W`ue_;q1@t!4UXopW_=aPWM1VttdgDNt+o8{|l_YWCIdTh;ZiKhgYT5w^!P#-wp~ zRwXQwgc(}AG_fX~EZl8xwkCikhL6{B%h%ku-&%w(6M+%k=mSoA$>qKV>dkubCoCfX z*EaCpg$4&ccjIv_J3_JE<;uFn?{B&8 z??2yH=|OvFv{|PwoMha!pWZV2wGy|}?Y&~Zo>yYtAPoElv7qaUJg;|J^q4$oAyg4~ zl04r2?(gF!Lou%@hHS?lUw;id$uID*aX@K%snV~1@6p&$NJzIiiOsdicb-o&NHu() zS*Z{zd>{#Mb=A-*VOoh?EJNyXUu=iz8QL3K2;vSto;T#rhH?N zWq3LmAvhJ|-!AjZz$fa$OpJL&=4f8$XJ|9b9KUnU>kIO+G> zKJp3t8w~)IozUj+71Q<2&i^*)8c~qbfqlI`h1{XNzjMc$qBo z;ht0r=Q(-(N^gfRG^!loMEr;_u=$YX#yYJ_z_(gpf3@ivnD?!M;gfinqmoX$7KBje z!7TS6!$P=rbJkMFf&jGRjQJ1U)5p70=eoxAu8fRVL!F~~*T~TIZT^Y#mYWI3=i7XY zN|L41B>`vwshkgPY1dmTgXZ>aT^t;T^VLn#&9@x5q1P-xzRfk80DO1{o?7pg7JfhV z7x2}XgZu{iiB0z#@>lLUjl&hRl&IlWX<$HqzV@s?&7l^kp8K6ycMHEA5c!wbb=!)> z0eFnC!7L5*B;r7vrf&Sfw4GO8^7C&H?j@Kb0SB$ddY5*<1I<&t{7{Ib*zKBI57?&}pX zdgvz`)ASG5Unx*LGC32u93O=qQ?9M+PJVH&kMKr}JgT&F#a{7jmFP*nD6uUy8f-MTX9U zqdn|rQm(-d%kVd$w%jSNC`P>BOn9RotWvaX0S7XV-)?Tf19|N(J<;)t{-VgAoOn`x z``p22@RwNHk23yaA#t0~crWLL4EwONbL>dbm>%t6SIK|rfl|1bbyMht?B$guPQc;O z9!@Z85aao-pn*@!lW+&rv#Is+#U)S^^nY04UkMz-+>o3m!3UBAZ7>k~6#l6@uym8; zEw)OSe_5rZciO|yT0h7l5}e{6Ub$0 zoGlgmqn?ufwGmJW3BM$-bbmo}+hXO2sCRSZb?o;8W&Q5n@7A?%`TC)wHMj0h$-h*#wF=y9r*P z-mXIC$XmdYdL9uk-;;OEvl(e`Njv{A>wg!_AQ7{DsRjQcZs+5svdk~|(|Pzzkv|QU z+x|zgd{3BFo$Gg!O+{^qcM^!-Ab%I_puE!{oZfn6>H<3!Mn{L|_Z^YWt@8{=?$dYQG9p3#xW)DBz5xTgxHs!sQ)MML|!|mTQL4c|3)iUi-*aJ3p#JR;|#)250aNHqjN}4 z%WuoY>M@bOK@dOw7X{|nZv^1JO|IiHu`i@;W4fpQV@O^zF}A80W4sg(9%y&#o9z4| z7fHNM&bMF$_4%u6UiCia4<4wi?z&5lh&>X~@_xAi3yI!V!OLIF{6oGP*sePp?+QSR zUvtR#4-wrK60B52xH;x*{ZzFt&p&@Jk{FawFwT81DW6{+c&D-Xi-v%k+Z(^@o!R%M z&sze$KeCrwffn{dDcq0F9<+a_D6KZf)>!5EQ@3LK`r^{*TYd4WYh~pG%L^}*d;idV z%@tu_=}ucy&1#3GLgNvx8v^PJFO0X@-!|D=T{_)(Ct-&>D{i!Fc}M^5IoeS5YPWt& z?c1cPQ}@SKQ=6E)?+U739&Q3DYJn7RaF9U;)@#>VK#6$0-0yhYD@yP7HMQ3L%|q;9l3>90Hc5Z2sJ~=QHm!cHM%Sw;K>TUuCuCtTp6Q zb{;N?!GHbcrmXx}#kTuc4G(e7FyZmyr@GVR&V_4dXXk1iEe@d^F;(3bf2=>nYFVVV zRfWwfeNoBwi#W92oXPLkj6e)b=?0@`aEE~znJOe z$`#~@0m0MyN@waTEj&l??q3UWU?RTL@V{2#z-%wTmAHTGBfrezoK@A_xDZ~*@Rfd& z-2e8kQ=6*L_P!K&*AX1ywu$Q#*NyvckRWw|Q9o@fY*oviL-5cn?f9WM&&qo1wpH?)zzR`K}C=2V6Pk2L(n{k@NaUO{WR zR$%RScuw&eHdr3dB<3dH;<+D63^wuOhi}!9MfC)JZ2*_Hn~y*ZqsyNp#?K!Q-Z0;c zp1N;hfet)>;5O!=y!iH&%=fcK#2s%=zX|*$l_-uOZW0g1inFfi6Z6d^krUZ3H>%8q zJ999pHFb;=(}&VPEu)(4s}BPmuG?+(*(AIu0}r^`EtTMTX>$QW@&xqgn=R`)Gb4wM z9LbAF8||!(90PY9-EUv5@j#=iyoC-N)I6Ou+mo>#wy;tq42+!A6JW^}_{c2v`yQ4Y z3(!jPGI8`!3-v>LRR};oqQb)W0ppyn7_1kzvT;)b@6K&v+bCE=ST8<{3H3W{Xl>X% zrz~&wlVaMiI+I$b-o<~&PydO>NVLsNR7HgXK2bi+){v*ZyT4n?46(lp9j)2owHNY% zTU0Wh;zKw(jcV?XWG&M`5v=~t!HXTIRaf@f|Y4`{X0NAL%K`nO%&Bj0<~y^!(!bybG#g^3@2;fy(ZJ;ww!T znMor>N=pt;|N0e8=-O&1_jFvAjy%9CIE%FspjJo~Wi2+bzUgQGno5*XT>~lObKJFKy07W5mFyyabMo^mR*nmRcj$VBHK07*RU0B^k z*6DF)IVhrUP{g@EH3H2hGgSA#dO?Vo?)Ob*r=Z7UgUdeY>d8*Q*%^lwC*XwAi%BTN z1<@^!_%gM={G<}}KpKC+ot*|zA!4{pZki-~B~j30kP zqOTH&VdKK2FF}n@%|SpKhl#AlOV4}8X=G_Q^zlg6P&Ai{en2*Kp>0`G3F?^&#rrB5 z5_PzWU+O!`30sq$PPv>ofUVFnizdja?Cxh>8+(U`7zHrChS}2X%aewXFY8eex3#vh zG%d{apAo06_Dcsi;{2ERajwm_^iLbDKgb9$s@;-wWuklB=b-%t*-pBq>woUqyt;g? z*3|FB>l|W;Js0S<8~4`f+7?Lkt9^j9?D9k$<~T|$JI6~IR0bxfyiFQn*2~VWvxpH( zH5NZzCKM}W(}};OEBv)lLcWY)Tf#dT2^)#bGt|OQFbMjkzlNvJGwBnYTBN#ZQoqYQpZblj@ZGs5a)fA`@0ECV7{(-Wi6xY-gj#$#Ib`;5z?97&07vw`d zmI{kn@MVY?du|;8Pk?;>tS@hWF;^KT_3#p?tkX+TX|WPlD=0Ri90{-@K1aaU5 zM4#@cHPzN3z_^N$QEC);od6cNw#s)jM5MF)FabIa2{##N0*|RpH|xl18&?@Ad7(Ri z>3R0Cxn4OFB8j-Z+uH*(kA&TP&Ae!=EI|>6yWb|gUhmWA?lJ5Xw&vc0S*k>D8Y#w~ z-{oROkvzfDmTwmf37}N)HnOZWr5(l5VTKefLK&D122r!eQ8)*)zy;AD8UV}}l`^)| ze0@yPq(M=su~P?%@pwAYis^F>szyxGnKvTlL%eFLfn3E>s7)Rd7JZx~G(Q~8>>DN0 zJ|7O)`MN}OpC;L&M>LD+Af=I+YJSzk)zia6rINO=OTMFZhqrx^cV!*=VCg`rh{RGT z=OaYAj)`2S-mLrcuA#fK)Y36`UP!i|k@OyOGUJKF`3@^$@1P#rU1qsQ;tMmW0*=|3 z-dE+T{k%gOeobX~r3bx0 zqh{zXiox38S93UMZ!j+}`o~iGqihVmm2Kx8bZ_mWmgcEBwepG_9f7!) z{Hf>WE>D!&um$+5S7Vkp5(cZV3ghTJfw!rDwKzCYo#%{43xB@fcu zW=Ek9s&45`Iv)22JK%+^oKBi^urC&z|IqsTIdtmc06#YK;{JHEqe zPZmGPWs$GP%XN-6orAphp^09qSj$&x#xlxCMd_2>7Ub8%TRv4ThK{Zm-pq zr7|^yJn6oAj*%^XyYWQAI=@A+s-q~Z=N-W1AiTETut>UO!BZ5I+yn~>4L({A&B4yG z4Q=HE4V5$|7>o*atyp2Y;xLEP)_(F&{^>=eobKl62gom%NUgCeSoeu;?j#o#)GCq#D>5YV5p}oX1ujkfi|gJynrzF?d^Q16Gd6*ylPV^T>qr%h+rP_)1G%@C*es zu1=}GsBP)BCZ3?ORTALFgwgV-9MT6RylcjvXe;>8S~(gr;O$hc&j z2r>)mC=!guY(4o}dPutol}2QtaJi01D(#16T2^COr(o_n9=F<~HQ2sZ+2zVep&=A; z8uv%Db~}u>nKgw?(nk$2gZ9eFK(h{lu=IX()H;)yGA}Dhb{^7p_aEctu^D=3Ls4KU zSPFW42p-^^p!qs>smnTS({j2(|CO9q3K%gb|7%J$qrD7`rJ+*M0t!72qbblHOAjjv zkRXx?a_u2Wp0%Qaqk%);+e9}@?%I}vF%6*ScasCpD~6*f%d7uEu;-XRDXN}}H161o z=1y&yT*cU7{=YKt`R#<9ijbo7@Nlq;=iQbp$*qDqtK_gN?Oj>uSIgG;dVsXtchv(QEBN0eyF@_GTgX!6W(j>J-SP%kBNNpqB)x6wo zVd~%@YgUa#nJ-|BTOt`f8-`z=uUciQ1cUh^c^)YNr5k&8F(_bLc$r~IFvyyYGG*a` z09<682laEGXD-Hei2xr{!_=IBjCDa8%==ZATz!O`gJd~MDURTsCKY8k|B?(J>f{)% zzc7lofnE>e8`A(z;P{a=fa>Gtd{yEwB6cLYy%l}F@z@DpoMt>eoVe~jZH)+TDh)urDl6HD}6`LNfuZGlYK3{;216nfyMEVmOYOH|#|&R_>Ej8bvGND#n!~ ze~XyGWTm;x@)DDkv$CxaT3fI7yGy~hTUqg8PczvEVOVT3^U0OM$Of3Eop&2TK={J3Xp{m8veE91ZbVZNH4 z;FPXq*hDc3oKCVPj_H=xH`D2wB3-@d=jkTigkJSlV3HzhT)9Y`Bz==?mmqF9o-z!$nApn+ zTGbz_Q9QV?J(&i$mX3VVq+$3&IMa@a+Pxq`{R@S@eE`UFI^+7i3=8PJDo3Zzo(&#K z0^pM%B|d=pup0wq2p236j|INbR)=Zwm{ixM5~}_5q$BKjwwb0>h!R(E((GhC54p<1&VI2;Klwqy$Kd<}jw_Q@X;)m5p(tg2~tn1bAg(pFtcm+DXg+lO|nZkl1{SW;tlWs$CXBqk=|D zm9-EkYKf}hV0Q>He}4)O7ad4cRn)|%i60iEggl`K(Yn^UdR`=K@H^QkM&C?cS@+@m zv%a^a`{T_Ku#L!DUGN%F6f>h%08_~vx|HFgct?}>{ZcRD8dA4XMC~bw{oYy!2gNYF zNHVkibHAkk2011xiD^sE+-~`{rozR}xHr^+-va)oOcJda0sp9k^+w_dK`VpxFvM{{ zan?=IeBMWgL+V1_HE>r8xtZCBkGHqr8m|;jg*aAl=jx$8nOoK9S0veb7&M4x1LZhp zy&1fg!-hE4K{F=?vZ6R~Q9y7u8^meW%pjUwRFB~H;zyS(qIhaSqOBfY%?i0P`WW4) z!V?kj39D=I)ay=;3=>azLbXq`^4bI%c7djpk)F|-hma|}=(uJv=_e+3!fA%&brtTw z)LyM5-Hu7dT!t++RZNH8x*+@xh2m-iM`L#iO<}bu19zAh^F{SU>LD6uYxJ?j;#$(m(==g;xf3QO0peQ1 z;Qjqb03ZF1%O3{=7B5s=L=J$nrM&dZxCt6$fH)UKLU+u^L%$d=stfB)Sv&LDJ5Nh) zjl3}$BI~$Bq9IeJdS_rj_3lL~d1RVQ3bR1XNCF_e-JG*_g4rCTR2hs~dMEnDA)=IR z?b~>*MBv0Fvhvx%2fbt4P2>%`U_%{soP9BoP`E)}SMadv>@*h@_(6Fs6~zaa7G7n- zS@RFmFT3;0h~$Ix(?!_)4C8~+*rGO8c0HPE{n`1a=Y`E{g3gTlvBgXVtb9V^SNvL| z_2NgIKXe%hVo6$lY6Zip57wtOShT`%=7xcAh1kWU#l*bbsTIU5+od?th!5|H zc5cIIWZ8gyk0R0r)8?rg-&}+}o~vTe3ERfWG1(lkf3@F@y%<6~iMZ7zJXixsII0D{ z5}@E1HxBYamy=}(0X1C8$e^H3)G<`^0A)vkC-A2v>BR@Z8aMW-LuDJ(N;6I!dg8f# z_5R0RVXPD+Gg*n0TC9~w?>U=%3Z|e4)PB|8oy*8qUBH4cA`056Mu#MuVeAV&_{nb9Y;yZxN zK}9j1u009G$lAyQHD_a>;HkCZ$TGVYu6mV6J0dP??Hwel2>mfFPpKx%8mFm1tW75t zxpBR^6n~b~!!CtGoj5P)+=Ho~ye1xNoxEfNOVGix)RNQq3ov|gyH4Hym(rYkmeLWv zY&qwzJb&()OvRVNBt`=MN&1rhV3YNhS-Efdi68o9M9M6cqNtKdxY27Dr^!R!XEVU| z6$v-{X$Y)tip-}Glpk_~l{MeND78tsHK{0Tmx#f}DyT|!lhxb_k?E1aj4_VVw^t|i z%io+0q*E;q9Nnn~T3P{>*-=TQ%?6sst58yr z5u1HWSGI&!I-$Zc^k{~16BUmfNTWgJ5y&#lxw)tLBT4yKD^D`i=m)ZR$=KoL+X)ff9*&EFrv}d+~cpFd={3Q`3Di#`~Bp0&cn4B`LwW@+LKK zzC$G7J5Ts9f508bv04b`nmso88bVe-fpz_1jCm-3q&L#e!Btn-v2-1N3`qv-Q?i5G zr#FWrz`T$)MnV#0{iwGD3&7^37j!*er37y=_&|G`r#z#zXpmcPxb`-p8?$E4I^Kqa>Cl>(!o2dOjYY9cw^k6m-kPUUV|_3hO< z)IytmZ2Srf1ONzEoGhlio8YtfTHkm^V^G0INmh_WoNV-A>XXbAA<_4j5|q+|EG?C5 zPm)xDp_=My*oFFy$Rlicc{G&3hduys$pg}C3F##z;lxL3uyQgT=0O&_`D%`Nz&xX3 z;+ASj4GBz(zXpNNf1Vxv%=qf9lYjAkvl!ZBWJ`a9GD6%L^DK(w8TpOax{7{hCnn$s ze9653HwXe-1Wn*R@TG(Qfs{58-?cw6L@&mk?{P}-H@R08{sxgD0X|R4lW@f$U<>o= z0?dKURkVHTf9l=!lC`o?DZWVobUOHoRQdW9T2b!WUim*LcihY=4Xpw*P+t^!AC~0D zH35~{lD?l+k|429_zlvNbY7I#!dDi1=`0(*p$v8Z5Qlzz6Y%R8_z0zqKb`ZNzSsl& z&DG!`U8jHvFy;B&cY|W?S4e|{I68nku8}gX&<}6h5)R7dYuiX*RJtLgw*Mo zR|%8FWbsG%y;?6S%eKSLhI}=~j0)K3GHOL(FxzCTzCPQiCgR!etS4cRw%0uCkC$9#P6T> zftjznUR(`k`0f2C1unX-F&7XELY{Gdg8+RR{dz(mV4r_z(zP;$62Ujw+D^mVLD{x> zuPm&096w3*qTdF7l!Gw!tGEe?9@=l4x(A+d}5g-w|tAhJj+j@cfrlg%BqE6Zip z$z=3+SDKl*J-xBm(e@Kml&|Luckv=?=JjNK$z82ZD#R^WpZif^hdA!_I%e+fzUqQa$VZ18I;>6TcO*%M#DewOwTgiP#3g^?|a z(-_tf231>;*{~8959Q+L7umc0a6X8^IP;{k6m?${J#9A4OcfQM)PK zv(sKn_`=ccOr4?Ek63H7QiUKbSDeV&%8}Ngg%(cw{Ff&fXQzmE?uGle>r-o*oo`T$ z6K&1m;_4I2XUM%^_SO-un2@p$DcqtO;ydKRoL)I53{N@lyhbEEMNVSN#$Rz|{(g$K9JK24|)z*gA~nN^m1 z>3HW}w0qKV_ZCr)+qq!vdo-~U!Bwz2BsR19gEmM$1D#cvM z7;QqUMwd3;K$X>Sh@-{MLpL4=pNOSGxu>E-N>3$Pf0`rdngV%AGb!LgmA z^JV4LLbbVt&O1md?n!gunmo@ z2+z61lP55)|IA~R_{(l27p0q7wDEW2J~-+;G3FKsUI5SGxjBWCcxCs;&#;MDagR-U z1zqlzI;K{4t%KH}B7%n2%QQmHw+UtWinMUx1?I8ySA6T&?#*e}POsPPTX7lZ<@$V+ z&V|-n=_?X!u_q#MUh*N(vITsVxfXrNX66KuC&c=Up}f0zNQ03@@YpGmH3x|o6$i;z z_(hJ9!)Y~Vd>J(|Q3(UYj{E3M=E+!=;sfY_7D$-x1G2R?r%sKLhyy*olyU-ypG?Vhft&?a>m8CNns=xVqEwr~zeS;CgW& z3~Z}0)zv(Hx>4&aryO$q|BJh~2#d3ewsi{&PJp0A;S!*5*93Qm;O=&db@9L+rn58{Rt(#(<}~-se$`VxK9EPhYEjK( zlF^)LnGTWAlXh=k5pKTC-&$@)Htv8MVK37*zT<1l^>a#&}# zQO_mW{@(7Qa5d9v{%6`8DTtARvvOSO2Wb&Pxw%H0XxnElcr!^;{m!3A=;~e|ys~dB$QxI{EM-?l7yYaPYl@ds^QN1K|w5-tqc-4emuOZS;M4{;sD8{x*3% z+FZ-g^+Vn^Ptp9i6F|-#PyV*hw+5r6jyxi9M;^T4u9Al2tJ;Jy+GyEt@%fl5vxm_U zNs4h6Nuul-*>WV}L8zvpBZ-xfS zb(HY4o$>R~XPmn~MCZ$m?f+<>WzX1yy|SMY{v-#LE7aVK26!^RQ4{qHztOLroJ~bN zlt%D4?Ipft$0+ErD+?LNo&-6@m8=C!Jd3mn;wr@?A7Z(WV3mIOIimL^;bk>JLL-c= z-@U3+loc~9j@+?cKE;> zwI$Y#$3QTcH?iQ6DN?qnzI63goMUP&QU0_Re8^atY_ZAuSu?=e=_zD}|ip1+xenHIXA!MvY$SHqn; zQx8)r{{xi6Y<%{{_P8YuMmf9}wlDAeL&v%qYjnh!h_xt5*7)(_GgaS-co_NAT&%G$ zUq$({_dLTOl`)k3An;&TH@vYNO6UHsc{_a4XRcb!Bk`D@W_zfX=%DFlmkenU$2xgz zOYl0wVUzSq6LeAb&*|FT5B9u_n*`;61smp(GLe-ZHV{Wzg|HI5MADB#+~1Y&Egp@# z<2j+?QtbM0^gxfBlC|`j<+H#OonVZ;tQfGGUb}|hcBCkCpSpu`3RMNiX$NsFz@D9k zSNZ5?6T`7ml>Fj=Qhb5Xr}6=eX_7COMABEL4?A)GQTdfTG9!0%z{M2W_iIb@0owYqvEoy?e z5Bf^me|X9++{evR9i*8z8txEO*Mg4sO|1yo(_(QGi{u@+6#L5MQlQ3=I{X~lK??su zY*|r`00{)?S70M`VxbM3T;NI{by)^XA`_jtxGt!WXk=(t_*E@3B;1x{&2iXtD)sW- z515UA6tXnh-s-=S3&aC9QmX!Q-f?n@5IzmPE+{!|L248(S*- zK`E$sb9zL>uom>4+SdyKeg5Z1`jo~3U9j8H@vf`bARU%Bj=8N-Il+pgnaBY0l1yZc4g=HreIZ(Y zY$>PN7!wvOKJreiiS_IFofd*D>y)aq>>7F(S09J_IBDD99d3$h;OxG2;i%OQF@}aK z<8qMu#8~$-Wbe1B_7m=aWs({bj* zvVXF{{k*D0tzq4K(j4nsOtFkZ8S=~c@G@VEz6<5EeFx=w0TJM6w6jve;#gl~oE{?2 z8L`nkf@4g+g`BHJF!J&TV~P{-*qO4|qt!w!4pgDf5U9_%Dag+$#&yXWn+(ZT0Y?6O z{pX(vR?hUWDm#Ks7DC|;Ubfpo`8;<48x9ZJN!AtFdu&&Ur3?#)-=_e_zjK8xq*It2 zv>~pDXpueZcINp}ftNDsd+&J{N(vr8?>>s?m@PLw5qcb5&5Ty?t397D8lhsu@g)Zj z09k&_iPv)T8RlxPx_$qn(nz2!Z<6)3br@X)siDGnOhZ`+Udlk|bTQ%P;V?2cuf4c6B&@Dc_9W2q& zf=d?<*e@P^3N4}lX63#710J5K!^;IQ2d6C^UmQ>vPb1Tm_ zZvg`n`w6XTE`aS-P=U73fchA7@3=Q`a`F=o`7G#E0i)fDgn`sF5zW*9FAJnCwfQnzf zU|ZmN)=l4JZ4gUcw67hV=v)J8O^RUCE-C>=2sK^i2^Gr<2=V1Yv zSM_m-6*yjUE&&;ljIBjMg@L2-q?|7TNEq#zLYY--!_Xfv;%ey` z!+@8RI?Zzu!W~dVFM|{PBc5*cCmEMzsz9fo=GL!wr!_RI)N*8kPQ{oZvccOWG<|j8 zZh{D9sc7{}@)w{^Ux$Ab8B*5Fh4gIWFos$SlG4B#TidH(aC!(2;BJR?62&!V*c z!vjO+yQF3Z_7Qxd>HCEW2lPRz#XVNi>aF7+Bv0WZP=CI$n8WgTQAI@VxUU9#g%TzV{qSEIG@Oscwgm9)2$r?vQ#+1JC%DsI7WnYL{m3)mOC+BH=;U0LL zD`Wl*LkUx5KUo&|ghQ67y+xm5?D`MBLc{4=Apb%8gids_pS>P1Ow^FxjI083PXW*| z;fcgIXY_LeIXw#c@uDih{TtwDvUTfC1BD~s`t{a09s!^nV4PCUoS9R*l1O33t4QBZ zkgLXU37?f;l}9JCUIuGaz}UHTVlo*iL7!vFKF{)SRq35c3~VlAENq@nzH>J3`{{)> zt%h86kZ$G5_}q9Dw910FNfsZCN+Taje76*lPZ1pA%+hfF3llguw+uXSYG~En5DC2-&=@5gyN=Z}O-4wW%!g zR=x(o{J)Ode<|8_1%9UazRl8{2NG5uaTg_3d<dMsX&)QW6FAhrNjF4KDcy+_}@NrVMyU5tiLAqxP8GwRORQMpl#y>jdCm zUBRMAfuuQi0f2A^Cii{XBu{1uwwcGUYRYLWgXGjpVJ}x8H{VILXhg$LsR$ItA0yW+ ziCWQ$Df?I>H=0mCq5e20pQ^3xVvWkGdQ(Ou=0{4xO6x2%vckmrwMH!qg=?F>cH*n1 z=C-z=d968)pEAj6D#;pd`AEUll4)l)3Uf{g7rxeXt${Qmnz=9R2l{ryABwL7iU`;; zL+->g;?d~hs?B6k47ly&jOAl%#mpbDgC$KF+Jdv__88}dv70mWhbBMyTjS8cSS@en z>E{DCf7TwQ2w$l-v%HI(TWy3FgEMI0D*T3PU{Um3;=n%uAx@0`Z&5T0>MgtSd&4_W zJ^t*}S|l1vS=N*>M{XM9oAIGg<)vt3HYb{AakLmo=g;@ucb#A64S5b=F)WawT(`Nr zsoJ-+s^gLh;MugDf{qB&q@inqsX#wPiiLh`lb(}H@)$2T=~_~oS((sH*@mQqcUX7M zshgcwaU(~qTzjc{p1#=OgI?8W3Vk<&YpHA8KISE#gf*vK4+rpRl;Fm-mfLh`9sR*~ zNMcPh7~rYXub}R!`N1P$o8?1PoAAwRwn3BIFrG3k#9!HOy)Tw8AWB3?`av+Kf#sx$ zW1Tg?LSn%DaS?;IfrkxSrs~kT2;EyG)wFZ`^wW|&B{(u`_ffF)vlK!)qj8STOg{*V zH^7{UjMlC9k|A}Um-35Y(pIvAbQxolbov20YE&|lj?$;JMuYdm+Ggn@IK(A}@=%7B zMouJu3o`>UEg6o`@Yo_oG>%e~Z-7_@sMXpIyXcpjge45P#9?v-8$%cOmr|2KVZ#$_ z3a5-~&fZvL?>YidfOx>Fvw9o#66mXeeqf%xm2vLseka&3bJ+=D`n|3VqotB#QvE+0 zDwpda%S0bdAoM4S-v<`Q#zlnE`fSHRYX)rvccdXUS%T{}LExs8#1ftw8Z8CrJ+r)M z@k|==Xg*o`$g&+B?m45_fF)XgSJO`gK}p4*M*~2jp73zUn+6Gm*LuPklspN#r5kCI&;m0B7hRMBWU{PA zNyiZRZiR@0htWZLCjtTq%SgE{`L!7R&Sq8%(sP_MoDF_>X#m4Hu6%})91>Qxxyd^1 zQV)aDfAOo@o*%KN_sMBnVFD_23a$$^wlQSe^>&n$yfI7yAp(1@-rDe5pNol9dBm4o zd%BaMy4!I@(~M68?Q*4_BTgkz!6b8~>ixq5ZIA0@F*16sFUB_R-wSx0LgE42BBF~m zqC~#agm%q(iJbu^*cTGk@esDQ*-Rr+#Ec zWKl39w}|Ix0GM914}FqTBw^k)*ma?IxqP+=-B)CSNg54j+99!>7@-Vwa?C)){6slD zvT(X7r(ECV!$n})eu<(Ik@uOzs($q2bEACtrf~Jd#vzr`0mgxmWStd&cD3*9eTWL? zC33~tr^&S<7hXFEJdS23o_*P9KK=Prf@acLl$Z|7*B)tdw`7TM|Wgh0l}X0L#v| zEXE>-3IE6}>N&Tm>i)qkL#}De>Y~`lDbJ~bkbB!56Ow-3y0!BYgn480nE)dOhBSfp zGqBDmyd}E`9dRWmx3-gLHi*l0xSsR()al4C(j-iF1!d>txC=~TBZ2iQ{qf-SG{AMg zGnfsh{wd9VHeO(PFzMXLSe!n8^~YbOEUyeoA4blRg}u;DUC-|CONUqt4{^K)@%L9G zrpJ9Vmzms{LFRK_(hJ7qqql$5k(|VnT}95wD4_?P*Eh@xQ*w#z2{XmOf9m6ie|(n=*;v`T9drcv@p@Nf&j*;qRE- z9pvNy5j&f$C8aFqT%-obEVNS=y)-ap;r2~~q|?nOZ!XlpOSuudf`O#10~uM4u5X^p zh3+$?vk;rU-q|4Sw{m)xXMaH9)ud!8G1Il}5;4#O@bzo<&I`-wP&%qj14WXr=+e;7!+$`-6=2 zhBb8P4`ET8u$q+}I_br@Yo8(7v|)2jrZ#!9zT3rdUAKmZsYgYBcoTZZAw4qbSgr-4 ztYZ?f+##OWn#Hj3d(>qkQW^-2Wyqp?OkIf>F2dz91wzvK4LR?XzT21Ch~?15J*@Gl z``4lXPQrx<<}6c|pyCr5EwfQUN2yETWcn=y6E|Z$SQ_2KQe&fu@76LlD5q zj#wfY>7-UPv)-kgkeC8N3Mt6nE@p&Csxo)F2}t+nCu{DzW6C?KcF~cGMMuxK_tbSI zwHb2V1<5ZydMq^gNCoM*9Pf_$dC&a@T`w>lQRe5h9EvdUL|Y882onyD*TEfQ?a|iz z`u%qYQlb6movEbmGIFx5tR%WaGQYTOG3$p?1eO|aYQTXz46XFJ{{aG4(wY$z7K<VaB*E63iuR5G^%H(7GanVwo+=Ux&|Z|tnt0jG{=vMg3;{_~ z>K8~VfF?(`#z4n(zfFN=wOJrbpmHk8EUR zhwS#HJP&+gGdDBu5rLHqruTuIA6s`wKVzYkup6RlObGR||~ z^Tm{!WDvHnIQ;CvYeeT5*9EK!XzC*1#_SJ%jdq&6;#pQ{*i1ucFCpDTPppj%sj;gp z#|6xlwAdI6)t-vN&Wo!{F5&|oJ{$8-w0C>UJDNGR7qoU{+tNb$_yB-~H^{>zb^byUU>M)OV>7<#*WZHX zE#W(#$IvUU%WUOuH6OVQ6`n+oY_N4lfk$b_*mexncStVdL{-9SBZs7{-;{kL)dbtn z)?U(d{{ws;V!FLkwEVs^0k0jiy7i)qA!2p%{&}pt2X-Y~02Q!K)m;USueZ#$898IM z?{3Yl?ZX?Zk}Va*?mT!0-kyx68vvfdThQggaGa3ZpMcv|sOAAL!3-m@riw~_07;fO}_?d3LMtZBTp!(u0@Xy5VEMv9M@s}<8c z?Jhav?-;r~EtM(WV+#tzeav5!?2+q%VPwg{BR7G%2$7~>fF~5qTBfn^*r2(28SkN8 zy^9j*fZuO*Nzn)_Z`ZoOjJc+-Z(Wlw4y?NbRg9+Yq-(}hFpQN z6(}PEV~_zI-PxM>-&zRc?vyx{cxR1ST@|U9at?@ z%1%~s##dRMO>ovs>wFNe#Wghl)wHBN@3fis5HZtdoyN-Dn9Y*Q0?l`t1`5eyi&$AZ zfWu$J);QUev&^21#LP3KIFV&M+Bhk_CpdUl3CcJ7Hdd*EhhTD1s{9;HX$=c3`q+I@ zjw~=_$MIeLj_QLBVNUvczNg8?tX+viQ~{~$Ns_eAsZZ7n!7gousTI$kZM@%4DUoGe zzPfe$>RtXO1C{Uz9e&_jpM>nh2BQZoMGEdAkRZ3E=+C%q8f3;=;%h9Pa`*)KoNt$? zQz1Na=&o76<)<6k^M84NSwQ|qHi%s2l_AKt#}YV@&7g|gC8>ox1rFS5sExl zwHptPHU;4M>dZMsu~=_utLbkD@uiakIdOXTTX?sUA%-~lXaelXmj*#gCeGuZrkrCo z56nQfO?;VN|14ls17u{s$UWppfCyy#xH}Sv-JB%}?hLZx72=kYYLUC z)-y`=r2B=e23G|fDz~=wV%@#MZ`1$y@X~je)`z#VYtW*O7{aRXG7pWNAp+QCO^wmA zLW{Re#Z`E+h53m3l9GQv9EbErn9y#Lr))i)Z`t@dgEMHY@1obl$moU+-8t>tv*_&n z3(HcpM2AY1aXzZtCW*RICmrf!Y^>{K@*#cac8I-r;z>vsEVO_heSjV(YwM&i8yD#A zd}#842Ly*5@lttP$*5^!%^>HFd(jH#41O>k*F8+bMgHnvlm0vynwKp9k-B&oXx?#_ z_Sl-vuhGwU*V53`;p=hVg1siGjh-hjEQkc;^j#(sv!jWA46dReJOx)8 z;t9#N2{i1>-e=wg-790E&_zmlS`|=~q=$PhnsrYDC>2?%s*r0a|CW3IpR!MXM;tv{ z4r4eUSn!2wYbHS&ekaDFpb!IHH$-HQnf(ceZ_3Di@)<9}{`3%wO$dls*tU1ACa&$` zuX(!AG>sr53nBlxZx!^+J-%7gTH|XvnE#$H6|mRS{dxXynaGo!73Y|!QJ9y}#nA)e zrr<|Rv`oMNOtnWjP9@SvfvA@=N@uBuppBC_;ps0oUKlQ0k{HTR)%}(3Z3P}WgS|v> zZ(K>Jw#9J>kzK0@O>j&smPfpQY&TnXR{y%89Q&*nfCT^55xJlJpzy^zPI0eXXFc#` zY@d|*i!|BHetlm_E_3QwGL2NuLktmxg4TEZB=d0Qlrj!_++0BvWa~WN!?<1m5+q8Z zvl?RjR0Ty*QB+3W<~FcPzjv;TrLz~p3~tC1^&b?9_SHiO7@n{#sYluv>N0$_#Qj*p zH1J9L2q;#x3RFzf&u>kk!x$lErjb!=>EPTD7h%xGJr0M2Y)>N+dqITT71P$o=ElEC zpva>4gkp1FZr24tVy)1ut;SvcZ1KBuD`_%JTj=?WBFBpK{+`GmWcGx-d4Kd1XKz@F zH{9SAm&F6cm6(R!B-?q8krpUTL_!R0>@ifD-!RM~-(iMYkB#Pyo;Bf6FY*e743z@+ z`VF-?h{hL5tFZp})7A?o3F`lR){)5*54<(BFJs%A;dq8(6Uk_-R)mPnldq@5i# zfZE}Mx>NGwrWEFf4`NS!rA8;I%s%a)r;}QTk=wCthrbGcKJY{PH4tOe@xndx2IrV+JS>V-Y8O#Tx}p;LC`c?HGO#XYGg}T{EDXpxoKhEvfuz zXj@gN+q4De(kEpp&UPtLozK(C+!2{3<|0htJWEu%BdM>cQnhYqr3fL*w8Qd<73`?5 zT{BED_pm-lNj_@)}r&-1Cic|M)6mx#ZLmyc30pB*XZHmAsSP>Ad79<#flO~M{TytzJb zM>e{|q|ZD2=^FeK=dvS@lSi41OAfg;6h4i(9iTWLcCU7}Qc!fw+t%sM@nx9Mp|Nc*QHS;k~q+sM~pldi2TTpB^Z z2f+}!nO?gngHW)hY`8Qt2fvHj9!9a1Y2$B9=AP92`+1;VQ7N6)ylh^npAI;j&H*do z3mf{O-dJPnm3VmXOLESzBC%_X20*lo7;^hlz>ghAw>v)D7E=G+@RTD93tKfb^@gRX z@8VAA9h?-%9B?>;b5>b8RF6}5Uz_7mKyUipMxdR}>QJM1Ps;CRA{Ki4P538aPCEB@ zKS4I;j7gXznqjsoo%E(rELu$0v7_VoFZD^#BpY-AXNOV?Wl$8pG=-f3kSi-d#d(#- ze6L*k!7v%@pR=zQP#DOU1a5dH7=loFrHq*@yV2uK8yB%xEVm0)?xO<)C8sz7#PrwO z(;Dq!A^N}>Bw2u-`!DKcYBt}AWQ_pk`5Nf~cdarv>3Q8N$x=CJA7%XEY!xMmx5HS* z<;T?eaT-S{yQEbd8|2(!O@-a}SS0!9Bid@l#n0io@2LTLuY3Mm2OBX!JjBgU?GDRC z>A-K2hi$>on_>~job_3kJJMQ@WEM!|P%D&~kynQeJzWBja`e36sQu3ghh>g{-yv?s6I7qwxsVB#6CGw~4zsU{YB6 zeEJagX}Dp57e{Q!v3L5#(K&C*T*bmr)4d8pD0HwdC_bYY=b)j-6l2o{!=mazu8cx# zY>V$~g2aIKw6j3u;yZ6A?3!HlrzN8$!8BwRIb4X_UNpf!BKv8OO_rQYyYBYro#uJE zwz)=*SLW;vad%cTm%f-{BurB%0?$;?S%Kv9(Zfl|!n7MN{B_KVKr)KO66BI0m`gD# z40RTiabXkm9D7!rHY*}IEAWuDPnL0njEa^Tw*J>^Uy43-=u1U!L?1p7bJ8?)xy%hw z(v(Atl*NF+-#*K<#*aGUG=&insi>S=A_|Md8sV<3Rw9&PZV^RX9ki%Bw)M#l#fiB! zgILii=K}-$rD`y(ZMp*QWFfA){0*6?*$w+tT;pwif%WFb+F?5A_Xyc64(sMOVg*$M zwI0ZY@~<|-TiEC_^T_C%+j>#jiU6fK6izgxK$x6Wk-n-fnH1b)4CmG(EnZTA*cQ@6 zCF>bR71Bh~Ye+ZckBY&*;Hm9bL^ZiANx*n9=H`m6R+-D+azUypxD`4bXjAM3rL@&q z44m~zS)nPAkG$&Y1kwu?L|N&6(-^a(G8KFOlSdj2ovQ>x7%XaM-go)>!_y{|lHg zcmvF&zs5KI2WWg_q}+N#zP3`PejUAoocz@}W&NLq|DS5M!vE>~|6e|IsCZo{*xZuV z@7o0b(MXyU{gPz~vOE93K5er5w+TbJ^35xKm~EE5uP#8?LX+hr)A6+REw{0qF+PaYl4sto4bwp@*jWc zJa5#xyM*bJPpfF6+P=BTw&3HjoYN?lN9o0H52oS}(nRE8@Ab*~aC=6J)4RO*seVFZ{c_ z{6Dv!v)OF;&t9*+G7j3b4<@`@+m}Ih_UtvYjD_@88Xj?+(tQ(q&&N}l#!>$}M(a1R z=sq!2KUvM%d^wDivyEs^P-gBVzQq0j@}XtOVk0Io_g-YT3NzU>-PEnU8Rgt4@{au3 ztRv#OeyhEPVd_k|j59p~pU%`4mK#BGv~fBAOl;nKWRu&{ylm~S>Per>U*6>JME0Qs z+XFvS&cgxLu`A6K4o)Ln`^$OyS#gDd21d|)YVA#IH4Mx(pOX9Npxb31dHpRk9bj$k zHN*y0#%w)9^i(YZ=?9CreMgosS3;%OYf-%RUON8eKebo;Y3$=TZ(>;8NPk^WgH*_X z`hp`9448y15ahnHwdBSBQybMy`Yl+x8UBly%~6&+6~N_Db7P)YJ(naA>v`mg)PeRK8kYoe^q?DL#KyCO1NOaoL?4G;MIj zI`L?%=-8(B=^^bBcN7jcRHRFTy6gAR(HaAJc_Zhj4PUeu<{2EqrRKY=GpQQ>@yy~h zeK}NuT8I=3sm2e}kLj9?(bFl^^^smNy2KA>4pehuLr~pX>(J}NadsNpUhmxH+np>q z4Stp>cT^3iFBa6!6ZB5fl2cFd(DwtAC5Q`Fx|Y`;BfZ7Ob9dOA9fd_ZV;5d5JrXY1 zSd9pRXWL~tV;5%0VtFDKOJd0Wk?QM`ePF;(wO4UZn2ncfOX_QgRT;SnyKsl)MDlj` z8Iow!#0Y^61YG$Trzy0=11i4ZqnXoz_5pYiNO13WFZk^~{Wc(#v&Y+}%?1v@ivIPf z0HC+51yq@b3JBO)pBg5!ef{j)S;Llju6U-$zTceDI^7l^9XzyaL3_C9Q~-Av6&G#MP>is`#Opsm zLo->;k;{s+Vg~DGD~oA_V@l8WYReWbeo03C6PSl18>%>o-~eCDgkEhRQ`H;|NlZZb zN8Xfk z+Eug4BdWgVJckdbcGyx_V&gdC+Hnn)4q^Az`v!b3-ZXWkM`!WYZ0lp=^<1HEJn6hu zu8pW_3#*D=@H_c}Il<{UyvkW?a(k8ufgd#O6P!ze-orI3@)8TRlh{=X7B?#p*iJRp z`w@L|;Q3D~-U@-8tQ=Sh8a2^HuX(fR*7T(ltR*~kU)gFQWLIA;Sw{dPL5{}^U=hju zo87tH?4o8p_omIsgzus`K&6RJwA#*s#vNY2Q{!j_`Cf*6luC4a`q{)!%wyC~lN(Eg z@MEYTM2Nj|Q;As}JEx%Ilf>96ra2x%LLqbJkdp08+OF@mJ(H`2S_UHauz#}O@5P9n zt@Ea~vnn!I>tpJ>EKfv3-vws?usr0d_C)xpSu{D@=d!=Ggl6>hEmGTZm}<4NIu>4F z%HFRhgcL3|jRI|;XodgeoL{QUpLe*u{uBL0U`gr+(w>Mv(zNbWQ0_181Nn^zf)c%i zH3n9nHs5%Y1d+7P-Wr*4E5;F5PO7obiFgvwjn^Tvb-Tha-pROkBTqv7O^NjdZv#2w zWkg0yBT0n??efo%S7Gl~@7UtSy}Z$nU4b3y!>$Cu(>(FeF0mVe#1;X;i8LX&0Q{T6 z$B8y6cskpHvm7AfL0qUu{jUPw3Hlz1eJFbDt=mx(QXRLZ;LkF<`opkUXjq74hIHPN zdOCY&Ns@gC(B|Xl?Iv|=a~Pktxf~gIn;TW?h%eGsCE3s*BEZ*2)cdjD#mKq({FvRU zG2)>q0-PpGO>6qD$qs0VvH~|Nie8vYZ9FpK0DAMD%F_>#`ngAa&+=$CWtr!nnq46k(DczHeF7Xsq95UWcGJDUtFjv?`nnUqPTrwpt`*Z zV(cMXck6sN^xNCrJV8UI4}pAU!>!jxVwbFIq3v{at$M#}VP1lHSzgUnj24Fj<0v=4 z>}!At_14$W&8)}fuLEBh|5{z+%m>|4COI!Mec2LZk1Kx5p+5sBSaxe>43nG$S){g6 z+z&hyDdBhW>CCkHwM?v%^7!^P(^@^tFbCVeC}Iw)MrVS^vc*XfMh8S zxLNCAGzoeMiYey^6u6M--)ZLAJ{#aX9^9^x(#BrVnpI8-MF|C$eQfd7c7Qc+Ns!t^ z^R*bgBgdWt%?%5MVPRv+2+maN9^VG4&yml`mlBgk{P5`io}XpR1}Q5aX9M*AepQ{o zsb`U$WO!r9peTsLWwzu%51pwgIPXst(B*St8yY15;eLm0CMdx6m$uOwbo!EL&5UE8z&pmf7b9?DilS_CWY~Fc$=@UVv$db-YIVNf>c- z=^0^<>;B+IU#2*MUXx4jejK*Lq5om|uB2q;zVk(QD3Pm}o*7L_2b3RX1^jA^Zsw3I zjxPLBT3tdfO8*$aN6DNXByWmCqvYVq0C9jAQ-`*Ci6ubqsm_|9Flvc4Y?gH8ZegQp<66W zB}Etz;Wb`p5GTOj97Gmc7l=aK6yuBIE$C@U{4PR*!4jK#)AYmRE~Zhk`j{dWeq2k- zHm54pr4wf^5*r&es!NGK=~>e&=sRVmFZV1ZM0L~j4A{7IZS zofOd|3-3=+^fPkL-nj{ICvgrJCy6yF%E5J6FHB@e2&oYo$YL32T$d`D$h{LL~pjtU?4m-?F%5DgX1WMRDU0gt8r_>3un_f(xuZS ziexF3cn3)mR9#qDRjh{RA*x0P91LYdD#)P@o$cf_j`p0RiRK;=Y+Sp$Xg$$w60(u! z*|n0T843NeKs*H?lyjKyoW$HC(Cn2O%<9;~g{aSqq#rF*3P6#?SX@07B{4sC>hz2$ z1zEtVUbzM{Dq*2Q41M+4aSC&WWCBOuZozzIo35WejyYnD4D=MrLg!-RrX16e^l+>J zg=l-XfCF-~%V>0Hter;=k1KXbR^qRrNWvwbpWl-=*PBQwmio9Vk#ddv^e-`>%p4WV zA3@9L^M^6;!3v*8DBL8OE3L^Bz4DveSU_fJ8hQ~k7}P(t;+sVq!!bZIh%)T*id3L! zgF-{bgq~Yv)jzqAre_N9MRrQtxz41+imfgtmw@Oe`MB-$dQMD)PJv7MF^jY0dA|e) zla<8Uj>0~qiAXyJHQX$r9WRI3H#mpfca~x$T>mUq2`qi5^%6Hwl@)3cw$pN%RnKQ( z7c2ohN_WF@*&cUOxP7-Pp60`gZ!X^!$7$%d+Z@pBWMS#wvl8iBs;pATY_~yvu+bie zy)kNntlsia*jruD!^2sVU!$8G2Z4C1M%BApd!48*>}BQR$hxhPQ)Q&1a7?|3n{g>+ zhG&>6!q3~j(wKgiSmi9ylVC|__i~gUG;GY<*sShUPOef2>?0)iF}?$F#1>eOWO2+a zf8pdM!FjpZl*Y|E+&`CU_kxp(vm!~$0Z`3-P7)^AC(`9!M0pCA(je~=(g$)p{!}ZG zZUG$tqX>0cHCHWTL=TBsW}&IH&I#|B2a}gO3Y`&?6!Gjnq~WfAH{I%l7?n3WZMDB) zMeca21?;46CbwA!_g;$mNHkJQB;YkX@8Cv~$4?pRlyt*>!;JMY|5aLa#nVm19Pn)88CeWe;9Atui6bO?m6SE z>wc#=PvznjZL&nWGXkpqp9A$+G1bgW?X>*!%Tb=w`g@UK3fbtHal1kh$(kPI+8Lf@*`LEj!Z)eM=)>Qmfjg ziUzmgXEs1EWlq&N=_91XJ@2TCLs?l`83&1a+aT1`dyh2}q!cH(@h&acmKQ-Yb8k4@ zJ|>QA{!;%q`*`a2(-JaR0Y&z9T8=U%J#82CD|I@L=!~!D@4qWfT{luNea=dqfreX% zCok@o&4G4YYF*f+i9nLJ@%T@(KuIntO{5_iNOeP~(W+x}inQ#wLgQ2oYi%Kp(VBHL zrKaAQq&|F(ua=ugGxbytroxs>4^)iYvy&ko7WgNgT&#g5JiBxkVL-<|^CvrD+fn^@ z&S(YN>BV4s$dnzHC@B)n6MjhQaSMj5~H}0zu~ed9B^c96}YB7i^>KB2h9JR_x zvxnqET||hMVe(-_VoL`Z4CiJYaBi#R;S>DsmJ4E3FD(TUb?`^U5M+~!oXuk6u@*Nj6X4u1XmX648{eD(61SN76i-{stMfF}*aN!xOCW+8!ZAWHz>W zG%=z=AXG`in4`v2c~8Gxop)Z5F5dsoC?kM}0AMZPzGH-$-mtbtViEVbOAKVFk6ED@ zfis~Inqq`BITcCKz-M}lYlKu^YaHMiJnXXlR zR6@N5_s2;19WJ{2T973kbiTjUG&oS3^cTU*FsX}262=06S#DfJc)aTd0(xBLRstSM zqR$r(Yga~(2%29thDWqUu!pKj z7+rnQcTpwkXAJPioLZi=>tMEG_uJsS$GYTG{t+aH@wf#=3F$zDBkyBr#H2w4_APOk z=`jKzSY~!uMypsHWf<#5zq0I8;BT3)KmTrZdH?#{(=1_hc?fF$bo46%$Kx<(Jo`U@ zmV1}$;QsUHAA6Us0l8!QHiL%CtgwFzY#od1tD2%IU!4vXm9oFXr(;TztJZ~uVTGTx z^V@X&xAqW`1+<-i~qdZgcEq>MKIcp^<@@JS(JJMh$4Htb&RceHd40 z+2%gWmsUSTy@ zdYBPDg<*&x0c}{*;LB39=PAQt+=cGpZ0{t_IUKYuZ9mG15#HIN5zx{a6Hb1hUSD6y z*yB|Ep!l7_({N56XF}~ji9=5nw)W-yd|aInuE$>}e5WK;%3e}Y?LyLWqnoj>e$}aP zIO(HCrvg==Z|tEB87W&#BfNfZA&gckp>MWW(Y@+Tj=&xt?BS9WiF5B*p264=Fa6fn zY^^@1-(fS!WEb$4PWGjuR4aGd4At9S236S{vjPwe@_~bLsfT zJ>|Y(gT7#pN24}aVh6GyRvsM!?`Etqn5UE!`ovc7iFLqKKb;JL*VAk!o{KH`iYJg% zk^sm}U0yUTPdJ!yOJLgz(^e!KhZE zV;Pqk%Z-#px5YM@H{{ut<2UGjp4WnsR{h|r7nA1elepNwd*(w{p zFa|2bYKF5FOllQeY?mU&bblT}RsT{eboIx_GxZOc_xh_3WF>#b0+2ddF5D;w-1>%;hS=F=8mJvUjD1JL$a9;Pj=j z?>yZs?+c`Qg^i-IGUTkC!hYQ4g^gm;Gp`dZ?UTm~>+3_Ukv!Kmy9xyg)R^zR-{0tC zxz6l>zW(WU)K0%+#lFsU!=+}+UyVo3<;f4Tggegc$J1AVF5UTF{IBAUe{2F_Y3)N_ zMMqvT{{zg)nh;{X?ltIrs_0{kDg8_K>+J0UWP2O=FTTb0b^ZqsdE?f+B^H_JeoCS| zcmDh9KLDY}zyI<7p}nb1`K%p;MC(a%O(y%N3nCa;%Vm=vlZKEF1s1|@Mfl&)okxlF z?d*0R+L3}sIQ_{o5*SrI&(9(<>OmP>f(#pV8{*wW>p5KgSPLIGYq4Ei77$<&QaUsO zXaj5qjR%`~KkZl&NoiSvA_dzrPVFt=iRC4s)x7UQVJP;bKM$9tAW(LewAyq(#m;O$LOq;67eVKe_%I;IHP`el?e04|5TaV+}ITeW465hr*J0< z|690M)P7@!4e(~uq$!ilAx2k`_g!c$#og|YKN?+DE!{o_|BOF3VPY=#r>~;ik{|tM zU9E+9|B=0oUort)@Ly3sGyK&^HQG!1efs6`S5QOiJ4FkX#UYqdrrm!4YP^)KHR~7s zP-)jZloVNYP8kZ$dm2KjWcfX)9pSLP{?H?VGdD43R>5>$g|0$NRwLnVTjQhtpJMH?{EQo3k2KV zk~4b2=^CTrTldO|R4DB#^}-?Zy_H1)zRT5sMm7B zt|qSt^a?0fqYLVLWw^eah(A`S<6Z%olIgpMplRuSZg=EsBqox{Cerx$ot=^0tb-^L z&t<8y%YBP|k|RHJv<~-k+G@R}lf!QKzK**}@+stDLUTj*H>z0mjsgG3!vSNbB)j~o zPu$|UTs{&PqKHI=#xEZnvi2=vkz%}RXXfrOk_yh+m2Em;N;F6tWV9_jdHJ-fKEN-- zQEjGCLtT}@!df0QsrceO>&pE6aH<}~$FyWoB@!_-4|A99CjGQTL6Vrl(90IzZd64z z%(~R~0p{=HO@pdK7Q$+AyI-FEybv6|RMxkcF>c{(A5756jLO9SEH~=sM?%HG0^ZXy zJ6Dh0P|Ig}*|naMAsw?G5SYl%nbvBez|v6P9DM}~+dSM7d*VORlwdfIk8D1I76D6@Qon!DL7|N>PH7?Df)*$y8N@zNN^}S3D zx$QltkXFFZ%t#G+Yi7MdRyu}7BpO%2!n)f`8rQcZ9olUl!AEqy$nAd5x$q{4x7a4< zW$B0Qg`OBo0U;Oz2D#mUShZTzlI|Gfajl=EFU>vMLyxvg-d7dO{8%M>nOLvn{1S%w zD3+9vxw4i4GagpT{xaFuy&4>@azH;Rv6cKAf$ztojhe5|i)PK*{SbFnTjCg9_2XhD zK1;0$adVS9G}81WVXuTy|3<(PiqeXq60Na$;{Wley-EyE0$t*(FKpN7h2Zj2#A)(c zYSe+2zM0eUdH=WU`Izj5hZAZSX{REyq|!3uqVZ8WZ;-QP9S}-ylt+y~VYIx-#scG% zQLL%0{PJ=6JL@c$9AXX*eUi`K_`5O|PEmB`M#wYx7ILR1MM`aax5NL0z~tDb?u3?M zVnt?MgU%+Wkd4NGT(FO$x5!x*X!>npf86}RN!`(tzy?}XDB+gS6t61PhE&Z$O0X{L z_={|meS^nf+~uSJR;OD$k;UoLFT*bfZP)zp^Sq5eyuwbiQ~A0b`qXd!?Q!c-#SV}D zWOx3cwGt)f@{<=Z#**Go-`Ekc9g2vx>$)k_$& zv~_4cF7hVt`u6K<3`NgO08eU?#Z@g8Wg&zKbNZm4K7dEkC9N>LD|R3m?UjPrlGce0 zVSj*W+*(cbj|U2@ls-@S#*}NODe^3M-fqwam`zAVFTC%0qv!x~_Zi_3EQ{b*TQ2B~ z!YO-c+0L`ZmCmW(3u7W`Yp1YPDt(t=gp$BO^+wppmqH5taXeW|) zQ}i!R-xp7w3Qq}m*gs%w%^uEuV2gl-Kod|LA6>E>Yn+(yVrhmgqeZRqNyA$HfF|An zgCl9kev+%DSvNV|!JNB<`U_|n{j0t96|Q0x4H=udM9Q!2mi(&*28OH;HXGs&jm}h) zw{kr!{kzKFM2`dz6zc4=;a}o9j3s>JFJa4k5WO=0ia)q~b<3j_`}U5WrHza94F`P^ zCupT@&JnnjsL5e?LOEed;L15a&QTDRKsM-{`$sYftOnojmBIx?avzD}`rCj_duSFpU15PMl}LM`v-m#(O$M4oYC!w-w22FrP!mwL=?o1KhuxSE-#j8BTJRRU=WVe}&rRtxig&+TI-hyK}rgZ1?5 z_FkwgjhQyTlJKUV$jhh?KRb=ohOwm64a#5Z=Smu4_81E#{(#yk;it9gkA<({r(?v+ z2Ox_~KFVkE87+*aP@uF~n>ytz~ZOi0)f; zL^rtUPDn^d@<@Z0cM?c`9MdJ}x)jTa)NpeS1v*}JvmvttGxUy;*RvTixQA4VKQ=@f z4Wc^!++l6%XNU-7&QB|1Q$Z ze;;cV{-rCv(a19SLNdS3`nygNkhmv)_qanfL~Xsz9mh|6zZmjuT1Vd@kZIseV#N{@ z7g1|23`-%D?Gpn60*J7CcCJA>(Ymk? z1xng{)Jgqxc|yEgDqI>Q%JNZ(x_ds4t9{fA+bT`7ulHe4mTOfb31m=3Fx(la%giVO;>ePn=|%KEfuf~N z&1wlT1COXbNlk^PENZVkXHj>0(+*Q665P65e|psSIG^D?_5K?~W+aRku?w+lMD^$E zVhu+7JjeW#Pqj<;P71q3+)v3{umi5PkJzm@O8t9Wv57+H0NuS~=eE=C*Uv zDIaW=r55L1wmW-5r{YbXS`kDa$2)=&g7Edh0x&Dps$car_7B5`j%agx`lW&_pPEAz z-|5~D752&s*L1XYa=87I0TvW&4ZVX_Kt3Oq1$`8@Fp#bIv1S~f7TY|FjwntZcZi@z z$v9I(L#d>ZN}0LhdLmBLOtti@UAP!+NFRKNvJh2ybU5(cgAPbzi~BI9aG<`d!A(1% zWeW7Am6uOeHnK7taJ*tU7r1Vyqh8XP{B=Wf?A@l$TBlkOLOzn(d7b_7uERShd8fbb za&_tIixd9+fa<>gYXhpcz+#7712CSRg?YPBm{+GeI0qt`_2g*Y@eEJb>fx;g&}h~* zZ*|Z4Povpi-c;W7U|pFr#H%j_+0l@=sl4yWA>8yZZ8!6$|1$x^x`R8fbw*RWWodHr z-WMY26@GfRRbsZY?_Kf{g2+s-LiuS;lZx|izY3CgLLG3UFKSJ@$6N( zyuIo(ym;orlXN}Y7ls<4{mwJh?J*mvH+Rf59SiF=T?_Lyn@+>7HpC0X1yaAN$CMvQ zn`yYcUsp^ZxCDAA?q-4`zaecw^QgcHlx{xnd4w+k>oxv&%SlWYj=^WwX~Oa54|0r zoVxpUZuVYrr2c5RUbhOmw#!g^2bkn6^AzQ;?!y7zgn2)Z-Ou=|N|CN& za#YLw$iG-J|FG!pXKx(*(KZ*VND$F7KS&9g4)UH~myPxO7f=4i$sZo7PzARTyz~J( z$=l!{%~In0KBk{uwKSpvJ_WF1STnvg+^Q%_jnHwjoAXam&|M|Ta*_gfA;^%korC%z zZs4HJ;AOv)2w+&~Cdv1=%PCDBwdA<_&_gZ7vR3n6==JHQGmS_FqNT_5IkHis{?_1? z$WbKM<_nKImvhB^zz^eUe95Ommh8mp+kOR=5hF zKuC{A zpW_i+x9Ds}X<*I2QQ#(;JVM~FqtEBxFK_UgT%m4}P{>Jbdd!^A)aSvm`U}%t1+;iD zB@opj(!Q%bwvFF+@U?JJnS{Xh6uG%r(aL!zqX$avPUz?saNM{|(Nx6Cw)zX>S|!B- zqID+&);(|9(#W)mf=zn>wusOmu_bClsdl7Rj`bhc`4 z;cCi`vj<#Wt6E?~--4CzGIZD<_+4aAqtP9rwg2yMt2}?;%&kDf%i8~)3Surs?LW!S zOF5>zs^`OxY05ADqDddN2WE=x0c&n8>Iv6Wr?0_(lw0O(MsF?rr^ z&5U~^pz@-!EKcjP$ZrG|=IZUeKbI6|G;86Plw3gQ&zNDso?t~Y`#-Lsce(;x(-(>J zWtshc&0p2{5*j@LJAq-jYLKVk)7N}p>D=K`V538>nVsS81lEKe0^dMB#LS+~YGt(D zE2p!6uN(L0cLxuV$lp@|Z*LG>?7>30XA+nRj`bon7hF5=z;l6!6zf1D$0 zY<=*@IZ6Cjx+hrII(dzee>Uf@rPx7-(Z3NAQ>v6_wf@`(m7Lyx{3aZU48EJ(!(pE% z{iB-k!5`IVqn2@gBg`&jC4=}ZfSKgy|6EY)e0*_>se4;GOSe<^XRA{hZx!!dz$frk z^*4frQ%LeY-ydI3k`y)php^-H?taPZzh+t=oJqqEG)>;2?*Uw~DcgHTrw=aN0f*zb z$R#zQ4W&ijdPGsVyA+u74X!NVg9{N0?)4yR+@RjWo&Z(=<-z|Oy3!E~P34p~pi^AN zBrsrGb>^J^k4hcs!`grquhnnw&GU>bx<5#N*;TyNfN|J-83x-afY-y9KOR)u9qf6H zNeu76tCA~s|54{-b86$YZq;bz)f6zMd-s}D>HeT9Fq<&lP3^Ij+?4~v(B1jFCph6} z4ohD5n%nstVwP)i7O_SmeEwJLEA&cLZ;SKfhJ1i-6Qz?0m+MiKPWkHG{NNXHUapD2 zSNN}wafW6^SJ^m{zw`{Rdz@`tox-O4zQPq&bYsTBiaZ*$y>jl?8uPOROCnk~FIfP3v7oc;B2kLPXDqzT_=^6jui{k-s2_=u!WUn=5p$2TFSw>yFyod7S06+3zoV<((=+HI zgHXNtG=Bx}DP3QbIFlXIC{&(Vt@A4%u!@bA+oxNab)A63BKx)QR*s3lc%dE0PRCOV zMeNrXc+OjAzUvV$^+3Iu)kASt>+Tb|J^x7$))!)Bz=O5WI(Mg6wFfxITe3p*Xx`z^ zQ?)7ZR2@MWSq=TC8Yr76p|{cP%25F#TZ9Qj0c(LMA3OwTBN1C5Ms;kx_$T%u=(Z5MbU0}?Z4EQSQKK!TX?-jRm%I=*p{$Diw z$EZI~5%reF+e6SyVrym7oo8=ITQh|l?fRlU(UxZ^Q-HbeqGWH8`T#G~YGH8B`YNT? z-UjsmTu{biBCY%xdqTrl#1pH9W;T89+_}6F-KOudVW>Ghaq(*R)EwkvCT%tRZaGTb zW-8aQ+SJ4bfM0J&>g!weH)+1ACd9`r5q4{`C$*UV@~g@ zywjO=*@MQ7gR^>X7S11HOJW5XlPS7w1)D8wFhaWn_-$ql*8E0jIszI4(RqA>wiGsVz?4b(NIARd3%yyGf9P^md8V~)W;50RVHR;U zAj~RvF6s4r#YVbCx%{Mm`cd3CBr#!$`P(4X#_@&AQO&ir)o%pa>4QmaHNIG=5ZBz8 zO`N62;ra7*N16?zOO)lelJ(!f0;Gag!(-=~XYq>_?#wM>OK>(j^t7RN9{MDKx6G|N zX0!HvYu3kMrls|@wtIQEM$RQ9g9m!4R$H8X^(iN%)A>gBa6tNA0VG4i>!xA}xqYj7 zCo4R4>7B5B)Q$P+chbior<;v)zPs>_jO3UlFsjU(#GHloxgNBQh0b4{H(S8Nu(j0^ zz$w(%_#D>0TYQTTh1K5RSTlq-k|B5G_#lv#eWjm17OVd>=XB@byl~8@#k??O>$_Pc z_iO{t4O*k=yRFjx9K`JICzf~=%tnOL1`2_F@CieI7 zIEG^3ye*ns#y@)CaIbjh{|813bRiXvA;v3vdT~mA@_Kw931R)|-tEk=v&f>beaCME zS?~AEtoe!=L3Ury?{-OU0b{UTuWa%m-|>=E3$*}jaP5T&?#{c1L$1FXdkUeCApo)fD_9Zk5WeO{*UhX0a-Nl#t%F}kfZ;I_ z=r2|08f(+tn2sr0Uh@OMUPkY7Zgk$HFDx{j0Nzm87iX+$b$bAd-Hu5|e<^FdzX(Up zh6;NXFl&4~@|KNEDDzUgxmuM4Dew|BN57KUL$4Rzqs zl4J@w0qd--_gw*D`Cb@T_W&}*oOogMd(8{Od@t6N4*JlQ@Ji7~$BJY-SD*b0A(o_w zS>r809qe3vx;J>wE3%lf_Zxxk-{^la#=MI{C&K_cDs5fbTILpRPDc2Jo%bLUB0HjP z8jk9LNMSTJwcfm^vQdfcu7DqP^RYvXn}wGcW9 zRlzGs*)n%Dt7|fMw{Sj*gq!yd_4P#V%2ED5xMGmK^b!Ge&d~`#2{dKT%0}2`mPgRh zm&%F29qKEMfLZz0RawjZJGd`+KUtpXUG=ua=c^`!cBRkUD=Z@WQUkD$;SUI)TwBX@ z*=Bn1F`Y}>#CH2xj3=WEm>Dme53Dx?cup)#S_cH@9V_nd1d!G20o`zT4b0DMuUcE8 z!72dqi#c2%bCh317N5XV%d5;er*L8&P`P@SyBn@sRQ^Qw4w9RMqpM4k*`7O^EnGE2 zGdRFvbefJ1Gl9F#A4jad6{tXhvszNEsLoFEuJ!=SlQL!=o%uCWabO+MRF}ARe}iHE zJD;qD+DYWAW7xWL`I9P4z_cWPo*8qbEW`IEl_Gw$M%VL80ij9VN{Yls> zuE~Zrc}5lW$d*)RYqVB-loW*zI!)@C-H7dQUCOwf+I^f-5hITPrK*n#A-`Z z%>e8Emfih?)ChmRk4IZN*6)k1*$iN=Sj2PwzPI(UP7RhB9qD70n*hFRC@Q2v`d)wn z$z#HypVzBB%Di@;&c&MtymRt;x349}QeDxb6uUXljR@z9c(gfW6=$71?wvg7HzRsM z7cnIra~bo87Lxw1`!?3bu3*n$hELZ_12F5DS-E_xc{O3o9=|T*K2486F3|9u{CUf_ zTV`)pYvvwp&nlF&l)G6IpsC!o=HIL_fnI1|vnSUfcx*P-&iNQD`%Ec21`_AyqL53J zc2hjo7Du~wu&$GiU8p5B8D7hdNLRaMsC&)})y)$S9ljJdfVpSvU@^C5upD zuckmbNiKN#=nuP?36Yw;Ivpr=23z#>HW&OBn;oc@b@o6OniV#)C$;0fSHD6IYZFdQ zE?VkzfASFCey_QE+cMIxy(*Bt<|*|XVT%GB;(vQI!;qn@hb=O87k+Ry_!}X?`*ItS zDh~dxN=zR2==B^A32x3u*Ay9aWut3_Kaj;VwQ4wd$qRWIlE@CqaB-u}Rts z!{L&c?)7+L39F7tUPtK{Si1>XrB6x%o+kPaT;n>{r;W{LG*6YM-1fzOlyu)0#y_gi z`D*^d;YuK;50n}@>aTy1c0avL0enBb?5_22Duwgeg;k~6Gj*6RxRCf}skpo8GXFOM zHaN+ec@Y=TH*(Ln?d)*gZ~6nmCAqf zD21pbdI!xEvS=esP*pa9*|@Xl$bn$SxL~rf2=KNpW0WgpAesRF`(9h3 zN5ec^IY-IqBhX5*By0jdcQSqgalAl9D_dP%5@-oAIoSoYb~G-cr-diseFxS8y&RQV z`a+0e5i8CzlY0i4DuC=9Ol;xgzBS+L(J21O*|1R7xV10Tnp-1uF4z^h~|V zlDjQuMnJG+M~^65zK^^jSbxs#XRoDUwmNrFNKh72u3Xi4|q;iqD*pNS?5Cxn&pv*K#zye2B6gD1L*?(dt1jFBOOR`Lj0{=nmN;OC2WxBD2t#*Z$DAccqSd8&Rve1s(z4VNQGL>|Aw(9@=k>{6pvXDI=b-SblGXg7klce)M-cWzU?b@Y(#9Itp3dBY%r*9O)Anp1FD?L#OfZ( z5P^7A?-Ww&Aa1q)C=cywJcSIF*I16|y5N=M8)+GV?e2G>m5Gj(~33B>gVuUbn&iyFioc!b_xUu3Q3B^T{c zL?nJ$ec#aX$?L0V+F{ri!XXj4?Fh2?{iC-bcAt$RxcgHz#zE3#+|6qTIo3=;b=J%5sHE}Dg?fsBJg-&zMN-&-#otWLdd zP?^_GKn^GfvQHw+wU_kkSKCV=TAGTlZ_WQmN$-10FuRNsf>U)%InSPYQMTXGX5~&H z+c)1Pg}Qc*$+WU&(_usG+qCfW=qcVyo+eTTcXq|!2*oD3vLdikyK81)ThNY|>78wX zz#WsNn(kKnx24Mg;(+J;u5ID$&-%XnQdHp}sE<*s5!#NEQcFEJq;tl71n>46=It z0j+3@h{}Oecd*v#Vlb2wv|AQ!@lR6ktMNKiu>XMjXaRzs9(iSIRGTQ3{y?~1W z?Yw#WS~Jvkt+BWI*cNvzXQQw4S{zY{e=XT+G{EZiO2Z-o_FIp7=ERx8bun@<;)8Xy zbLj@His`FP0#iZ`6CLCWVfJ5Lv)Dr>c^f5;=sD~oImn~i20lA<+TpQuptE+F$)zK9E$`SwLZ5X31(iiRia%WTremTvBg_%mH zev~~XQhIH9>Iwr>M|>ZIkcru?F9a6Kdmg@!1@~h2{#km9I_KQ492KGjmmGqXJ?G}J z+xF>b(Jx<71_mOHhIA^9OqBf8A=5~uT$}eeBd;$18i_UU)>p4!NZY4V*8Lc5bX=0+ zLh9SkFg;sFTEE76ciU2TO_@FZ6&`6$grFX@S73@4f*%Nk$~PwTARKA6WIHGaU@DpBW7dnrvBoyESJ4&gCWk)<0r@ee!jUe|$I&<1#o(0NEym8cc%Z3B^uE;s zPDhrQP!9L{k9cknFtsYOIhcsVR2sJ_``6eML!S1#FXda?av<#pm5vc9byT+iGP$YM zHfNwS6c(y&45h>ospIQl^ldQ4l=gg7?McwqqDI`1LJm(H)sOWgUAP3O@g5?IfEmMnx+pkef|HYvs# zpy)`GQO?9*E+)mPgo2jOD5K6{aBi}cO(IfFhz3H>qa3Sph5 zt@Y9`?I2zcBCgG@hFZdw6Gt70^8xaJY7_x?9?+6%02s33i%f|j_#{-7^c+2)A?7n! zbdKF8HzSb__+mNqtjT>i#QR7B7Qb@4Yy)$-n`U4JnjISA{U0?uWZE=FeD{1*hyAC$ z6gV4%nPE6{ATkY>bEWn=E7((LH9}KxjS%W9>y8a8`8kj(OjV%OO&V=y@ddQ@;KLW& zpqGz6=?+Gg%D^Oz87wFnA4z`vj&K-qcm#Gb&@;n+a6T9{`e7c7 zpOJ{Ls^`{Ctf@Q5P1_*Opz4_?qfD5cZ?(Yhx}pl^h<4DV?W60phJ-@n%cVD%2Ba;` zW}d)W&oVLWuc5D5!-*umBv815T`7{J4@j%ZeP*1FrE>| zz8DHTkpG1msaU8wj-|aNXGR^joOPH)uAwk)o#^)QY-SxHS0arSPnUIF_6oo~@Zo;?wr_9L!bDZ2S&3Ea6N|r zxYxyPeEU@@{b`(T7RuoRATbM5`2Wqp(*ocv`kjZ{0csN)@y9xXq1(~Z*PK}sJoM)z zag4?+8yE~ExjQd;gr(Iml0Ff#u+YyDxs3mSkShnX870|u*72Y$ofG#8&_)<{Re%rWJwWDW90jHq` zIm}V9Ipnr#siRNH^iTsRtn_aR-SJzPtVnS>YUYe*#cXiXKH*jv_ZEE+d^K)HVoDa% zD&-|_bpn}TTxkB(P*%N2Ju1D&cRTd6Z8Mmp+CIp@i@2Q<&Q0~!Qb z7WrBUp@c!DAm6`c{+j?_DE`> z=D^$KTw(uUR`bOpw>icR$8omZs+}icY-nQv$x~kumz~@yp^h(7r*J|=fDs17x5T7(gq?ZXm z3}!jO?qGfuleV}$py|*i1GI5Q*jpIkOfa$miyr;_^$3jGGL*Y$8ieZB)GjghOr?oU zX4qqMUwi0o_^Ck?i*@_~YMq6_^>PCfi|G?NB79vgh)iwZU@1-+Y1E$pUyOS^{?YZlpf7psmT*0oT?ZkbF!rys>l(syh z2|KX1VI+M-v{P)Bg{1tXHPQ_IL7h@VwzP!=7G1qF-jLTy<@`0o;lLD}XQ)}}GON|x z$Y%l?#13~5Id~T;tf8et_+i^;TP{ly+=OaxC@iF}?pmI3*+g<`_Wa|yu%LaVLn4!e zAI`TX-rAm&KJGX|_LyLa3al^MabHy!8QU%JJvwK?H`r2w$sE!rtmb;gA6W5j_-NQt zS3Mq0YW!lSb+dkMnWU+Qc*1xm+346oLe6BF95J|Wuk%w4Kc~JmbGBGi5ogJ;<(rlw zeXC-s;RIpNQgja+FpPtB{MYsjF^cj=vSWsf(Q!A8B9>IR9W~c!x{AChIxTg{o3`$L zED;c<1?MoKh^>D22|V zyVVw{lV-Cs(rnS(9fc0I0++qmKe~fqwy7T@FM1);KKCBLiz^e_3PyjiVwT%+(oP!nxm|2AH51c zGnh5Bj6fW+Pd?@&*uF?yLNp86n|E5>xOO>DJqNq0>B_p=q9tWT)f@{Fc32vaq7X}G z63_})k=Q!YA?R9yb5f)#q;l4pWw`n+U6K{Q_u_`@&aQ;1^#+I$5_WuPm6w(4qGYE0 zAYnok{Qenb=!2)U5fJREJS@2n{KY&h&9ACZXWMpa4j$N{g!VWhw049mgV89%1|?2S zV12uti)XBa7Iha=jsDm5g9dsAGlgq`qCb~O(kKyVe~$U+nIcSs_$V~AV^&YqUXTgC zT`KE{5o@p;tNGv+fFGHB>?c($i={5PYEi|8HZ&BK;IjL*o(AV*{0T*=U0u`cGVWf5 zJ5SR}Dzuf7*tRxC>c^gue?@>V@q1+QM1|~z8fYrX;DTDM?P9CvZZM~GZVt$}x03xl z(%mn6e3NCstI5%f%Z;9J(<>&7lfRw&1Vi`}OrIn8Hv+!t7>b^%vw;Dc>?{AR81m*p zQbTT*iAOs|BYinm#}uO&LgtRagDZ1!T;GCJYKb8xprwX@ZPkv=r!>*;>7Ep75IlN- zH2C`BW0Umi&z^ zSkKna`h2G%!P$DcWP_2lQd!bMN^4V34oxiB#j_BqvXHq))pc2E8CH{GlEaIzoJIX+ zoUKk@<6|b5yuILWgin_iil0H%H5Lyf1>Pv1Q9x30TrM)OflniGCybrfZP4SR+cGts z7NV9O=5GfAt=5C?nvfNNSg(ppBr}LBdN1d8Y|#NxLbn3A=K3rS=nR@!;bxrfaGakl zU5Q-OmE`wgNt#w~x+Tljq5tvw*rtrwmlaGZ@?3#cMkQZD#bVTPvi$2S3T-rrU0D2i ze0)tmxa>E@cz{iw3U^7=i1kQ*wC(c^@ELozLBFC)e%@a?Smr0W96=peRi!G&R4}et zpeKQ0ru*!~F2dkvx=Wz>iPXWnHDF?ptFt;Vqe8nbMxox|`3Cn@Eya|6YT)yPrQPvi zyzmTes~)+SW5ghoIU)Nd&+(7kA7m)AA7ZF<+ha!PVK7-5`G5b?hWr_X&DQhNznfta z?|JgX*thN>DvS9kQnw|IB}8-j#K#1T9<2(VF9mJ88c$j1CVq%_T?Mzq=ehK&7Q9hW zRYV}y^;w|CqA6&q#Ga{u1vh6N!(LmsnLOUh4@2Vcf8VnfQSR7FVNLW4q)D1S_`X)I zlvqlz-><8VG0libak{~#S^RsAG*hm+f}l;VP(jF;Z_)7>%niGXt629+(P`_+*9h*y zRKC2N&rPRJsb?HY@9Wdq#x1Pn;yb#Z3q}whhu3iTtbb>(6Bc8r)Y>DBe*Y}=v*Rav5+a33SQ5n}im z2qX!*(^H@jYXu{q&T}M_|L)2}1$be(@^vp3@q$bE-WrMn86z%$v`+p1m@WgUGOB z+j$?D$pI<+0vA;U(jr8_1tsdN2-({`x)<_VbW^Cc?D&i|!-qc?L4C_Aj z5|P2GVby(*!f$1`QUBq17$!ho*SBG8(XLo=y=Y|1?S!9PqjHPmxY!tVNYB##tInPM z$8JI9H8oj*R1e_4Q!3s{mZVOO0DUW{E8s8mXc9T0R)W!AM2pBX zdzCRQmrMLb|9l>(019}Ro7I6JklVBv?YLdgKO+(}@o6Vz%(Jp+LkpD{*_qDLak8Vs zmAl}It>T00HK6pUu(oY>&P~5nj5MZdVhD?EJpkR-3GSEU9rnxP*Y=M z1V4HKo?Zst4c8eRIc66c8wvu`3YezKmTyz^mE^KYs z-TQYTrLI%ntL-J=W%UAYsFSWvwkPJI77}}rqdbnNX|7cbwAykl%h9&z{;pxFJT*rO z>yEz(U+7i~oQ!^U#eUk(wx}6+d|!#93*-zA@5@ZyP3YapY&o0&FZJc9@)EMff@jRVIVwV3wS1wsVySjk`V>K!GNtImckX5ckf=I1S;PG|scBCI5bj ziE&YsL4Pd*D8xzHs(2j@YZ@*Rhrxa$e9ZPVGS|7bBS-&&iSAb90oudcLA{yxM%3Mx z-MBat7ZBjw!a69Cme92p5;~-&I8f$Sbqbt{1Z1f|>UC2NCMa)ipgH2W;A%3ydPMXe z1qscL8F6Wuw|bBilPFV<0S{Pc%JP|v`GpeT6zpSJvRbA5)DfqPW`aX zwQp@~avhZfI_hV!q&R+4K{s>W8nT7exZ=^ma(21;CV;?y$}*U3_cucF@WMKEFSxPx zCN5&GBxJ&GmiXL0(E)!=s}xqWt7uo_y=io5(IdcWRL#>=(#dz>QP7Xm&}*e-;$;(P z^5)mup8B~Hlj(CKAXq!pe>fG8|H{zOy{W1s(;6m!?(-Yropf>V+?*Q^x0qmy$h67e zfsy*J9|uF?hjC50D=IbDp_C7J%XN=OcYpeT%R&%Z#3m`lww1*b< zz-)NXTC&+@dau=j!pN8A>B}X-$&(MLf}P`o7KhQVY=wL9mmS{t314b%FF7jouP&W4!nvzkj5(2Yrmi$&I$dCn~~3> z_CIaQaUhz1bn=pTU-Q!QOyQ!6n)^{y|DprsUqicXgj5Q zoQ;iQDb%g&R4k5Pc0b+V%7}Q!>9Cb#U2|^2ZQ-KC=Dn*Dj1TAuy%k4T&7*tzDblWV zZ#&KUdXh7HzlevgL2L;+MtdL${v)J#iWv~XzwME+z+bw+8gbfiX?VXPVE^(Cvl(+; zHs(5iqA^dI=jYC{FJE76ZINR!RBbWnxcJ!H(XG~@FW+^6_l;P+)A^ZBt>+Hm2#M19 zlkE+jJo!au+2@%1+Re?1r8*gHoLRg@*N+_Si2>nn^8=7Cm=+iFHJZ=De?`)z|5y-P zEB?oH;DT1SLm$sydf_Bkb{P<%?D?gkK4L4@IjX;H%{}4vR;JSZ z-$W}jb**}j*m>R~fj!~&wX`jG07&^RsUA$SaAjpG-MoeFau;CjDw7?5a4e2~OOPQabYV_P@Vyzao48 zAEZdUDFDe{08%(N$RnyR*TQqs*(VyKL7y z0C#NY@V#Sq`DOsg*_{)2c=YrA?rv%NCx0}$rx&?2G`6j~CjsZ6Lj* z9O&*YU#)4gHhaH#S9ZS;YtM<1#jQx;*8@Wy}X&!QP;p4 zzJP?T=&;;tQ+IBBZqa^jdv#Nu=ft>JVM+Kgp|+abK#EubV0j_*$4OzHx7 zsOkMR9LN(61T7&Zb>xd|1kNcA>5GWN0={9ufSP*)ig&{o;9JwN+C+zLf3UZ01%mDU zx-R@*l8amtZTuhXy8t(oh4T)Qu&eU9%mLsY`4znuY^3^?^D~axdo6+5Fj&ME_P^Mr zOQF0Dz%jY!R=qs*PMYLj}s15OT8`~8U`xw|Kyu`k$n0|#x!i3qMp5G3!@KIXMoiX<3*YSIVF z_>UIIBQXV}TIVT%%}5Y~tDRr5%MCX(_{ta-^>`|K@T)SkD>ni*%+hNk%!jTOH*JcRKM$UYgOM^T*-{0TL8+o|VFi2*<$-XJkxX8KL+q zBc!9c`P#nu`R$1(LPrruts=vyRk&xghm+|49jwnpbayx(xbL1o$$q zy+EH#W`(PxMlAoO26oBl@<@- z<~xY-O^(5~bBEF(vQyKpU9`wNGnZ0Dte&zO6qnq)QpTG3jbNN&aeDo0$l^rs?;Edf zKIyF}kRxdc&yf_HxT1z2z~4}?;RB3~EDfXi2bG_gJvyIF*!x{lb(ytqmteCJV-kw6K&<=1_2GxG5bDIPqfp%Ss8Sh^VC*4lT)nH@4 zY7Zux9Vz>NG;_J&vbaq1?VQtRlAl|hprG^blAP^iG6U&^_8JQ@UQ=l;d9qW&s{T!o zY=P#pw!1IW&^^T3qqOq~jRX9f3yN(ZXH=ZnLy7qyqSx@M7&(h!F5vL_CbvuXiuwSR z3wSXC=DgP`LBi>Ey)^&>bG~nkn?De2@?2!bbKps7tero17F}tEYRcrPacEc+L~W_W ze&-!+c`?TX-Dhfdw6Q|*o2*IZkV3m`Ca6~4%kSbynj(s~zCk$2s3h|`sCRGI>F%*Q z>qh!`gL4>pa9uVJoT0(Drv?AUA^UA^Cju=Bc-RGAWC)YIVR=B6x(|u{xgq+(1+$CU zF`=hz>!<*t0A#=2p7}Qd+wf?S?)&;a=*(vU=XuZN{J#Rkh7a^2n;%3R6I%iaoP0UrNRAFcnt;sc{&C^UuL}+OubVdoI5o3o<5_lnO?Dl!z^Zm{l-=E(Y-=BMqn`^GM z=DpXP*L7J}NiiIV8S@)>oUmq;MD&NfrEhI**G_9UPH)+lFmz{cJy?MWV4Q#XiuFU$ z*=+&jh3Z9&r)yQL=AMLl_s{)Hx4PBACsd6b{NHcKfnCZ4A6M_+qTWlkHr2sl^uOl+ zH243H8F1M5yu-2WApG@jYw#eO;6Vn}IvZ=2$fnz}|1(P5!u7!bEjaw~?-{B7w&-5> z6A5fMdd5Ctx!zAM6e>Rn^^tfI_C_CXKnvV z&+CPgi+QAZQv44@-y_e1%@XY!sj-W3ieF!pve=P~2qpOTFKEk_0P!8)mrV-qjL?$b zSGw(g+<7JTX%R@-n$<$L;q~54PVmw6+y6SrIlr-}b&mZDJa*lGS{L=wzZVIk_HYZt z3}sJU)BfLP?*YL5e$2Uc=bi5kLOL_R7v3h zsJo&w#0M?{kbKM8;NbCRItm${3e5U*e z`S?R1X9x5ygltU5^zRanXcux%q0i2ij^D`kc2!dF{vG^3%^%NQpFf6w5mEiS^2*#d znIjT#>vIn|zo@rQV}I9uv(HL*V}M>(wRC*#T5%ur*D4c;=_BL28Xzo?vo)*vB4+y6 z35Gv#JHubWR^IJpzDzT{y{?U^cP_z4%?AlN0O`NR zf}K{0FYm#V6$e=8C2AKYSE&NkJtM+BpubD<9^ZMa46r9@tpoROx!PcD0uQP}Nc=lh zr6#y_etnWJc_Yo(3hcJK#J)4l*Hu9W)Ik4TlZvCB%SZ6^ z2b^9hWy`@E^xuoG8LzrTXz$S(%3@}E)xYY4*QuS{p108iKnKsGFtOqA$Pi#n(weKS zyq~w(xE~0<9^MOUkCi=tjQYy^y&Fw`j76hqb0u0AqF2r53tre8p8nvyYyKY%{T$~a z!MiLt02wb-{mA!R|J!@*KfOo$N_75rZ=Qd9udX~+{`xbGk@CKu<=@^#K5kmQUyQ5M zH_Dk0h2*ZP{#|?Qm9cEovclEU-^u^eef#@kw6J;amU12Vax~ct{{}yMai~16xOt7n zjuqc`z%Js+`QsuLfc~dctLor3t@C^E+VTDvHNE#2AFG7TNjeL@97G!1>;JiMT0$X4zxVx&R{m_L;-Ki`LiLZNHxHvf6V=)Bk%1q%JOR#M40F2hj(@rf*V#Pz z=ba0=TY-DmgC}nY1ULL^D>{LAU5xGYe|@FxtT^c6U10tZ^cJ8LJ%-i0 z-15ev;z26d6hN_~%i6YR2A*4g<+4NgyTUAC%_DMvN$-wQ^c)uIg{{wK-g#L(Q=e?t zGV;~qYFB$*3-hO`F8}#+;<2hL zr*}ug{CdB4sdXO=&aQLFeL>#gJN6XWTuD6QHycEs=`f6(WkZobrV0~1n~8pbNc+<;dyObcNGL&H06&`#KEUqs~Z&3)!Q5qV??s z?O|kszBhEd!XeqChr;tRr3Lx;KC`GeFwK^GKxyQi%RVdCF1N$5``)IOIaAO8 z;NXK2V53T=9|e;CGO9Dp8#tr=+#U3K@Aqrb#>TYUqA~FqyO)Nra8Z4XK$gj~ph(Rw z7F8F2DY%GLxO~Db$MTtZ2%{-qcczc6L&+jp$K3mF2*6kEDvqdNpW4ZA;hMFOHC-)_ z3KXZ9U}hU-ymPSIiTu^LWi~4~UeunTz#hR~4JSqpamLI-kQD`&tRK1>4%VWn-Chwa zJPz1#ys+ch640Y|{xZ+k?m$cl@*qMsEbnH6SWrervPG7Dp;WmY@0E8>1IN7>OIF%3 zLJl5+94I#o0r6ap5zEGmeV*mu!Ome``o7sDif4@s(vJj!soPt*`OVvH=*{MCLq5b3 zF|JJ#k4s1kWhgS}2u7?ng77_loTg|`X?~0|-vp=a^OhZ&rFOB4w9~S_)$UTxP_!U_ zpLm6x| zT`D-c)!b~|PX=|knT*_m>4WKtt8mZ`33NkE5Og4XS9;1@oBPtec))kT9x4l%w|X8h zzZ<|NS}>fzUis)HGgErMogKH)jJ{1=x-fY+XD@#2CHc&sB)~C`B2}EUAZiy`05iYK zq}f9%@Yx`TcQ zL{qfTpc2c`7Ojn!_15hTjRTpJ&>+n`QaI78Xrd@r)_fim1FE*kOUD&*+trDc<*HOS zQw^Ex{fVGNp<@iljo?aZeFPGf0AqiFuJ`mrI_6#`vLOtVe6As*>&Tv=mo73ee>^?< z5KjhN;Gv^tEU=(){*t%dpkNv$+Zj!%Sxyxy2XxUthFduYdS-2)ZFFX`7=w4=@Eg1YOhTyt zUg&!vHqLo;RN=spI5OM;LL0AxozbuHGKXo@+!{CiJ$*j4ptPae`inpdI#e+CLp&Hdm_u!E z-V4p+zdkYJ8hZ2 z2n3K#9O)=aoryR2&He>f(d+yLSN*O0naK|IYh9~`WqKdbS&IAZtaY&M4+ix_=}fob z-yjE0jXe#R`u=o%Bkz^y{s#iKi~9QRC17#Hi|Y|8dIsDY=VG9H!^Jqkj~G;t55{jNhJ?|I`~Xu5jkO$c3jN$ z7>boRo+{lK- z;1lXt6cDl`5M$^vB+c(6AFK!IyV+hCp{p=CP7RVyXj;$QO;#i$MSl?(9Zom!vmmnA zN~p-Xm~tG@c_M2V8d4Dvk?!pae+P^;W(^R9N=Y&BjS*Z|c8Q*#xX=~mC8mQ|@|v1VdwnPFjuu_cVd z1Xk_nSU$Bh@)xszFDQk}oWEUYzoQe&cZ5oiv}{dp>|NCSnAdz z7-XmQTn(@7r7Y0447*@_Dm+SCKHh{Da%BUwaK%?bjVzV}4hW`5cl8(u8ONH+C%ed# z#822)ymJ~w+9k1y>GBTZPdB!SQn%Q5IY#p2K>H{Zo0~m#T5)z86(M+em=Qp%hP#he zV!9AP8@C+4kjOhKxC zeo=1y+qU_Pn6znv_|7rBy-zXdYe_sI+Nq;C?z7Qm^oo`yEB0aq9;l2Xz^YB1Fsel~ zu#OL}6$)lq&QaDp8}ZDJj+vr@$oT1+N{1Agnc>n<2C19_Z=w+Neh#^D6x)u)70`7C zMG}#8IeAp{waR$|IExs>+l}(roaG7|Rea~eMI(BLl__Ivre1#ku)cG^;TaA`2u+-& z=7Q5E#3@iFyT97Q_a2PMS5JWag5OdZ4v(rt-dxMeJ?#P-KqCzPf_N-W^EnJR2_6Fs zmnu)83&9k|hORfYn!#)?`J{tEqv4(g;!#bFbwO5iLVVVS35VFjHygz?7MrP!iggDo zv5hHH*NEJc_uF7}AOr|;tQ~w6}Wi+f6$Vu^j^;Mi>2sb33O%^>QHHOHs zvIbY`hLwoD1zD~R1VaipX{n+5%GTe?XaB`vphk_Z}IBOSRBd@3?<)ZX1fv-Hu8Wp7iq!Wn|V zh>l>MdcS(6Jd9&9cheUvMz3r^!B)OvW1pkULVS%($`!PQLva+Na0q$;2))Y(@tvj0 zr5q48`)^vVv3tr6VY-Y{cX3bw*kMZUd=RpgT^r3-82K9I+NTL5{3AggTpeZFx-mAn zd@Gdsxmhg@2TF{pVC~sDTyMs#p_cLXMAgHn!{mYE`okN*!$V$Cx%S8Lsm@s@?~Y z{*L_mX261i7*QA~M2F#d88{Xeo*)5GLS^Hc4G+r^k_{Fu(-YSpgB(rhu$0W5_`U_> zhzlDhq+rpAJ3#)OLUIgXBE5TiSxujuVKJpF7hDhV!O>Cwx%`;!1NEAJMZ0tE{1Zy&) zz&tSRX}FEscG1BZOLxR!a?fIWhf-_H{(f)?Aw7u9t7OKX%f*&%>>NBh1 zNl{%fmOw5;2VAH}+;FN}6DJfks{m*wHgZ{HJZGU%W=%b(6+_IH3o&VUZMbT6 zXck3mOk+nu_H;M`y|U)4arMQq_SN<^HmK?~`pF6(*kNOa;4t8xTYYD`BJ9jfU4-_I zWOTP1N^Zhb=NaTcpCO!c@ES+*AN0P}Mebp|D2_;4GLz@XAw-RB71PS>sc})drz(;m zS%_q#eJ&+`Y z7MN(%be?uVPBW(u4w6Z|x|0=h+1H~62Jwv_&?MAI4-dxr6{f_nw|Y1tD?*AxY0@hT zZG;qtv90h|6Xl3Hc{s%Q$1U^&ugMvlhkrI`gSnRF4_qgm6|uALQylF$M){l=4z0s@ z?71%}Ci64y4OwIR^~M!l7q?355}&-_#+yv(7Bj5pfHtr&`3O4rIJ`aWTU7QRYMo(@ zG`{*cBur5%6oF9y$D&kdzc_TW-kkb?Rg~2KawzCQ zEfluelg*kqwpgGkyQ3oMKCD=ozITM0oTn?KTrQ`l;J_U(QR68M8Dj+Bpmdac6l?>l zA6jnw^cyGk{F)oCSoUFXC*KUp!#Y5T)d$Svt_ck=3K&O`kJ^EPK_HZH?hrF~RL2fw z3#?E%a~GiP@=x1+LO|I)p65s`#u7SwD?OA(jwJg&xBJn1zo_;^ecCYgWglPLy;hL?paelu0{8 zq?9llwYHTxVwR7V1Vv7KF1LwA)WMx1sb&Sq+C`x8Bb{s?Xf=#Kr8m?vIhsR>))DUB zmPy|(8)tHj81EC5QB7u)=~43#g$PUUE`*O}#u@8o7T5}D;JVn`X~sz;X;CJSbG1?h z^UqtbD3d2T=NOJ4uOh;H7OfZyj_=u$9;5usY!|_jpU;=~VGCT8E;^;p|9u-Ot}Ok; zf${t!((v~>`U#YXb52nwq6x6rkTx0$Di@r;Azd5 z)tg*l1y=9k$WcQ`6PZ+FtUveE07E`AxB|(stT~O%(|9SuWWVk`z`>Unx6(ac5T{5T z#wuhKt94*w9jtG!lC6;-$RE!>&K{>MohOykH7(}pMzeh&aI%vRT8|M!N=s{R=?{k% zM~8Z|az@-{ORf$*EZ1x!RWwL!Z0D0aVQ-5_Nku}Wq;$^`XaQlr@;pmckQF&00x$_U zf?#E}u)sCu(x1)1n_CA<{zM!L$1!_<*pyT~h`3{wA>6mG$SRS`Z+9X*L}}sG%{j99 z&^}8{nJ1r0t<0eyEN}g&*YbVy@sJ?DKE{-04|SB-=&9$b(`|jA;t=lHSu3$u#wNC8 zb|0#2xVGastA#k47V{mHTn}eI#!sV9)gJ{~J7&n0q{T92((`;Yjh8$`VZNCY*X&&( z@1jw*0$jL2cd}A6HOJ1d*k$SSiQ(wGb4u@yHO%;oV~B~s{y5BKJoiHr9tIJ5Df6G; zPFP(aI0eZxija@9c}-OQkjJJ}jG|hkjpwdQVpOfMp@wL-$YIZLhPDHWqt>O#5ufWH zCvX-ZD+hUR5>t|{04fpcd@JWvLYs&>q@{z4hu7FO7twi?j2bt^gwWL8i}0F2T5we3 zvui{m2CBIQs0px=F=b7g6Y?5rU)r@0s@9)sE-auc>0HMIj+6yTXO*Nn4J%s83(8A9 z_RG+*ZVPpv0zSf&k5iEvbF&~~c#dlXQ>c|ykbkJh!=9)1&ygRVj!^WNgDTC#u>+i+!OUwees;;*Uww&}sqJkz)E49adg`ZhYptkO z0lF+TWm{CXju|OZj6IB$2%dWXa*h*Ol39TW(LSr1zdu2T*E7s$Fj0#)ha7?GCnr4L z5nHneS2jj&J)M>yQuvZ3AkuP05yoY1{X3$?`z78Sv-K<)^>sbn5NE8GwG5$3HH%rKtB1@KEekfvLTO;R zF(z?{L2?u*3^>U3Jq#G)YXl0^E4vV}o;Fsy7oHwhevcG8IxzWB%^wsocXC#Fc@)LD z7P68;k5#DkVUdVO4>#%wI!K-<#cQnT-tcPoBhSlpy>81xEL4Fd~(ldk}g`> z4gWfp@(uGhg_^h$@EvCY+#Qp=(@aA>W>p! zET#8>jE65Rk6o(9rAA;l0Qg7q682!6%8~z9o~$5mwTbt-x=>9c6~U550Fc;&S69R^$@ zAvGTtwKKK|6L=hW!<}%kAJG1L$}n*6ABeQrK#iBwe;^{WoBn}t`K9g=xC>y-$QLH( zYt$1Meqq7I^l!cMWm0D*}~&vNaiP(-@LzFitTVstr?$Hl|^C`XA25rTi-*d zbIg0(7VN!fY#Oi=;!xfYRnULx=tm__f}~PSK=oF_uUm_oySZ9>_-YZ*UG2ntS-d@q znxtzU8^^-IUr^^JA;%2c4;=v$1GO_OO9X8GRhv?$?~-z!i_xjfXNQx0U!wy3_g!A^{cy$U%a+Gc)@oqjLFv+T@87YWG$li{WAr`M;3p&g%QAJmGQ zmB>KlFglvcF&jdcPRkwk_j)ji&7$NB$b~z8?mIH@l`|X(6sea44C4#tK5ynn)GUv2 z4wQCdzfm1uW8Xhs9U_B2z~%fc5)Wu@+`yAFz%0|P!K_W|=@{G}^ca>EXMCv_&~dyV ztt4<9{mJFmVx0&sbXa)Sz*w+GaDw0(@eg_u)P0la4SRGSa?cyD@1+~!X}t}27lFe* z&Lo1~9HK?;_t0l(&K5S41(wNmk`MUmQ&=$%==G@~8qba|JmZDlk2mj~306CgrxY+E zbApJ?o|>IzR|F$4aUeM|?GQE$(l`$VTVMA{Ub@@TUf0IDuWGn1R}t(o5IjrIsS*?n zR1RVq$)K+A4!53jqb{GV`~WPpm&do;rf)vL6*)j1n&qVEXiF?JdQx*gK^)6t+}qJA zvZt9&wV?=?g+D%pMdB_27bTWy#r(-l~%vd53vSr{7WPw?yI%o&@*a_FVE?$jE8dv#tv-sXnTm+^nEy zF9j!-t0+c0ZDleSoYpTlM0;f!JhcwiKv6p-NIq)-9&rh{EJ)@?CI?^B=|!358~vnS zY!LZd{IX=$TDCb0p{;ku)S(Qaf?H+S>fSP;`Tgm5JNVsCSjdzg(eEF19&qEZ?r`SP zNsPQbZpMO+M%64mC(0Go6&7+Zpb_*tpo^)Cw)>OA8+X^69`z>Ml$Y!H4q|ML-OTXN z&1_;VvuVFpVSZK;tn3WQOKP`!zTMntMt$M3?YyU$Wh~S=Ts;;-AvHl=4@XiOFbRQ+ z6~y&zvoY9@ia;{MjIfx1-6u4msnMXz?3c$j3d?R$+(J#6+gP3?YZ7AZ@ypWAFGuFI z*a1$xSdUTE-b_YOp-UzBLjOTI8%Bhw%);4q%+tHObWwgz;N}jAd98)L2gkGvSu~o~ zHs!3}-JJqaqT$e(qhhbq4w_67xOym@EY1tReryt4k>1^EFvv*O+t|55+_HPw8i%tu zr0iS=^-vbJY&7KQSA9*K`>bG7(U`O^9G^?#VNdXS#a7y>Vd=hHnsrlRLnD)85~Eal zdH1dC#pGh2j`b?0*~U9A-SnF-|o)_ChcSOWUY3#3}PG>P0;b~ z=H-p|RY7mMx9$y|ShuIc@?vrLxBS-6U;WqBzidsf)kwYM=qAz2R^9SLz)sN&GR#hBf>E7 z(mR`}cHs|#7Id{kC_ZH^qU4BjU4KRohrQii zag1gjS@-Mt-nD6Hg}qT)T5KB|&rBOG=i7sm}e|F`5TOKQGtnn~!b-fyKgq?>W8 zWya1^>CMRVHS?n(rV-x#@0j1uE6A_#pulFlol&oKe!Fg~ht2qkufM1~?o|7F1*wzI ztHF9Sf!}(5cl6zZ+3!en9^sQKSZ$BWP~tEbq0+Gpma(viKHJA%KLqN6};25z_VA%vrK?#FQl;_Q$H^{9%-1C zYT+Pc>gZHn>d>3a1LCx0eP)3egJe;{kz^x62%$<1@IjndzKRz7Cx!-6_UM zrWzIsmLvtwSpA3vhF4XT8N19f7n^X&?QlxpDT^Qw_!S}DDxPuRANCJ~NVuG@nC`u{ zZJfVB`&P;6>cx@~M`C{r*Qw@*TZ9E>W;;-aJp35q3e(K|MpK}x!}+lybU(b6^uf;F zSbryR=}22a!-VaLHFZd5cS-R%KGS|2C(Xi(e%^8QN?7_(0l9@KjKFi9G%X4mCHsyK>6;3(pu ze(Zwf_n8~f2E=hVHdViwGq)v0I4J`WSQnaBEi?41fCaCAIN4Z;S*s2VGlHsE(xGBS z%3`x#2c+EhEX^^FX=~K)xiWE|y+1?xTkc!V z(I?i-FAnGkj_#|(e#$b#sKK*E{_x0thOhc+lwiSsoi70Pk|~~pQeJ62OnV-U{gejM zf4sy%$kfxR`hR=LG>>!31)GI7a6%zMe1;xb~+}$9Cep;jie23 z03DoV%k*M{eaH~&Lgh`XS1WR@aP)ibE)vM9~vX=Ax{VcWcY2fWdN1{Q6yCg z(>%O+qcqjz5@0Enj$}?zDHozWLAUZS&&JIEI^y#!MM3@^717-+-B?%%ZUMZ zM_yc__#SJDjCnIY&b`W+Tw)9oW^?;*-@aBmd^}syHwi$O4bxkA%);49cO^Bp&0?2v zUj$56V|#_$$5n{8PS{EuO=9F2s4-tvBEL6fLjYBS@f~v$XPg*+#3p!H!wT)whr zoKt*1g*7X>;+SMMYOY7dkFLmS9ulUkRm93EQlZ!k>cLfV9Ibvprv^Hiu|``QccV+5 za}oHoj3e!LUZW7#x&YqXYB`*p=NY8@ZyD7^hX$Lc%J9KJ^P!qLa5eB*ps9=e$=1$% z(Qnoj#-hHDzBBKE)%gA{JW1RzmPrjCrPc$4Ej`-f1t0A#fNvaA^e(_v-I_L|Of1ds z6+#?aOyj2qV-kp7UnK8{nM9UPp2!#;+&}j$5pcvmd=)bAGP-d5)&%!O%@D$di*R2~ z1omdc#^t+S7mU-I$l5H+g8P-+@_LpA#Qf$@>-Md(>X*;=d6Bohab*H?lfTIHRA4QE zYdlg+c4H46A_UV=pXrJA*iM064r~8I$?gD1dN~_`}OvO{@J2zgPb~l!@XAQkF z6srmhi}OoTI$TakPm$|~6#jtLOfHyTn( zNJmLe#Fd#GO|Icr+1^}Lv1-mQZK+I$V_SPV50);W6?N9VgK>E#Y;E^NYD$#F#KAH>i4$#q_}v__`Y`Ptkr)V)z0 zM0Co?a6G+N+@ou@iWh_uV`a&qI%W9yo_YF!o;!0eUXPp7U!>O3h_fu(y@*4g6iqc4 za}Xg+rx4nV=I#r!TUWT9(!qFWnD9+UHSOyJS$^l-_mk9g9&U??<%0y`pGcIHUG<55 z1*c+3<5jk~J9`v_O5VGESKjiLmB3#%1XNzlN%f*n6ZFXu(GAnp!`Xh_yH4>hC~x|a z%JXG^BBziSwjayh>_^EaLX-zbTs{9-QH;`khg;4gw6kX31R^<7T!(0bbn1zb9Pvrw zYZg@quPhZ(*0~3~x0;_|DpgdAis9#1h6q`F~zfNPx!AvT^#4rf(f zEVH~V8S7~g1aJoA8Wrk>$NJOzum+;!5P#Gg; zEPc>WCV#@qZLzyl*JUo)0@QyoJ~~(^+GK4Uy1p=aN)7SjYZFvT#1zx1a#w_0_s7y-&?72G!7j-1w zR_krq-oWN%5tue7ol}&fE?+$f$=KC@p=nwMcmwWA{-jROL!h7t%^5XEh&pOS%tFFl5-@ ztA|kT6v`r#=dkFD;61np^TXPI}?~_mRbWMlf2h*`_oHGtcFt z1Qn~$=rQ)CO(I#RrD56aLSvkim3TgZ8m34eYn}IW>(NeS}iJztM1(ga}ONA-_h+?QVX-sM6_* zmu?)qz?}BjLL1U%@gA9>Z&|H2!zsf7rd+wbO2VbnUSW~-?-9qIvZd~65c53ZFNYmL zVD~)B1k}r4fy_ztAyouYz`D~@U26_h%g@d-O`coRlxiNLCeD1f2a0rp(bA8<$Ko?Q zhQ1Bb$;j>bE^4tmKSA-|g&Gt7up2q(iszRK>P?ja zd=q3;b>kE(JAGHsNqDyI7#w%9O$ou8c$fbe%R~oNC~g=M>yuLv3M2b-uO-xB!`4lo z1TF21;Coqc1So3eI!*XG?y%=|E?XuFG%Ce*^PKF_l;OG`7n>JRYfOI#diNZ9T~XjX zD!m5@Im)A&9hS1uB;Re0?A$|55>wXW-WN0)xEkbqon1Bs2~4k|0M<>PKCm36mv8vY zjwI<|$LF~n7}1JX{&dr}S;o$jiWgL9cyN-I?!uofj6u(j7Ma)^zayNax;xfmuc@P! zG0e+E7UrJ+lZdUr>O`VO%z)`*!Q3rLJmvNhy zXw~6Vc%tST(C<3KT)5=VTG9&sCeH34W2tABtZCDv$f04{C`m{yG41HQaQG83vqXr- zi62}4@yHcQVP;lUuE)nbvpS0&Kvp~|wjlh_FvNm_U*Lq>bl8@MM=&(p|M`SWDlDv> z6yS}Ax~PnB`dcEZlL(~eBt2$nU$Ve1oCgBrCNJCKbMz+;CHy8@078;wNSm^a6~`*j zD5`~?@qOW1(3^o)19oE1sTU|cbEU&R;C)n}fcPS6cf7(mZ~$Z*GN){@Z?H@9!cZ6X#qhreQcRPEMLI^Fz@3CPy4+)z?z-d?ut z#X&R2``dI=WY9o~7A(V(fztfFSZ{i%-u6=FdM2g6^Q4Ho-4BipexY4BBg28h{eQ`+Y^X^S0Gn&Mgvwp}~ zXtJtMzI0cfwT&B)6N*v9C+gw1L1)~_43I{wTggWn{vcEMujV#XZN&SsPe`GH8S6Ac zS~$tG$BLM2l$PN1tUe-(V&atLb-=T$tm}hg$AHunKlsmAGC3>`y55%=Yczq}vCpp= ztZ|da!bj8AAQcFz#X&g{?pM5_n{`FnqQ!!B*ot3F{O9(YO_fR(V6g*Z(qwy1z}VG- zeP0<3Vb?K!=Y*eg z_NNm2iGy|9#9g0}e9hJlGjIB+XUL*ziluBEn^ZslL7DKyz!vC~X7>Y3uL*vHFt2Wk zB_o8fW8P{Di)5|_-f5D;a}>o&`R!Cdhy?<5sI>xJ0)8^*1jjDfcZo*A{lL!?c2$zl zCfw{l?k6>0MXV2Ib0|sAZ;NzwBsou*)|eKb(l*`C*yd^FSR(J-Q0Da(sV%%>W-un# zKTov(TFUu6#i?N(8ym-cTTI4dbzx1Fn>u=1HI3qL8u0l0J}!6o=r`=o`$B z`u@6w;!A>ZiMynn50vKaw*ca8919s|-g*MhHkM|MzVpz3Jq!2^U#G~IjWh6rLfPe0 zgaxz0AwLkLq*@&uR&ZefG2BQ)?-Hs@9zflAX-6E(M8UKGYvLk(EW1hpLP)_QN?U}B z=L;U%opGB0G$?opC2=nw%C5uP7Px%KZ`Ka$b}zwS{AL);Ny@xH6LI^>EMmDC94?@7 zpI0=u9I_1^RoYlSjjfVoZ}d`<0~O`N3I6s=+0csjVm`3o582M3Vs4%QD+A2eEs{@n z0+oW#L1Y+2Vid(IYWa7}x>8i0chUq0fNRf2NOG=N<_ZE7IZs9nU*>zlUnG3IC0ogY z!`Sj4>V;*klNGbd#;P{LiX)XNQgY` z+_&q0Dq6-dyJh+H+921N{4T-mPthv4vpJ5!kzwW(>F&WYyoTCgW+vyP3oJ#tjrbw75+`o)zQ-Nvg26n?D^Iijk@9=2 z3?}S%KviZvu|tervr!Z&Pr@L22s6_Vp#h!By^KmNNA97`(jp1v0$LpnLVjb1-D4O{ zm=^T+V;mNI3d#P<9s_;ehzdtrs4rqtFvA6W+AMo57pYfv2GCg$o~=>O>$dHbNiw}> z@h(hj?ejSdr0cFMOg89u4U6!%;e&m>ind+J{u+Hzp9&vLem(^P0>@bZR=05x9`ZV1Y*G z!vSG-@QiZsXyX1YLjMGscq5fwo~H8YE;@ZE%V5vRM4sOCAeyv<}zaoVhWiipA?dn!D~2fpS~ z5)bQ02j$2X4OfL2{At$AnJB(Z;_?X~JTzG0)w*W=@Rrnj&qrKV0)FKp6rNVb>}9@nVrp!)-g zieSj}q%+kZB4$Y%N^81SM(V_8FZE`??~t^q91#&ZvOlv8YIcLn60o12qre4Osc?p-W(PR-Nrt zAKaAJ8lo~efrWO~?z!>Xo?ULVJcH2YUC!;tV>a|R1+`pFQ8Ib9 zOJhWB%$wI+JC#z}klaEdIYvhNendU#zKxPqWosZ?X3)* zjBAY9)S6oRxI^>hU3*_89JW3{&|ddSq~2rpRYvYqf*YqsIJB^lJgbpeYgU6a>Vq0Q zPUG#uE%yWwv}F5`;Mp{c)-W1dTp#TxMvAL!xH&WrgMT2tyE&21HhkFd`Tj_EeT-ZB zm1v{O#!p)>TJ;x3lMr^5qEcMZH`Ohp=!OQ{kFkY=Zi5&bAt8%Q~wOKLB1cx2doD z6glJwAvaG{1-&EbErE=63bJ6i=%Y(qJWY zTuezr`2w2?uj<R2g;heKV5z%T?7)ldW5^vFVLCK|1d6_Ct zre37Er%>*uMd#;D-(Pq$cMrLwWj71XhLMt31R~8y{C;bE{Xu*hcfZSDqfr@A$tlI? z(E6*&iW<8rR`ai#p(lLdqmU(F=stI^Pj9^Hv}} zLgho)BvHqTA624G8QsH!mxyFRixZo-wR-FbnUIsN3*5aWdJypK+7GAq99e0q&^5Zs#06k0$ zOGoB9DJ0}&)ol9b6No)==!hCbV{Ej-AMltrCsCOj9MVFTiTqQ=d93CyO{_5-ZM8Ce zSYESFy)-w?p&Qxwv)W%L=|3i;1I+d}TxU9YHD7t2T5sn2N6%x!6{MAL9Z_+)pdtGk zY}R|1bTfU^?)$ae8Mr<@@4YIh`Fe@h*=XL4`ICUBSBJw`<> zLKwzF@+l_86mN(7wd1D{*?F|QeioW6Xh&0#A%#pWP(>-*-|_>-REnV2etMq9?4Tal zkzC+%ZzN<&9YrggDnmFKxR0e1h-`1=DII|h$TAz6wMQlyVd6P(gxkpRBJT3qoM`C? zBBWj{)5usM5z->h^5Ot4ljQGgwC+@1UtCQ2ss1ueoS`JHL;t7%gsX$~{!YYW0whXF^F4TZPNgYT&QNc(@hid}4T@^>d4D;1^+1-GADj9&K^CinYrF z;N|!hMbN>~@qLS#iV(-WNn78LqAZKz+YszWbGAk!M}m;WC5M23CuajT-UuJL*-jB> zM;$$GH)lB%SbAQ?--~C+$N}B`L`nMTZq~L-^%?MW6MFIO!V3Ik@O1PPQx+ zFoJ3jW5gX~+2!r3x)W1xKFR`Fk!7Ov!-%u&x70aiZe@2=ueEMNlJ>aIuiFv8FQj}| z3Ha*aThts-;NRu7vz_@wLn7^?&?*^ZR$F;F0g>iQ_1U`n$x!qB6V~7sUYPAiVM!2E zFIc!b@y zu7c`Vjz2Xrj|biNSJdfv5BUGg&|DXMkMF4hn0*kq+-APha^h5x{No$$o)hco+)aRU zU%n^ro0!A9RX|Z%*Jz;d_-iIu-$=$77y0d=QBS0DkS@CH7xB(T_CF9T;LPIpK-rg` ze;|bZff)P;V%k_&B#G=2EGzmC1g`tS~llW&V9ZpU#UqSlz17qY={q*<7w$rNGADi)C^y2<%6TzGJ zAbkIP_83T;YRHdYf;4gihQ@)_#Y+BF7yA{wSI6Usd%a;v!4^*lRs6+*WOL|doX|t zL-l~c)bj*^vcH`}PM8T!p2});L=77@N=(^>Nl9n9CVpQR z^{p=4#h$U(pU;p=${QfZhYaGayx_~lNdzCV+S`xxvn;~h!m8#i?{kxfoT0h<`lyo!d*FB=M~myo#erZueQE2zxlmUFszei<&+rAk7hS1+WZy;^)x4T5au zeAzDc>V(LZB@%qr!`C^sa5z(P@HiiapD-1T_D`CHP(?8%6d}i71f#V#w*9n6M;mDg zBAHlhsBF2(RbvpZbt0l!$gNNkBi_wH>D70U4zdj2CN(SYbb3GZV1l02)XgQX*z~SmaUDkEUJ0!h{Ch_>1&|fplUNer`#gA!;u8% z7^u3ime|F+A!!WO^GHpuUaOSl@|x++a>k9FWK<0C2p>+DCNYi9FigN7R&cpUN(;SF zIH}hhn@q%kz9|?spP)Ip%Lvsd%KPD=Iag)I_*wYJxiEXsF@DY;>(T!iw3np2)xmO|{*p#Adw%kN zx~gIf@P3ep4D#jomDFc@GXKUEC=!NF)}JNhw30pS@{=90Iri*TqH^n_JSDVQURy)E zNaRbR|6WOA54k}$hMVSvCaUQW!)5~jU%OOV)~q5;2|&5s+3wBICD`SSVELu#3`3%& zpnxiqKnj&nO&i|b_m6*?G)DZ3D(eR5%KK7&sKvpOBX6t zoyb;k>4Q|#QYYolQM_1@e3(6LwB$j95a}Zb4vrG0#92>0b8dO_3v##+MDwPKZxOuB z)BcW8d>O0%i@Ue%iYwZ}F7e>O-QC?S5Ug&s*fIZe;d#(A*9U#@5qDEY~PQKa@?%-&kMVX6%%)mOePUnH9 zx$?sOTfw?$`mDE$^|IrxFks4N$|0-9-aG>5Sbb7bY4`%Sd3d-ps^XeaWP~L$L%{wj zD`-<5LCyW0d23@)kHQ2sjRq&ha`%+Mw-=&JOs{2*j5fD~qEQeH8BAn26ED~LSoPg; z(e-oDztkJh1+0rWzm!|ZuJ#`8S3E$O+}=G%Ch4NQo>977TC3gE~mp0$8c$4|h)*Tx?;(E)!Kh-vcd@iRp6H2zUFmtefa=jdFJjlGA*lzF>|0IQmHEKgIV~oSBZFjag?sZ@bedhnWx}J_ zU#(U%-p>|gUzI}3EsX8#WM3F5fu}xGKS1w8j^nU6!g1#?{{ayCyNaeym{S&+I+FyQQ=bhfYm6V4WG1Dp$m!hD zS``TovYcS9dK4K1WskQER}C=f8qtK|N)N>ywmTHtQq^GArh(>QA0kGuNVpZ%%GZBC zed;BzUWRpqTB)-pYr7*_%S9p!xV$EjKVwjsRC7+$akV%jjaC&j)sPh(ZomLfX#8aP zIg*(NaVY{27yV=Qc>#Pj4KzW{;WV;-9_%4JicH)lq$01l*N*?7wnN7&EPXPezca9n zEQ4B*F*LwMp2nyA4SBQ&^{Tp`;cTF5bk{&Ci4wyza~U}rIFhq>&`x-8WsFs#Pl(6X zMPkD~oinI`NIkN<#yE(O72+!yXidoFinnxd_Df7@LInt9;vub2rDIGE!RhtSZ<#cU zRHT&#`6Bm+YYIvjL=sz(di#X=6jK-2ch8TkQ4=~3QQ6QcyO8N{WgaD2BstvBbX7g! z>hR0W+(@^Zeh|<3N>Bh!_+64FK@tv)ejgjUNCGE>Ow?|gU2IYxdQT<0Bc&;-k%$ys zJ9wJ+?O6Cv)8HmIA{bex(Ij^z;Ahp>xyT7GUi?}gZSrStwcT76w4((pN}Gexy5)XW zQ{Zz=qD=7=6)3{(o~3)uMtut$3B_U`DeNL*uKfKXHBs!QF#RR=zs_GhZOl`K#4zfJ zKsE6oMKrR@pzp-8SFd+gcN)wUs!P)ZDx+cJY*3SCO*!Qv!$Cd-4UtPf=cgVt67*>- z*E8NeCq;Hf{T3D=#v4@`F+`IoDel5p&l6JI^i?#|YsCpgke761y4HA4Y^?=1I3K}+ zJj=&GsGe@{6MO*F-cbQ;AQ;0(nquA%TpBtRJuWpw_BZ-DoT4o~*h1|(N-Ag8l52=< zL(F0zOZ3=)6M+CtW7@dVY6@n54`(6Zb*+a+lBQk6FO_Nm)dqd1aNC4K1V3a`nSv=d zF<*@>3KoJB13u3IE><;));Zy^AW?Bhgs^wFytd!!A`yXELjRm6b9v{q9pT5-8az9O zxT!JCD6P}C@XnARmi;4Y?-U1)rM4!K#@B7!&;shw<6xTx?o2v$TO=oS4D@U;H8l&g z&Zz>;-s+dYxIekzj&{gQuBmnKj)JK3Oqh)-YBJX=L$UkSxG$^rATY&0O~=zA)9XU| zogFTF483R4>+;kLY|lxXZ*eHmqK`S+&M8G^!jtu86p)w4WH7<;>pWmI-Nx#V4gNl& zIaNau_|R3B8^NW~!jk3VL(zw`dRx4`;-xCC$lLq`LwAn zB^fd1rGgr~7^A&-M}o?c7v9V#2OUqFu+m8vlyz`)-t#i5zo?5KYJVmoh{AIoB)W9B zfWJN{9pHK&vPkRtDl1GT6ta#^KB@}BB}%}+!NEEF*Y!Sz?zTxLAyZZ9lYEa%2pdW4 zhyYx2)F3-#R!?{;lRJDM*OmcBnN!KrpnDMn4dk1JNus@ZtI7=5p~oo+%Uye8518PQ z=Psz(WDce~ksNLfL)Y@6D16?lB-+b$I*LyMQ>{kq14!yibtT4~UMuZlw^!}Evd@wO zckPLyXp1DnXKX>lLr@@N+$N1_1;!|g9w4$$J;=>-{71U>f>h7Aj#0)IN?={0IWcCl z!5SLMu7D~e&vcvkfOP}pt8F>ec;#WJLEXASl)M(H{zWar{d`SJaf!f7by zvy$MpJO&g1H$A3>~M=;c4<8d}*2+XI`=;!P5-Wi~nFs5rUnwaijKf3csy zXxz(mH%*bQ(SJG=G(@na5dPG86Hx;n{i7_xTNr`a-YUXYe#sP@N`;GbRm0m*fg`S1 zY3>EL)kH1JrzKmIGl!ozrO*toZ3NPfcwKae#Fa$IOy4qz)6_nS1o@z8b!Kq(Ctr3+ z`5Z+d^Cu8&EU)OnWn}+bVz~fSNGp-p0dS1_oQC9iAvQ5_a*S32oVarBSB{=sO2E$& z=KNdCX6p`5Y4n7D&`gKkey!9sN9Y%{1B*=z-l~~B5729q6zb--GyXB5=Q5$$#CS}= z7}b(Mdm->ZH^AbayptGCQ<7MYRv%+LHQ{0x8SPaiI%1JS=6RX!h5M7`8@oH;In6;o zaA-j#BBRn-{gNjb_-DVMHacOombEi8ag+$j`6QR9mKt*)(Z^)doMLW1jYPF~{El&Y zxX`Fq;)7+?+RnYuK;iX{z+`~x`^)EQ9t%2Yg#>tdCSa1NdQ@LFk-Js%@Qg_(d3C%w zA{|~VB?5N`qo;J6pevca`BmOgl{UK1l9mL{M4~Y0`h#+%9DrI(ELxvHaE5K%#&Tew zIUD9lTKD3=RC}&DBk8+tmO6@ZXiAw(FWp73`uxppho9Cu3N3nuJ!6kJP=gwbkxYX#(Ugve-EH+HCL`wVDy z8`;K<2nr`?G5jO{OjZF{tH56B`$(lr)U=|1bLf`d9+BVEKdzN*eT?LK{G~&w;WPQO zk*$X5AwM7c>>}{%X;lSHO$+J$T-=bNGg-HB-yhvoI?bLoi%Qy4-N@5(Raq(lSH&2g zq1<0s6%M`6j3HSkKn)nkfL9h|Qm3!u6~pH<+KlFn$)bQzOVYu#itN_7RjQd_&%{#4 zCiFY+o?p=6E<9-$Uu5D?NyGD&ok3rt9!ay1AJ3saPSvXkXRS12iYf9BFcGEsRh>tB z9qInov@Ql=A?n7vQfx*m#V*n%v9&H7H@wgHWy+WFkPw!w#z&JA4%O@$RKQyeDI*Nu**4VMVKiyz_ebs7syt%S*W`G;qh_@&r~lZH82n z^$5A>$l7d)_(+R8#zY=!>7TSe563Ij9SbexT5^EYzy1(h@ss<_$+c=QVZM)dL!0mF ztz8N4m`7W}e+-T&tDnjnbv8XdO>RgFoB=zXc1RR!Pw zSK`bJeb=kROYzAp4oIwVvlzn(S0Nd-w4Wi;8)Fgh_jP{z=401d;CPIF#F#Y@6-?1V zwk$0mBe|NG+S#&NaUS!MyaDqpu3(DNZ!4x-j9@&+@+; zhmgslS;C}~lZUcnsTG}<$CU2|AqK9|;)mmpFO&_=?=2wnblrdXb9#+7|&`tQwvxxrB9^tlnVKtWb+U&c%- ze*72iH{4m!f~&+q4#`9_^~S**QSi--I$>L7Dz1hxjdWCTl1(*f5fUZM?Ip=@5|>~F zonhw=KEyK`+7=^!fsDSPW$~qQD4MK!=tZdo^iJ=kWXU5@qFXD}o;GiCNt-BBNIh#8 z@tZFoO=jW{vP^g{Y>X%k;;1d(!WUeQ|2z@sVjLH~?)*7x?Vp|Nn9BH>N`mfnLICIY z!Nl6hbqksF`ATq&ftF8!DO7zGHEy&tIUumIOxVy2u9{+n7{`B#t|8n98b#j>lVc*- zJxyiwDdU*q4sL2XS}FMhOvTHO*#rj$%qBF6V$s>+$T;dXVFzye_R69}YAkd4M)2D} z4J{4DSA~!o5|Y54Ng?EvcY6Ist_6^(&^UzmYn6Yo^-;6{H)%2#4fdhpo`_`XZCV7` zENxnIbz!Hvi3H+(RrwK@mG5XA2hKLv4SXt98K<{jzQI2gA0NHT!#)pOCE})`QHAeT zZ+g8_IieiGdg`|<0pW{mJnymMPH(*TcX)F59`Jx+V0pzGOpd<$#Cc1Li@?Wpbi4f7 zNQQPA=c--sm02T`3CMM$jqQwTc@)#)prp zO;S6u&;k>3B^Mto>b9M4&sjOs&E#GG&g6VfKAY3*r=Bb^kv2oOch!861^0@dCX6R( zY7$x{xnvcO_goCaNK!UG+7Vi=^VZ0JbRH+-kdQ#Kfc^dXs)bgW{s?svWoj!|labM| zYDp>ETi~@Jtd_xw-!m=YCxqH$tzk&G#)=M=_eT{;7Lx|ekG(N&!SFLWl zRDT&cS`U;GKQ<{KG(%{Qw3bpfmc)OK*3Z6`Hm10_Sw){u0xxV*`3{G&tiRPYt<@~D6#B-PL9J|-(+*;-C~$tV%j_|l@OWI2JE z4qX+iG;~Yq6zo1zAib*!==p=wLHk?NyU(F%p?4G&AO&z@P7O1Nm4Ro=+~;ev<-pal zyy~-jEV9$24)Sf{_9Z?s?Ny^pX~jY+=s>MO==ei%2kS(VGEt(UZU2nj`u_z^T!Kvx zFbKjJPspahBBHYt>ADb>6CmflrFYN?00f10>LOwGBd1X;c{S7>l*j7FIOGdvS>I?9 z#@Ri+rI{AQkX+mm8)a0hB%U#Tb$l_X*X(uQ#5Y z-u}Eh1$8p>ni2tst3uM;HtB`_y&BOjFN2JGy!i`bqp7--Q&3L5Xo@IGEPK05S95ey ze3I|Ls{T*NFswr9h{RUdu$@~IH6DG6VQx)cdMmo6C25A!^KH&Q|6#jImlbhtO>9^Jfp81%hrh zS_|mLtk@fI*AEjEmQWIf2(~|$;o&JZBlRq~kKRBb-~b5dj;v4IcwU{?fOrmTzW06d+e$R z4#Ec*P{0N`24NjGO;X}-CE%~uzwN1OYk@D#TSBH=A|ilOQA6BlPK9_8b}EouMbs+S zS^2B!UXis4mJB-n!u5*1WWHb2|7o^oF3GVBbn2V@si&7~WbX!=@_i9}c=x-zlWo^( z<;NfP+fRdLLY*ef|L;T-Uc6Ht_c!&XKkF4fmn01(egXZveVu0Z{CAr9?Qh5Vh1JT( z|9M}09pgXo{};Qa&tmZg!zrwPzNH_}a|J+$zh%ARnyflR ze_p_6o4NTp7WXCezsbQ3ORw6SKXk9a&;KP_<$HNe5MHu+_zNHVuW8lpL&@Yd<^S(Z ztDAocN^rj?+{y6_c(AC_*{>(F*n=2H#l9|soXxx(h0R^jQ?zlDU7*>+MNqs+UxA2B zxxLL&9tF~W@9Z}$*F$?McOY5qJ6oMH5=eVxCP*|2%dbaoFZcQQ=`{w9?&uOd%i#)5 z|APm{sEv<1BCQiyFslygmZYtMU zX{r(&Dek(I*j(yrZ)p(eVfN-5JR&CH#vg1yx&&MOOS#w)@UWnGjBfOdkq6)-NuyZ2 z602jBALuUGc6%@Set-7-`9|gUq`!#7DA3ebw1G+^ca?@l*lDz8E%9`WXOch4u+mz4 zX|gIXs8fy$D$`ZWSRwzk$)G{*caV{EG|QpeCR>ZlPUj*Y@wmMYdE^Hg{0yd$8t3PO zEeJfDBiEQ)R5{A4p%kHhFS%TM&^E#M}E(k z$E^-RVj%#*eA*O5)JHuSI>_s_g&j2$`!{eflMFMjn9&gQWgMc z36byRt9G#w^shsr=|L0S|1V^&`K5s82CC!X+za*Q3v#UEW*^zJ?_=?%pGo)5Op-Q; znJ4jKnnhSD??&V3fQ;JB;Ue#CwKOIjl`Iyg>su8&M~T&V=z{%|c!LiL2I^g-OHtt@ z>{7Pc66lxjyn%fj6|HuKV}mqmy!T(uVpcNyQ5E#t21Sdy`kaQ$nT;x4oii(*?+kCX zY{a&67WCp4`W>`hoWQXYQJ;yKhz~$t>mJ?_m|u9W>ocZ1DvJR$ICO-L$@m;C#l$CA zy_Ga^cSWhb2=2~gQ~V$=6;jbk+z+ZXQECB~v4C~9R@-s`qD!!s3bp|jLP%lo!CJQ`Z zC!Y_}%EBCL1Cv1-!e!skB}5Wf9))zs;nln_)}(`mMntnGCnl_1Up(qmyKz$P_(oi{ zqWh*+NZpnfk9`_{CX+H>m1(kcyBV-%O&pIKUa9A6l&Q#8hP6)L!-? z)U-nN<9sAHHEHVPXKtRob-x+6(bM|WkH5Rwt0ZSX5A=p~_*^-NRdpw*K3{EPxN6@U zL`KY6S|G$ZSlr`azwizYdj!(ZL_MANSX-C3_J(?!AU({n{>?}AkS^p3l|7YqEpyIj za9Ew%bO1-|-}!-9AMcW+(`vs^ca3Jbn$o}(z_IqLf{2M5Gm&k0{lPh2HzPddQ&hRf z#9QFZV5VBJ;F=9pUY=)M5iSzHg+&)1OA0ZC9gYg)aBeFdid+e%xR03UtQVV*I!(x` z2y*#I7g_qiX!BBPDZ8dAKeJCPK-lL|*Nhm+sTx1s(hgU`h6bNE$FVMVI+?k^Zh0Xd z?Q5Y8Zam|dUYoocgRX&j^KuuBiP)jMKoro@%h`+|NkoI~%StO19vdE6<=CGPu~R{Bfz0?Cb#u&e!D89=?f(aN%uT>(LMkEo$yj~_F&f)zG~jq zEr~G40Y%XG9`SpK#S-2iRdvLtSJq0CgWS?uE@+R3#-M|zLr*aBXRzZqHMAU)ool5N zQjZfH`W?bk|NFW<1?9!4`wPvy`YBU}jC|eOC6~W+^_HZ4(EZ$uL@_ex%A(gV>NLTS zUtS}Fl6Y%lq|%fJn(hT{jGTiN6N9zbqfWy{lZ1YlwQ`3{yQm#J?2!Owcv*esKnshn z0t1cIq)t$-yxHf{9#u8bp7c98E}rV4sBZ4EzlsjHCNyKmqUcwSS&&|_tLzPuqAZ74 z&w9rUbscS}y3d1V?KPz}QBnBLPzT=OU(33WR2mz20mu@C%F{iU=obx%Gb>ENxfu5S z)-tRPwUBw8&qXhVgQg!$*HpPjn7i81t8zjV_!!112xlzV-eKBQZygQCm}#^QJW{pO zzfBhuPXVkQR!cW&DTC{eap?Na4Gw>?~Altz@reXRN0U-3E4>&T3+NL1)(l z?E4WK1;KI1qd%uJQTCU^5g~Zt)!BtEzIOHkKEsgYhv3(8>fuYbHG)+^x=X2h0y~KG z`=CSY5TBX6T)Z@IPQ<~bBKih*#lvx-FJ6ET<}>@*HzUKZh2XF`=uTSsdkBNjpWv;@>uzv7dx+m}ffeWbFJL_& zhYC$i<0e>w*RJrV>jzFhyaiOB0uib3>b(lA&K@00+N2H6pcVW;kCptow*MdjB{j56 zMy)SJFU=OQz@L5|ahFRup&*fOR>Pt+=rY zb607nQ5W(tA}10bnGu~wnF~0mo<2fye^ZtY1rMFQZIN*+&MScFnTL}M#)QMWJDLnN z=8N8~Jg8Je`0ObeI0ZfJn|1S70ZnUIYD2otD3MIu`WU9Q8cE%f5E;i0-A4=mDprr~ zAoa2yqq(nRHX)6iT> z+=#%xPVg|Flo-(%s-fN_^0(WBYIYQa(K$IxrI^_CPfE7>H2DF=7hZ0acB`rGq++&J zdE~cg-wC>m2}P43!cibu)I8InI)LrWSg3AoxS9Xhnp2$AQmj!Y3m=p)x~z8-y7zNx zc81^AAM$!k`N}3TO*@5WP6iL+ZhXo-bac{Cf)YnMh?=~iQ`AXZY4AtFHgOh;E(OpT zA_+YOuavVONBMiI)6Yp>}_vs39d_7+iDH6a)&tstIY#Ustc~)D)(jC zNu@NFaD_q@O5qPQQq5MJaw8vw6ZdY1Z&4&nSQw0413ja zuP*Q=2b|3_#(Fpdu}Knamh6l*LDlv;%*ESwT^o+@=H3Hed&}2Ed-2xEaBGU@Rg8-j zWcwJNV*A3SIF>%f-4vFcAZ64$nrj#yJHmu|>LJZ|V+Ehl}FK)POOxNcKNg3eYBYhbet5I$J5z2598VrEUL`csL3fBVt$2Fq8i~k zOMb>NHZ=TXLf{M`VrcF4k815&O^2_N1UpT0!_nPpmXz5Glv8O`(w6T_BDiA{sJCn} zJ!YOMNgUMhD15SX;8CyaXgRW^wKgi> zl}6+CX2@e#gm86h-Rp>q!B&OE({alf&h}$9fpIeHVy9pe&+p;j3H<=%a^G#-*!8l7~n1Qg#hV|M4@!kZ5)h3a3ofvH-B0zS% zk`z{T&oTN7cX;pZESkp#b#J+ztCh0VOCH}7zdZWugo)cCL#joOTMeHAvcP_(BR(g; zV#!7qRw6i0JLgNO_6k0JS=q8Ht?<{9^~svKjZf!T0nSZ+BvV0!f?x1k?H$HstA!bK1U{&i&!~8^PST0x(m9iMj?5P zn--D^+W@mw7+TU|eSo2eI@BZ0pI(pWE z^CIA>N8{z~(nAQh9;|{_UEQ~Ua)J=3AT|nd1Yzu&xB_OAN#wJp1@r5uCBbjUv6Di7 z#?iXhqGgb@s|MpA91u-CmnFzDw2EcEiD|3D+#m23!*SY;*>?j0tT3HX=NHut>14oG zM`ZJ79{^t|I(OX4fuc!NZ3iKlH8kLoEd)7iRv6>65??={+!i|B{6pn{YHL5Wb z4JXvo@KS|3?mgqJS@{>i#fVUwqrd7Jzj#>&13r}*NdBNkwX5a$Mk#jG66rDBz5140LgL~YT0>SVd*-z7 zOL9mj@vmCo#_udf^7ezzoM^T&Vmb-ruZ^5a$5dr+K=kFI7;YBa;Hgl$_5p;=Df8Ke z-cW_g5uU1IlYTtT3p<&riKE2OKbZ`&`w<@vi=?)tN|E_|6AGwb^h}3hOGP}lm7vP% z*{+G{y6d_pYyAqXNQFv0%GZ576^$GSMd_bgGm6L&0{;~~;k4!vX19=2BD&{yUtf_Cn?-G{F%BCrL55dTpPT4ZtvchWK{-oLk zy30mMJYTlmzpFH!wYQF+6dE?9NBhg>1-GtKj0cbohbx#H!kh#rX#z-d$bU?)@{9?A z{$Q_$(No`AkvTX+?Uf9$wB^R;0Wu$zSUz(1^?}wey2X>!9J9jv(<~_-U-5waRF#JAZbJbGnh7gxVp+f#4X+VW|{;u>grkmnCDru4!WD zB{&Hvq{V$qI8dqP@THSZKw^yQQq|3awx^0Y-XnNRRNbnR?UQWR^j1%})D>Tu$npPv7@i zH>l_0KU>YAnFx)mVW>*3L1bXDvJjq@i#S$!k5!Coa( zrGn8W=8IX*7;=E1Cx!nE>S#BaE>0f&y%vQwdwh{$p?o`z?pONj?9YI9tc42dLWEmb zo9EAR_P>%x7cfPQjJONw8kAy&WKH|{1B9LH4EMe^%xHWyoDqN!c_?+oXKm#lO{+qTrsues2uT^Y^n8*1Et^9+ z*mhJ$=5A(B>7$3`H!X6P=;Q?HM^fbkUc@QWpTNd@={O?$nsf5w;ha?96F}M{;-AuC zPI18-Sug$ME_xY=saeRE$OM)?aSA(3H;6?uz==;1eG#m&MfDw!i~fy%gnWU|*wk~7 z&m06(^%53Q82$LN$)9b}aBF5dmD!MWwb2&1~k>o+|dv_4=NGY1qE4 zZ>KI1>fYf|YU=rl<9e5Sl?VR;a(}z{JzX*07`xD!#=)%))54InOGRlZ`ATVilI@^b z9JX=)@9BHGK#jC!B3DWF1mr<3e&FDiyX405*B!f#{d}*rhDdJ*k*0$od3f42;MwrK zOb^hw;uvXw(LeX1Mf4ii*uviDl$O9|Wj+~zWb%KX1vkw3loU^^WRTgN739*m$KD~? z>#&ZI8fy{Xd2te?mtBQVKC&*NeNO-VCdSq&om+Ckq3xZti@zZ9AyQ%neF0_6jg<3e zz8O3zF`H$EYna5~iCbH%@v%iQ+oFq^3VadiEn$5Kx;km)OybaeQ?6*GMDqEzF{|V9 zgXAI`+t|G3WF>qnA(^HHdHgoLHIfOiegAMt6{2sLfP1X|@u9>GiI7Vj8E#Wf1xZ7A zx4QDKpUk5cRsgH(PR!3TfVzjJr_4l)8fx0I;D!E z&lM724%ktbIjZnQSFt-hcz9Iy&r72fEfnuaXTy8T%(H~7B5z^H#G#CNt0QrEYdJ&+ zZ_35>^ry$*Hf}NnZeel?EIHv>+_!7xDPQmuMk z5iT5G0zBXBvucKpQ5^KTuO(-F&EjEVR5zGC8e}xxVW>rVpln<48!3B4rZ>!-&)&?! zucxEd^dl=Jrn zj1WnFFwecsc@NeEghwd2vKqaNa>0rxI1w`n(e%$)fn%ICKPi3J^6!Y(dL9nVHoY8= z3l)uFsi(FM#iAOreYU{LR{ynzWu&sU9llHe*t>!Ip(@rF+5~ z(?X|ly_B@7tr<$>h#z-`8x`8Cm6|t|uYq#!9db$#EVF{lEQHIF8@z-NJ#Y7n!;~NC zJjfMD#*OchS8+QOrZ7;j_)!U*Q<*LlZ7mqx))-5~rKi2s?v$N>hP|%f!F<+>lbTBY zJbo#lr2g`Qdnc>e3M1YgldqCBAd#U%!%iwvn6XWT&KkyJk}BABspG1%_UD`z@Rjl~ zJ8dqhrAuex!ZlsYVT=z(gU6|tDO@cB=8IW>xGMZ^%Hyqo&#)xbT8^}DCT$Q(PO*vY z<8B+OseHVS_uDWD`d9Yw5_rIFP`Vm+fU8yuw$GxK=x!9RNG{531xRnAf;FEs5)*bU z?biSIZWS>P?0Uoy!k8sk9Mo|BP!qBBpqVn?B9bnE$47%C*B#u8^7X#LJ+T%n`yyK@a3TMktK!K%NA?=@dxA-bw1xqOB1$?B8)`qymzI+XhtRm| zr=5}diKqJr=yYo}Cx)QdB(FQ13E|85PnCE@_FEpBw#yGr<9pkTf{-GBx}9~rX2-Z- zcgsOjf%ABdODSJ0MRYK&@~$G1n3 z@Vs}Y%u-^R)D5oGWqQS|@k~h4`rV>O`Wm-`0KRY070{4JhPxbrC!$NrN_yn_Rpu_1 z=Q7-?tW-70U2BUo*X@Qf5Lo8&F@}atz)rf9|l`=|AW+b(f3v5g}JvAA8dG5!j&|yzUlVOWgVe( zc3||sX_RI1k5>G14A8Ckry+uGwPn8ds;8O1XFWguFVytQ_lI4Pt%91rUxxk^ES&u$ z`VW%EN<+D6!@^l<{HOAB1ClQiSua*?g6jW|--cQDZ*^VOrv)A2?*<5duJu@cEd5LI zZTI2lpRcRG;>q_uEe8K9>FBsF{ueeGAlWF@-bGwily~CGCk?`lX z^uaC+jRWDDZ^tX@n_k!aBHN#Z8%((~_$gIQBAbAYtm`fv%fGwH5pUsf#|fwEu| z)Kx#3W%&Q1^3G+02zf$I?=2qBt-4|MyH@!!I266EbiaO+&Ow94o}*(w`@IeNHJ9~j zed(%mM|=a%d^cb}FX0~C`7r-^=VkAc{DsFu#OBk?Z&hCZq<_KCU&8-EDog3(B7YnM zHHR8S>5=Qah<{rZ|7ZChq=PU2pm)Ll0{?>){U4<4zQ$0NcWd8&fBVOEMySv@#e+hd zh&hTBXS=n99>r~b(04iH*1z$)$L+%cK1$6wjyxaIF4l)~p1%ut6)T#lutL82chVEt zRFjAW*HO9x9ymqKfiQt$Zke&}!wflwFRM!Vqm}TZAw-jHMceYi#`fzc+HK49??h%p zwDhFO%TkN<#=WjbfgZU|A;-=Yl>46Cm#fbIe0rEYzSdufDtq^JI28A=#$hqZ+ODu6 zOj-;8DVCt;wNwFHYy@sjmQK-*B_>1f&}YmfDxIL-55rLzzWrftuNiD|n4 zYhik8MegE_T^lJ$GsqpQxv=#nur?Fi_fu!k|hC|oU&jgc+h2!jc%xzcYE@ynFI+MV{jSAk*GIU!l0 zKe{}IP6Zd@;GNpgoi$(L))g|dXtHf*z0~eZ$+2-zEr9UB8cskfu$J+vLy;}zH*I1j zlGg?{#fE70Sj~B~is^V*?Gru!O%oA+OdfiV z7U?gcn_u^NQyeT1-K9e0;Kh!mjrRI}$&ym}du30Yz0Z=gRZW-V_Iy;o6D2)%FJ*_^ zHYwxYRbw75TvMJ%iT9`p<;Hf$>JZFM~uV>+J z!m0Y#y#KOlzuyd5xo&Z9w@+JhIv!*9mHFRAG|=7cfBvK2XW#iPz~Z0PyLKF$)uF6) z^jq4KC*Ji2u>T*`+tig0NSES28K2qy>StOWz@E;G9=?BUem%$cU!#Qok%3Q9e^6!i z9QRsnqrJ|{hUaUmS-cR0B2XR!K@GuWtXaK3Vb|4%?ExY=LgaqMUjX~IM)I8c8rjrn zg8@4-O#nB(T&hd$j5H1x)x0c<>5yEUIF;+hz+ffU@Z!;uWYJIs(IQF5Et795xP#Q( z!o}WHs$MkA2N(>M6~l;hOxBCtkTI;2Ap3*C?~Ymsqx_$EPNAIT=27Y^L;#YsxI3)Q z3&0i4PTTZ3Kf@1cDw_h@WQpZ%54-6MEpNZDttAt(3LH zZ9XNdRY!jro>dKMB@GB&6wp4?*sPVf;v_Ocurz0d4?r`+cGW`989Y~yq~N>34t0?k zl@*~QZ_!bW*e#0hP3stQ9YnrF@^56Y&$5TfAZG0W(TGm(q1>0YQbl>z>G37&_6kU@ zrE)BT{e6VXqjFH$lCV{?kLzhgZ+x5(AK@gW1=RsyYW~?F)OT_{k4mJYa64s1by0W? zcBq|@VWuVn2}9y3&Kg`Bo%>~a2(l1ALnFYd%##oBM)dv%{*=vy{VW zZpC_uMk1_u;^~**m#qS`IWdACq*={^l{~i>v|`Ki8$u>IRu2La0*5phC}Ae3{a!%* z56zYbyuEG_66KjwJ|%5De0M1&``;xTgD!`NY2TGh zP_FV?A8RoyF53i523-nG>IqxWD{~6xH8Jfa7SXBU$VrtWId} z!@|^5KZ=7kwOEI*><*I(q@1C57!pZNp8fSzU)y7579^PytWkfV>K`W;i;#GtloIL& zk}gZ9xx9kjiUw3^3+(D-(3_ie(MsCsVq#9Q;v4t3j^n24p#EZ+w?t};e5XcsVh9zC zR$m08-OUYs~0U#zIQ`+z65M_rjtxDRl}s9=-!RL|f*-;ca;W zjg5`gBq|jnCnU1&SRO$OJEWP;x(2iL?L{gV{`d2E07MXZAJ46yYs)cO7eql@QxsQPZjZgfF6Kd`bBZR&IlTYU# zh4japSrniua!<>PxhgUr4hHZ&u#)2cfMhX#tV#Y&f7~;w*Vi_kEvPS;@{(LU1t#KkBSyHhtImeDUDI^r6a9Xs_10 zLNhg6DF3EK;zXidI9p1Y(UM81Yb$e-=Kc31oU&(+Af^Z{TWTbiNU~(f_jnO}5+E2M z>ynn;)?*kt6`7RCZ7l>2NZLwYk zpKt62_hKH`e@t5^#XKv5#ZEh{qDbU3D@z+|+wB`NCIioGM_jS1H&`1AP9n&oWeKbE zqBE^~mMLvcr^@FzA?K>CZHIEVluJq`ARxTyJ`%>d=1tsgM3m(D5riqv!gweRWhU=(4nLkH?Zz1 zq!_T;+Xt8gpdFN5uY_UKN%@m2Kk9B>t*R0vQ086(=fj}ABz!E(WiO(mZ8RDn>Pks6 z>auCIgHYf5a3@ET7eChp6`cfUIQCF(Q(tjQx?FM0Qgn?@3dMNSaAZ{#5oT>z$&`UV zv4#1fD4d~+JH4$l-`xCk?YC1IsCxiX?6ml^CMDgTsdL)ZV)6to?ejFAx!@BBlxgCAQDzaI*0mZCflYOl5=gXndH`?KtzpJrVYK=%OlPJH)ORRnz|F*} zq7bh^k^YsmQz}e`&V{JeC>|SF5hPjj-uvFB>EPnAQUx)C|4zb2Gdbb7%MB(D-zX0} zG4tRTFfpWXE7B0=Nq9Pw5^jIMb72zpzVFENBiqY933?Wyxg zQfwyTI5geFzBU4UrC&}3hJ-j=0OQ<>rCROMWpp{c>}1nj!sM!EB9GOs(W$rihH?(k zrG#W(&Bv=V;MOcx_os$k#w_KD+?lExwrLO>VdNQreUPL9M1$bQ}_06njcf7B|7xy>sa>K@n-hF*Ws|V@%SV` zs0qG{xD+pG;Yr8#_GEb}^ipy&!o+%Nvy!zJjo}ze#M^IhZ_ICqr(UoZ->TNbTOZ*^ zl<90oW-&z$RO_EP=fzwHISlP`mhjdm32ex4BmN49AAw4(3K>Sre^IM}Iyh9n3slRX zHb4@JSDE!p9URQ2lPDv< z1I(w!?T~8AC#DADW-8@75{orqDkTT5OJ0gNi;^%9Zi(Wwy+$+pB4PlW9Vy~SCnCCM z=>1E_9BQ=7#T4||N|1lZtDjZ3PK>4aY$k)C?(@f>VKYj$vLaZWr1T$^L!ME}k227c zfS}kPEc6Uj&N;}oO735Xk?i3(ugao82NRhF@57X%+K(}3XwHml;fjj!6nyytZE4e> zXImmf@w8O9N3NwVP~Z(6X@u{~H(EPO<{zjkQ zhwKqoF_wAg9E%oR);n=wyGTPxyjB+?HbFgmv_)}={pCjQ3YgcziZp<<|23U(cN$^| zSM@-zv05FUwjZe|a$$<-B90L;d8llz=l`@quJk7-mPNksAmBvxE7^QfG^10tYJT=E zOf!@swz6=nLVJ8mTY(}ZiY}vHx(sN8O|4>DTZFzP-Ub!i#|pY0ov(Q|FM+1xJ79lOA39O9^t4NVsn7kuR{iO?3ZjVTE7;pORGN&N!19c*#3`r2m z>-xYc6W3e}HEIyKPQcPGYOd$Nx+i2`$YjYt!1)BrE-nW4cBv5i^MuIV+~h)Kaz!8$ z&O2A5*h9(|!?%I-tb?9?!UbO+q2E}i(@Q=AA`_{;g6y{XZdYWJVP z&RO=Eo4lqpnx^vSUYIXVdf_)3rqt<0JDN`YQkGk?s*%MQVQM&uV*jkR8fF%&J`RPA zvgzK#HbXATLu4Y5!+WswKS(8aN-7&SP2=WH$3zN)F|op$;Um7b3NXQx3L#Y;zd{2S zFxH5JRP_ojKl|xVwqwoiS<=Bkb~cf8$GvqMQw4V zq)GdWh&bBv$Gp5-upd;42}0w~1Io|OquzDxl9(s?7&2K1Q}h-YA@;FxMw6H&+-`b* z2P66gV$04@ftza9j;=F%vwPE40%O7q!_Ewlcdq&T5xDaR7X)R4BPLp|2psiGiiGED z>{wyk2O%+%xsPS7H{riN%^#(M$bDaNn~gL&&`AyURYiuX^s6BIZjmEX#NaV z882c#N(!ssTvGri|CclcQLMoS3$92NSUf|lA+yjz%4sAuZLSdaw(+nL+k zqy8%QfNu462xBW*gGdE`z1WydiAhj~19sEzCL}nzFGDDoRtgx1m9ZzV)6Ym^fFpN%B9!lBs!J4I#5WY9CKpUtf`L}@l zs)pV^N?oLTE@9AqZ}GD@66{l?UPP@%Y{G-#rTuV0(x zaw>q#Xzt7&pN+orQ(QHkzbOjmb~2tkA{7na4GF-#QwN%;$EftTUlI-htuA>#Uo?k} z)l#(-WqR%}?oI+boQVt*`=2dj`*}pMAq*G;lNsg*>;Bj5q4-6td)lerkMuQUSgGPC zLCw_Ck%e@wxSD7g430U5Kb&PrkT(}PK6dN08UpNZpvBn62xhZuIA;B+5iNVeVr++O z(-=zo>_wTTX2ka|C!4`5fhvDc#l0(V8=2quXfQc-BSnhbs8#SljJJ~6uZoL@Rs%*f z))>WpppfbFOwvLt0t6&=5~PW>W2X+je!NtO5!@Ffp)a;(Dc$ZON13mke#W-eHdRhm3V);d=0vTqrd=a|n@V0r zX4;zZJ%2p^51}$83IW$lewUL4-x#GRbP(phrbX$~)K*abBQrj|g^AUqbMYsBBl-P! z7$DG?D?n8$#MqePuTH_wQiZ&@29mEl-SD!~A4wXK7?vtSYnKi^bn=Y379(YGl!!JC z>(_6Rw79Z58J1?po$<6nin-K&F$&^3#&1WMjn`(^5a`jZf6y!+^diR@k)jCUH{@rQ zB+#i&xf+g-ZI>o}@J`xHE>^Z=2-2Jz`z41o-t{9_-c}Lf;;8&z6uv3Xdd1hpuej^n zQ?|2*C4@&2wH^yQ$IDz~3Ps42gkrU+yu~4uFLCUJgp2o6} zf~*-kT;~&MO*x&zf9tk>&?)>FVNmR!h^{EgqL@o{4YiH5nPuK1AeB;LzL)!9-M1^_ ziQH~puFSQ@ZBQ&Tv?GtkR{rD2wWfO_*~BGLS!BGB4P1;_lWJB>r|-q4X9~QsAW^5i zaDK|&S?c#O9O}r=zbqs)VbMxJzS2|{zK9T$+(QB8hB?#uL_3b#9Znn!bMOaj$w`wz zzoB(+XhE6=GqRulk-b9H7TyaC(K!cLEZ?fiHqCGhlhH_*p)zD!=F!A;P-F|?By#ii zBo&8?qw^$?j(@Z-1gLNQ`8oHZ9pb+D3Lve=&Mpi4B0#g9KqA50&^Cs5s0sK z5?u^y()2&L?o9(Gu;@ncnkKVU+@1F|A`iTFB#JyUlDRt1Z?Fs|b(K1bWLem<7{32v z=HzArQE--g*7*0inm$C9U*QWHUuG?wYwal%e+Q6uZ(g=8)y7X|dWrA$JDh!Sadrca=h8AU z0jNo$E8==-s@cX=dEH!JV?mU7{_%5(kbeUDc_Ys2*sCniIeqt0OWx7Q7gn>(Q6&u z`==Y1qsm3yk+SzOVVEkxWpzn7qE*Pu>k(mH!H{N3APCQpA=J=Wjjyclcv5JX!2JHS zK(tq>vzd0lvi(QMdQFJlF)jUuEX;mS{9vvE$=yN`1`&X z=5|S;eJ?W=tLG_MACN+nrN<4=@|l7<_iM~Voy@}bt(7fHv+v&pw5JNxJ*Aw|$e95i z+QbbX!UzIs#Oz#%tR8wssBw%AqRh#BX`JLg|Fy&2Xj+*rNQ!r~2y{&(2L&0nN1oUuf~Ju=A>sV;5P#(EnE7Oyl^2I1b-6iXoYs4^O|zDre)&&fVQ~1xD+Ebf!nAkNff)AoiwI3x zZ1=rStYl%nowweWVo3>sUh+Lgsij_)cMPqyrN7xjH1cye`|N?-c<>9#$t%R*vX6SK zH_kI>`vI+o_n^_UEan4k)wt6g3YbwWB$pgqm`Q6y7JFk^%qmWtuE9qeW8B8d1A4JC zu4rI{x|Er}g(8rHCWhxx0W z%|Y)S$rfq%1|F7B=ziSq{97H~({ZX-tpkr;Ym=hxcB_9|r33G(1_IZbAEn>49AC3| zHUYO@;sg$ ze709#c&he^Ioo%$@AU4&4`C;SpCex(z<9MAmD#gbiPwJM57ZbR8~k$@?Io}VpE5Or z{e;r)doR32WYOzc^dQp(tc-V>zF&-jNz@t#c1!`CN;QuwCU)G;xM}sO&zMdtpx857 z=FYuF5B9%D$vJp`2nC|IF3O5Xu5#2{cfo$lRVI|pztONJxMdXkha~JKp8QlS_%S#Og-DV1=xHe{L9vxyNHIq9 zmhuhai$r+Hr*#6#t%N;n8%*I;+Y)GI5Veq@M#j}i;xvGh8a7xXg7&l3*tCvSzJi`cRXZ%X7{mTbgN*UKz*4Q3n27W+ ziM3rUvBKD1bE-n=LqVdawQNrT7;Tp8#*M&fTJht4phsLadb~IrDOE?%!xsX$;tP7< zh((zM%rAp3Thf0_w7tbgQz9l<@~Jy5*)hy3L|qs6LeV&nkZeOK=VY!j(&B!f0DY`t ze8EhTTHy|coGw{UL%JB{jZ4b3OzElp=p_Ach)Bji0U0;Z+*4J~{sCQ0%WQE!`mzA{ zkjOX&N=loX>ZK*a2H@ZNXftHholrLJTpAL_n#U=jG!>D)Fp{g(EM3xQWJR3I+IYlA z#5n9E&K{G2v=!VeylIT4>iL3U_aC|@o_p(WTLaXuPo&K9r1ec2WfqW?O~ zW=lJw`1{%tRk?UmxkS+-41-Eue}rFX7iEXQt{3MkjzwV#?Hl;2oh}&B6QqGlkEAxN zrOsWd%uV9koETF#jaMdKd=&dbi%|o&lvS#Nz7(-UzPw4arY6~|VYQ+nBDv~otT=fU zxcoqAvnx&fuI`qy>4V=eU~sWBEOx1zc9~E+TCugmGJvrMw=jhj)`#M@Ro{0B2GYD~ zgnWdo@how3O5vN9E3ME^;~Wj{(h{#k3~nk!lS5(l*fbsEo%(#pkm?WmpM04RIw3kd zetr97118qRta@kHweoP|XvNgcdwe{qxIR##c6W~J&#xSjyWqxujOz`ay`2pV<^r|y z^EPG_bWG1rcSr=6D7pm^xb?spefmYaat84c2I4DuS5r?VcjEe#78rM#n|e}d?A>5keKd$;~93OjaifelMvN2rl@UKdF34Lx2 znf`C)zc9AdKY=k^G)oH8GD#0#A+8X+utxCugwg^nEEH>w5|zP)@-KQg$$bu8An-VIjE;1@d)Mpcz-yC`ooTytiefY?`Q{rNK(*9pM%I za#9K3hz)EOgn!CDOOmd{JjxBJGh_DNn-Huf-k@l{I1%8w6ZowYDsu^>7G*XyPG5g#5k^IHG?#|W&H{f^i@Y@^@y6> z3-!U&`*Ml+Kd9S9&yDT&H&y>Z-7a6!@?zXln@D`nkdLBoOTM5=gxgr$_aO=W|GaGH zBm7doG>tLYxAY1D-RWaE?DPtdcP>;OR&&sDr$(%F?ERfujrqCOsnfzQy@?F)!Do4m&a zz!lc}6S1k$mpGogV!c(uxc+NZFuc0p9vI0;d-dMs6(XwaL2UT+e)k3Y#qgoyD&`f! z$Ors0dRJC-{Y>tq|Ar?HX|+~dzZ54@0|qEp%~_K-ZL#KoHexzL}zgIT56`94dtFBd1B;aGlTv{7$sS-%ohZgT=Nb zJm%vap_HH6eu|7&=ko)!WG3lJ<(%*%&R>4a#0x~*4`StZM^ap>7NE}6sHNdvAbmW< zs}@$5maZX&aQA1fp{QJ4)kbs6N5>~afRro_lP!Zx`x|1^6C?ILa{+Jpk()y9?j`*Rjo@_KP6e znHvTz*y%#)$s4ukL4CJQq^-a?HK;9V|B6PIAd31~cwG6Rcz6Tm?%2Dy5ixJ-?_d59 z>5FvTYSoVjSA7_$Q;s(H8Tg<)U>d zYhg<>ue$`HlLIR$r#RcnZ~IX_Y|Mw<+f6u-Nc{bZ3@=r4um)}J@y#e58$TIP*Ov2S7 z2b`)MB(G`GqUo`#k{f8e5m7|{bCP4blUi4hHc4z7*lNl$736O#O6SBr_Wctb%-v;| zAN1dDP$MMAFSS1vfS@Njqim4nXoOM5`YK?_t9w34wYUAM|D1(6!sT4=v(^R~G1O#n z8*Tg^u?Qh~9CgljtHD%ap4`3m5NZS^C0b}@j&G(a(}-bG5u;hgE=ED3X^qj;kx=jxWt>d8tYf;_2A=p;V-F6a<)wk0 zcA7Q(*}(LQ{RJx-TFr?7Mr4JRg*hz`PTgMp1u(RF$^2}S zRYX;d~)#Om0nB<29(oOZue@FOg6-IRjvB!WaITWu^ z^t0Q_!3EZz7PT`L(xAGtX`K{KDR*VpJ#a#yf3)^#nJDF-6Wn~!lL2D)G1G36#8#zV zpvh%%yrXYFo}(p+c#NP}*V^8xNmzihF>Ic!FCdYUuadhkMW-GvKuU@o9pDQS+`dHM z$)@GMFm-afBD))0FGG)908dY*+GI#0_o;wJb#^x^5}B;S2WgnkC?dYtZ)V$12D{Jcg;q=IL~R>oTR5Kff$R8llV=UsjB1KeA~Jw?iaH$2~-K^-fsH z^lQc>7J+miZc!p$Zp`qsVPKkjpRH;#TtYjin8961A-SioCVQl1EG9}! zKnnh195us<)llTv41d(5p^0syOc@V z!4Q`ZzwusWlkG??k04__o_0d_pQ%b3_o0vIiE=}{a2%sdwO2HRu}r_Y&*9QS(aB~- z*~#arrqa+5{;dJ6>g?I0BFC2|gI7N>=aj%kI&+4fnucC*n93NHQN-@}doI-ZYIO&@ zmjT`5QN`z)MS(V6vMYcmdHvppP>lFZN88_P_+6)kxbk0|jcc!uHjGw#zb!m2n;y@~ zy7V;ZaT_t(2r!gQYg8BMmf2STiq*PW!=ueTzU$WNnOJx^Dao+mDcvg-+U&v+f|2@{$DuCuvJAR3(ZqcmCp}V~a?FxHQ zs3EGwBLk`#;3|K*sBJ;5r>6XpQHYu1j}s)G&qyP!gbs-tYhHuMxC^}$7=;c~*v`5U z_c$sGsNPE(v+3%19DpZRWY;q}M(<%`T@8on`@Py%k@mw9dz2^g*GjMA#1-?QT0*uG zOz2Siep(VxY^4WDyHksiMt_Y#IXWanY~Z&JH#{w=38v8s&p?_f@BH(p-7l?I>W4ay zlI|@gLpZ=w6dD_0e03>NQISk_sqa03{;)BVaJQ7OFF$tl3(-oUG;RGiYVnmIeoBAM zHMO)t_Lw?}qMHI@Z=8=b##Y8GimN{h%W53p$P@JU+iB%{dzIe{-6dwLE=+edcThK$ zglAw3Nr6)Y#X)};R!T#CmVm0-=L>wlOcHPXyRu`z{J=N^0n$n-hE;D1GOd!+x+d}Fxq9Y z*Ts2`oQx^hkQKv1P}!a#fPF7M_dGRT(w^J<`z&1=Z+oj09dWZ|ASG z%Cp+#zdv?)4Cj}G#V4pBC)JPuYRY41LY*qn_TB+?9=R>D+(b273yJ-1FeW9DW)`D_ zUnhmL7y`O+MDpAcT=2yUl@zw&%IC+QSekw&B*JUOAh&IE{!+!ORC92y*>GDGhlUo` z+$=4c=;adjeSt1(FpfzS>D3?RjM$ zxN9HOMcIy$PFu(SMs=sEwVZf`plaY{rNKv1g=)OT9OjrI+84q(opI@E8;r6Ye^Mj# zQ7mg-vNl@rgqmu()$OQ#XjaKC4(X&o%(z+vq|u2nM;wsKxe`W0mq3Vqh_>WWjqhTG z`K~?iXX5*pGX378i>?6%-JNyIABdVATu{gnskk=Q4uD9XvjfRshSkqxa$vUs`?WFP;@^Uf-WTE|RwA~@h#0}D{vNm7;xiyOGDXDysIUFvxJM8aUSVIWa*Lh_~+!j{AhesrmgZL<(L?IxZe z1igkK4zyvI9c#4J;u6W>(aN6Q=h0PSI;RtCUr!aCAM#O3b zhQo;rk|S62hdpIJVk_}@aC9dGRx+HJ!vIekKTg#EO$yH)@Ba~X5W854z@4(CmE-3~ z46){|uc~W~^dB8n(x)EgtAywE7dM!0Ad_x<6;-qCZyt7&4DF2nswElWJ~c1{?ROcP znAz3nZ}N`?$gAU!i=|o-t{0SG6IB*Ad|d0;pa>~8TKghn2`1%$db&SUlZ{FfpDxUC zTsRy|p%Y9rsm2Nn^Wk&}qqJvNrhal>0v>N7@oQqv0FMC+1N4A<_Wcg3G^_#sGuXjR zU6Co6h3zw9_YFP*f*fm3;mQgp(g1c~0fAshEz&>8h|9`qwswNSvAKu3{nQI+{4C9I zky&7$Br|hisd%BlO$AkeLGVQZG_ZluYvFKJ7$b)N)SLl8DBSITWWAWW4ho~n(8Ax= zw?xfg*zD^8fILZS$)LGrul)3Ay@^CZh(Sp4{ zO;wB;pa2%4Oa`*P z;FL>&j_I{XD+#|M#a7^%!%q!=2lhn~JPs}y?YU1D09hv>qhhqw(t^VFRlgSCu8?-9 zV>(m(oEB9Kv-H7d-2$EhV4hKv#Fo(ve#jM%x4NVWScIc!XPEf$2cdT->xvsj?g~p? zL&-QsQe=4Y;Zz_m!mOAg=PlwGAX~A=oa>VCt>Aq$G3fh@2$PJb)QM@@SI=&^e>sux zwE@Ajr`-ETL9y`$U7j67e}`27hXj28PZ3h?3Pt0?@mhP+^nrSZbclatOVVx5aVM3V z3cAcm+&&29U4QAHmJR&R1Efl{qg+*62z0vZHH34c3C!rFEQvXc%bhASn(kz-3J;4u ziA9eG@mR)pI!@yBE<`Ed=p6N zfMJ54-JT$BohO`*YqE>Y8Wz37Cn(@TvMHMnkeo1$?3 z%OJZY*tm8y7=QK{aG|^@o_K7{nlA8tnzvBWAQ+h$MXUyH4eM*v8r(5pzGp8qvwRPEim=8Z&N?epFHUInK9Uw~kcpviBQhSb~6e9vP?n3yOqnWri7N?s{1WtD9GNt$5E{Xm@y}@Z3jqL24Sv|3D%|9NO3}Pj{YJ~XhV@5tWxsOH zH4VEgMsqN`VsL|h1_Hy=E5sK<5!IH%_Ve`_wX^am!BY@pl|oy{DFbxcG*59@9-CtQ zqRm3Ry#i0;G&n&4E*~>}pn>j*-ib~)hkqE5&=YP;ZEv}S-C}rI`S?)q3Slj{)cW6| ztV2EY7oDnSXQ{OA7;7N60iK$o4k>umKH0U+z1a% zXXnV(R8Gms#r~0*f|0%PpMNop?5A>h9@GBHF!hX03F0!kD2&swuqtUAFu6IU(MSCd zmS0m`ZqsWR#dZX2BW1K=?rMgvXu6P=x!@+m8KMXA{)j(a-Gb*S7LKbiY0@`% zuq^e^<)HQSJO9GQPUZ=08k3igm9(8H)5~E0JicNrY8lIUWjm;}3Fe%wYe#Pu* zLrO-~SYG_L{?pl!9bYL^1dd09IwS{{Jt^U;_%lSGJFCmYN&2QM;PYPb(_AHH*lrV+ za0a$qf~5@i&!(yJNH9zC2lSo!d_G^cnC~!kgr}rYOc5O+e9}&YKVxJscKZwSk=Z8q z#*AXS9WoDV<30m$jO8u!Bp51N2$h6VIw7d4Z_5Mi<|b~3Lb^K#ibiKnEJ_!P3p z;-8>$m1kPVSlDH&kxY}84JLxQzefV8X_rfWtiY6kWp zbMnx>%Tg3(Zuin(5$fFB#2lrQv!h>J2+MrKgz|4Zi|?sARt3z8K9o76nZQ&0Vo`Qb zn1}$t2Xk(#wpx_&&~;Py%w^Gs8L;viL0#=EXgrkNe`djWGx)F4<~n z3yX5d8=_F8G3SuQf{QEmnyhV(PBKg!QLxE>TQ2CsnVa)Cy1-zbl`-b~2gzXm9@8)Jik3DCqluq!Mwrq)DaF2ZZK$7*8}&LDB67I58r z>W|ls4P@*8ReL>ENnQ9s*xs9fY7uT@Kfve?r zV|PWDweyHP&TiAxxc0Dxj#Qk~BlW^?D=3qc1VBSp;25TwhB~cz&Odu1YM*5epfTjQ zp0gn{cafbNh3PVpnXv7Pis#u3J&{gZFn1RwJqle&Hfy|A}>vZLm7 zSqY{U?;&n%ImVPQLafZ8EQ6G7gHU^qwz6$z)vj_obALFxhi_7tV@$mB;~tQeSk!+u z(J_7-&uuh@^b)lyu<_*kDJ8`?u;Xk{>tZ(T)@9a8laBK){TqY-;tno9K=>>Ptw?CVL6>JLn3h zMJH^BPlCT^OofMS{abM9W3LJbwGB_xz~|>4Al5u*t4}C$23GmmV43e5=nvSn{*Y2< z%OmtXSX9S}Oh!1cQTVmpq#VV~7(%H9W0zdwYk*k}r*Yo-BCHXmS-Iu9LIg)eo;oc4 z#dK-V$*6{o&ccmJ^Z;0gLtvrQDVYTUs_5&_O1roO-23xu5vM}3JGO!376(h`d# zfmVLeF2cgKP`9&z4r}t;a5~xJ*atQXOw2*NP1%d#?(5dct_$H;h+>A1y&8EXDFrmN ztDSys_o&(r&Nh?Qc9U)KSNaXITL4LmN?zTaF@k}&e{cwDXGCf z7RAtz(jJG907m6u_pTXPjcKhF4gsRUq}}1TTTlHYtJMN=%?ZvLV3ocM@3}&90>8Ux z&K#o07|J&>&8LvNZ2tZa6ma-(?DQBF0zmKPbx;$wi*>VuInlvH>k+M7c{9`b%b0hI zP-)WBB5YRwZ&$Q%vLCCP3j#kA$P^;*tChufep#HYSFVSB<-6Gx2gxz3ZYh4)3Ar3n zR``e*rtUtx?A$|`?#uGY^8}0ve>eNv<@i)%k-7U1EEH)ao(SJk^kXBkzJiFsp0h7=ZRYM-oqM{jbplI;As}3-gc> zf^X4xl5+L?jRs~gwIUE3_nC*8 zaqQ;>$08C9j z*$_VS<*05%NKMS6l_q>}HK`kCHA70!r%1FiM}={n3}7H1B)`w2ScBTIr zI2e35__)0g$OIK>f%900sN*P7@(S^HRgoF_yKD4$KfQwcQi6TD&U}$hmMmyGhDhVRS~5w+ zbZRkl;}?ao7_5?5nNY5-Ts{$m+e&yV-QCu6L2=j1K*)-ZOua}<>pI1Z8^#K~d|F~Z zBn+msBr}ysf^r+r>?oTYlZ`yrBW8uz+w<2Bn6GQO52WO|4y>Xt%LC*rHyr3y1nQ*i zYmGKp9kEHiru1OSs82GA%MrFJ>SOn5$aHwD++$TnE>x`MTbO@|R6+i08>ZiI47U?5 zvg9EK@bE*S&v7eRW5?|E)BJN1ueO&hO7vrIn~3_yPMZn8y_3OcXUa0^7Aa<5Ay+(^ z)Y+pp9~x_v{48xBP9&{=carB97{70!ctUdocE>Y*vxbMN5GK;Ya4^6PKB-Vp$MEvW zpGifXfU*z^dh)s=cq@l9nggT!72)FLkNa{zB<(cCGp~+3xNDxum51b{E^*~4aQQe< z3v{qsWPU5yGuP4RHZ}eZ9bXurvkJne{PR;s{P=?!YGZ|yF?FK394b6v7nmd@Yk-!Y zZGH!^FaK$YIuNqZL=QbG4OzC%(nF~^da_BJ$!A?6($1PKS(rf1y5HGAEcBdB53_r# zdLC+SNt?>V7R*?N{{9mDoLrU=|rU58^mOw{hPFH7R%uyC+cC0LQes*XxF3TE<9TCd=tPSGCNCI-U?q%!GJk2w_b?3!#@ z)lx=+5Mj2kL#FXMiBgTs8oqNR(k}&x&D#X=Sq^MfwEPlVRiYh+vrw*!4hN#5QyG4< zWi zElf(H+@{+|xMihD^X_E{nfK`~wC$HbS~{#l@GcgPXz&zo-d7xSi}GRc7{;$jj`?g zTSqKCpM2&p6_w2nzp)!7h=$vaVXBAWalUaIsfYEs2fjTVykR)i+I4-rUl9O|a>tEW z4P0dSwdhJ~U$Rcmn3$5y&p&N*R+A5v=xAlN~Ir0!QY zlV6DCR4yh>nN^7M5-^T%i07&m{WmC*Xe4hTPR?Aw?3PoU_*=6NU6!3&QgUZU%f5LO zNCq6TMFlu+(7<*2w~nwchP$%oic0?5%cf|C#fq_8hnSK-r{hjz@5bxLvl)1>3^oum z$1wT|K!7YQGz(suBpZl2H@4NrWCYV4hA)|n*KDy|E=woDPn=2;gzVf<^h1PgFTr3A z5`3)TP~*vGYM?O{s&Z(lCSTNp{taLQPQX6=n24k=Bq`#VX&s=;bV9p&sU+OxsT!5_O>f5ahdU+g{&n{u#lCDGH@Q#cFX-|Kx7 zowUqISBLZ)M~$pw z`t|fav6On)$CaA>INB?5R|T?g`XMhr-CSUN$f;2AC{g{QJ!xYXqOw@<5k;Soy^FiM z6GG0Xry~8R^pCwf*5FJKfRwGdf7l9~t3U_P#$x;cr-Q`nr*Nq-EOTiH5*6MK8!4OS zXlg2#gUD6Dr+}stMJ?1Se%VKgQH2e>ot*sNkJ925LWI1*=)UOM@bh&-Qd_VWYb0mZ z+Ed`_;@@^TiMq+7A&l za9&MA3zsaqRz)c7+tWh-I+(HfJau~Y5CT}Tl_z0 zB>#_@v(4BxOY|*Pdl}d;9cOz{u^YJKFR)~9T-#TNSJbldRYu%)yEyoR@hZ4%ELs8iYZJx`cVRWI5MuMjKeK6#+4;un^GZRc{?D08j* zMYkb>iaxveXG;IybSUNAr=fX;IH12=U&L-@dpYYpkbBznMi_hGT8_SH(c3$DrU`tI zL}-7A@Vs;GJynrAMe@#ng&5&nD;9|2bgBYwyh4b%>?;ONh^x0x0k+#uCy85C1Y4LN z;$I=SR{g;!+6-zwr{?YB%MZ$aj?q}O)}CT*S}#x4kF)Ym>;Ltt`pOYReEi-?K)qMK*0?L?c*Mra}|s4E;!YAl&X_I<)!ZzQ*C`! zKb0fux!AAB+|m|t+&d1_%{3}3HmCn<1)-+T(^ZmYELSgb0;lF7A$e^x4|nZN#k8HPXi>&?3f7 z&b92eO(I~ce5ExLn>5+w4DESk`8do&`Ju?ZVW6sC!^?w*L18=d3C72T)59>>J~erDL(m45`N05&lu~QC`_0u+b6_k8D(chg*#_Q?UG)Xv#-xf%g1SL(=H`05~WXx zZzHzsmY?9>M49HSL<#!2NR(Wt>gkSzCo)iaEMCUEoLJus3$-jCWKP&5!kU;|AMG`X zIc@$8c5!g2r+Jirw+eWntEW8@Dtg%KhjOxW`cs*Axc5wUpK~CaeEWK6FH!3DeSl9r zjfc!i$uh;gGJjIS3t=XDPM7N$_43l{ao8-7@IYt6vi)tqmVFW0=HDV00C)i5&gLDZ zWBRsS+{MF=JvV_^G*sqt#mijDF{S%kK;AWM*+&kzwZ1}d%be`ILi|+0A6_Y0s<@~? z8In3o?Ni~V%!bBa9)6iCc8L$Ydmlw%0#-w{rR#}g3ErN6>nf3D?HC@7QA!>0hq}@B(4$EgwbR15*B!{^nm^3cu@@kz=9b zG{TJy0cvohiuz05yo+31IeqwK=KTAnTd0LV@B>(j3SoMTH*Q9tSnH@aJn`EdJ@fxM zA}jIc_{wbihiH(zhhfeNil9GOd&z~p+?(glv!KPxe6NLoMNFQ@R;4DbZDnWY_d6>M zZ$mzLm{!#hzK6Y?GXJ`KizeC9qkT{Qu^MbJ{_Xq1$~ig;J{#Ji$vp5l`_Bz0@D0bb zqW*q))_5Qv?>+;j_jgICY{W3&6PbAc&) z+2%^+H5agmaX9k+m*Y9lCCt5dzA~!ugtTz>D{cUc7 z7$>S_N3ahwU{MzWu5@pr*bYa*qAY`JPf0hm`joAE9zDFB!a<)N4U5~}Y*Hr7j-RQG zDZUN7?eIV85|}HQ{m)vP{X=}H1&$x~Dh%RV_e{6Y^B=X7H`#)Rpd$T);+l+3gFVrY?}Gorfa!s%je;+PdLWw8umEND)>LZPIW^$|hBM zlqgB8n5VohRqUWb<>5;GH}`b(0&+?PFakbFMnq`hgD@=o@>uh5jlUOlvBq7YR2ovs zDne^j|6FE9{<$nt`F;E99G@FB)8fHyeP{alIwGkp$cwez?F6+?=eY&(TI^-%SLEPA z`$D;3|1v}4xs{$oal2%xLN|~qk!oWh;cX`^NiNA%Uw~fV?a(Z2)l}-%aKOiSLp7~e zvMK^TY|ag~h0*ERROHU;Y7v}Ajy!&(kG|$o+Y4(KRa=vMKl`6V-%wr=5hpfZ`g{0` zx0Uf(rBK}6(81|FFs^>71#y_uZ0K+XVH>eWVS4RI4Y%J}v*OxIbKP0$&nOLW<_H^A zbNcV~ax}Y1qZc-+Y4fP=FDYUkFRbT>*~SL`BGs2)aq*jDduM>5^vHUop*<$#q| zNAh*u<4S;5jm^Sk!&LvnUf5Yf@|D?bTch3>FaGoI`WM!=@j;zU!K^JW=~sw-(yFk) z=ks~GY!!p|FT(F%r{DJf|G#tf{}OI$@m^N_kq7%r`9Fc`xY@DxLNMy*J%7hNVta$1 zcd^f60h*jw@Qg}ju`f~W}@W(a)Eh2u>Yxkr<7u~ zc&vRcPsO|F8M}RL%{6f@ke%L z+TxkmA0xfZH9hl+-=q7 zW&ia<%|;?n^-g&MYVP{ctj6J3sL`%m?rOW{^;A3~)YbZ$diUmgvD~ll_hPw6aOpU`F5lk9 zOCEnv4knKj9>2tjknrZ@Gm2vB<4z9Y8*R5jK53cExZ5A$ZV;69`8s&<@%%15!}^Ph ziwnKpmu0>|Kw2811Aki^u*{5fb6U1S zMpL|k#zMQ)B5fbL+m!in>F*1i*z;-Cp6<|cds)YQ{XI5#f0M{x^a|m+3Py94IvYRN zi15B;67)xVh49!v8eV*8Ax^i66;Qs(e%|zXiDm2*`@aY(s^zpfIP7?)e_2>ll*Rn&~WLS$5?@9pO zE*7;ACO90$Vm(wJcRfExE|2c&oh6|rko=IKu~z(1pxd44p7Q^(REiQT#LjI`mr3xPx%lY5y;>2Ue@YYCBUtp- zDvY6grPYVUbG+@%p&Q>)3dQ8n=IJ4j_F^T3p$+bzPe3`~RpE-N9|T|NZ=_WuY&O=7D|-A$+j;>8-yqx009qs3;=` zuqHfh~+Y^%zwwo~tkFVmA8^hsoacvS2r{ zxsF$dWv)BifIsXe$8El7VY@iq&GU#2mMS=b9Lv>rDAVIR^;K=+!ouMU;aHyO*>Z5Kig#h(70L+XQ^#z{0-X-Y_K9h9rl^*mmQHnu z3(G?`i*mW4=9-&|-XyD_*yl};T#r9k&Ws2E+@SiK|Etj*@KgnWH}Fp6N5LT18bovetg2?z^6@veR=7sblw)b+iGy`0v5ER4h~R z{>KWfb4HH(TikWIj%JNf>D?|bt1u2n2vP9rrQ+ zV!iuLF{-Q5w7AO56nRezLyhHO6f~e^Qwz_O;x$-D6?E@T)vC0g-o?=_HOo@(K9z() z#+@s2-p5DzL!Iu$-E3BNwQX@Zu_+6PhVh?RN^b_U$COU?pDae-jcA^*^VPi)|5k1r z%pU(Mes?Lqwl?>-Jvdbwt*ye$D7d596`;$!qAlJwYN^S=4EmZ55KA%*ocT|&nNM*yiKUL zNwsy5unlJby=~wHc%4JVz16*3{#_dwu zuc`8^z}6i0`(?gv4eY2%z#sv}!uK^{=c6T6-` z-9%lTOV5(BXm{W|-CNIpQxo;5P%3TO*yY~g%IXjNj%L^($H}&D>9B{37}{BNb+q|y zHvv|JQPY+UJY$VFFJabYjk65$ILxQTn$T-wi8&R22&?XBkBR0(D#g&x!Fb_Vq^U+t z-%~&zpO-J3QLZ~}dNnc$bJmG36Fye9$fR^ev?xNdh2r?4^Y|vf6<*~I@Y(IJtl8Em z_1SdM(=G7+yrKPxwcJ?&P0Xywx@oh~YA@f}zWgsQ1l_`|%&H4-2EQtZYE*r+;!tl+ z82F!y0;NA-XOqtg*?4FvGP~>CwiaW&do>9niCZI{+wdKtMG;GvXxJ{P`7a76H7Nl z+}~|NopqHg$Qw)YAP4xAAyYM<$;zar-&?xo?R=)XwQ3Z)Owl!;3|gk7 z>lbV*jsbA>I>GKL)y>~qJQk}zRiC<7(EjCdSoF9hQh%H~Y`$3RbS4?^>Ev)jXLS3E zy3Hf{i@N1I#k)JnqlX1rJ8G^c`~YNi70Urc)Pg#?3-~+l`riDOHqW4%ud?o=H7~GU zfW*h3y}aaCFJ=0R=uxAm3g~>goLgI;YTFmk7H(MOm3SZca`Oia4nz>@{ryS8r>-05 z^P)2M^cLS$OZ#ht7tdpO@YisrA6s9vbsWWwd-Zg*nXx?9=PGOS@!v@tv=&iP8+b5c zp%L2s6Ky{zp#p`NwxLBmy&UEmJF~l5tcx1Qb?k1$`~giRz7YFS4 zzt^vAWp>-M$1zj2fiQ##(qg28bt6Eqk91sSP?EV&mc{%7%aD}hS>=Ai=PzO`{+)ZR zwf=__Yi#NwjZpzD1y8QD6&V7dYh16hrW zOb@nN+#EmN!Ku#SCp5ym;K!vmIf88L+UbAq~-rb{(z_oz`L zihH7w_)~BGF@>eAb%K^oWrZ(CrzK<#!2`uMCoU3ou4alm(z{VvS;8ZMlBg6g25Z0R z;B*1inSlFPoql8H+qEfNbqcRDi0YxlFPJ;x_8v{15gwa1<5hR&=$)%3hXd1y-+!67 zgucvf0WDncEOlhD=>;+&Q|HC&Gjo?9SGB}YWsubpJM{PMLnOjbf3r&?q71>H6gS`b z`!KB;fF&{R?ggTXeorQdexcjQ=#YUYQf-sYGT>#kbMZTvnS?5kLy#DT2bgE2ZB8|G zb~sl?2DpYs4Yi3K1`frk#L&6}dW#C@Pz40l1txRY+J{+pgMR0LxJT@hCME#ECC0#x zu^op~W>%}JC#sW9SK=*I!@p>Ezh!zM*6&>2_@lqoWJQmJ>O80Z(x*_W{iRR&1NQyF zaxw!{ZeqC-isq$PZ9e+VPGxt)3-nHgt4%axT)_E^58wCddptNi6Lbl8`~ib%XFap| z0~Y$G@HD>lwfJ?9pJcOKu>!>A`BkcG^Qiw}lZPoPGG)5_|SI7D`|k-f`z+M90~RNMu-oz#~4fhFD8F2Db8J0eGLND*SL8 z=V&W6#j!L>9bYes#BFOtBWiLz1i+LN96sjrBr|&(r`%`Z#DYkA-L3mmVaa4OS__kF zB+LNgPo#|(o(_`SSU4mEGv{XbV`JSqcH*Wq@uP6k&LV)811V-M{Twv&_54*cb`deL zOva;;{zmJ33O11iF9@vIOB19KENYtJ>xRLfL=w6N48ieANddS7qd9xCLr^kdus9b4 z20iMy&C{B|6RyW7$Cpt!ObHTjK|!T{?N9cu#Oq8v;=m;wk}9Yw(%zv$zYB9kQD*I$ zM6nz~2lFpnvKxlqlF6Pfuwxt3EgUDUy$tN+7f(0zM-k)`cR$xi>AQXwgRj%(&O zg2eu@umRA=UD2x$4EXGeeFx>f!OYkQVCdGbzlu`~2uYo#`yM1tHY^JeItLi-8M1%4 z7GN)Qs-J-eD3?RGg4ek)$V&p(lMtL18~V8vJG`ezLxfx)c9qO@wTio3z_8~sqNue2 zF~RojKXgVFI^2#D>wcgM6J89{ConS?rlnvHwa@k)KBz@T+Z|@^h-Gw>vjuhHvstPP z>sI<=jV2Oi>XF1S)^@N?5&N3%5+4^bh82EJVfDRi%Y~K?bR)c{h7fu2wR1*e5un-#!5)4(wTXq>buN2U@rSBPRh zuRd{Ap5^q%7`W$sa%M^Yq}ad*s_TyZy$(<7a4&fm#Gu0ZDVmkBFiL56P3$+EaEi~z z(=cA2I71Q*g!8LVt7s*>QH~l64p^j19{;{wEvq)m!q2=hhFO{x zkl3bi)Ztp6m@wpU<0OXWRQ)Bd(+{BxZa8ojC8 z(=^<#gQMujt?3GIaIC2!0jP)R^2SGo$vO))Z4AHIoB+x(EW>t>wfBr$+Kdd7H$bb< zt+%16bZ$geY{%VFWyT?+vBH<@51t|xTd z+?+LV8E7AaF|YxIy>Y`~=3iY>ME1+%_rf8LRfXnM*UgC~Ni7ydPEh5_yqV13#DwRG zxY5k#zm<@RB^jl(&K{V|)BfU*SW`%l*7TIx-+|Sc4%|Q)*hz_RUBv2$`7&oER)u6Y zJOc@9KOeN()O12eyCOEM3|<~4DBYog?z3Hn>vAY5`4|dC#LHPW?FJ*)$8Y8lL~+hzk4v9nz+H$Fu(nx@ zllgcH*fOXdZ>zY|Xip25&W?b&XRPOOn)Z9>WT0PR0M!WS-8x+mHGZcoYd{~8LQ6<5 zPgzzF{y!JQX4ytY+m1KcUBTc}X|!-leU$I>Cin9XSZw>V8^*EgYwMsZy+yZ0Er^E4 zsSP;TJocD$8(ndm9}6a047b z7AlP_Xm8$OqdlXV*DT=YU%cR~|HZWZBzoa`vi6zh`-^ESq`u|UE3mwz{Qo_s?G@(4 zQoij+kEZ{EL2|Oy9N|2kcJF`zInzRQaB;7h`u-)RO2(dynwS~caC=izQ!(;}Pr6zGLW~bG4Yf;Gi!3odfIX$Mpg!LkrDvBR-UOO%o^{=UC zvd>!1jeZFii_KjOPFiYdZpr3SKobMTXENzhI8PG34BD{~1c}?T z1=VLZ7M*jW34yXms$~#{Vj?e0^ujip`gct*9MT{CMBvHrN@w7EOVI+NHrcLSPS;cB z*0m*Bo~Ou=4^XfdyTsy#oK4O|_GWzH>bfhR`KD&?6C5;b#^F4r@g}v$0K+(fUiG3j z_uc7pcnXJL3)3X=yV3=mMRQH8FPt>fg);1Ikl!ZTY?JzFs=t6?k?WT(~_qJekdX+~zz^g1cgFMKlz<9&WM}IWRmMtWK*B zAH`^a=tpjAXSMNEgICdQc=kWy`R>QU8PhdS*OijFr)uF9LS&CHe?%CpX)C#KW*lCD_`!x}Ku8+S{H_d61($ zv(b2t#5I-N!jjOpzg;wd0uAM^Q?TxtVAtK4!senXL@gaPNkii#!w3N!I>KJume?y> z9EzQt6_cj4M5gGY_1-%;RS3G$_E$&!`Q)d;P5VIh@@6xTWmi|qs2R2tZG%*dP5 z>iB*%wpV42QfNDm9Z9o(zKGC=>K#DAofXIKe`a5z{<>>UJI}^9e)jRo=$C~dHc^vt zZlYMf;7=4CbTRcl?0p`FdH2%PEnG_6`CwEbo8&y^$ADzAsUJ*YxvA3_{ok7u0i;8F zhLmTu^EC{*>@>L6@L57=fl;g3JV7!kUvw(v1!M^}Jo|L!?6%!WLE!=In*;{D2mGrUJ(ShvK_3euz-~A~pmI;t$7K~bF$oxRWPeuzA zF2CR@HV{u{G1p`6l#+?8rR&1?tSyOq5H4?(Lq%W+iw7_f6*tyVBpyjc`u3s(20o#{ z=oYqfEIqIM-T;zxAAZfr0&u5eWs@P=@nfIa@;TPg5f%nqNJ35)yXS0mcGX3=-1l8i zVQ6wjI`S8Y5e0gm6$TICT@FJA^mqn$YIqP!fEY48Z1pf>{4}EfQh$Cj>VyX6rrr9G zC+Tov6u}N&aR0qMF8b&8_%&d=hRp`;eAT4(%g-AqJw@)vUbPtaE4epHJ*G9i4qDKS zH~uPA(8>1rssAJ5%?tV|pt1dp_)+|X-q$4q82gg&qfDy$YShb_>e;qi1C%fNOgHzj zYlYH00zIDKsopd@K zmdk{j_sM$bI#JzwSo-1i7CE$ELt!Zw9J))*kt1<5p?JTr4e=;^CzOqDO$|QG-|ejI z=MsdO1$F6;<4`u~v_9>?x~k>z+2CqATyR2(K+kuEh27pCUJYs?^&2pNjj$q(67)SQ z1&k2Ogl|!^X=K8IlKC_=hzPCuG&`-31^v4QWVaPc&Gi+0zoEww_lj6t8vDR?HyS76 z#x02r?HnUzi=cU++|cDGpew!>=rB289xE5 zp`!2;g{qD3?`36Om}s=Zc^6GC2H`h0;{hg@Wy?TViO=0RQzTUanpuLBQ{W^ij?pI^ z68mexV~Y%(3m;%}Sk$n$wd<){aq_I1=)92P6}~vx*@L;C7ikqPhTu!kPx1Y<+Yt9V z#!w2uSIK@qV3zEzN2`;wP*wrf@nkyd0QF* z1>0#w)q2Gv&z1&XXk<3ekS$r$Ai0zaU}AUtzMH0&ii4O;0CnPt^{~ zjJ<6wU86}-Nw{p0rK?+ASa?dqZxRX;+XzNZt)-BgQW^XlX2Z#V&3J9It&>fyv^{6f zOJBiuH_Fe1h9rFqmceL`8^>Cr$}sJAKH^fF*W&5vCt8S$$lSPh<*~Asn(?$3qHSdf zT6l^k&pFTUK&|6zXxGx}#1-1BewI1JW>_cKlQ_f#m#dqtNEekI zJ{Fsu2b2FTF^+7#8{Ek{>Q`_R14$%L*`w@-aEwHgHhYUXT+XclIx+-5zYS}| z7$bn_(HxaJ6oHU^Fd4;ngmbuYKU@Q|v!l9YmRJM6#zC3W z5t?YM&3H*&2ibuiOAZ1)A2hijY3*Mf5LX9riVHSe=s6ur-OI0~t#`MHlaU4`H>$f#6oJoqgV3tb0thzq$( ze2DhrF#2HHRC=8dD_?2^3wjk%d~DkvFvYm<_SHIjPlPM=OS4}yVDfrUp78Fpa_nsY z*7gg(yU^NJC2-0bNT81yiV67Xi2Y}Hkxa6r54@!<{7woTlyWjZH#sSvEch+#i+&W@ zOweRxty|5TL%BNrA)gIDaecT51$Vv{r&(C){^~CnQbHYN=R!vu&XGzhY}sUrh*pg@ z=~P+VlD#k70e9IHYrr`7%?yFNVXJ(+6GJm@7EY|l2|xb~KX2YEEexnz*2o)sty`E{ zmr1&pl3JZrA50w~OgICt&zRF3Rk-+=eixBK>i`4f$rn&ygBtdX+qI{*7W~0{fgU z-F$$m+xH>$V>Mq!D=ku@v0rQT-`Ky7I&h-2WCDqXQHEL%@nG6L;^0yUz&{%>C;#NS zW?MV5hKw**#FfZHW66WWP0^D1>1uRd&&02kieCz*30~?9&iWzT)PGZ5RXkzqCHo*d zuL9mGX}jj$(8-!3tSz}th`TVkvGqtr35!gJaIcId1gC_RGq36JgN8qJGe>WAdph1^ zM!oZD?}ht#$LLr143qEs=lh>GZz!7hq47(sh|5Y7O=VFftfJ_pRVC&(%Cj_yqbF@n zu8Om&W)4ydTMl5R;+wVaJ||J$L1r02v@wDek&;Kmjrp=8eeq(lHVRA=#6eyJu2(9@ zfZXf0m{wN7rKpuK{%hI%=<^io=%OESQDtIAaNphq*$nzZ@_y{0v6aE z{YEpB7G!BRYRAS{mKFnn=z;)b{3JI!Z8)Wj#*ECn!GfcYje7MR{!Y_rb&w<)s#+0c z5go-*7^&D_N^PyA%EAIoSPR{GE|&;t1PG=W^>>zdv^=;uh@0I&IDg6Ujt)cGd4N6F zX-fxZLuoN~inN``%n8oz;O(xF#4k;8Vm||$w6X#){G}C=Q(=z?J|p&_+)lAV=rwgC zS|aVhc1odE&LgyhA&Q+L^#PnaQZwhMc3IIGBxKz{dd3Us=BD)Y<&o{Nj!zIxKc$vy zNRCrL%kiI$SwH(%V9qs?j>!wDJ`<&7poI>na0}<559WppE)az=)HbXKGiisp$rr)e z#iznfLPLI!zORN+UrKYtbz@)+PTjXuS#C;}4znswJ&x^?mjUZh3_zr+_A)|i%RhoW zEQlO8Oa%ePX^e)ul1^hcFvW807Q&3e#4=b6OP1!z7;%S}nm9g=bSMNRM#>>GA+Y{q+*Kik8By127Y=ngcz`%XCbd0lq1Mkn{ z5aT;J44Wqvj!I|_WNh_kw>{?Of^*_bRg>Q=!)8S%dY7X(!!qvUr`)9+>H4y$6RJ4(T)p&NRZfo?ahho6cc+|TpjWiqCQ{4Y6CfP>^56@P zd@cNeHhNS3YiGvo=gWj<;4=`^!>V(AAKz=Q60gq=;O*>dC#O4_z3M{P>!nbib-j9= z^kPw|>ZF%e=KD5Xs`@XD3e`KRC_YnB{GSSIJWCO&s&c!xe$@RxYvD&!O;5xBM=^~huQ;10 z86c=0=%i)=lyn4w8u9!A>-z&Xp{LFlkAG_W3d(Oqb9(D8tf__GX$5Ae&Nag+JiG}*+!;0#= zwf}`@1xZ>rU3TUPhGiA1#$%HSG=liTCxR{Tpa(L-upmrvB^tjQ4kn z7?98+OBP0hn5_c-aN4_cG?@9cw6r!U*l~hFZqxWd4q*p~mi-%ey<+L<4fc>dj2wup zwdpJXJc(I8%l=mRL1y?QtX^xZM?AK{6DH|f7fu9^c|_vmT^OEZ4140Z=6)S`&9|_1 zyI%w}0Ht&ZnmI6Pba!nvwux+VX0QYEvSM<{4{9+HttykI-n5ygDN~MyNVDrAV7j1V zIj0F!!6Dgc%M`mf;oy=aPp4`PcAxQAdhMBDNEI}TrYVnB72r^vF7Gcl?X8a2vkRe%1vr zK^t-v7;5hJf#ECIXtkHEV6nvW1`r&eaKcDs949EbGr%kSwUTV@0ci)<)+$2@4IAsCka!5d+!cUoinXF7@mz$l zbR-U_qV8g>t5pv?j5WA@jS1x~!MuiM4c_rE81rn}G5s1#R_dcaiM4f#b>reX^9NE1 z91TLjc@%WlbjCMot*bK6Y@&jO0~^$i>BRK(81va=C?a9O@U|!irDhCGc`jmg>{wi7 zsW6ant??$9BTpv%T)|s9_BhvsU_NT^UwfoLLCeq<^MjK#X!{!-s7tHp2!_)T(d5L6 z%ur@*Ci`uD$f>S3jqL&bkbcC|*(N|Lbx5G8}$2FP!PFTh3DdC8pOEU5AXH`Rf=V1 z>V_9rE+fiogT}r;1np-V55PrnwkhBKqm8FSrU^wC&4dHZbpdj>(Ie_NU5&&dMp1k6 zqm>)9qQqv>OkQ^E%*JUUYx$izE_oFFW`qDc=YuAr87@Y#*2y_d12EQ-k6#Cu8oQDg zw-efxR)$l|=~(E90xx%m%vEYQEfv}Vr#rVF?x!A8Sl!Bn0_%LDD)ZP!-e`+eMs5lx z_&BLZw7Xe&geED0+J3AiM8^AeQSOxp1nUd+HLRa`eOH5NOj^=0%&fqk13VNm6mp=xAwgrMFHK@M0p>@=9VJzA z|14i_n3Pc(fPznAXT%tudZ!*l|I*AURl+6`Ptl)Wyf9eIl1nX6IKRJSjLD!Es@3yL zbCH2%pMtT(&!O%5>>SCf9wYo$3P+rRu&_221MgrIF;)eW;7Tecp}QrS2xfkQu?7>( zI7YUVIySM!S5GR6l~doImD%Gs9F}D(=TIHi zK_t%LB{Cj4o;*E5QYhhaN4As6tN2oH+^=O13UVvN~n21ZOEN-z|9c4qv9K` zw?E=2TyB+Q`?fv!0gVJs&i+NjKxljlD3Izbg-ta;#J9$S0mz(`ys{9k+=@YxkA;#xp1&=3fhJUnk{L=$yn;5%*-diz%;!Ft~(@&H;z7v#7kVDSYWi%Ei_e-Uxe@7 zbjlnfT9WF3#Tg{kW7(LFT(b|RwrOY1$4^+5E$>Lwd4$4eQSMR>W}4`w?HoSlFcUhx>_v}N{WA&lALfkt(>*q)q&*NdF@Ea{-Ui*qIr zC$6J9OVkc2Z4BM#kHsj9oH!6ez~Cg;Z|*^B0|)4Q1Os2!$2S(V7dzW@!Ogcgf-r$Y zTMwg_Ln7Fpw_z?%V`8tiNP`R?_om>ruko&8OiBBTkm6#$Z86~#qrTv&e^eO>APVV% z!D3fm`k5?J{2MZnMqBl5Dt~!D&OvGM;hUZ57r#I6W4ha3u^i>Gz*VL zkcSb?3E!yOF653oN}*q%`4m1wh+%M{pf+-BwtGU+qo(TR9_bWZkKtTH_;)68r(UV# zkr^_C(7HvQo5`q3$8275WI12(yQFy2ks*-_AqIiV_}yK81vis@o({($zPPVROoH|B zQauY_25pOFNp{_a>`ll&iJNdN@;RawE%sw#W$7-KNkp}nCzQz7-oSy0nWrGvrAy=% zM9E@hUtmuyYBsK9mE{MZ`tINgcXTel72F{unZjeSQ$1{CX9v~ZHf(KGek)ehz zS6N7FE*Zkt3q9<-Oc^@#37BKZ=C;S1SrOR4E_NEni47$)Rw3|!fngArWQ(7Xc&-zS z#DhKv_R1sa@$<5J$)X3#rhF!ud0c@|$b^pH!V(Xd&5ZR>rEnmtvlvZC1&g=n$s5r=0`!oV<-6fJFaSa>Q8Zi0Rk*va>MK;z- z$g)YpE{n~SITALJnWidbKH72oNLuYfNc)TKhU_%J83g1HK}}3N8%N6eu))!6m^w+= zfGzmt%UGIHcqS~2@%+KDOS5WmadGSXB_O!;IoCfUFTRd&|&MQ~?P zES1qqxogrrCR$E?wbK^HOUxx1SjX-)-*}^xa^vu5k2MmCMm}fR7DV&D*sFe_<~Q}STIMqz)~KQF(WtK{)^dpW zMom0nFquZ$a#;y}NIrs@El--7&w)rq!;~pdSf_qN>*@8-EE~;^to=)3ELVSHN82jh ztoL*0B;r(HIPl?x8u`5nU7C?mnTCr+etkH^YOf}oAt~(Wq>A)uY2K)S7QH4`W#sG$ zIqUo0N3yb7MB%e=d>PTIyiu+~d1$GFiij46C85IqYppI2e6d@F@X_qI1H*Vb$+~+;peGUaG5ju2Moq&Y+HAXB0|g& z<2;HOgsCR7tfAZ8jR`o0S~4B{1BS>NK^?f?`bD(IYQ|eBgQtpmbBjH3fXqB1>6RA)Qjoy7^3t zlOjvo;%{-Qh^8zQm-XoVi-l75MJ9_U)Y}=BjjWv4?2EH4y|SG9!}xJ-&c23y*_ zOAsxB?Vg&?)bnc@)h9j|6}*GSg}S=yq92Thq5v#WQR$&oIAwjs!2y15S;d=+-ifCh zmuF(vNZjo%OViF2-zz3uuJ}xYAq41V`wLOPSFmkC#y|F=Rh+&8XZq{fv-i%=UROJ3 zOgmF?8iLig441TjZL5xU5PQQ(I7+9k%Mi}z=}9!3&JiErG^ko{ye2ubkxNmzU|;FoJX6klGCR^gvntMv0rowwSbBnBJdY0H-?3z zw3dl3-^f`-ilq4YrOP7z96Fu_xzrUUe`0Sjz$rH*oDq~!|2TA-ymzYm9N~P9aA)3a zwby7>?CG5y%sVU953@XNw=QFeeWMm2iL>t~bkV@iyve~^@;G`r@qE*$lhJkBbs7G8 z$+mbpW)#mh2=Bk$ZmA^>st@305ZBt4?~Es-xo*GXl|x#wg`INerMACYXt0`IovC%3 zj`}xSh(CY3W@7O_=-Puh!?(QV=F=wr1wy^N^j91`zjwaza3b}ljrt}gNqm#?02aT; z#)Q7skJ7u$72W%bQmYP^^epQ+%r)xNiS$_FY5{fHrliXj%?o#yLCbXgJmmnbT5VoS zHAsL9&qDI{tu}A^39rXz8n2^UE#Qf6TfFg>ZYB>LW``Bt-80&+I8V06ppYCFB#*ma zDWLT5)+(FPZp*1nCGwMe>b?!VnVgNMZ6ASC+k5}jhpA_ATGz-o&$v5g5PmhfiwXlc zY*g>95~zTFV2&!E`uxF5-wn8n^r*6v?gN=!@JzM6Gx3S@X}%z$KjpkD*Rm_=m~hbT$46 zXSdp2t*n>tPu|z{7(Hroa(eWSt}QOLoLp1-B#=i4ND6tC&{NbQb3f)Aeg%4>9gVy|`y zGU2fF7;$#NGZg)e0C#5K~r?x~nyi@D5~)I<1@OD~Os-^Ub|WbeD2*{|kH60G;dJqg^w# z&!ft~`HGEU^5%`P3)T(gs$T}J56>o1)4~Zk(%1F@DqGOo8p5$U{^&g5Y1lel{j#jn zk6mawxw#TcPVh;CS(_=5u;D9USyuN?miHYGaT05Mo`mtpd`gu zUml94Lo#S{a0zj3Ye2b$y(4^VJmcmi>wWCqQ?EP!F>#0r^DXSDkFSr0h%-$?GdgXt6r*d1qD?okda+daxwOE5gNv_{WNlC{G{l=aFG-9#pYd8MGW- zl()VPOT`x~2L(*l?+@S$O3g{GZ^e6@sYQ;#3?#VF@Tdz&xVH|Lt& zl5!SW_L4KyK4i@~J5PdNZ(eELfwX#sC5^N#g@=ictRQ>^#w;xF^}q7GjpX?;*)WYq z#*UK*zF39!sEa4ut8>jRfV{2r534Mc($lFMyqZEZ?ron>Jf6uljdLqaYs*}udI?(7 zSKb8C4|=`Rc)u&ooVaJ#oa{iGko`h@xVjR&tq_XFl_eW=_@Fp*a zkDmnTv(mNzK8|SjZJUj>kM&McUYQ$u5l0^!fAU~ooEvljG`OeME+uEEUjB<*X+`x# z@jvQNaOLc-H;tQXgF4j>{{-5=&D~)R>I>xqvIkbs1Ks#~*Q8E|gnX4p?hYS}S?K>R zFKNgHje^qN`PGY_1Vxx2py|L<=XFwLh?xoh(>4robvzp1m#f_58egHs%BYsM!^Yc z^5*E-_Cr@Z=YPbYfIvM&dF|=;gT63YK|J7VCn%buGt75ouj~!{h$ASRFqweei1yrymqfURz0Tb=LvanX`AnC(wY^d-8YigPOapv^A{fnB>wH;?^rCCL5-- z_jdW--iRzc#}~|#bBF)S(CI%<&Z#dAYg)(Jb@a@7Cqeyo4`ONobpuZLf7CCakw61n zQ%nDMGN!b2pH$Bxfkt5&r&zaOlt8=zk*0dylB`lo~|&as(xvTn}Oi z`sWGaWk?3ONU1Br7{a1oAr|}NVqjQgHfNVDgXxxQcF3K*r z1jUZj&VRW4)nCB4d^Z!WAx zb(Y7wl=rXgXtV9DX4x&Jl_Nmj`y~9U3gb06+-u|AQ_(noGUEs`+-?d^SaXqK%�+Aq6zvS(1_vfzM=xts%toUy_YVBA2!YL zA}bHRYwvcyzP8-1;WU{tgZO3li*8U*VN=_E!%dkK_HR9^5mt=caH5&a!RG>rw*jBf z^po+Hhuy6wKNqYkpZ+L=v2D)k_rltuurg@Uc;mKdyu8&fGpQH9E1+Ys2<|C1Qs4mH?4R#V8TS9k|UzD;Jz ziD)`t1z!_5gon@M1eH}R;SR1S;yya2JR1hLJsPY}Ag?4<;F~MZ(yzbNIli&l-gwP7 zthuIl-|now@{P87e?xVqUXhB(bKg*l5Dg8Ugd;qBHc1chv3~>Iv!)j3#{B}$R99W= z+;iEe`IBTe$62B0HX>MZnkF20UY@g=nsZ|r*x_R0G2Zv>oSSOd;a0cg!*t4k52QhR z_Zg8b?UEDHQ}U|V86~_PrG0^_XybMagcMNz+hBdvYU}h_ucNmf8#*2BInCTBZc8qu z;*h4{89 zQG_(FCH7m-T&FcI=4$Yyj66#U1_uUCuUWEwZG^nBHz z1(K2Bl51RSrTA(ja0Zz@CURR2hC z9@Z_00_(-m*Ega}Ol4=;s<_lQJHwBEwA$NVS8So_EgWO)0}3ri7QA-v(Er(s=;eb} zDfuXZ2ZSKoi+Qn&}~KWR`EZ?)+PYt`eQyxY33jbdoE)^;BDQ;!l~!s+`-Q`WGARcwG%A zHEjOo`~C#vp$t#uh<5*IJSij7g*xi*sf!r;i&cfe_={Bqf>?iaVBFN&Ez=iIGwwUi z?wEeG5p~f1e#LsdsK&TO9Qp%hM%x{E(cl1Zn?7j8SK@z?IN8bb@p6$SUr_(wi6rWO zjOBk#MJF2%;x~Y!&+$CQ2^Oz6%#-<#o6|pF|5S7WEpKdc*4E2k%lpshQuL{hP+m#Z zh8X4NV@P%vZ0l8!hgg1wriY~Yv%u=83}T1=5U`9)qSj=JQQgZ6BM>Wv83UW(5LE|9 z81&Ta%$}08#xS!Y?QC%_%CP;;B)gc>h7O~tLt!ZBIALAUuL)!S1uVXca}~PA^fdW4 zz-5Km%@*VQBze%{powj5MBmN0z0e&Y>L*uRN=tA9w*mo#nnfeT(TrdjZt77oIdbk7tP2A?}m7)^5$sEk-uWwT#tsWamjM4GoY7aRL|z)006tbEf+l^~m?+KRe^r zfH=t;8hW4G+Z*Fm!wQ|EO-fw{;Aim;I20Kyb|i-pRV-Xsi(4A*^17upKI+BWKhAuB z!^0;FO(JxXWMusP67n%|uQE_T2Gt!tY;_8(GO}l_f7nJ{t+7XM(I?Y-^TxB?bxq!D zWpmvYVZd7Rh@{0b{eH?Qk<*uJW9jH~XeQ!Bc<3~#^dP(P>*M|df4ec zuX)-?eucLtYB8Ten#A6E{czFP>5SN7t|;nxsk-DO$qULyO)rBku6wA<@hJlOHH-yt zoHFD1h5&+cuh4Z@N^~Q2rT+oDL+I=_u6rwkUBTbg5C%qSfY{;+y$1nnAf|Q9w?ZEt zez9(m+pY?MNOKy)bjugaw?yy0*6bh)XLrTws z{AKu4F*BfSAlli*t|ika%2zJmHl?`DSMm2?ME-!;H}ZD~KgGQqzG8!Dilz8h;r|9K zfgvR&=u-xj^AwzO&i3B;#9sR)VyWG24dXCKz5<=Dqf zmVaOTreF(nJv(0bSIh9p`6_MJB5`taMMJ0i;>q(-?=YyNeDv%&<`Lxd0nYI7isBEL z<m)k*dS1__8gI1G>3q)8!+mfXx|Ub$h!warfl(H!n003vtmE_T8n=rQDG! zYl6`x7BaQq24Q}Et}jpHnr}hE4ko0lJ2x@e&iamxb<@g`wyoM)?STF)5@N2GvL4~u zda%pU#^+7vJhF*|64TaXiFKTEf5UXs!W6SE!!>ZZb0ZW>&f-? zqOYeK^SChAPbcny+bQ z>CWjJl(g5T4?g_jSn#st?u9ttc@~u^?3l)sfsDdq&)V{HwfD}?_!}_DjLp9q@{e*I zSAoAcW_Im2pfOxAR;Z3MK=J=K>b@$fu4Y-62pS{=*O1`u?(R--cX!v|1eYvi;qDN0 z;RJVgx8UwUb0_)t-sjwh`*_E_=ZQhh>Z-4+o5Ac^)%E2$y} z#|Gfpz0(5U{Ji1x?YriPWAqtz=3X?3Ev4EY4@YKLZlFaft0Y=Nh8MKfit7$;9y$dT z#f>k4`Y*wMj8uvH_pEHYf-V2;wucb*#fC+Rtyv^6zZg7pcYK{yHH(6OnW&7ilVQB zk88V+-w0VK&v+kNDyC$Woq!dB#B|5tqW>m*0-Tj<0C)=$&ws~l31z-nYaTpiDSHHh z`@b0iWu^dYqSd!4{%>!SHJAKpXwvSFQmP3hZM}cp6%>cIoCiAt)=Rgj+W)aN!}tR8 zWn^T@BY=~vf2Oc!yRBZnB&YS_X++cds;@ZQfN~= z1)>?v{~>Cp6S61K+(0;2(D<_auphrE_c^$e=q#dGO=-XEOr))ID)-BK6}n@Ty5b65 zvqGKt;#+W)Ku2KyR?EL4 zV4LP&d}4b-#B%uqq_(U^XQTg3N+!6S9obitPl)#qa?>>3E@s3-)+kCc3V7usK zx+rgbVhk zD&q?*SxLL`4L60>LfM+T42BQ}jxY2HaZ6V$KGXCTGfVs_TX5JY2r<0@^sp!|BW*H= z&kJ6R{GPYDfqKXqvrsMreBNR#AhSwuCocKvqckDj zL~(W9W$-(giLsQ#^e&kN4xa?_=F9zfHjC0p0w(WLu{ICmCYUi%sP4zyo62`k*n6q= z1Mwu#C-<)so6ij14os?_e$3p+O_W^qiCNTJ7Xa;s#SPp;)~UI2S##=^n#qb{n~^!> z2JXrG)N6lM=VhJ_CEkwA4{v?j+-fOj(}&N47^i~8?oz<3;*895qc_1XwqllLLGEAp zI2T$2%qr855nKwLfJs^Z+&HK5cCPk<8`I@qz5kc6xg?W7#`U?Ac`oL*E#3Um8&K@S zE$lPG`Z0XHO?W%L!r?Xcu_2Z2Apg<|sXmu^PQ`0Yxx6`LOEz|}grDGuu=_={B3JJL z#Dh&|MtT?fsXmweMeRSbk1@c1;Bq+ zR=&fNEb5?pvolgOiQ>9N6X!uJ?~@HNZHnhNzqnT_jYXo(sir1p&Hul1%s&+|TRgFU z(hPVqBCpH>UJCeEeKP;y>QLrCp8BT)*B>aLLu_be-n%Q1b2IUl1Q;w`7qBkIeLIH3nw}p#DUud_M{cM9b0v)6)HmRGL$o%UOSX|1FO-e>PDE7Wwx*g0Iw2`ROYMLr3~z7LJ}hBt-PUSsYn z><(Hd?~Kfh_cLfSZbh576V2Bh7Me_x2H(xInlHktuq0lZ;F?Hd-f>&U319WJ+6zq2YKV|2Sc-LT5%gJsO?nK%gc=&=9-yy4%iO6o zPfay-%k}lMiqehczv8ipjTNhvNsir;e?pO#S+HJ4SxT0xO`m3G0vEv+$G}e;UW?p^ zBik$lIn5z>5yi$}4I*o{N&_YdNhjevcl2B>St0}lC)2i8`uf95tx4|R^P_%>1|O@) z!k@m}123Q*>1z+7Yu7-R-qZhQS#Z#;9UrRc#@A#&wA2W zA@e5rKbV-wUeFy!xioW5IZJq0z5}idd5YdOwu`0yMn4EyJBV9b2?M}n{@Vh=B;tS>4kEPssV?mj#Ya?^Oeb=D1T*bBs7k--t5MTt- zUdJbO3eZRXgVt*2^l2AM$Zn!YPYgG$ep&AHq(yIS(K`_+KmKc_^0wyE9iZTFCVRpS zzOpL*dfY=d`#_In0vPms!)GV|Aot=*3Nyxq>?3akfz6F3^XZCS0{88; zn+{?&I71fH*9;D_k7gfnmYPBB@aEq8H*dFFa`vR} zSt9nO6-;y=y)AbLhTN67{ip}sTMbsy)^xL%%YRE7?ehOHquu%_q&HESHo|CT2yy>? z-p!p^AcJ+tt=lkW2I(J#eeoNU*4C8<3G#vWef?Wzs=%E5QsVR3iH9v>(jn(+%@M%~ zb1=}BzC!UbVjg1R_fbgxYs;3st~#+BD3lgvXJw`w>yQI*xioBs3b1H6LLe)#Zn>VL zk(sYN%nYqVGTjm*OX!_!4+;+dKyg)H2R5DdI%FtrYmXd5OfvUYfF2p7KqoM*6NWot zNtj{>D^|w0i*qa;xXG^2K|}#d^)%<<4>=g+zcd?+YZ|}hUoeD3--6uAxg{Iv3vKH1 zyY#rLp*YkK&T0_@&XfyyHL^c(y0J{CtHIIQFW&yJady@6v;c7O_Ti`;70$6WIIFGW(p3RO*uH?0#f6A9Zs zO4%HwLS`O1ykY!eYSv}>LfnJ3Q=zV^$s3|f zfR-o~Kyatjg;lV7gU8>C0DPdS`M@g^*FhChA$^UsV=LP{VEoXtZB1vnP?E=TYdGMg zu$)pM`m)gr9pE#P7bz0|w>Sqtd7|3`449e#T)5F2#!P_r_dp1(yN)*$PdK_6kfP7A zn)u~jeo7G&plv^Pg$6~cl+~nV{1NQLS) zziYnp%EZ%i+#Yb#N=Mj>1-O}+o&g*sq5K`cUtIh$-s|spSy`!IhC2GXmhYWQSKIfV z14N|?`Kxu$eONj4Q{x7>=-nr4(Z4_$5&Z%@=1!^-e|ap4E{e*`xwmt?2_!Cq`JUY zh4n377k=9k!5k2VB0BKpq#sBVrW_VOxEQ}KvX?qn5?r710Ti;KrGj6H{9<2Owq~DG z_Rv$!N5ZT_pm5Lsd}h$rlnPOn)F_f_5fb-VJJWdieHzJe!pS^$(>tNV^!~zjO>x9= zG%Ly7IjKYJ{z7+6HQ2l{U+8RsZFg6(ttAOEE3Bja9T`Kag-(7wq1A)lF59R9M2GM> zSc=|m^sk^G)Pcxex`j-6J+Re-yq?P3?E@Atl%1za_A2;aGuEkH~J7BsV?p z2xPv~VJNgVHt|eTgaQ{Yo#<Bi^rbn^3(hYpN5z zDeYvSH*a~=o!J;nM%nK%UL@j^n1~69hzV~bV+4F5}2|bu3Ast)^%HMlY-Za z4rq3j6>p1kaJ zs$=b1;a>UeOV0B{-4*rc`Ffc0r+#&HypwCkg8}gFZ8`qOTZ(Pe{U6Kgc&7nb2)xx^ zY5DgNs^5h~KrObFlvi={63CTliuBD=ZyCfi35N9mcSRW<_o^j{F4>62kUoi|4%#*> zJ6Y%VuQXH-thP0}25D)X{CSL~fW^q97$Bqce~IkkA%||MYIx(m4*I@VUQe(M9FjWr zpFT`_PfbzmFMXJ*{s4WLno)c1T&{deuM(7wZp8vt+*6R_bAA%_^lloDl~@xyeRF&# zGBp?@k-Ymk7gppgyheCqgG5s-R-e$TWK@J%2BX&JRUZPgyA#*_hR`2ft{FpYPSNL! zM{j$wT7>i;?p#<7UsnYP5^==0^S31>{(%xpJbmDfN!w}@w^_4ZW)}=BYntcmWVWX( zwZQ!zq=OzUg62s4U^9Wk@zU%LZu22Lr&q75D{Dmuf@%OiwoFqp&cw@Ytxjfqk)Cm9xLL zbZx+a!#?zM{}*4&I{i{E+A*}$1MWoRFLK3HYL1O0`E2paQcPa`*k|u;iO$_^3~9P^ z$X$D(yd73|!iE6qmR%v+#i1pywbR$RwIfbiL|KT4NO-)7ne4_ILY>PU#ibG9PfmAn zp+M=%&Xj8m*oVBz&L|zVG%|R*&jvz&d9x`iBycJcMf=J>+bRibd+zeS^58ycr!voT z30$4-o72f}NQ-e_d8PRWDv-PVCZhe2iN5O(R6+Zbu-(5J?h1kKx!T|S!^`nMuHo)W zo!wtXl>fhe{J%&HpTLT!E$%oxY5lho<9~`%`EL%NMiwg<%lrUVnjW=D1n7H-W?X0b zPbP%nGK?QPZVJ7SmzgTEWSFStHMDg6W0mCz$wSZr=VsvH>4LP@X)V<3q18-fF~ z+#J-V@XYqguSFUF3tw-r3!nn=UtPoz^&^v6h8;fdvy_>$MkJu^hZyYZC**N4 zBx=iXNUuZn{sT{Kv;W9@@Cs~Xp|!rPE(Y|<=Lw!tQ!!8xWz1&u0V2x}G%JS+BH)kr zk*#lMXjc?cg~!{~w%PoFUHCqI*pompe@R?yd=TFS3u<`)UC)d?b4-5N3*vmdN>Fq( zd_#tHOpMn#YItM8mHgSd!WkSa7E;$Ull2x%Tfgt35_GI69uM_vP;b$`45Q85Qwyep zP2Nqz=`#W)r%iEy8JuZ9N*NZuH>H z3@?}XRWO5IS6jf@viK#t3pe=2cHlwS{XyWUr)_lMtNk`J8SJQA8cLcP?ct&9>xbla zRs!NbP*L9xuQ5>{P|Yi6?0{@9&dOR+^E1$pxJhc;2kv ziFbbCEof7|Aa`&w*#|wQKHMDtfhwze?7zr8L;J~x^4*(MdC}sZ$KRiyn*9!G{g`-) z9^am!1a5zXSGfUue_PgG@HF9Xw_Up*(i@JU3cjNLTJAG?EBXAP_?VSX)RxaWovD`3 zhud4cSM%AVO{L0Q@70o#bEe%PNZz8bzr1fGsX)Q|{^q_N$LcO_TM9Ge2W>t3K-&k( zW7dTE*1nO10@e0*xizI`=@^EWf;(PK@1GO1JYWLGY;I<+{s=GT%xXcmlI2YX^CrK z!Y_8zlZl_nfp*uo{THia?LhjOU7PNxVDHT}}QfW0uVeb`O>q zw&m8F76QK)MdYL2102|^@RPnv$P$+6gWm`WRkgM~AHQt#0GV_B&S{1d+mrkbb;ZAna}&Lt=CdR)u80e%J~wmrh6Qeykn4 z%0$`e0~r?X4JmtwBQYTV_^J_rg1>>hl#O(1v$EvDCtEt!uu@;eIBM=9K+bwWJGZ3s z;NpSE%*BfWiek~n#)4AisEn>WvVg%Rif~*?O-(#d%eSyXpRn?=(q%p8y=^_UwtLIA zox=;YN&StyKWPXBXefD<`p%bluqEbu>R8%nm~>MhyWOv)9hC#NM{$tdHs-P9Ukh^LY zoc_Le6es}m+Vfi}owoTML)A_J$;9LOgnx8;+ZOt?7}EQyZ!nVrk_5=a4w0AdKLX>8 zH3U}xV!{XyH8y7zkNszLA0(@yj+JfAJK>gwg$b42!aIX2)&vu%b)I4%sqRpdQ|!_Q zh}a611H&$gyo6gq-*Hj;oGESxj{kvb*8GcG?>qNURQmhq-xL=e0GhH?3fPZS-dU^K z)_Xy29rB4NfMdCQST_)#f*l<&!Qf?;lVjMdIz#)Rw zm5|#Q05-wJ7r>9^U*CoQh(zQp%k78+t}shz*2#$nGuBpltoQP7XY{7tjB94s03iQS zq&O{UF93XSrSXV^c-vdt7bZkkRDBqDp^K;2H+f1P{M^rLa+20uGG+A2mqv=GH%>0f zy}aCk8Am^q-5WmHuG(;=@vf@e+?;Khi4xpuT=m?qv@E785uQ^0qnGUlYHo9H*Q$fg z<6Zrd-EOy^>VfWc)w~~Z{bS}J_MGrbLOUcB&yyq6Wq)nZ`9(R>gRi^Rgvu#EpIbmZd#qL>yt1vfl=NuxOWY6ZiBY%CjxD<`$N`ue z{qLpzT(>!-F&?#Y&8#{gcT}{NbUo4%&iibC@7QY2>RAR%SkGq70TQQldcp=Ay3JXU z%5+x0AV0R_)BaLCeTbO?9SRoD7@ia#4?VLvQmhJXUv=Nlv@Fi40OsnBn@`v-sNB2% zB^;;=b13~QBkm=qFCd>S-~npyj>j;@(_*oC=^L?sfY|YpS$oMc(>luGGDub}Hnv zZRryhX4k9fL13Nh*^shS7pw_k4Rc3)sPjg_Fjn8PNS9IIGgF2w9+tv946@y&e9&DQZ2g+x`$200m z+`}G!ws=P53r-HGe#v*HCjQ$@Z^_?-LUZT@ij^*B8pNCZ?cWovFF5oM)HXN&1MdqD zx>Eb`%tPu6yUE|&O62AeP(X=6eHG`TCNQt1i{enYFq%?Lj7byB#fk5)v{1dNB5oXg zY!b{B_`X-AYuC6)B?!TTy6nS;@?NANcnGQ+obRZ2?~U+o+pU{?Tbr$05MGVb&?l>O z`*lOG7{Z5n87!j4x~W=52<@lt`oMj)NvB=dy@_T>E5!8Kon3bH{uq%Vv@S7{HUln? z3;fZ3sg!B91)>lpvn7PD{-(qgLiTwd6wuf^pYY37J4B_YcNjkXyjB{!*w#`#`*Scm z-o;#+4u(V2XId&JQMWhJl%Ir=0*B^1Fj4*FkMa4XTL+x%ObE|Ho*<6aL5Z;%iHtpY zPuG(goxe4q$Sv2g;R=3ij{AMP;oI7SUew}l-KN`bSdsq`VZB#*;ufe!thrMkTTQ#= zM9?*qHu1wMo!Fn;RDCY-Vo0OGP$QVB%Zf~F0KVE+9@M!+-blH@)TK=`LF?lcN6N%y zxlOj>Se0jHDV9ad^F0Su%D0znvH}cN>PB&bA?vf@3XvsKr^&!~! zaxNCKIF^IqrVj+oEd#?Lw5MF;y+zd}?WGN%XJi zuBsLYT7(xsJbpciH|E;pTe1{}G-8o!zVc;rN^`DJ^r*ciJ=-LvcFn+(-8NlI`E3p@ z=fR@Il$4R?gyH>3B@T_2orV?;NleWbLGWGQJVdt|;3V;ex#3wHN+KHSZ0Wjh;q}}C z-cvQ)rRu#izBq8YVwVTE+1PWFF2@{}a{fc@<0}m?Pu>E|ldUegFT6hV9`kgBb3I8fkFqIT@9ME}vHghBB z9qRG?&eduWD8)j3%)L(f$u0ZP?0qvSc%!)VI;i38ZG|I+Fwuo7)XPkQt;i3jkx3D?rnZQ zOnPT!N7>~Fueg95bFM`uZPS-Qsq^Hj%R16d)J7Xw*Zh0DZ*wbvjK}fErU<%i2sMX3 z1xerlfmUhnk|e$@#s@A?g@#d*TSLQo7?VgS(hs^;k+2zAj%2O8$Yn7~g7edtJd@7F1cZPNopu7;#MU6g< zcX@PzP-B|q9bo6-_rIk>V)S$wTU>^e;!NEdwB@7|)i1#_L7&71)&mh)%Y&k3cil`NPv%X^)C zf=?sWzL^eeV3%f!VMp(ojH)(FkIx4Al!S!mw3VOw%yV`AFlSC8xXFk86cvLuH{SG1uy^ z)~mEv(eFN&|0*?CcdB~~;g@m)w=|n28x3ZGsC$zZp)Hux*Kl|r#}j!ptA zE{NDlL3TQ1qoE7Ovv0v=Pyx!m60Q?e2!CQQdjJnTO-hP%x7Sp>c2M$ALRUVF$W7lE-L5)P>Zvt-Q9z((B>nX3OYN9%NeRi>k zQ_mx|E;b^Y7vbc)qcy*PFE=Ec{QEv@QXbEC}kS3D7; z(Y$44;kDE2)N3p{o}PP^lS8vkBbt((Iz&n`fu%GezJ4a6k%hRt9zEmVy8C30F4d0w zF{}prFWZE~COt-y@5Z|kX6hDjqlIR1f|vE{fQkkKG_!ajBG)wG0|6Hbi#&&Dj3aKB z#gsJ*qz;!UR*wRvap_kimecN04C+F}S+qR8Rkm-ymY3Q#_a@*2S9JXBZ&gcTnPeJe zwDaY?8lQrCnPAittA`EYh{`{N_UBxpefQ_S$carD!dHjwBf<5Vu8126;I%Qx=K9*l z1eIM*y{wJ+17fY%cH)4yZCemUx@sc<-Hj?0>wc!Y;e+{&k*QUo*-B5b9M`4~*I@b$ zTQWYWLYdCzv1gPECmUj3;debEQv7Vv-3qI5 z+J*8{+Y+>bKpHt7vg9vT3jxHGEwSSGN-!!c4N31MvWhwb>PQ^RDH1IvL{dp)i5CT# zsSkGXMIyq&?>>`%Ekv!mWzz(qQhw+sDgw#Q_Mo6AZ!N}O9Ll^2_&)c+0g)J~w0gn( zr`Z(Vh&>%X!7_CSb&1V2PyLiiRk$Ipn70wGQHXmn^#pRQLiu;L@=3xX)CM73fhQJ$ zQmAd`LqkMUcIb6Tv5<0(KCgdXZ(Oi~N-4a^3h$9gaCbXywJq%koEL@TWe6W@MWH(t zqLK3+(ZKiPO1yW}jfLSG66+>Pm^XX+ zWb==5?`Bz`4imgM0v@wnQQm9&jBoCk3H;oFD5tHDQ_IM^2HOtKK$JKueM6YodBBz( zIfV!@CDoWdu|1xESW_!_eI?ohzDZeUli%~-huGj6d6VyVI}Fp^MWzcGE9O@ zt|^fY<>nLR0y`#mgSNJxt<5Xm_uJx|FbuDtAkCzUQVU1HNXL6oEI~EaC0L>Q2JR%r z9;hIw%;tubag1fjnsaOq%Lfr$H5bA0iHt<6iAK-w&Z|+0g7_hMs4x;_XO|nGN3zY>=4U?dWemE1?oCm%xFT4 z&!FwMk)xjMHkY4Rrwq#(%``HkLmQ=rBoYWW(rQS>`M&j#o-kya$k^(AgVQ5T&JZ|T zqy}0-h1EtNFf>2z^>o*3xY+&Ru~~M z<8|E!BgAB*0qS8YKlJ#H9mN?nL85Om*y+SfrlhkpFe_+X?9Zc-3N;pk{dK4DJ@lH_-9j?-?e9Rzzi!J^hhf++l@A}JKcwLkJq;IWg z_>DMA`B>xN?Oz)bRmm;mp!KYD|EQW5)!@qIl-8X?KCaa9r!m=K*hz{;2*039$V#vJ zmev;*Ek`td8!fdLzYnWw-|(x%fN}CB$IyX-x+r)ylvd~ z^0?i=boti`WU3bA@B*DWfex!3W#2X&Cg*A!A{nvExn}pH=)E0bSjOWgp?IfcF!iQ) z#d@oq<1o9Kt%rh`h-vf~1#^^UceA$A(!Nz=d`@W{O2K8A6?cg5Bp#WFT!-o!(R8-$ zEmA>8m`p=O0W}ErAx)!wnC1AA7un0QfxR7VRmB@TQ;NN>zCH)#$nilQYza*eA`Rbv z6;m?r-HQ`L+GI3MqU#se&#B52DDk6B!Z=y}L!lh2S#gM%_LfCP{@V|+Rk)1VL}x)i zmepFztyUXF#vAiTyrjiUa{}9%PU;%1x(M*PR$^&cc{gN~96~7l*C<(pU6eWdagBo; z#NQAr(^JE9|0))QCrBRj3ZWE?njK;+DOB}vbP>T8I^;wbXW^iFw;gC7q$J*Ksso4O z;=aGuhab0y)lxsq*!^){*pqS}ncT&N{RXbETi|;$|?K&!n6{84$CX~U{u_IYD zQ=}4AK9WM$jTUsml_Jsz;WvhkR8p}A9caJ(TCes-R^>azvAy<*dRrM5Q@^Aa?)>5= z{7f9dATr^>>9YFLPk?w&l6!uRwV6wWo5ls-B8tL2iGztO{rXiHnM}hW&H8Xp#0BDa z2|UJz~!GS+|GoAle?6-CDUg zwM3zR)^KNz42C+I{@md>^IoBse#EBHa~-_HCa|CTX`@BNNAb&iK(7T|iWP;kt;g*M zPc2Ky?peKVW3N>Kvxcl3>?j2|*KUYmR869fiHscUP(pUVWjcYu%=XLP=cf=%z zVOs+4Y{LZWE=a$vH%42S4+kJ|?~}HiH!z9uR=NrVz@NTm)UAyWU#t(4mij8q0x4=} zIr<%b^i_sRG^kpOq;?O@;BjWqR>fwJDP_Gs!n1O4%(04HL@|?#nAY!eHCb?%f|s=(#Rrxzvi#{on72WIFj}@QVAUz$qvZ~=yC;d3wGR|CXL!Ca(QIVHZ zuZ5PU%zfs{jc|hsl1xl6Np5irfl6QYx=Qr`(CnaFAILF+yD8)-Z$@fLx>SxQX*xo3 z5(}vAtsa7^mVVX*TW&l^j$mVy!Wd%|R>al|W?iVpG5s>mCj>fh#_Q$P&|ShHq!8u3 zU5*E7f2LIwsMn$RkcexJIHMWO+|d(k6`!-x`-*7Vn8|!9Z&$!+gOz0}ldAp?)Efc= zCkfzkP=~NmOQ#RzTU-RlCN;&SWZv}jAqE!8x8ifHxVS+Xsnx8XkX+Qk6Ac)Io)ajw z+9lzz*=1-sVyK!KTYGL!3RzgN1CKDGyo2pYBNqwNYMf&M%g^_DTp`nJjw!!KLA}h_ zlQ7DWT^VVHHjL!r3AJ||XnE;!xx}(+ptqnC+HzTlEyEATUod@1+N!tKDIww()<#F6 zk@j@QqbKnT$H7f&lohXn5GnW4To9e0=+sBDrmQWfyaIlidl$T7kD}iBz+t_}cZ>mI zp#K%lpTgUhsYK*vsn(DG>cT8u{dXYJEl_!IWNFp16AQ3z>@`LkMG(Td596bJUKznO z-$l3N$Frs+w^+imkRuJxXL4&8eb3Cr$M0?Z>t}esYR)>?tapboYI--JOD5Po{8&p%oX1g`dWbmkwfTw$*hI0jrfD^t$Qb`y zPJKGnff#!#BPL|Vbeh4sinq_so!0x6T*?6DAwNO9{<}Qpt|f4meLiVReEvJ|183n@ zqJ)q+M^1n9FRgw-))7^Wkd_`fiGRhk5M;;>|naqh9j`M$3p+X|0fQ;yA_I zce(uEY5Y_4;~5%adCS4PpL)Me<2z;l^l{O#c$4dOoIksR5UZ&bEI@(_o@`-J9$9=l zd6PVkDa`!bjM>X-9*WXa9q{<--Q;wq2a>60zj{Y1mC>^KU0P_|mZb8)8g^cJeyK_# zPhEM#Ok%@#@atY`F?pJhsy_8nse=Nspvg=f(%OJm1r`<#c?6&GB1Hve6r+g)BBZsP zJxW814)(qH66xAfG!MrX_jO_s7;mgAXT~|F`%&LOJ5B5hR&A1{aH!Bw4%TSome;t2ukX=yVnZIr zj>>VC+J2v;ogPINOSbS0$xQ}}c(XKLrd-mP1Ojis#v}RGUa~Q3Yjs)wq;~-C0DE0x zGzdcvvFfX_6&U%;Tea3N<0L>>X4$&IUTK!6I-~BGe#PEG5j;N`%dn&-htJ|NG1`Xm z$A9NuZCbA-tW$#3`mu!OaExj92;^ln)VAGo8ff^CS}MF!(O6jvf|8U8*Ryrnir)|$ zRgjL~LKI%*&#U+=EB37D60+_M`waQuYi8~v25bxIf@>#I`+KRY3>1_joDv1gIGwP+o;19SPhe(8%UEJ~R7OT19Zed{ITW*!T;T7MmI`Z@c5ou= zQ;kQ}Nq(T@%$6uC!b15Fu@eHnf@4nuTd_j>42|13 zs;Pc{xJAFItEr;Z*N({}d!7I?w(!_prE%|~KIO-1OUoo(!_aEU#EZFH)^hBiTO=Ej zvc#dIF5;@|%BmEJJ zuWumY8~h2+9|l03t=|>@-`wJL;QG`y$DDMqjP`Pi7y91v`ReEo)X}Y<-Rmc*KTr_0 z6WbZz86%*rjVoJQOywl&Y6D#|lU(sVGs_rsuOnQVhnDu?Cn-H|I~QLIODC};A``LN z_4bJKJNO9ILZ#ZO*L@$}e|^<@#0}KZKcAn-r|M34?>ymqaW-;u@|L$$t?9^;tz1;a zm^ZcR|1Qd>v!{D&X%2;mWRA6?XTb;D-n*o|`gt5`!EIpW+tY+2E#aFc=5B4I@YTLb` zL95n?R1~<;suv4wTC%_oHKWA+%%y#H)4~zYAN#oRc<(*vz2*IETbX9_2dWX04saf^ zOY1k*#p9Wd@lM+{d&@gpw=77bx&C17)aWhfRo`3w`KE!mB%SM5D~oYn>|-TKwA6Nd z@}&1yr)iswrz?=;Y9`PpA1^gBE-twLHc%TI9oywZ(f_>+RHGoA&=23J$6fEWo0y%$ zTGdVRn6yW=HyPxal z$&(?61zoOwQ3rH}#3l;;#JD~?mWi-HawDx>mVh}VC@8r)wafe`U_rSg+VMEY4oE)U zkWA^7eiW_z%?z-JoBzAg1GvBXpHh!cdby?lQ%gM#*TmgU41mwAkie%Hrnzg|pU=ms z7eF~~osZDHh@ZugpZ*`HkvM1MHrKXx`e|V*5zid2SV>lBoY{i3&l3_-Xw^I9X{|Db zo-)Zl*dU+vo%Ex0QmlD@(!t{DV?vQ2rsCPFcVOCc3iUk7iE}%O^t(yl+WfkZa#i}l zI49#k+A&(>UI+s)BVl9Q(yheZjP7W)O$#p)OqoE}c+S+0CGsv%a`0Gu zt}RB!fR!TCWSQzk>~C`HYBa>5iG}K9RIyA8HWi_@e<@a7(uc6L9gltz zSDEyc4rwp@m8o2rA~qIwx(&zx?Sp75Bt3uU>_mOYm1BnU zkDhmz2{MnN%j%%Fq9S z68>@YBRIvt1H+VO>O@GM=^Sjn?%cCWCV{(DsK{Qlr)l#hx;JHUb%3+z%F64&<0m${ zqdpA`^$o<&g1vmr!uW?%s0x}qVH6GSTN~0CWHj+5E4Yrf@)_lNoO^GhZCXtS-DW7G`O~k zEv8vA`P=@;6U?vm3^`4?Gf1FlE=u^%99?hoyAKn5sUoyoyDqIEeU^66QFL1b z9tlc6YKG@$D_HWSYr)17+DZLxbH;7f5b^qJ+tN^^5!Q?n9Vxb)OSwdZgzU&!D!D?i z{t2Jt;Tnq255ltQ8_GUO4ED|~>CfTnT8VV3J5G)sJP5S3ylq2sY1y86d#Ze3MPi;V z%6|I3~{S#340#@D_do1lC( z+*ATK@J=l#2&}UKSK({PQC~D$pKocv;o)cGG{OKcRvX*hIujOG(-WgoM=@f=0uADD zR6=&_SV1>q&OirLWTVko>0()YB89opK87|e{KUl9lbrcZq+vGr<0D^~OyQ>~oDJf# zc&NsDMh>{hRh`TEnDS)y34Y#R?`=B<p(ncsye}rUj^3y!~PdN!A7{Sz2kd*1N_NS~_9_1~{V9A4up>;Dl~027+oByL9KS zT^UFgdv`od71=CiF_SB-pN)M;>=Z=|_Xv}7!NEIjB5Gk|7t=(e9xCZ3jNAr-Fty7^oKl8Nhg0$N7J#=Lrg~Ye{mw zLNcGx=@toVKA9i7dr-_V6(-Ju%yUd{i5o+)Y_97zeEnvV-xR_@@3{nx%W`09oH!kc z1TF9eBieDawstWS!E8ma!5E#Q?jd9o+_QJt^Ioe^J45L%vUcBcy`ur$_ASu+g3HINk@$xQQCO$QhJBcy7`3)m z;>vQp&C@&n?WfYj$~I;XHxY5r5p|II7MuwZmsOv9LkyD`k6gbDXN@dMbk*~dcplG8 zh3{6hBZ;V*Pfo>HWa#Z80kxe*H?71lahjv}+nS#S0#`c3r1f4~tQJ-+Utsz6bkRN~ zuHJJLpvUrtfM9t^OcdkZU5R7%b8*FAft};J?FrKf7-J-baC(~z?@jtOj1;45tVL6q z%f10gG65!ODmcpb-rh?wrj0P4?=Lt4wIWi76w*MLVcP->Z?SWSo#>#`=s@>nd!6fe zLU{hz)6;LHz;8O+gcfL4S}HW&nl$&RWCxFxYHw;Iqob3c<$VyfFBvZv+6z>=Kkr^4 zVCJPY+PpA=A)b(-miG9uX?!jQ{hOZ2uw^byRhAgIk}P&LeI7N*&L%xU^BTegM|2g~ zn7=HzdEU6y5V9z##hC3J40KO^{e5e|*_NilE(hXMgf5hA~sGQF?EH)Y)ba zww;(|@K2r&GBHK37JO6YT~(M7bHtc8BGsHT zDdsS(C+!NK+%Yw}w)B1|E4eiWX}HQel&`I(ns;H76*)+{n%e!S=u#2iW)ORz_3Qgo zUq)uqCg*b`hice2GT(~0ynaE?A51RIGtI6a6~yo7scb?4-y*Vqz82c^&9~o5jJiq* zz%kaa`$8Z6eidBKO{>8-qnXTNaK~aYhH8pgtX&VqVQV`i7xmlgbFEls^g`cB&G#`^i$h z9WcUlw9Y=~K7mG@V@7e6Aem(KGw|~EAU}j-ayO-fjo?+1l-Q8eLrjvuV)#3&D$k{H z`2$9x=7EN!!Ur@2iBE8!QK)SVP4YB_3BJBQzF)<}&1~Fw2#~VGdP| zEiM`rf(5NCv?;$?u0h9(>&u{|WZ(RgSEpP9f8Y1iP>;#9s?ovK0$P z)03ko5iyoKTH=&~&^Dv?~-92sxI=%#x0doSnbD-ujrso{_8 z0{))_&4F#qXP9+UM~p#>_`{ulm6_ve7>-OCtjFg0FR1^y4k_uFRNke@>f#_uf}f z&B42|aHfhBs4XG%)R>qvknk0@@9g$B+SDg*!kx;}u~lOpuo7YKt>VA({j$-cjr%G5 z00n7;i<}{`VTC0BplO9%Gh{#!&Q&3UB-2FwCL@Do1UUwLl63-HMkFvg8_-oT*YzaI za4>GdmkgJv%0lY1Ugo02V@bT4KLW$Q4phzE%9+VE`*t&-lQ8Sq74~*IWdBX%IGXPL zW0=&LEznn9MmC*f!uQ5-c;`4i`P{g$-C-t!h073nBJn6ZsU_;<;FomDb_ZX>G`&Qpnl>l#Yj4x*Myz&p$mgd zX8ps82 zs;T#WOud?URjEqRvU;u6efHknz1H`=+)599KiqPbQP`r?vJz zPz1^OIv3Xq``~12k?=E@x>swahG>OBz{yOBh9}D*Q}yJ~WBF&BzKg~X;WKp!wm}5{ zS67}$Ii>>xX6x((3vcS8{G>TvwO0@{5uBS8>uw`vPESupxHKk7XcA=CNyyXJ@?b3b zXv*+z#})RwB_H!^S;kYMQ6HpE#s?C~v1>bZ5Nf)#u)Vh`IMw)Kl@IrdIFVxAYN$u4 z=5%FCd}q}zCq?{tQ+b{nM7>3C4SGff;$NT``~`Yq@r>-b(G^Pjb3qIB_UBes1M15{ zWtyb^{NGW_Av9|m_aG>{mvR|hpL_5~BfEGD*C8h+n~@tNpJ%!*BHCbTywso0lke%5 z=7r)q{n7T$)t&4ytdFK9qH$F9YVVyT<6`zm$aQNQQ;4h*K+`@*#F`)48_P>$hp1cT zJQgL4jA*4v98Ok`7^Wt;+NMYFFb)OdZPHAr$f@_&wXiDWt72XJOHRa;)o64{5Jq;U|Lb8rU+H8imwOVG`( zh?|zR$D+%9vQW?zg>N=eU&51@ltUlGl~@eo(7)Up2}?Ox#!+1qnOq^L8?M9>iey>|P+RwZxXO@Z<9z6OvF1C5iCgntt zym-)ftLyKqeBQ1pdO@!Cy!G;aNcb#j`u?+Un?uOR<`d}fN1((n=wG?t} z#gH2qUX>f(Iaw#d;z#86HUY2P8$g5;N{;z++D=TZrlO{Ln~I6j*My#CR)R$rFY%ie zccZyB_VhKXdswypbUr6$$(QTmJy{p{ozw*++2!aL%CdTOw6_)xeLOB0!n4xZR7E48 zCnI63(}B|-90f7!rWV-@*pwKW^9qsl9U0<*A*_N$rTaWH%AhU_`{|esF8c}3Z3NqZ z-mHVlNy~qRmXeWhKK=zShMlTLZerIiAIDI01p~oF4}rmU zz*2-+z<#sd@LZU#nS%IUNEx%H!;qQ_-4;p!y%YBkK0BFFU*(NTwSm{hHM0SEnpo*F zFSYKB0DHiR^y;6@XSfcwydsnamP!#E2CK(`bSrh;tciK0Aa+CMpDWm-31ZYsWsVVy z&P%hg3V-UuZEQpm-pzQM`k67|m?wT}0#Biei2GBN3XIJ81`)kZG#im}5HnpiK{*-{ zZ?AeKLT-?OCCA^u9W{JZc@9s4mjCjjc3+1I{Lf51k8F<=yH2{0UmbD0=LbXVM)eOr!x|K4J>^zxdkUX;wr#*dhViB zZ~&8wruwHuJG~2M>o^?dF!VYMBs`FPVx!Y*gY~#5zMENffUF_p5@YFD(0X+39#r6Y zXvVLv4)2LyK`5_ggC*_Gb0hK&B_qFx86n+KBjSd#vYalJrGS?XMbOhQV{5B)Dpr>5 z?Nhx>l_&=s6?Zu&T&6hLaQr9Or6gn$pL^I`*IjcXnP?h)GLJyHaF1o^_ zuYqijNL)+MG45bB^@-{=w%zGmOIdx$m+N7(0)N(-WjWE^z7138f+nqN@w=O_Af-i4 zruYAc^4v7Epw%r!m@rK>&`dLj-n=-Q`Y${;Kq@iIBi?M!ZQA%A7-xmCHB!ktS}`Zg zCQ&?3oTKjAvH1V`N5v4T7-@8g*v!N21_f}t!7r3?^&Zmp_j2RYV;bxET|&=IF5MY$ zw=wYli@OcRVBcb5^V`g4LDO88p9W_Ie3^NW~{}r->$<`O8d-? zcX}2d?$FKQlfC%e5Y~VQ4zXU^nrqzKT5*mNL91s(<>pwRNv8E8n2}|%=1Bce3@e!@ z8TMRNJ+vU{wvpIAkb>&JtI?F9Le!`U4e}u{pr|lb>e=0d8MSWD8ZJ8edA#Di`zyaEW}$h(hVZ$bZy^49R1F*1 zzEo-zK9Nq1L#hOUCy*wfxbytf+86>|&%bt{(Uy8`389Y>ayu(NtU}O3_zjU*YgvN) z+MBZty=X-Irl*m3T6+Lxcd<-CR0-1WQ7Rii*ooeHISNBP#{y)JVyxN^XV3Rl4gm|IyFd+zsgvi9d^26T6}0u>^PT?5 zj&#)~mz8|)eo;iq2Wm~HZ+}>D73=(^uoD>)#7$6#F^WnP1m49$G<}| zT3)sx4AQoFk(YkFZXP4TdFTX!Q%IEv2Z0oSf$-stNvD4APSGLX40I?}ub-3vB)?s! zl|mU?7~d9Y-^@Bp=Ig>naKb<2<5L$pgQGZJo0(XWnM+Y~plGj_R{1+1fK&zR@s#WK zcZ=-kZ9i>qaOoRJq`LZG)BM0KGO)1oav+W`kvPbYE;SIg_E;nsr4x`B#VawO`jmUM zZ1C1cO8uaZF_5pZJ9ltZ+1GV`1%E0zV5r%GV2Z&;R$s_4_$fLKp4I{3!OQch2-k95 zt4>5kCQ}D#ip*7HWK`O-a?5fTr}YHf@G%#p#h$v%Z3nSuO1b`mfzJX0()hx>&pXfO zf_(Ka)BJw#s(jK+{BHA2)#*|%<+6fH=}==6=UMzcNTcW`WM%kebJ$2I_*ncpV(o6t z=GqlDOV6Z(tYe7V>yq)S-ngY~<(RaeU(FM)W~Di3J>^_n$t6N!tw?3T>!yDlpH@kT zaCH59M*_gB{B#tET_`MtP|4v8Obc5G{tbQTt>6iZeyyf$cp1eEH&^x6F?ce+sVO=B zDG2Y`a|~9t9O3!NcsL0K4F&P17?i?Mei*sp0){cFNFYJW_hkl-^#}n)xtKkf`r4Uv*wa);?hpAx=vtnuLbUZCZ@P+M^wP~ziGDS88z`XVuKTlCJIGZA?4O_NkxnH1FH~&!jPza z=kUZSV^R#0X#LOGn*Oy@Xm^%uY`$c2w;Gq?26p#!4M&5!2>;Xd=}Xp9njtGw?yGnM z6$)CUBH7YL;i{@cAh)Kh3;-x+98y%4`>5ecCtztI4ni_y!PrO*i=(2?N>v)C6^9_OsIqWzIoQ3H?!0iNmLyC$*=uWf& z`FYFL9X0lH#SLRXAWU`+OL6~v-?!Z}gFW!2ldaT`ma?%cne}_nmntLM3L`9`9L}&y z;pn?yAbE*(CeiM=r~Ult-UlG(IEbJtzQaK?lw!_36f2oFfj42L^y?U7NtMzYFY$1N z0LH8##jO2b1y94;_&$JcTPqWt!YJ@;n2>j=7f$7M(j%+EJqVyRAopSg?_j#a09pfi z>>JdIv8gMf|4nHPoDOp!m+w-*3+=F;B#WDRU9j`eHTX!qHOq^FzvhC!61>^l^h(z9 z0}?1mzvI0R1ztV-6TO$y4Tlih1zWb(g9Vl0h^HmH(2tCuUoUj>=vW6~MR z`MLYQ5e)Q=TXvJA(I?=QtbITeOwLu7y%@$taH1B@yVU-0sc0wpdRkg*Pfx!EOuVEO zyMm$ilb94ce2Qy=RJqx5!_6ky)N|7;AAi-BZhyuM2EdBiw3HvP3OTga8)reNgI`ea z>QBG!Q>j)EjnFc@o{fB`|FZ8@Qld=eq_k@cS*XAx;MT}J12<-P^p5pp} zTQnOsCwO1`1sJVlkLI&#?S%_)Ams92ec<(`6 z7Z(^0&KamPC@|d`?-X1RONWo+YV0$+fYOVm^X>#M-ZQ02G>~%*KK3}uCeU%Us-jaoaHXj|u;L;I<`EsK> zz3yE*h2*|nJ=Ie`(4SP@%8#Z@R?B9We5VbykmdxWoaS!>l<2uWNo*JfE#va$k<(xQ74Z4=kArNsv2>pPEEMfE!=NsjSW%Lpv90u(#V6dt+{WlRK#{8L2V&GtGI*(d)Tc^GkTjh-W4|c15z>CL||<3&8L0 z%(XA7rY|7w{m#ejP6N-+^25&l`J{xsu%Q}`_`3Eay|<9HadQ^ZR}9Nm$EPOE`7)-04F;$|ZR&ZS@ z3&;?Iu)~!1ywnc)Cdo%qf{q9VV%=Gl+z7xyM()6@wgZ>$SY45Z>^kE1G+9EZ!5cQU zU~1P0v9wC=$xRb7^fj<*@QM8S?iw>J(r>`lgBz}pygWajjIp|!-WONb zPw;%>%MuB4FN^;t^f;+bH5F8ShAsySj`Zi&f~za!rB@yBEn?G%NYVru;jYruZkN0b zmCMdH-&-I{2Y$QX4`OtJEz+8i)WiJ6f`F?weTTMu^x5iY>8i-K3bf++mV`Ky0Xh%hi8<+z#5X~M7be^`{rve2`o}fvt2#4= z&+dCCW`dZWZe)v8c4|ll1Hm}0IM~8(c=Xyhz6YT9?GX3a&Uell=Rz}vr^*9YYc0f_ zksU;_tg(pj;=5cHG!VC&Vh%=>GAurZnS3v-WEj7!d8f4Jp?N1C{yBzcCvf8H5{-k3 zrf*4e(rcvzGR{!GMst6=#Ym#os@`wO@0${Xf}VQ{Da5{f_T~$e)LNrc^@*0)4;GI= z64O@qR)x7)e4EC#U?D2zCtE>&FT6kwg<>pJdWo(~oBSLqRVx>Dn;SE-+8T{LhF7UG zwUvn+J@^bDb6pnOPXCn9O@)Fno8=afim!q{zIzO}+n$$XF-pwV57Ki%D~F&nX|$JR zr$|qM*v+!ZyU&MP@Y+IKUHn&ho;`N3oMe(Rl)JpcwfbQO_o#DSE9&M$)QHo8b}?ORz%l_$WX+{(dD zr}KIPc85*(s^S(+FaI;7$0$>!Day25RZ z&PD6cyd@XJLR>+)K)LonzNIUi)h0o;CIEMvWeUe{Nq6?biE&%5XWq$VkYgi;_Na>c ztI{vYthlYQROOqp#xWKKN|;9EbL*G*jOqOY(irq~1@xIgw(lpt={6Oa`ZNVL?tn0R zf(U*!C)HUsHB~ZL2JXG%c;fHK*~xOG^pk+D?S-VW@gPIK71)T4gJ(qfJ1_Ft&_Mry z^%0)gK(0bKb?5OyLWYS0$C26-%!Hx$V{`Z&eW?jqtYXcIYa9R`a5X~`VRiIHDb+`# zVEF<;E_->HL}yJ->;p_&Ta%hp*NyMoe|W1R($K8$K%&!$Zxh2)tmhL zKU{gn*wN?#}d7zqo5SE}e5e`Vf<=EIncrH%|jm-SBj_#--J69p~qQ8$!8M6sk z77s~SE2M%qIN3DldSviPI8KSl$Tg=z4Yt*?85hqa;>8u8d(OhgTLin*W9GLlgxQl% zs+FwhJ}C%#-`N;YzO@s0#InFFW3^GlVRhE2GMhD&X=j!pK_5G)rH@Pa-3?S8Q_2Pc`P0IcdKb3Vo$R7M0gmoL_qfqWxJZn6G@( z8v;VHwf#x!C*&ri^2!%d;sFO7FD%Zy+*IETLNf~#4zr(Sfh~+37DXha2~|G#`+cL* z#bh@`!$(dwfaaN~vt5QW5q4{)D!HzBRqvaBC@uIdhn{Q+xvSCQnbFFZe(T?UnA^O| zE+_A8j~YOb{d&UI&4b^qUnFx)AdIglo5y8*&CU`0B2ga5sYZ5vB{`mHj>o8I=K~HD7nz+<>7J^|4 zXA>M1?4yKd?_hsXA{$pCcf>VA5FHD z3Mlx5Pw6IxN&4j%SD~R=`WSh$TiQ_^`qv1Y=6;_kN4|fFh`})ud}@Nt2)D^jyUNzR zsEhi+upwb}UaiK|IUCMGP8l;?nl9v6t=-H6zIC*ZB(R~OE-=Pd?&I@L+Zp)$*%DE% zx048IAE+*g-diN2N1bgKIJ$?Y^jF!<=4qq#OOCw=O!t1O=?pQ7D;Z_4Fu~ z*}KyV<+n@~)cSDiq>CL;B$c^+tvwhAl?e%urP>V;B}&styxm`+(s%A7a_{4EYg&|^Vot0Br!FD)4G`avb|H-#;f*HxuNC&ETva1hz9ZZT4fvIV|i7f@A@J-t)(m-nJTE45+ovH$up5ex36 z-WPlbgZR+}jRaeS#KP3jiFoXaeuw<&Akpm80M12uw;%pZjB5wwX7termikl#)A>G_ z;<_JxvW^CJ_eTlZZo7B=c!sk*z9h|HCD6J0J_}2G5i@5?;M*>tnbv@EwUtJL5wXZl znC!fP_|Jmq-;<*?<*M9F9d`DOA&1xa;R+BqH}?xXVkG_SQam#V(*^X<+5`G0`JXumKWV*xsdxSR zx@G3oc7fbYSf|K{(NU(vLJG-mLnC^v(39pslz75QP4Qqudo@FIOpH${@s28uy2L-w zR$>&Whaw7lR!F*&;UxcvRyZm$BI&^=T#Rv+zVE%53|(bqEvIZSRsOZPC{bP}yWWEK zgVnAKPr9bYB%QTB&{m#)t$;q`Y3~bi1)?A_D_!TWXtmwX=a{=QR!U15t@iJ_X3?0$GZCM&k+EQe?nhLoJXfJui>?#uQ9XI27?2n{Aa z{b7wa9o_TzA0hb%N<|IA*$xqMoi{1tm@J9!*u@;TaC(0WM8yubV<5b7LP(1=>G@>a z?Ah^JkGlDDYmT(fGn_cFVT?#F#6@&#as~^c9=$E}06HCRk}Vv%IY>k=VUK2=NbekF zN%0T6GM4LaMq7Fj_zVKwOE2GTN6`O7M$+e6(uzB+`^r>eY$YdbS8dmX)_pMInxK)K zrK&^5`zCeBfXCp9~%-WT>Q=OYL0HPtgVDK3kx)Q?mKzw?-X zYpK?AlM+6`w9-MTJmS0H8UHmAg!Z&{aJ`M)X@?0J>$4&!g>2n&W}l^DmbxluG4Q8P zQ+!m-6hVn-;obvKk{W*T2Z=I<;{WL}N|qXDagkR%7Uvrzqg_GP=wvO!%{pKk9l{#l z3;buVM5ST=D()q#95#x0VVDWBA?*j*G6q^@t(fA(Ij^1m9F~&l4o=~mI3Xj9uDbKA znAtwzDhDNO^v_!2S!P8prAK8%apvVLz>!8_;Sg9GFYST0zfzf;O-W6{&@$m=8I)m7W??(asnsWTM+Yp{%6NV8G*9XMfS z?R?+kR7Q333l616U?VOUM@*9$P^ZxUo_`}^KXvBi9X(2EKPmpqT0P`r_<-CS^hR^) zr?(?#G2U#|6_cX;dqUZ5%6wruMQn+leQDX(7?HW+%po#wE5joIKjvz1=kU5Vrf#trX^7oqaCiFU22!`&)UQ3@WXf5+ z?A%NJx3fg$2yN~39xkf-)Q|I}NLHf;FI-gPrV;Hg5~N&SICnQQ<+(|&xCN+ByO{_P zTfDm*skDb$_4Kxu{FhrCsefzlms@C5+038TPxjsX2|cP$JTtj$ZO|FzCcMk6gMDZn z8_-(gPhaYw#aeUmdTMPH-;ZYSuWK3&`F!O+uGZ#mph^@Umgt(LT1Ru-AL zT_%;XU)Tm@x2~mNA34m$`>M&Y&eGLY*U%FF2O80S3ZC|_7oavdI8bfoa!-nOPGWbU zHumPjLG=v2=FopVskUS|weX!|TnB5Z4gaOGJbCbC6X46;;`0X{%m3vc`Uy}O*2`$k zf@I=;*XR~!qCNuBiPjJ8-|-bZlgBA@hrg}5#lEgfCW-2Wk|Rr1QC$;RmEc7j#yH4h z=%UEli3^Kz4c6)!e(&32R!Kw`m(P6ZTCH`>{v}Y9`ip7I^Z8K=_0fhI@-#C^PYZpb zCJ9@f#sS&mS3y~9xGgL@{FDN$&!B!3Zqovh@%sCc-IO+VeT>o__)oGy)m?aTiy;bS zU&FYn+wyHBOW#h=zAy~D{JmpeTk2rMZOl?HXJtNai-O%5`bq%l+ab40ldu?wt%`%E zh9jH_I1xHJa>ih|2GAN87X3tnfAdaRm~lorzGl0)<%79}6cMFmSo~b~ zFcy0QV@VY{JUlF!=8dZ-f8))-0$R%PA{Jlio4Q9 z(hjOj*ImkKE_~FewxhyFs@3x;j}~_cor-f$-ApElHXZ4VP-wd;4K#OHqwLnEVL28- zmwB%fLPB%tquJ)k*_JE0Jq4PAqu9ovk-<(UQkQ9TDvDrLSueT>uoB3*KkngQ+9na5OXxq}SuMXw+ zpsRot5{V?-Ayzm}%;Vf5WC!wi9LWlMLnnt6@G>H1=ZGs&je78Qg2vMz|{SzSIA0KR<^P#WR~ob2zw4 zC|I_Yz}kyI#?VC5+&5e$BKFC|dfnjS&=K}--8v2a8uO{)-p6XVbw>cQL2REuCkp#Y zXWayu@ADyPxWVjCMJT43oF?7s1Q7=^J?M2YHD^y5$JEZAf-gumm*%M6`A;0bY>rOJ z7(oQh)1Aq!14c9UxfsOHsynR=h=)NRGoO|jyAM!S!@4 zl%{V=;Iy^$ish$&NB1CcMux!Z zZF!l*8WobErpVD-Yj(^DY13B6G=ttX3;L%XWPJ^ddLLi5!{ z`9<8BUyGE!I)hP+LH3MjO-d2bW#Q#Dr~D{29Rsh)*PJkW{}OV`fX)NW=XFlAi=7wp z*eJi{Oi*}(Uw7J@*c1mgeL_Js&`3~xX1A7MXUBEh5*5Tg94f+-e50O9ELk}KN$!%r z6IqkYDf4VA{@UjAGsg&Vd?d7uIF=37i9SG-Skueajj#WpQiQsZEAyw7JZtDLmk2 z90hi}w=bP1hh9@vMTZNNROs(%8`MO{FpE)5WuhTOy(2^6+j994dx%Hnr>YlZQfq6+ zH4rX9^F{t!vV1zWlLVLQ@vvgQ7P&poST5z3YlcqD?b|4l_x{qEOM3D|HcUUVypO3r zNyg2EO)ty$=k{?eUr|Bm%tApFdur>!kgs-iG6odrt&zd#TC9c6t}44_{O%(3)lH_g zLY!v##b_Gp29gdvpfZ~BSWAoOciOBy*O8L3+rnxzNFRWv`}%&75MB=Yelcs%aH~5h1HjG1{t-S@z8ouz8^UKlRSBJeah0aQmhn?xtHo zM}RMRrR6V->q5n3B-Gh^b6;g&nScEt(QxS1_ZU$qt0I0b>QHGqb}zFI zej0#1%T}elkm&1{zjvMH-maPD+RxQY9Jah3G6HEat7%J(Ewb7v(oJ{&uP(Ya=I8% z3|{P{8~Co3NB0g2vyzkl{$nGq*xJZN!4X>?fPsc4V6{CA8d#-T(_U?%${)ZDZYP!49^*jMM18IIwUL1bszdEq`X=Q0xRI2xGnn z;SDQ8=L}X{*3-u-44^-j!6gFY;Al@um=-%OwEytb?pp5FfK%hpDAH>s#zQz`8R7I! z9VPY_$R}e=3A@{Hxo)^EguPEKyR7PIJ?!JnkvW8OgOwqQ!2ssxM5!*=3FLOsTZ!;2 zY{TN3AIJ4_>6)g-+_UZ9Ko&497l53|ObcesPaA$C#9Rhod8PARdW-B6{AVM9Gp9wR ziW=bSiJdf_@s5I1`5v@0LF&;$c@Zv?xG-{V$|ZBc03N7Y&Ew5^P^gGv`(?l1LO6gR z0U75vo#bs@Ho08}v<}lbIc~TIEwh7DBKuqQ&U}r?_j}NgywajQgbN0Swr-kochM#!5++Y$#zifOSH>e38&7a1CP{pp#pVMeJ+B zdeU~Q=1a1Rc-@N?H_}sAF6k3S<=YNxK-(I|CiH8SIPdBc0Vt$W^`nbtLE`TDm@G>|BK<0ZWB#F2D31+FD+xFG==atYrdQ zQJhV);$?OMeq2PtZB4C;84)cP=fLw7zc65U0MuUzf)8`y7fM$$=8&X1@W?B))}zW{ z|7aSb`s>3})mz_12r`+V-7I_SMumrlb5qr!MWcX465ca~r=DR`)q4Q>4HF8dh#OnN2^kt-?wq#B21#McO9X!rG0yReIk!t2JgR zg{_;2PJO2!OiUZ)WRcQM)5)K0Waw!!tC6Nfq1`c4WW2`_7uZQ)pKJXhcRAM)w&8XU zqDr|LxoU&Rt+iZ{^EYK)=&2x-+KbJYkTB72Yo%YXn1T+;FXhqRXT!yAl@rH>x+nai=LKO!;~i@9Hq!5m;r z2UF}Mw^)x=6}7L=fnKz?aW=dBtFV0{l{R>qOIec1LYLH@BcSH(iCkOhJtz!XD-m#T zal0dP@Qt@bVcx+#dIB0RVnle;^Pu_q@`Jp09?BIpn}6jc*zA%%$V*xAn}Ft{non{M z8(g7JK;y3ISEl&^oAUZSj?}@9?fdHK<0(@p8H2eNY0&ff&u$NDGE)Y+=F z?K;CbXSu#jm*(KeIka{3(bl39v$j%j7_`Qv>~Qu4w3#?NYZ;TY9hjzqzWhfoOj?g$ zg7>iQmQluJbp{`OP`I}s0)W^iU7Dx;@wj;DeTZmQvj5`IBF_|?3Yf`KP7A7cyUfzB zeOBb#o+H7Bh4;6jSS#(|0`^^I>Bl}R*V&_~h$e))kHUC2JuWc34*+2^o9B;x(jJ8s zYsCWzO0#0n2IBlcm;r!;0gdBLHM`|M<=ZD-_qZT<=u^_2i`o2iuabTpScHpI2ilSnl$IfwUa@k@mDc+U5~a4 z3h#YH^J1oaQa@ly6`kF3fE~pkv{nXp`7SsE@?^f4aPD!TDqZuf=-iGpiG&?`cgigJ z>^tsOyJw9<<55lK!Kk+*5V>Y2zy-ArRZi}IT_7;Gd2?IFp+Px=`TGFP95(i zQN?7nCHWBmQUAJ#r_;9F_<8Rj=9bL7J~zm_q=mw_Las9$BeAvb1o)lc#ueFPN!$PX z7~+YcyeqKBnn_#-)Q6g4c1)4q!0_UHZ(D9c0a} z>P6W_(^RGvN`ah2>Ja|N$J@Dail2mZu;}-9}xLgX%hFl}zV^#WV z-kW6aKZr@POOlUTReb;F#rKDk6PF)(`kj(mRJXnQJ+L3GD00;R0?gJQYmkb9egPAx zM3|Iw)HVpsBgIIEMe|qq6{tOqUb1I=v+}4e6Ie83rd_41TFoQuaia)peVL~JZ?c^qg*30yOC)*x9--H29AR0%em@KTcACz2usJehCEI(= z%N95R(J95XA0JdV&p9_QfPX7~DMQfm24b4cfIq0y#1gqDaurhE42g{3LoLMB%fP+m zn5HksOkM{XLNU)nwSnir(z=Fy&-UNHYv)!Cjgw2w}$Z_ z8Mt`=Jtq%%JSUg>k8^UiHj%_%_Wzrnlh-%7jA!($>m7)#ZH1<7dP8WKk9FFbx9nOl zU{1q4`o`X5GL+wM-mHvlNq*-p#K0RSK|cn`w92QBK1 zv~3FBgN{~U0Auo$$gvF=x_J*ecL7fHr}!a2*@s(|dypOf|Jz1#Z6XL8mwcSyJ8kfd zgUkVt4?XaAM0HY8;Mw@S?XC^jGi7;4FpQk{0iV;S_n?Vnu*(We=c)p>nsG`R&KrIY z+P??gDSmf!umjJK9Ky8kK{s}?w}4jH;MF#0n9eD2fY-lPMyFvp^HenasM0h9`tPlV5I{?!9Yx?=g&XqtKhPw6{vqBz>sF6#tWcc1HHPympDD+uiN&3 zMmG1LT_&HUCnw-t6JP{8cK+k|UBt>bfF${^NKOB3B%KS9dk{eUd7k*qIe?S&Jg*zT z6(h{JYCZR#bqA+`!|p)`jK+6ltl%o}UBx}9yxE9=cAXt;*~+(dL)Zn3s8UWj+XPO1 z{7##-85yp)0K5%^zMfCDmWbXixLo{LJJ7y>DgqI>t8yxs5YmF%HQ>vme1gxD^&V6O zcqaqaQ9Hd|k!*brN=VJ$JiKn~+_AoLRQho+@b``CLV3M053UB%@_>!U9v!bY-lnF_ zNo@S;0nGoH9eh{JwaZ^U4@L$aMp}Ph&TJW*fZu@VA6h5yKov&nX@<*C@(hAMy3f17fA#N zs{qfyHXNYrd$-yxa|SvBW4fxj zu$jARnfzB+0L*;KJqXAg3loIcmbc+HRi6NU?SX+8-z*t#-Dcc_NN?4Vmvq3ZmrBGX z71wR2u-^*l0EQO$_#V_*Q0+a^bx?9ra!fh`{o=54T><02ibpvpXgl^O`db+!v{6rw*6Jpt8M48{q`guFm6G=O(&TD0Gsi!(Q4TSXdhZwS@GzF zlm~qRQ0~A}Z8_jE#T`jRI8of^_aJ2$@M{v1 z&bshsa^h3?>#wYp6@X<=rt12*S^f(9~P4_ZGK(^%q<1n< z1DIchJd_EysiL3%uZsUsoK;oO`k-UvgO2eMee4W(Mh^oBC=}fR>Hu_Vj} z{xfX%#5vcNDQ{UA&4ctJH@2+{+>4RR7D1B-eH3}K3w{B5(x!g^p!>je;Fj%4*g)4o z(aB#iqy*p{{so=o^S@#$;`l>=g;eZ10pJ3`=3$qGmfMGbBR#VoeL%wUD5R}r$4TbV zkwJfz?7!r%V_xJ`xo@L;&x!$!H|YFO4??cKiYyuu2gq*vQj#g>KyEE@+vZ~Hvg~hDUdW@~0h^G- z#ly#P_*$<^;Fn?js7KLS{kF-);bk6I1uzwp7l*5DX~4o*IID)bK*s~;)1koc|F2*r zCica%kQy=Y%zo!qJ)pxCDhF3BQ^GfoVoCU6zuw+xv28sD(+$gcqsucyZU+e<{oqrbQ#Y^yn^9j za{M5G^sCM|@D?=%yH3ttDz{WT3TVu!0?tWKM|SwkhB(_UM40;RZ`*F$d8BD5PcB_7 zP>nG9lzId2%OHe*N7in}2Lj0<||h zZE$}({+IX%Gq_U=ngAfAx7$PoJX9KAxA9Bc48mukU8(X$n>nHaQNLkWqH4;fWM>Ac zN-NM!sk~md!F{?Mv3$?1gyAovk^32i>h;8*`orlKC#r!T52Qx#U1xcBFX*x;F04+G zQ2F*q&UdqzUS3)`s4vBO&!n~H%glUM321K1;cq$bfceg@?8Da|r(Hy-AL8F$Qtp`C zgT#nE6`2kr`V4PMo@X=uKWBpizRjt_taj-PsG={-3qR;^Kyh}b#nxaq_4I$BYP9_|G16TUo(doUn&onN}OJ;zm zm99f>_@C3|UmBV8SKd|b9{glK)OIXc29Cd9(81~z&N16kvE6VpfX|uL_*3E@X!pv= z@m-~=b}K<^FH_VZf4lXNheq2ff1G#K6{1<28)_f>PF(RJQDTpLMIH1!zL!}mJIxH5 z4V89X6A@=c7^~nE{y3Q)f4rHqe)!Gf?e=*-Htw@!v#2a{^*IU$dXmlC>K5U()A(nx z=`PYwelhMm@fuqgF|4oVlS=4)AIX4?Qk04K?%)vPBDa3YdFORUctzMa!M>`LG~+m5eYn(}x!2uuTH8CLT1u6y{3hlL z+@~Qa>qg^e-9>SdSM$nss)~8(d(s`$=wu9H9r7{V=)FnE>w_pE`(_QO210bXyS>Mu zvXcxSoYf_=S800b(}*I;d`m4`$leMkPs*MIGy5GnS2qu{c{ep05p_C9bnl|pEPQ@ri$0wnA#RI3e7Jk5pRB-bMp*Oa9Gy0U)BOqOuFy?$u^-ea5Hoo z7dQM$yMRJaOC9sHX8?!Jubot$J0+7mMk^^o9v(y%{xLd<+&H8x-uD;=b{Kej#@eT3U)KkPI9$VpjM;5oS&K0Fc9>KV z`J)&*yr1HrFB&Vtr^Yk=sO!V9piw=&zw)}5d!z^-J$RK$_Uh#yeiv;KR8Pc_Z3o@6EUI*#mh-gobhhN2745?Qv^di*3fE=)FbSRH^{QBQO{e7?rq%1mkDM5K}6 zlxHr2kDWc#g#O2MgeyE|&8^k!3*Xw^yW#KN>V3bGSk`YDi2H`oXA`JoPT8$?X~z(6bX(gbvkc^%k?W$oR=tQsY>du` zcblA0!!p7uF}J~LbS}YlGhAvL1Nrqoxz*1wVqbMcJnKDLS)igkM4mEBb7YEleQy17X!ie--v7 zSpy$I_t#rNNl`@ffRDe5_hzk=4v=&{NLZ!L5KLu`vP21rmAwF|NK5?o_~Sk>N~D#Y zy26$4DM5hs{p(2bK1sXM0^c6muF~<;vx%V9tl)$o`u=76-Q4)R#Y7=NLj!qD)is3Z z*{UJY@-FG-9x*B_LmJfHHU~Y%}4fr`l+CJBiotOs)Z12cx_lfR31;?^OD(Mk`is0dW;SC3-da|>6uxy zW{}-GGBed8S$YwS?l>)mL%r1ZO@VVm&wSOD1?`KgQ=0gmJ+0(-x-8W`6m;Xwhh`YnH%}KB({%*^bN>y}K4-wP&4+CR4jWEL_I$ zG~CQtD0c>h)kx>#39$x&@TWhE6U&*3x}8QeUrJSqQb!7+B#f@}n1g6Ug3D=b-=d&e z_gV;GYY*9q%n_!OiplGC8~?$|r{`j)KxNw>nm5@Dl0`Xi>~2VgkI3GwXtlmtpWWDM ztYuua$pNtLo}B~)e|Hfct~td*z6ZS)d^cv6W!Lz*3S)JfzmFppomPR#*C$$vQu39W zxyOsB{^jg7*{6v#7UGqiJ50S1(`+;Fbx?@wBHor;=YM1Gt-jid8*okBp*RFdakl`$ zp}0eF_uvu;PAM(!u0e~tYp@n~*J3Rsc##4vQu~JU%{ph+xtXh3GdJ@GFH?_=(;iD`*#z zx!FAx>$#>&G3HxCJClufm~u0aky##7A_BO?>+MkVxMxWtF$nt1j3AiavbU&~g^v`?6hG8w& z5t`xrv_Uw9Xb=2awan*Jpt{yE)FvmED>HT$Gr@a-6YmaJbg-ijZ zeBz|m4(3)Hma&NwK&^ToNdf}Mf$G-I&e)ST;gsiu5J16TSA#7oj+)T+U=O&JXSxtV z&*uNmo~TJ;RoldIi|94;Y{xFwB42#}OERx_7OI+WDER6}$O!6LkkI{;|W8J`&n%BYFxt{_0g0~zwVGWr<$G+}(w9E-jJ zxreae2`0M?ts3_Yoi>A*(}QJZixu+*ObmQvFVQjOl=YMG3nC4xLbMOJSGJj5ljbxV zEzC{9>|4)#!&%H-p6R}xXdS=9{N-rYhg+oDx2 zqiqhyNig|<+@W-)_+=^EFEwK6p+ zED^pflg^u7%63?lGa%fp(M%W(5+$eCEJ$C%_5mPa*Me|5NW%GVP<5oglY@?`dj2-( zpnWbpO5bpoHvop2E?cL3!W=w(>4m7Eov_2MwAxT`{}TNLB*{f4hG=H-l=ZU41Y7r( zZbYfSGNiD%-8A)D>k|)kWD={Y=e54jg)(M`zo4eN5zm%9Q@oScxbo1TK zd)XtcWYi=LXnq^7rnkTX9iR4QoYEg}?&xmKtd@Je+)P+poPYF8GKbXpt}?p-y|9~= z-z3Sg6_0q(7bytz^YrsZL=|Ai11h6UuJ5R0#9`!d-?iK^JX0AP>M!>;S$gv9iohfe zlAX7#r%dJ-X>$@9DLOW_m?_PojrRc^Mk%7vW@CspyohVrCJJ~=3E{Q~^0h+wdovns zJ?`lK^w50ZwLy+O#+3(E5=xEI`I3mNBJ<2b51VZb~`gL2-r#9I)GV$|)>+=MOR-v zUg>O<9DAfg80jFZVVwCWD^5Nebp*e1EpO_C#zP*)ku`{)gX*X)qPbdp7xnmD^b7@ zqiz{d@h|TCku}}x05v%Lc)`Fp#B)GA5#%poSRk|dYs2^yAk1H7a4~dH+e=pNWH2li zgI&M0&Mu@$MFBA-#ngvoQw+-<=Z|SuaU1v}Zk%*aNSFhgdIDEIFLFtBxuHp>pz5Uh z%L{yDBzbLYmqXB_=Q=LP?QEQ)G<;qu`+HmI;xTf52Kq<7F3|}dgY}}5B{;Ln1&{`> z&Q?@D`Nm39cM3k6a^+Kvp|2PMdUSm}Vlx>9isDCsQjk~~Lcvyu*5^xwxt|-m)a?Nt|u_{Wq6Sn#~7axPkJuH)ai01i8X8C+@No<=6w!B`P&8iy?3VgFP zqU+%_Ah~$slyznUqE0+pfF;{4w78t3ql7y8>(d3M;iGI7dRmh3zoe_?LI!wZLcd?+ zplGIHui55vGL)*DN#-bWB;rw4_pLi}_%4VIB|hY+ypOaQ&nKDaAk##WE!oL z%*`{Z)po_E;b&Op@}cS!B}(7ymZp4*?^q-kLRLlQ=+hV``X(1KmWW9nMczYQ)TJn? z!C}=v?oHO4Q#R-02A6H3ti?6v-n(vwLA+V>*`UGQ{@x)?gyHr~5t^75_L-Y*3;Ckp z{2%#s{2Hfq*GAtb|FSlX352|6aF3a!oLkktxpJJshZ*u^VhW9e%;f1Di40ARWS;8A z(Z>_Wli->iSY}a{o*nJZlR|DUng~52Fc6!F)>ALXh&xLFdTI_ySuIBn7W(K=bC`!) zO+}uH!CY?@0OPOQp_e6+c1Bgav1H^KVZ-{y*E&CMl>X)K-8e?c5k_IxwjJrwVK-ef zZc@`abR4{LC3|X~^l6w7c?t%L??mVIM$UJBt5Fxv6Zyc{gNj$8Vx7n;w*wPjo2ZXD z;Q7g3rWLlTA?)_fX|f*iMWJ73gnt6A<~9}ogGQRfHszw>JXiJbmp_NcRlPG>oc5wp z#j3)Sz}w4rL-Iuyc1FKod~NcP)r9r1#RrVFd4p4LqisRpt3T{8gJfS-Lj`7^@D>3} zj6%_=qy!08kDLnT6X1+fiu#ud0TS*RAE*J0cbQn>Z5tCRE+`XF4rk175?TD#Jh0KR zW9Yf<-UPzwTx9V@TJB_b4lj>aF}PI6{#AcrwYocB+r))Td0wE#ycBZOB2h7Lt~SoZ z+f`^5l#84?63Ov79O9>+Dg6Y_b=YShk2Ix8fKt!Gc#AuhXg70F76)G>d1S)A?sP%K)gd29%B#)GZ)}l!yh5%!2ly z#1}k8Q)B|=E`-5Z04JHQtgAv#!uSCAw-b_sU!CQL)TYuPkQXaUc~$As_oaf>ew7bk z%cZUQ6!4VjI*r3?#`#QQjnUS!jK?pMMfO{i)718}KAV83SV51d5#PgGOYX$c)y`VM z$(4+ncu8vOGj zM&-B<0+1$lz$Kz(?+8;sOqu&EK>)m@^DKs&kD2rmGY}nf4#rnu6Nj3tby(>9o<I!ojG_s7ir=6%0*3CugF!*}G+o}O^?{r5IYU9Xnx$8d$ z)W&1wu%|C}cJG~wZ{prF$cUN^N0nPvlB>zh!IzTJzi``pIMhd^ns?su9LRXkB~Kso ziDi#{&Rn1Z=v{(6?Iyjj$fW>lgL={ECLY zO2)=k5>Vu_g?sGUyi)s@_Uw5+BO9>*Ew~@)i2Fq!3w7mu|L#+{1E%cV|5W!~9CVcq@3BHUU@)0E(0WGhlM zs+`~WtV1vVl_FkO80l<*-@DZ|658JetBnBZlp~H}WoKs@qOncNOvob+`;C zi(y+%Q|UR-iHc_E9A!&A40H`mm4D}zHwQ}QkCeoN=J!L8M+PAm?a+C&6woUz0A-Dn z-r%yN_YoS8CA5jVib2k%%e%s+th)1HQKsXTFNtt4RCol44P&ymFsWRXK|E5gx>}ZH zaUESu*G;`^WX{rdU94izyq1KcHu%PCw2iY)C|DPw0od`Aj!&uTee8_UT98$AX^dbG zze;&FPNt;Bwq|?*0l-zkM%OZF0K=*J)zRqgyx)$Q?5%7^BqqH9&yvPSWDUWyHIg;Q z>GU}OEdjqW2xItc$<%q`={VlRui)Ot$n;LDebsLu!`aTKqYPYX!Cwq!FW-O=F1|0T$3Y%}#@7E)j99?l6GRWp{ zmI?SSU=4w$=y)G9h^dB7N>ClEq!awb#TloXiK->X!m9f$L2TGE|}qkzI0Vji$uJ z?p+<)+I{o3alqBHNaZhnQDP1hV|w~I8=TzI2U+xr8s4Z79<`g;kv1laWKCR9rM5U) zM&LKUKCg=Hi-w1pqy3gOC2yfPsO3YZ$oJLrCtr(vtA_fdi;q94097U?8`_h$6)z#r zo$iK6DE1{ZZR9=YANDNXm{6|&kNQ@}wv|#DWId-#GE4-Yj%~ zYX-rxi{cc?>EU?A>HBl#3$@Ks&-sqqejTon?$RF#u-<;KkIlsXSGkcdtib`6yffhW zEo+tT`16sfg2Iy*Z#*KbCl=#xSpSuGHdRH6&R53|uF2IU))| zGL0VRorX@-r>kVWR0!|F&-+5{# z;DAb;UM?G|0j0z?mD{%o5qjwH#)~37)#t5yJ7+yT!m|NvEUN1Jr)RucQyy-|E7_w! zGEIS;#;}4*u|Hq`reyg+zgb%Ecm)4PolG0H=u4fdV2h^jyAPN=pPeID ze+1a~pJfp{?w{W?$A4)b_|4wl@V>a(IM=$oHf;d(WhUXqc7SsVhb4q`mm^)B5E&jM zbg2v`x5CCR3h6(24SW7h zsBwEcwA&{Xs9U>rDmaoKj*Z@sDn26>GWz_U7REhAYqz|v1>*=FCz9CEeVp646OL(> zag^ZC5pLBoWd;8oB;DB~R(0VBRxOjgq8M`;0}&Q%zOl8w)>$}n^zpo30np(Sheo|r zJdUa?NIZ&|Nva$84ekvxb(XfK>KahF51rUcSHJIC8JpChvY5u(z-F*;bwB>4ytWDS zcSJruM}Ji38lIs5JDKJB3Bv=^Q8-Gks?t9mAq1~no}uJOl@Nex?+2%2>z)k@iL+~P z%e&Q0iz>e4^nhM@U0qtT^^ij2JhtT+7+wtCg^NzWeIi})37hUr`ySqX-;-X@3bp>G$6lHlvf9)={aefn<}qFHCfY66A|WC>=kpFjd*(E; zkc=@aQGR-5YD?2M!vQ$+_DQuxZBl$;E`_)jLg^A3WchJSZsS<1Rx1T~XZoIUAFqZC zttIyUBkyAns{BjnN8+u(&)Y;oJ=$;OLS0WV7$-I*>^1j01OA);?!CEj`2X|Y{|EA4 z+se4V@6ULt7?)qu&eGYe$hCrs#FZbx4N_WGtAh)E)xu^Uc$~vup-fO)zHdAOm#zur z-PLx~9LZ1{Ku!^~kSg*v$%H=<9@6($DYvS3yq(DfKD0+xzO^!hhpAtvGXhj+DMge& z)7K)czDlk`NvZ)$slf9eUD(0U+Jl0VRf^bt(Hi#I)uvzwaiGJ%Ub=6#7M%mpb$oH_ zgzDt@gx;*YNZIEaUIqH<`Iz2uHqNekMYcG0-rQc9s>3&>pgES{WMwl8boLrOaJ3^H z9htF!sI_pYp^U1Fm6!xHy9v1Y|KYggO zC12?9WHG9@tsa171+3D?bRcP0H_RP9jA9N5HQ>m-KY?D%hL$0zj@y&8X2cSb{(hP# zd*Ia7&iHT@1||GvAOns^Ccvu+l@|2TZ1!HH6G7M3oQ<|*CH*-gynZbV;{fkI_LtTV z-dA%{mCK<&gf42>4Vkl%VS30v8MKr0jmx+ggh5lcxuzbf%uOwcz#$9t1~W1SdyZ%Y z>=F7GZ2`=^9XT28@|Tj->AEtQ!hPMOVv`}mY%3$?jcZ+(OnT1hLna7^u7nmg<>964 zjGO$9&!@tW6Afq1qWrYE?Cx0q8!Yct0Y?vCI0+Z~#(SRxmK=WNq6fknI^kUsrnY=Y ztaZ!Gy>v_04=F4*6S%3~$bi4K$`9CPgRB+OWu?Xh|gPm27wf5a&U9D^8RCjW1q( zdtI@^>Ut*(DK_F*?Uon8)uUFzwq@^?g?MRED?ZYd)OWf2oNGr4c8Q_3#GwH-6bXCFT?0=t^6nOCjbp z2yMU~-?wJyb6QDlWe^Iq8N1H3+FWi&y-0J^03twtvbYYncKeZ1-eMEyqW(L{5M4yf zWwra#AP@O2Tf<(m?&=snOGzMXIt2K;wM~880u)0*bftz?VMp>;u|I`-|I+K-2m2aShV2APVZXcAfu4#s<4jqFtYl~W# z`A<(qTh|a*O#dvWw!S*MpSYTUsh8qcO=+(*rW=jd(CoS#4G7f>tG1)}GnDkeS%$mS zV5F77BK{Zg7aMyd9sD@u(H9n6FYucw7Z7~QQP!;tS^5@e8|6(-XVmPettTp@B!x$x z{{y9jcg`ZTcxr zN8;M_P0aA)ot{}eB>D$x$+)O9AfdgSF9BbUBDbA>;)BgGZjYiP|21Z+#<7S@PNr!E z_u#-!UL#3L(TKJF!?V($&G>z(}k^ZED z1VaK_5iSYApW)`uBpS7YXy1%&+J7XFPn!9a@5Sld?!K2=Akt6%oFpX!XsAszG!!RJGK@M=NZsrl@iSPfGoN`(xuw2CoT@!iUcG2Sk)vom%a&hotpV z=2?pG@=Ik1Rn2oC<{zK4oOTSNXOBd+lZEW`YVd~7gu!k7`rA+ zQU!eBE0Jh$fm~2Uai*bUpkcq`Flt*^JhkEY1GpODbZiLZ3w|`tbxrG(xi9vng8_ZE zH|J5K3=-7E{_=}9bMU)7fDNgdX?aFLH+HCuAvgnhHbBS%5q1FJ5G=y= z`K6erB{=mg`_>5PX1JnrF|vB!RgJQH&Ba|uig&&Mm#tSRJRhv(kUu<+BJTCYuQ`IB zt>CCxZUa1)X!lu$GwqzsC}CCZ^{o(aN2RXma^JTqDvQDm6EBK-S*<}xvEkXVR(=(< zJ*40gth)5ETith#4kBm|wVL?SmT1>(BL=2eoD&dRp+%H1 zLjf#(Yl8E=UEAu{+HswTwNt8w2qrzfb-y6a>KJEW{3T-+?Yx9!w^wCjS!0a38mX4= z+c$#w44EhRfw<-v=BaJ{`QuaV7}#78EnSncfewlabj-G>))A(i?s@&Ma!rAGu}cxL zXd5*VA-Ar10i#tHn-rC6EAl}_mYPHhR)h=(4C>H$DeKOy#12)JaR;1|pOe)Mjzxv^ zd3a_iZSB&zH$M_pSZ`KLb|~7p)wR7kUTuXkf+}e;uZ;lV3`S>lren z;qWNY7=FnZ7b!f}o=JbZ%YY9S9r9WCKYrCTfLj9mr6WY7#&DXXy0t2-;^%LQb&g~r zQz_eur9ox&m*|gfv9cZ$<%`EoFoJn8l%_hTcZ}M)_d@d6O!164kw1wCGU6*?nd;jr z7BHq&3^cT-8c^a7l|u?CBu+PNb+C`e1o{Y~GmjjnErXKkx`El~6`_0k`{j=MduwMm zdP>M&(X)OPvlczQxy?NCAwN%`Ruy@nxyxn}lD+9fQ@EX*UJH%76#yid6#yA$70*Yb z1H#-hskQJ^ju!oT;NHV$o(zWb(@Gl!(%>>>AO<#hmRBh=OI;qSp`)@$`SB@P_ ztX3O1{u~9JlaXmU4$w*tj6LUZrp|;u-7*O_y^Vqe&R$(0`)+2i?svqy?o+W!Ont>b z9ZK9(gkg$bOGWQDMs1%L>hChyCEwH7b062;00}kXtDgGw@ThsLIY)jaM;^JvS7tHs zYM12#2mgsKRq~%(MduG_5+Lb9f=DIylEROt8HTIoAt~}>$Q52)s+Cs0w<0@|^N>Ri z|2#T^uHiM#$O@kQj;ms!IVBPZW0>nZj|OW1^YpL&)d^zXv0ee`+Ed3u^qTi$+XX3( z0H9#e2@Y9Z)Cl_uZMLySoqVqueF}PX_&PJ7OrVV~HM9cZ!L~F>m%@SQ&cPJs=_ghG zTv4!mJKmX-vZ55>W$|5IfEGjkn!hsZWbOr}X(5OnS`K0;yT*H-Xtc}HNW6Afy&skr zRxP9d`I30bz7mGRkO+rd z&>Fg*>tyDITH(VsYCKlrJO2*cLDgz6zEA3oMOOU`LJ1jATQOe&W$)SEnY@pIplO#L zpC{pK&SfI)2Aj~$Uk&nQWYF64a{%ifU@BALLv<+Jm=@4S6Sok*a=Rf4rOM>s0p7mT zu{H&+w_XsJ^rr=k3QHX|pcR9?$Cb5p#LuUiFmqLVsezSaGTHY!sV!ii=y{F99JArj zg%5W|K!Gj1!f{sY)>pIDF12P;rCr4HuEv7e(ijuDghVXOstNpu_I|+Y0q{rHW+tR` zQY3`j_;*Ge$JgVlZD;@vQ*i+Q+IlZ*TnR4Sqq>=TYyBMtjK}lx+0(vf<{A(WZm&)E@ zZ#N?Gn0!_-xF={t4aZm`UGH&RD^^VhIwSF?OP8XooT!212tp z&|>I23hPYLRAfT{&T?COl`E=L z!yvrc#22~xa+J$TXhDis=rD!$=#k$4$`~}APv`LGW8iI@v=+Ta+#s&cC`F~FDVK1B z6~*>h%oxo_H8)zMD+93*z^8e<_+JBj`+z}DRA%|3>zbL01{*4!_5tI`f8-$PC!kt4 zC2-xMZFPYd?rLh*`&Hl^zd`TGF&V-U68)TZ<4+TrQ_yE$X~Z4n`A^;S*E9EY(NcEqHHA+QWv}mQxOk_&qA3`i43>)8xD`-!vV1X4%iWcq?w~`R!WN=9B z1*o9YwD}>J4=EnRx4v5Xv-aYB+Aibv_N~+fh;QsyuF;MJCAMPuVI9gfEmyqUsHM=) zs9~KNJ{-LqO`}4;C1XGI)T&oPu#PZxM!HsYWw$2wT_8vp*`(=Qv`v;@oy<`Nv`?}Y zz_vZj`9?*5$5J_UyGfpOTG)7fQDctc*kltt8p13Nn@;8 zFP^tG8l2A#CE!DV-$&QlYspVb7Q2&G$4`U*jy>dl#Jk(#$kl8@Yy3xcmM2c1of*_7 zGH;&uj z+<_&K1|3!%*u&Z^5Ih?qqvjz)PoOztYQCXRO@}{}2qVf9?lVNbjB&0G$g9(~ENtUD zAv^O!Kc#F-5YxDUPqX0X(yv~z*;0>+r&edE+E9)9cA)ZaDaCER;Gnd8Xc6Bq+*mHQ zi?clwE}5_V9;GW$`_uj{I~$yt!Mm#ro@udN4HjIx6F6Ppzc+YSY#K^T>h__Z31KbJ zSN|Co%wO!JLCRsRIn+4Ko0BmF>ePnL+E#q8Em4}J$d~;57&oSARPv=&S=?G5!wAS+ z+L}iurGYB*lP9TT0Qf8=8+8&wW(qW)Y4NERu$%iQ&`v7bsCA1p5@N^$C;rqx8@ z(}`fe>z*;e6^)Zpo!ZD@dFUq)Q~tovU(B$%8PQB z62FCL+}CzC!EYzp$lV%%$mI7`bE^!C=Y<6~a&H{xIsnHC>52l?V0`=MzyVLM%BbOE z;szhdxkD26gg}^X2~r%~Oypdy6(d4fin8`1>gvi#dkf9fVZuR3a~fr7<5>Un{ZVbN zF4qu1R%erL#oaJZEqgH68bRWiGW4N~iY(z=v@Ij;bLh8}JUiS5vJ8T$_`U7y3U(ob z1eW2?{kawXJ6OW&SBYXfgJ~N=O+t%OM=A}cg))o%iRyFEF=BaXzRnZl4!{e2Goyc5 z&`j<4&0-jAY0E{a^8|njnIS5s5Efydj{D(l9~}DbRkTLd;9lLaf1o1NY(vWcbIyli z8kYm#>JSqf5$%26*4eqHWxze$hz!lPS}c(37a1+A+({=^;qAZ(zu6>!HCvopyvGVV z9X$OwdEsZ4HFx*&@Lzfj{vmt4`W#9iy0l7j`{Y!`)e>T8y?eQ+@!S zgp7tUk#Te@rkO?<5mgzhce6yboPQxE)Jkf@$gTpT;;&M}4yG}ZlBxl)QYnB$NSdV- zf4t1@1I8UC`y73gMb?HNAH+3G zT8nIgN7Y>*Bq$YX>A22My;2IJ4K9eyZ?%Gk2h47?LTq3KBX{GK^Jl5lK08SAfr zX**SwGU;x~?-t775Bk`e9kwp}T*9O{Nyxs1UI~6MbJ&VaXUd}w@HVB;7+7d6EMxlVQqj5jmE9x9LB zj1}4y;b6+fJ?Gl6_`1kW2=OZy9qI0As%{CUgyG#{`jR~wwG7&wz| z>PkQIisjS9V=ICGK?`t%A9k+$I0$;$33YvWM_FO@Wxq*BrosLu*Lt;*)qIb%ZzJm;q6&sxx73{oH4|Cw;4*d(&tHnSMskPqtIED>pc@a7#Lo*LLCS zkiO@~VVGtlYK`qT5Ahkr0Aof>75E4^YY;-25Q5Hg%CsslTm>kT19q^9($Ll?va?O2 zFnY6i$m!hbanV8F{D*S_)o;~%fm2<;NLkLX;R#3{zKvFilU0tG$0W>jBG!5AYzn;2 z1#JR1vyIc)M~ah!ZdF+SJ)RNDk9Qt4Q_?n#v%z$T-iJ7{C|-kqwN>$Ga72wDA~&;G z&`#~9;RCx-^f-F&&Sb{?Mqyi--9w8ucKq8`^ZdGgpt0zWD>Sr`0QW=gG*}<3oO=vO z2*{Oi+WtGp*%W6uzm;J_uqG%W`*D%4cEcGbXQDt?6j^`e+0Tz0cxotCLjh9NPbcU5 z@TP>lFlU8DaLT)E;$F;}IetbdnHd7KZWoGu`B=PwMrS;UV97zX4K{VZtsy@}UuI^m zS*|0gCXajyE3A|v_bPg;T^+od;1>N`Ve>DOwASug_-p43wCMay$*Qbz3|zhV%hNoo zP?EFpv0Jld-L#1x6=_|NoP^0ITTV?nMC z%x{YWycoMp=R@`1!g#;WTx%JEjYBq|vHXWCo9JjEWgKvbqJ}L{;!G|$QxjB8o#?iy zN+&kjOQHY9Bg>_x5gofwiF$!G_=AAj@$$ygpH;r2xgv-s3&%hB3l8e+38ZHgz_|p3 z*1vtTv5#?kGB{kFvtPSkiZ{Dr^?7>=l{{Frtw2@Q%$5~C@3of1hWxAR*!*Eos(S67 zG-2+ib)&1U{_2G8+*3pWXSA@5)aq5`-3W@7?k=gkE8WqJTE0d*vHU=l&G3Zo#t|OdyDRCaLbW5lW>WG_1e}NO zwiw)OWb^zmcBx=+O%YR75tL7)DF72E=DB&b>%SarF;M7enAW6eu!mg-EN8o2x$RJL zX=57F%zVX=abSrjZSHt#UQ377VYJC+p@E_m;;w7W$-Dv;sK- zv)p$q%ni|;wM}2e&a{6oVI?XENdIcMHWXabVOLdU9mMT8p_J%SyP0>WBGWGNDfZ*fVZ24^ewHtR5D3xlk*w7 zyfe#XIHQ6uLJWy=*)j&~&M4v~T>C6&?FFM&@7`m3gQL<~%); zHy2g8rkIMB!zF0;2iB(I-dLod`8B#*ci*w1;CdXwn66zL3&Hl^YevpF=9cCnjPH`e z(*9jF<0ELg`d;Dx`S3kgyP4{uJigNf^zL-!RohD>N8jz0oW!GCsdemI+wUl|c7(3S z|6r5rD`u$}Ot$a*%qU=A57exBQKw(;Q?({ppDjB;xvw8|@`QHI({=I;(n}+0{}kg~ z_8+vct*|#KKh#2e7&hwGm5l-AIo&(&);ls2)+g%!iwWs3TzcNC_WZBdoMzSb;r|NF z#sBud_Y17KZ+O|+XPwhh(eNXNQdytudEWWX>6hj!`jD^ZZ1nd`%eD>gRni^>|EGvl z`Ir0qHIDyNM5^@zU|aF+yX=2|_dlF$CKDR07wdY?eXk0@pJ>w04Yfjk(~TeE(uBiw z`C|q^I#XV*pr>>42VRR`IIEw(I5KnBjd#nFuhfejNOUy8h`}R-SnrAV{uY)nE{FA9 z184SLvySE)R?3Kq^+mz35fL%MGPavR4-4$YUp>TMx%FrZXY}(mA^kXrp*>?yggDgzp|KgwpOy?Rw^`aAsh6DMmun{dpF&=HUCgSr&2M$>sNqfI#+l9op;Qb3 zOgUFCiv-55c8bqF{g!nacxT{k#sL|-j^CMZ@p+r-oF=;-Oj1#oo0}Us;2Mh%ZT