From 66211e27b6ae736f6367cb6717fda7d8aee4648e Mon Sep 17 00:00:00 2001 From: Jeffrey Novotny Date: Thu, 29 Aug 2024 09:45:50 -0400 Subject: [PATCH] Expand the section on changing thread affinity (#3653) * Expand the section on changing thread affinity * Clarify the methods for configuring allocatable memory settings * Small correction --- docs/how-to/system-optimization/mi300a.rst | 24 +++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/docs/how-to/system-optimization/mi300a.rst b/docs/how-to/system-optimization/mi300a.rst index 9e80f19f6..897db8b6f 100644 --- a/docs/how-to/system-optimization/mi300a.rst +++ b/docs/how-to/system-optimization/mi300a.rst @@ -145,15 +145,26 @@ This section describes performance-based settings. sudo modprobe amdttm pages_limit=134217728 page_pool_size=134217728 sudo modprobe amdgpu - These settings can also be hardcoded in the ``/etc/modprobe.d/amdttm.conf`` file. To use this method, + These settings can also be hardcoded in the ``/etc/modprobe.d/amdttm.conf`` file or specified as boot + parameters. + + To use the hardcoded method, the filesystem must already be set up when the kernel driver is loaded. - Add the following lines to ``/etc/modprobe.d/amdttm.conf``: + To hardcode the settings, add the following lines to ``/etc/modprobe.d/amdttm.conf``: .. code-block:: shell options amdttm pages_limit=134217728 options amdttm page_pool_size=134217728 + If the filesystem is not already set up when the kernel driver is loaded, then the options + must be specified as boot parameters. To specify the settings + as boot parameters when loading the kernel, use this example as a guideline: + + .. code-block:: shell + + vmlinux-[...] amdttm.pages_limit=134217728 amdttm.page_pool_size=134217728 [...] + To verify the new settings and confirm the change, use this command: .. code-block:: shell @@ -240,10 +251,13 @@ This section describes performance-based settings. * **Change affinity of ROCm helper threads** - This change prevents internal ROCm threads from having their CPU core affinity mask + Changing the affinity prevents internal ROCm threads from having their CPU core affinity mask set to all CPU cores available. With this setting, the threads inherit their parent's - CPU core affinity mask. If you have any questions regarding this setting, - contact your MI300A platform vendor. To enable this setting, enter the following command: + CPU core affinity mask. Before adjusting this setting, ensure you thoroughly understand + your system topology and how the application, runtime environment, and batch system + set the thread-to-core affinity. If you have any questions regarding this setting, + contact your MI300A platform vendor or the AMD support team. + To enable this setting, enter the following command: .. code-block:: shell