mirror of
https://github.com/ROCm/ROCm.git
synced 2026-01-09 06:38:00 -05:00
f9bd22626b
* verl compatibility * add Supported features Signed-off-by: Vicky Tsang <vtsang@amd.com> * updated and edited verl compat doc * added links to verl * add future release for sglang and megatron inference eng. Signed-off-by: Vicky Tsang <vtsang@amd.com> * fix lint Signed-off-by: Vicky Tsang <vtsang@amd.com> * fixed a typo and a table * Spolifroni amd/add to compat matrix (#430) * added verl to compatibility matrix * small change * fixed an error in csv * edited the verl compat based on leo's recommendations * updated compat matrix (#435) * Added a hardcoded link to the verl install This is a link to an RTD build and MUST be removed before publishing. * Update verl-compatibility.rst * Added a hardcoded link to the verl install This link is to an RTD build and it WILL break at publishing. It MUST be changed before publishing. * Added version support note (#448) * small fixes * Update verl-compatibility.rst * Update verl-compatibility.rst --------- Signed-off-by: Vicky Tsang <vtsang@amd.com> Co-authored-by: spolifroni-amd <sandra.polifroni@amd.com> Co-authored-by: anisha-amd <anisha.sankar@amd.com>
86 lines
3.3 KiB
ReStructuredText
86 lines
3.3 KiB
ReStructuredText
:orphan:
|
|
|
|
.. meta::
|
|
:description: verl compatibility
|
|
:keywords: GPU, verl compatibility
|
|
|
|
.. version-set:: rocm_version latest
|
|
|
|
*******************************************************************************
|
|
verl compatibility
|
|
*******************************************************************************
|
|
|
|
Volcano Engine Reinforcement Learning for LLMs (verl) is a reinforcement learning framework designed for large language models (LLMs).
|
|
verl offers a scalable, open-source fine-tuning solution optimized for AMD Instinct GPUs with full ROCm support.
|
|
|
|
* See the `verl documentation <https://verl.readthedocs.io/en/latest/>`_ for more information about verl.
|
|
* The official verl GitHub repository is `https://github.com/volcengine/verl <https://github.com/volcengine/verl>`_.
|
|
* Use the AMD-validated :ref:`Docker images <verl-docker-compat>` with ROCm and verl preinstalled.
|
|
* See the :doc:`ROCm verl installation guide <rocm-install-on-linux:install/3rd-party/dgl-install>` to get started.
|
|
|
|
.. note::
|
|
|
|
verl is supported on ROCm 6.2.0.
|
|
|
|
|
|
.. _verl-recommendations:
|
|
|
|
Use cases and recommendations
|
|
================================================================================
|
|
|
|
The benefits of verl in large-scale reinforcement leaning from human feedback (RLHF) are discussed in the `Reinforcement Learning from Human Feedback on AMD GPUs with verl and ROCm Integration <https://rocm.blogs.amd.com/artificial-intelligence/verl-large-scale/README.html>`_ blog.
|
|
|
|
.. _verl-docker-compat:
|
|
|
|
Docker image compatibility
|
|
================================================================================
|
|
|
|
.. |docker-icon| raw:: html
|
|
|
|
<i class="fab fa-docker"></i>
|
|
|
|
AMD validates and publishes ready-made `ROCm verl Docker images <https://hub.docker.com/r/rocm/verl>`_
|
|
with ROCm backends on Docker Hub. The following Docker image tags and associated inventories represent the latest verl version from the official Docker Hub. The Docker images have been validated for `ROCm 6.2.0 <https://repo.radeon.com/rocm/apt/6.2/>`_.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Docker image
|
|
- verl
|
|
- Linux
|
|
- Pytorch
|
|
- Python
|
|
- vllm
|
|
|
|
* - .. raw:: html
|
|
|
|
<a href="https://hub.docker.com/layers/rocm/verl/verl-0.3.0.post0_rocm6.2_vllm0.6.3/images/sha256-cbe423803fd7850448b22444176bee06f4dcf22cd3c94c27732752d3a39b04b2"><i class="fab fa-docker fa-lg"></i> rocm/verl</a>
|
|
- `0.3.0post0 <https://github.com/volcengine/verl/releases/tag/v0.3.0.post0>`_
|
|
- Ubuntu 20.04
|
|
- `2.5.0 <https://download.pytorch.org/whl/cu118/torch-2.5.0%2Bcu118-cp39-cp39-linux_x86_64.whl#sha256=1ee24b267418c37b297529ede875b961e382c1c365482f4142af2398b92ed127>`_
|
|
- `3.9.19 <https://www.python.org/downloads/release/python-3919/>`_
|
|
- `0.6.4 <https://github.com/vllm-project/vllm/releases/tag/v0.6.4>`_
|
|
|
|
|
|
Supported features
|
|
===============================================================================
|
|
|
|
The following table shows verl and ROCm support for GPU-accelerated modules.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Module
|
|
- Description
|
|
- verl version
|
|
- ROCm version
|
|
* - ``FSDP``
|
|
- Training engine
|
|
- 0.3.0.post0
|
|
- 6.2
|
|
* - ``vllm``
|
|
- Inference engine
|
|
- 0.3.0.post0
|
|
- 6.2
|
|
|