tinygrad/docs/tinygpu.md

TinyGPU

TinyGPU app lets you use AMD and NVIDIA GPUs on macOS over USB4/Thunderbolt with tinygrad.

Requirements

• macOS (12.1+)
• USB4/Thunderbolt port
• A supported GPU (AMD RDNA3+ or NVIDIA Ampere+)

Setup

1. Connect your GPU

Plug the supported GPU into your Mac over USB4/Thunderbolt.

2. Initiate the driver install

PYTHONPATH=. sh extra/setup_tinygpu_osx.sh

This downloads TinyGPU.app and triggers a system prompt to install the driver extension.

3. Enable the driver

You should see a system prompt: "TinyGPU" would like to use a new driver extension. Click Open System Settings and toggle TinyGPU on.

If you missed the prompt, go to System Settings > General > Login Items & Extensions > Driver Extensions and toggle TinyGPU on.

4. Compiler Setup

AMD

curl -fsSL https://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_hipcomgr_osx.sh | sh

NV

Install Docker Desktop if you don't have it.

curl -fsSL https://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_nvcc_osx.sh | sh

Make sure ~/.local/bin is on your PATH:

export PATH="$HOME/.local/bin:$PATH"

5. Use it!

DEV={AMD|NV} python3 tinygrad/apps/llm.py