George Hotz a33ac869aa llm server: temperature + test client (#15444) ...

* improvements to the llm server

* eval script

* eval llm

* better eval gets 58.71

* cleanups

* add temperature, but multinomial is absurdly slow

* claude is so smart

* lint

* remove slop

* no more stop

2026-03-24 21:07:15 +08:00

..

mlperf_bert

delete SAVE_SCHEDULE=1 [pr] (#7087)

2024-10-16 07:13:20 +03:00

mlperf_resnet

remove tfa dependency: use keras.optimizers.Lamb and tf.raw_ops for LARS (#13555)

2025-12-03 17:48:27 -05:00

mlperf_retinanet

RetinaNet losses (#9536)

2025-03-21 15:52:54 -04:00

mlperf_stable_diffusion

Training loop for Stable Diffusion mlperf (#12315)

2025-10-03 02:45:38 -04:00

mlperf_unet3d

[MLPerf] UNet3D dataloader (#4343)

2024-04-28 22:34:18 -04:00

openpilot

fix failed threefry (#10646)

2025-06-05 17:17:42 -07:00

process_replay

start ripping out old scheduler -- no maps (#14909)

2026-02-20 21:05:04 +08:00

sglang_llama

refactor test/external/external_llama_eval.py (#10567)

2025-06-23 10:43:20 -07:00

external_benchmark_am.py

clean: becnhmark -> benchmark (#10620)

2025-06-03 19:28:18 -07:00

external_benchmark_bert_matmuls.py

ShapeTracker.real_strides -> is_expanded [pr] (#12579)

2025-10-09 22:52:45 -04:00

external_benchmark_bert_softmax.py

move tests to test/backend (#14691)

2026-02-12 11:09:44 +08:00

external_benchmark_disk_raw.py

ci: add h machines (#11416)

2025-07-29 19:21:51 +03:00

external_benchmark_hip_compile.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_benchmark_keccak.py

Tensor.keccak("sha3_256") (#7186)

2025-06-06 15:24:05 -07:00

external_benchmark_kernel_launch.py

cleanup tests, bump caches (#11746)

2025-08-19 21:21:07 -07:00

external_benchmark_llama_schedule.py

bump line count to 14500

2025-06-23 15:32:27 -07:00

external_benchmark_multitensor_allreduce.py

all2all (#13902)

2025-12-31 16:38:32 +03:00

external_benchmark_op_conv.py

dtypes.index -> dtypes.weakint (#15377)

2026-03-20 01:08:46 -04:00

external_benchmark_pyrender.py

add external_benchmark_pyrender (#13378)

2025-11-20 17:38:28 +08:00

external_benchmark_resnet.py

delete SAVE_SCHEDULE=1 [pr] (#7087)

2024-10-16 07:13:20 +03:00

external_benchmark_schedule.py

control_flow.py -> linearizer.py [pr] (#12948)

2025-10-27 12:38:13 -04:00

external_cl_half_max.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_fuzz_ampt.py

Fix/hcqfuzz harnesss bug (#10923)

2025-06-23 11:22:30 +03:00

external_fuzz_beam_timeout_recovery.py

beam: add dev_timeout for am (#15063)

2026-03-01 16:57:29 +03:00

external_fuzz_hcq_mp.py

system: fix flock on pcidevs (#13123)

2025-11-06 19:02:13 +08:00

external_fuzz_hcq_signals.py

hcqfuzz: init (#10049)

2025-04-25 23:19:21 +03:00

external_fuzz_qcom_cpu_cache.py

qcom: sync cpu cache when from_blob (#14518)

2026-02-03 21:51:03 +03:00

external_fuzz_sdma_warm_start.py

am: reset sdma to start from 0 (#15109)

2026-03-03 18:14:46 +03:00

external_fuzz_tlsf.py

Fix/hcqfuzz harnesss bug (#10923)

2025-06-23 11:22:30 +03:00

external_gpu_fail_osx.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_jit_failure.py

fix jit realize issue (#3258)

2024-01-26 18:27:35 -08:00

external_llm_eval.py

llm server: temperature + test client (#15444)

2026-03-24 21:07:15 +08:00

external_metal_compile_fail.py

Use MTLCompiler directly (v2) (#7920)

2024-12-04 16:36:48 +08:00

external_model_benchmark.py

move frontend dir to nn [pr] (#12470)

2025-10-07 10:42:22 +08:00

external_multi_gpu.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_nhwc_conv.py

add fun with nhwc convs

2025-10-28 17:12:22 +08:00

external_openpilot_image_warp.py

openpilot warp (#13283)

2025-11-14 13:55:32 -08:00

external_osx_profiling.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_test_am_fault_recovery.py

renderer/amd: move in tree (#14702)

2026-02-12 18:09:16 +08:00

external_test_am.py

memory: traverse all valid ranges only (#15338)

2026-03-18 14:03:39 +08:00

external_test_amd.py

Buffer.as_buffer -> Buffer.as_memoryview [pr] (#14535)

2026-02-04 11:31:11 -05:00

external_test_datasets.py

skip test TestKiTS19Dataset::test_training_set (#10936)

2025-06-23 14:27:24 -04:00

external_test_dev_var.py

fix up some slow tests that launch python (#12672)

2025-10-14 19:13:55 +08:00

external_test_example.py

remove REMOTE=1 (#13722)

2025-12-16 15:58:10 -04:00

external_test_gpu_crash.py

test gpu crash on cdna (#15062)

2026-02-28 13:17:59 +03:00

external_test_hcq_open.py

test hcq open with pytest (#13124)

2025-11-06 20:09:51 +08:00

external_test_hcq.py

Buffer.as_buffer -> Buffer.as_memoryview [pr] (#14535)

2026-02-04 11:31:11 -05:00

external_test_hip_compile.py

move opt under codegen (#11569)

2025-08-07 14:19:17 -07:00

external_test_hive_reset.py

mi350 hive_reset am repro (#15014)

2026-02-25 21:30:18 +03:00

external_test_image.py

ops_gpu -> ops_cl (#12103)

2025-09-10 15:15:48 -04:00

external_test_jit_on_models.py

Pulled CLIP and UNet into Seperate Files (#5253)

2024-07-01 22:33:01 -04:00

external_test_keccak.py

Tensor.keccak("sha3_256") (#7186)

2025-06-06 15:24:05 -07:00

external_test_llama3_ff.py

work to make GEMV fast (#5824)

2024-07-30 17:41:40 -07:00

external_test_llama3_layer.py

parameterize the CDNA asm gemm (#14813)

2026-02-17 11:35:18 +08:00

external_test_lm_head.py

isolate the 134ms kernel in train_gpt2.py (#4773)

2024-05-29 17:26:24 -04:00

external_test_losses.py

RetinaNet losses (#9536)

2025-03-21 15:52:54 -04:00

external_test_mamba.py

mark slow tests as slow instead of as CI (#13736)

2025-12-17 10:29:57 -04:00

external_test_metrics.py

log_perplexity metrics (#10912)

2025-06-21 10:44:47 -04:00

external_test_mnist_data_select.py

add quick external data select test

2024-03-02 05:38:32 -08:00

external_test_onnx_backend.py

more onnx ops (#14104)

2026-01-12 09:11:13 -05:00

external_test_onnx_ops.py

raise if Tensor._buffer is called during jit (#14114)

2026-01-22 17:30:18 -05:00

external_test_onnx_runner.py

add openpilot onnx parser test (#15334)

2026-03-18 00:12:02 -04:00

external_test_opt.py

jit: captures linears, not execitems (#15399)

2026-03-21 16:32:12 +08:00

external_test_optim.py

remove tfa dependency: use keras.optimizers.Lamb and tf.raw_ops for LARS (#13555)

2025-12-03 17:48:27 -05:00

external_test_process_count.py

add external_test_process_count

2025-12-02 17:26:30 -08:00

external_test_schedule_scaling.py

some lil speedups (#14679)

2026-02-11 10:01:58 +08:00

external_test_simple_tokenizer.py

clean up the LLM tokenizer (#12653)

2025-10-14 14:22:01 +08:00

external_test_speed_llama.py

beam: add dev_timeout for am (#15063)

2026-03-01 16:57:29 +03:00

external_test_speed_theoretical.py

test flops (and allow wide ALU in UOps) [run_process_replay] (#5749)

2024-07-26 21:07:28 -07:00

external_test_tlsf.py

factor out generic MemoryManager (#10910)

2025-06-21 16:18:33 +03:00

external_test_usb_asm24.py

usbgpu: 10 steps in benchmark to hit cache (#10273)

2025-05-13 17:06:50 +03:00

external_test_whisper_librispeech.py

names shadowing builtins (#5179)

2024-06-27 08:15:01 -04:00

external_test_yolov8.py

More yolo processing in tinygrad (#9928)

2025-04-24 16:21:46 -04:00

external_uop_gc.py

add wrapper class for the -0.0 != 0.0 issue (#14339)

2026-01-26 16:52:37 +08:00

fuzz_fast_idiv.py

disable fast_idiv on metal (#14137)

2026-01-13 21:40:40 -05:00

fuzz_graph.py

Buffer.as_buffer -> Buffer.as_memoryview [pr] (#14535)

2026-02-04 11:31:11 -05:00

fuzz_kfd.py

unify to HWQueue [pr] (#7812)

2024-11-21 10:33:08 +08:00

fuzz_shape_ops.py

mark slow tests as slow instead of as CI (#13736)

2025-12-17 10:29:57 -04:00

fuzz_symbolic_symbolic_div.py

log seed with flush=True in fuzz_symbolic (#14597)

2026-02-06 15:03:57 -05:00

fuzz_symbolic.py

pin z3-solver version (#14605)

2026-02-06 22:49:31 -05:00

speed_v_theoretical.py

lower nv test_gemm_4096 target (#13107)

2025-11-05 11:05:16 -05:00