github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-01-11 07:58:08 -05:00
Code Issues Packages Projects Releases Wiki Activity

Fix: missing n_kv_heads for smaller models from huggingface (#2438)

Browse Source 
* fix: missing n_kv_heads for smaller models from huggingface

* a lil golfing
This commit is contained in:
Davi Silva
2023-11-26 01:29:04 +07:00
committed by GitHub
parent 96c12fdeab
commit df41a57e09
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/llama.py
View File

@@ -311,7 +311,7 @@ class LLaMa:
else:
weights = load(str(model_path))
if "model.embed_tokens.weight" in weights:
weights = convert_from_huggingface(weights, model, model_args["n_heads"], model_args["n_kv_heads"])
weights = convert_from_huggingface(weights, model, model_args["n_heads"], model_args.get("n_kv_heads", model_args["n_heads"]))
if quantize:
weights = AbsmaxQuantizedLinear.quantize(weights)