chenyu
c9371f0d31
hotfix llama conversation mode ( #3031 )
...
without contiguous on keys and values, it runs but the update is incorrect
2024-01-06 16:57:07 -05:00
chenyu
f88506e630
move gpt2/llama sampling inside the model call ( #3013 )
...
* move gpt2/llama sampling inside the model call
* argmax uses one more kernel
2024-01-04 17:01:50 -05:00
George Hotz
a280cfe169
move dtypes to dtype.py ( #2964 )
...
* move dtypes to dtype.py
* fix urllib
2024-01-01 14:58:48 -08:00
chenyu
ad4472e6e8
cleanup llama apply_rotary_emb and other helpers ( #2950 )
...
* cleanup llama apply_rotary_emb and other helpers
used ellipsis and other higher level tensor function.
disabled the half @ half -> half tensor core as it fails uop dtype checks
* keep hip 8x8->8 wmma
2023-12-29 11:39:15 -05:00
chenyu
61e255d197
use max for gpt2 and llama ( #2949 )
...
not using argmax yet because there's a multinomial outside of function.
2023-12-28 23:26:00 -05:00
chenyu
1fb815e77e
hotfix fix coder. RMSNorm cannot have float16 input ( #2932 )
...
* hotfix fix coder. RMSNorm cannot have float16 input
* update real world test due to new kernels
* more type casts
2023-12-25 02:28:11 -05:00
chenyu
b55b55d56e
use at least int32 and uint32 for sum output ( #2926 )
...
* use at least int32 and uint32 for sum output
* use the correct type for acc
* fix opencl
* llvm mulacc
2023-12-24 01:14:54 -05:00
George Hotz
64dded27f0
pad ops broke coder ( #2881 )
...
* pad ops broke coder
* that contiguous fixes it
* Update lazy.py
2023-12-20 17:03:41 -08:00
George Hotz
1765849937
new lazy, benchmark ( #2878 )
...
* lazy rewrite, try 2
* min fix tests
* pass contig test
* put broken pads back
* move that to realize
* no contig child fixes array packing
* so wrong
* now that's correct
* base children
* fix bind issues
* disable to_image_idx
* fix tests
* that failure shouldn't break other tests
* more fixes
* fix torch
* skip failing tests in CI
* 1e-7
* half is broken
* 1e-6 margin of error
2023-12-20 14:33:21 -08:00
chenyu
c0f76ed4ea
transformer kvcache and mask have same dtype as input ( #2771 )
...
* transformer kvcache and mask have same dtype as input
* don't use `=0` in cstyle ternary where
* (bool)
* where float16 test
2023-12-14 22:41:51 -05:00
George Hotz
b3982187d1
Mixtral Example ( #2691 )
...
* mixtral
* simpler
* global counters
* simpler
* weights arg
2023-12-10 17:18:31 -08:00
chenyu
539b00a645
move llama getenv("JIT") from models to examples ( #2671 )
...
Transformer class has a jit param so we should use that in the caller
2023-12-07 12:43:22 -05:00
chenyu
6ba6349c97
JIT=0 llama.py should not jit ( #2609 )
2023-12-04 20:21:07 -05:00
Davi Silva
ddeec24fa8
Cleanup & fix llama.py ( #2524 )
...
* docs, cleanup crap
* comma AI
* fix 70B
* this is why lexical scope exists
2023-11-30 16:00:17 -05:00
George Hotz
d87a246439
move to new cached fetch ( #2493 )
...
* move to new cached fetch
* extra.utils is over
* loads
* bump download cache
* bump timeout
2023-11-28 17:36:55 -08:00
George Hotz
7170a9a057
coder.py can write and run code ( #2439 )
...
* wip mistral
* coder
* touchups
* cleanups
* mistral cleanups
* clean up cache create
* download the weights, fix tests
* fix llama loading
* global fixup
* clean up all
* move llama model
* cleanups
* Revert "cleanups"
This reverts commit a71c5d59eb
2023-11-25 12:27:54 -08:00
George Hotz
095e2ced61
add name support to fetch ( #2407 )
...
* add name support
* use fetch in gpt2
* remove requests from main lib, networkx also optional
* umm, keep that assert
* updates to fetch
* i love the walrus so much
* stop bundling mnist with tinygrad
* err, https
* download cache names
* add DOWNLOAD_CACHE_VERSION
* need env.
* ugh, wrong path
* replace get_child
2023-11-23 14:16:17 -08:00
George Hotz
cbb8486779
ResNet training changes (update benchmark) ( #2390 )
...
* default arg for chunk
* bring back to_
* good changes
* new set
* unused hash
* fix optim
* new torch loader
* fix test lr scheduler
2023-11-22 17:41:12 -08:00
George Hotz
a0890f4e6c
move fetch to helpers ( #2363 )
...
* switch datasets to new fetch
* add test_helpers
* fix convnext and delete old torch load
2023-11-19 12:29:51 -08:00
George Hotz
0cbf6c1811
move things, clean up extra ( #2292 )
...
* move things
* idk why pylint needs that now
* delete unused
2023-11-13 20:18:40 -08:00
