github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-02-07 21:26:21 -05:00
Code Issues Packages Projects Releases Wiki Activity
2,966 Commits 215 Branches 14 Tags
fa1d4dd14be7fff8b7c3b848b06815d60e05bba2
Commit Graph

79 Commits

Author SHA1 Message Date
chenyu
86fbd413f3 update test_real_world configs (#2557) 2023-12-01 20:03:52 -05:00
chenyu
67f4e03724 rewrite 0 size loadop into a CONST (#2556) 
* rewrite 0 size loadop into a CONST

* check alloc size

* EMPTY is better

* Revert "EMPTY is better"

This reverts commit 574fe0f9ed28f1b97da5a81afdfd2cd5d9a94ff9.

* no ast is created

* fix test
 2023-12-01 18:29:06 -05:00
chenyu
7d26452305 call ruff with --preview (#2522) 
some checks are ignored without --preview
 2023-11-30 13:59:00 -05:00
George Hotz
d87a246439 move to new cached fetch (#2493) 
* move to new cached fetch

* extra.utils is over

* loads

* bump download cache

* bump timeout
 2023-11-28 17:36:55 -08:00
Christopher Mauri Milan
7f01dd04f0 Apply ruff linting rules to tests (#2473) 
* everything except F821

* enable F821 with noqa

* dumb fix

* fix remaining imports and (former) lambdas

* replace _ with noqa to avoid gc
 2023-11-27 21:24:06 -08:00
George Hotz
9e07824542 move device to device.py (#2466) 
* move device to device.py

* pylint test --disable R,C,W,E --enable E0611

* fix tests
 2023-11-27 11:34:37 -08:00
George Hotz
7170a9a057 coder.py can write and run code (#2439) 
* wip mistral

* coder

* touchups

* cleanups

* mistral cleanups

* clean up cache create

* download the weights, fix tests

* fix llama loading

* global fixup

* clean up all

* move llama model

* cleanups

* Revert "cleanups"

This reverts commit a71c5d59eb.

* fine, leave it
 2023-11-25 12:27:54 -08:00
George Hotz
8ff2e13550 From teeny (#2426) 
* changes from teenygrad work

* support not supporting ImageDType/PtrDType

* fixups from teeny
 2023-11-24 12:50:56 -08:00
George Hotz
4f8f0ac139 minor cleanups, remove dead files (#2398) 
* minor cleanups, remove dead files

* s.name

* use disk

* pytest passes on mac
 2023-11-23 09:01:50 -08:00
mmmkkaaayy
7f0cc4a4e8 whisper: support audio >30s (#2378) 
* whisper: support audio >30s

* make prompt indexing consistent with reference repo

* fix online
 2023-11-21 14:37:51 -08:00
chenyu
e9847be790 remove whisper +1-1 hack (#2360) 
* remove whisper +1-1 hack

* Revert "remove whisper +1-1 hack"

This reverts commit 5db3800f09.

* update whisper tests

* comment context
 2023-11-19 17:56:36 -05:00
mmmkkaaayy
08d09eb666 Enable whisper test in CI for more backends (#2355) 2023-11-18 17:52:50 -05:00
George Hotz
c7b38b324b A beautiful MNIST training example (#2272) 
* beautiful mnist

* beautiful mnist example

* from tinygrad import Tensor

* more beautiful

* the jit is super core tinygrad

* globalcounters reset on jit run

* symlinks and exclude

* beautiful_cartpole

* evaluate is it's own function

* no symlinks

* more beautiful

* jit reset for double speed

* type hinting for JIT

* beautiful_mnist gets 98%

* beautiful_mnist < 4s with BEAM=2

* better cartpole

* use actor critic

* zero_grad got lost

* delete double relu

* stable cartpole with PPO

* beautiful_cartpole is more beautiful

* REPLAY_BUFFER

* beautiful stuff typechecks

* None support in shape

* hp tuning
 2023-11-17 19:42:43 -08:00
chenyu
3971259832 fix test_real_world llama (#2335) 2023-11-16 19:50:08 -05:00
mmmkkaaayy
8235da11dd whisper: support batch inference, add librispeech WER test (#2074) 
* whisper: support batch inference, add librispeech WER test, add kv caching and JIT

* remove JIT_SUPPORTED_DEVICE

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2023-11-16 13:50:08 -08:00
George Hotz
70a65c201e JIT support in Interpreted (#2314) 
* factor that out

* jit is supported everywhere

* fix some tests

* there's no jit supported device, the jit is everywhere

* fix test uops
 2023-11-15 11:13:38 -08:00
George Hotz
0cbf6c1811 move things, clean up extra (#2292) 
* move things

* idk why pylint needs that now

* delete unused
 2023-11-13 20:18:40 -08:00
chenyu
a753c8e071 examples of new GPT2 and JIT change (#2261) 
* var_vals are global

* working with global ish

* better

* fix export model

* fix tests

* better kv cache

* does it run?

* use where for kvmask

* fix excessive var_vals

* fix import

* how does multigpu use this?

* llama kinda work

* faster and simpler

* cleanup

* fix conversation mode

* test cleanups

* fix one more test

* test cleanup

---------

Co-authored-by: George Hotz <geohot@gmail.com>
 2023-11-10 15:07:02 -05:00
George Hotz
c60c3b467a clean up symlinking in benchmark (#2219) 
* clean up symlinking

* make torch deterministic
 2023-11-05 16:46:05 -08:00
George Hotz
baeb77a403 Make the JIT simple (no batch exec, no cache collector) (#2215) 
* remove batch exec

* simple cachecollector

* remove cache collector test

* less lr
 2023-11-05 16:23:43 -08:00
Roelof van Dijk
36ab04ae35 perf: lazyop as dataclass (#1603) 
* perf: lazyop as dataclass

fix: linter

fix: restore eq

* use builtin methods, buffers to property to allow freezing

* fix: reduce diff

* fix: can't freeze due to KOPT tests, mypy

* fix: explicit hash

* can freeze if tests are fixed

* fix: typo

---------

Co-authored-by: Roelof van Dijk <roelof.van.dijk@vitestro.com>
Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2023-10-25 17:54:30 -04:00
20kdc
bedd028061 waifu2x vgg7: testcase, auto-RGBA->RGB, function to grab pretrained models, training "fix" (#2117) 2023-10-19 22:07:15 -07:00
George Hotz
15da96f393 print test durations and add speed (#2107) 
* print test durations

* decrease sizes to increase speed

* faster

* GPU/CLANG onnx in seperate runner

* test split, move ONNX CPU CI

* simpler tests

* simpler uops test

* faster

* less cuda apt

* running ninja install

* apt install

* split fancy indexing
 2023-10-18 13:46:42 -07:00
George Hotz
c36d306606 KOPT is over, BEAM is upstream (#2071) 
* create cache for q learning

* make linter happy

* global beam

* where it belongs

* bugfix

* ditch the kopt, use the beam

* faster lin and DEBUG=2 okay

* remove kopt, move search to features
 2023-10-16 09:46:03 -07:00
mmmkkaaayy
91168a28c4 whisper: make file transcription work, add basic CI test (#2042) 2023-10-13 17:13:35 -07:00
George Hotz
90c777d815 remove apply_auto_opt (#2063) 2023-10-13 07:44:14 -07:00
geohotstan
8d6cecb25c Torch eq fix (#1562) 
* init

* Revert "init"

This reverts commit 682bf2073a.

* kids dont do drugs

* one way to fix

* resolve merge conflict

* no more or

* clean up
 2023-10-11 12:57:11 -07:00
George Hotz
cea4cbfc7a move image+kopt to features (#2015) 
* move image+kopt to features

* fix tests

* debug prints (unrelated)
 2023-10-07 15:41:08 -07:00
nimlgen
d07ac379f9 add var_vals to kopt with symbolic (#2008) 
* add var_vals to kopt with symbolic again

* no copies
 2023-10-07 09:34:21 -07:00
chenyu
05be57f57f Fix llama with empty prompt (#1997) 
* fix llama with one token prompt

* llama is all_jitted
 2023-10-06 06:48:07 -07:00
chenyu
c99fa58dd2 simplify gpt2 example (#1973) 
* simplify gpt2 example

* kernel_jitted_count and jit tests

* Revert "kernel_jitted_count and jit tests"

This reverts commit 31a3c26dd0.

* all_jitted test in test_real_world
 2023-10-05 07:09:29 -07:00
George Hotz
2d0c1037b1 Fix up latest openpilot model (#1976) 
* fix gemv triggering for gemm

* fixup_openpilot

* external test issues
 2023-10-05 05:24:28 -07:00
George Hotz
3d5127038c don't create linearizer if we are in the method cache (#1969) 
* don't create linearizer if we are in the method cache

* remove unchecked properties

* that key isn't used

* fix default type is sticky
 2023-10-04 12:42:58 -07:00
nimlgen
2ea1dd3e87 no process() in Linearizer (#1966) 
* no process() in Linearizer

* more process() clean up
 2023-10-04 07:18:42 -07:00
Yixiang Gao
094d3d71be with Tensor.train() (#1935) 
* add with.train

* remove the rest TODOs

* fix pyflake

* fix pyflake error

* fix mypy
 2023-09-28 18:02:31 -07:00
George Hotz
adab724caa schedule2, keep the tests working with small changes (#1932) 
* lazy cleanups

* ast functions take in LazyOps

* op instead of self.op

* _base for mops

* fix contiguous

* start schedule

* test_schedule

* fix openpilot

* more tests

* bugfix and test skip

* work

* make sure things get freed

* fix zerosized tensors

* fix failing test

* fix ceil and friends

* fix openpilot

* disable training

* disable test collectives
 2023-09-28 09:14:43 -07:00
George Hotz
7ff7aacdb4 LazyOp out of Linearizer (#1908) 
* loadop buffer on cpu

* works for GPU

* sort of working

* has bugs

* gpu tests pass

* fix some tests

* fix tensor cores

* fix test linearizer

* fix symbolic

* fix has_variable_shape

* non symbolic size

* disable weird test

* simple cache fix

* fix custom function

* fix kopt

* cleanups

* a bit broken on the assign

* contig check

* only buffer

* need that order

* idx

* dedup buffers

* hmm, bugfix

* fix tensor cores

* opts device
 2023-09-24 14:30:53 +08:00
George Hotz
97dc813329 Revert "All LazyOps in the Linearizer (#1905)" (#1907) 
This reverts commit a5820390db.
 2023-09-24 11:51:22 +08:00
George Hotz
a5820390db All LazyOps in the Linearizer (#1905) 
* loadop buffer on cpu

* works for GPU

* sort of working

* has bugs

* gpu tests pass

* fix some tests

* fix tensor cores

* fix test linearizer

* fix symbolic

* fix has_variable_shape

* non symbolic size

* disable weird test

* simple cache fix

* fix custom function

* fix kopt

* cleanups

* a bit broken on the assign

* contig check

* only buffer

* need that order

* idx
 2023-09-24 11:50:00 +08:00
nimlgen
31fca43706 kopt works with local+grouped reduce and tests (#1824) 2023-09-09 13:22:09 -07:00
tomtom-95
7344f7c2d1 KeyError fixed. (#1763) 2023-09-04 15:36:16 -04:00
nimlgen
f863c12610 test kopt correctness (#1756) 
* test kopt correctness

* bump BUDGET to 20

* kopt hooks as setUp/tearDown
 2023-09-04 10:55:00 -07:00
chenyu
b8fde6bb0f Test KOPT in CI (#1744) 
* test kopt in ci

* getenv takes dtype from default
 2023-09-03 14:37:20 -07:00
chenyu
a2745819f6 faster gpt2 jit path and gpt2 in test_real_world (#1738) 2023-09-02 08:39:12 -07:00
nimlgen
8844a0a822 llvm jitted (#1652) 2023-08-28 20:22:44 -07:00
chenyu
b5d700adae update openpilot supercombo.onnx to 0.9.4 (#1681) 
* update openpilot supercombo.onnx to 0.9.4

* update tests for the new model

* comment out comma models from external_model_benchmark
 2023-08-26 19:16:08 -04:00
George Hotz
a6d842af7a move device to ops (#1646) 
* move device to ops

* mlops types

* 2 lines
 2023-08-23 08:30:17 -07:00
George Hotz
718ced296c move state to nn/state (#1619) 2023-08-22 07:36:24 -07:00
Yixiang Gao
8d6662a741 .cpu().numpy() -> .numpy() (#1594) 
* .cpu().numpy() -> .numpy()

* restore ops_torch

* restore test_speed_v_torch
 2023-08-21 09:53:29 -07:00
George Hotz
739f327d2d Shorter (#1582) 
* deleting lines

* remove insert dims

* if statement is never hit

* bug fixes
 2023-08-20 08:12:16 -07:00