github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-01-09 23:18:04 -05:00
Code Issues Packages Projects Releases Wiki Activity
11,676 Commits 203 Branches 13 Tags
master
Commit Graph

1235 Commits

Author SHA1 Message Date
Asim
1c148f2fe4 fixed example broken after gpu refactor (#238) 2021-01-05 07:41:54 -08:00
George Hotz
6842ad9ec8 minor cleanups, yolo work 2021-01-03 08:14:16 -08:00
George Hotz
895d142503 start trying to load yolo v5 2021-01-02 12:51:55 -08:00
Marcel Bischoff
42b4761025 transformer >99.98% test accuracy in ~30s (#230) 
* transformer

* BS might divide len(Y_test)

* outoput when accuracy is high

* more readeable

* fixed loss in serious_mnist for new API
 2021-01-02 07:45:09 -08:00
George Hotz
f9170505b3 if you like your transformers twice as slow, use the GPU 2020-12-29 17:14:23 -05:00
George Hotz
3f8e137b6f extra/transformer 2020-12-29 14:14:00 -05:00
George Hotz
bcb3ceeca3 set training in functions 2020-12-28 22:45:46 -05:00
George Hotz
51bf164b72 dropout, training 2020-12-28 22:12:23 -05:00
George Hotz
7b8fee038d it works! forgot the sqrt 2020-12-28 16:23:52 -05:00
George Hotz
1faf05ef67 ahh, it's better if i don't train the embedding 2020-12-28 16:07:02 -05:00
George Hotz
c3832e1bde hmm, fix layernorm to not be batchnorm and it breaks 2020-12-28 13:06:21 -05:00
George Hotz
2e89e75dcb layernorm fixes transformer instability 2020-12-28 12:58:15 -05:00
George Hotz
593233b668 log and exp are first class ops 2020-12-28 10:00:30 -05:00
Marcel Bischoff
ffff98db78 Evaluation in Transformers (#218) 
* 2serious

* load/save

* fixing GPU

* added DEBUG

* needs BatchNorm or doesn't learn anything

* old file not needed

* added conv biases

* added extra/training.py and checkpoint

* assert in test only

* save

* padding

* num_classes

* checkpoint

* checkpoints for padding

* training was broken

* merge

* rotation augmentation

* more aug

* needs testing

* streamline augment, augment is fast thus bicubic

* tidying up

* transformer eval
 2020-12-28 09:24:51 -05:00
George Hotz
65b07d2f4f fix onehot embed 2020-12-27 18:50:38 -05:00
George Hotz
d864e1c71a transformer is training 2020-12-27 18:46:32 -05:00
George Hotz
a361ef6861 fixup training loop 2020-12-27 18:35:56 -05:00
George Hotz
f15bec6dbc make multidot work on CPU 2020-12-27 17:25:37 -05:00
George Hotz
131e04c90c cpu only decorator 2020-12-27 17:18:55 -05:00
George Hotz
2f1b2c0a3b add transpose, start on transformer 2020-12-27 16:59:12 -05:00
Liam
075cf4bc02 Update examples to new API (#205) 2020-12-19 09:04:12 -08:00
George Hotz
e5f2055776 ane refactor, fix examples 2020-12-18 09:19:06 -08:00
iainwo
56d44637f3 fixed pylint, formatted python files iwth cblack on localhost (#204) 
* fixed pylint, formatted python files iwth cblack on localhost

* Revert "fixed pylint, formatted python files iwth cblack on localhost"

This reverts commit 07e2b88466.

* dedented 4-spaces added linter

Co-authored-by: Iain Wong <iainwong@outlook.com>
 2020-12-17 14:37:31 -08:00
Marcel Bischoff
da72a0eed4 Big MNIST model with PIL augmentation and load/save (#160) 
* 2serious

* load/save

* fixing GPU

* added DEBUG

* needs BatchNorm or doesn't learn anything

* old file not needed

* added conv biases

* added extra/training.py and checkpoint

* assert in test only

* save

* padding

* num_classes

* checkpoint

* checkpoints for padding

* training was broken

* merge

* rotation augmentation

* more aug

* needs testing

* streamline augment, augment is fast thus bicubic

* tidying up
 2020-12-13 20:45:55 -08:00
NeuralLink
1a1c63a08b Gan is real...Look what tiny just generated! (#192) 
* mode collapse solved

* info add

* delete unnecessary imports

* readme
 2020-12-13 20:23:12 -08:00
Marcel Bischoff
6785614239 tinygrad.utils to extra.utils fix in mnist_gan (#190) 2020-12-12 20:52:36 -08:00
NeuralLink
d901ef6b23 🎉 effort to generate mnist data using GAN with tinygrad. [WIP] (#166) 
* 🎉 effort to generate mnist data with tinygrad.

* dropout added

* working gan

* minor bug fixes

* more bug fixes

* todo reg l2

* detach

* logsoftmax twice
 2020-12-12 17:58:04 -08:00
George Hotz
a5aced8d47 30 MEGAReLUs. we need to lose 12 lines 2020-12-12 17:07:34 -08:00
George Hotz
bc5df477de readme and .ane() 2020-12-12 16:15:38 -08:00
George Hotz
da873cd556 Single ReLU in ANE (#188) 
* aneworks

* cleanup
 2020-12-12 16:11:34 -08:00
George Hotz
1d10559d1d tinygrad.utils -> extra.utils 2020-12-12 15:26:07 -08:00
Daulet
c7e95ddb21 Add diamond model test (#181) 
* add backward pass test for diamond model

* fix train_efficientnet example
 2020-12-11 09:21:36 -08:00
Marcel Bischoff
d204f09316 some progress on batchnorms (draft) (#147) 
* no of categories for efficientnet

* need layer_init_uniforn

* merge fail

* merge fail

* batchnorms

* needs work

* needs work how determine training

* pow

* needs work

* reshape was needed

* sum with axis

* sum with axis and tests

* broken

* works again

* clean up

* Update test_ops.py

* using sum

* don't always update running_stats

* space

* self

* default return running_stats

* passes test

* need to use mean

* merge

* testing

* fixing pow

* test_ops had a line dropped

* undo pow

* rebase
 2020-12-09 22:14:27 -08:00
George Hotz
9a64d13b94 add conv biases and max pool 2020-12-09 08:01:20 -08:00
George Hotz
99fa65f057 enable batchnorm in serious mnist 2020-12-09 03:29:40 -08:00
George Hotz
c225e62dd2 touchups 2020-12-09 02:52:28 -08:00
George Hotz
0e02f394ee serious_mnist 2020-12-08 21:43:05 -08:00
George Hotz
00312b8ad1 batchnorm work 2020-12-06 14:40:07 -08:00
George Hotz
102e6356e9 replace layer_init_uniform with .uniform 2020-12-06 13:44:31 -08:00
George Hotz
521098cc2f se optional, track time better 2020-12-06 12:29:42 -08:00
George Hotz
609d11e699 trainer works with CIFAR 2020-12-06 12:20:14 -08:00
George Hotz
80a9c777ba requires grad, optim in train enet 2020-12-06 11:10:30 -08:00
George Hotz
c66c27d22e get parameters 2020-12-06 10:45:04 -08:00
George Hotz
51daaa43d4 fix memory leaks, add gc test 2020-12-06 10:34:40 -08:00
George Hotz
b8deb36e56 train BS=16 for 32 steps 2020-12-04 10:00:32 -08:00
adamritter
5797e63d9b Train efficientnet should respect NUM environment variable (#122) 
Co-authored-by: holonomicjl <58403584+holonomicjl@users.noreply.github.com>
 2020-11-16 20:02:31 -08:00
George Hotz
2ffb8de1ea move efficientnet to extra 2020-11-16 08:08:07 -08:00
dustcollector12
6f033ea30a enable local images for efficientnet.py (#116) 2020-11-13 07:00:12 -08:00
Marcel Bischoff
a3989f9e18 Supporting .png files in efficientnet (#102) 
* to make it work locally

* definitely not working

* Conv2D GPU passes some of the tests

* Conv2D GPU passes more of the tests

* passes some tests and mnist

* removed unecessary code

* Conv2D Backpass works

* wrong test_ops.py

* white space + test backward

* ereased useless code

* removed default argument

* long lines

* works also with 4 channel .png files

* commenting out

* track
 2020-11-10 20:06:24 -08:00
George Hotz
db755fa103 promote swish to a tensor ops 2020-11-10 15:48:11 -08:00