github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-01-22 13:28:06 -05:00
Code Issues Packages Projects Releases Wiki Activity
10,417 Commits 205 Branches 14 Tags
reprocess_node_experiments
Commit Graph

10417 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
George Hotz
837aaacfbf Unpad2D on GPU: 2020-12-29 13:16:14 -05:00
George Hotz
02655c07d5 break maxpool2d on GPU 2020-12-29 13:05:57 -05:00
George Hotz
061e37de39 touchups 2020-12-29 12:41:21 -05:00
George Hotz
a2e6562330 fix max op, less lines 2020-12-29 10:47:04 -05:00
Marcel Bischoff
dc8fa7999c Transpose on GPU (#221) 
* 2serious

* load/save

* fixing GPU

* added DEBUG

* needs BatchNorm or doesn't learn anything

* old file not needed

* added conv biases

* added extra/training.py and checkpoint

* assert in test only

* save

* padding

* num_classes

* checkpoint

* checkpoints for padding

* training was broken

* merge

* rotation augmentation

* more aug

* needs testing

* streamline augment, augment is fast thus bicubic

* tidying up

* transformer eval

* axis=-1

* transpose

* test for permutation using torch.movedims

* another test

* line
 2020-12-29 10:40:11 -05:00
George Hotz
36579f66bf max op 2020-12-28 23:54:52 -05:00
George Hotz
bcb3ceeca3 set training in functions 2020-12-28 22:45:46 -05:00
George Hotz
51bf164b72 dropout, training 2020-12-28 22:12:23 -05:00
George Hotz
7b8fee038d it works! forgot the sqrt 2020-12-28 16:23:52 -05:00
George Hotz
1faf05ef67 ahh, it's better if i don't train the embedding 2020-12-28 16:07:02 -05:00
George Hotz
c3832e1bde hmm, fix layernorm to not be batchnorm and it breaks 2020-12-28 13:06:21 -05:00
George Hotz
2e89e75dcb layernorm fixes transformer instability 2020-12-28 12:58:15 -05:00
George Hotz
628d21f899 doc touchup 2020-12-28 10:45:26 -05:00
George Hotz
fafece9db7 avgpool2d is a second class op 2020-12-28 10:41:59 -05:00
George Hotz
593233b668 log and exp are first class ops 2020-12-28 10:00:30 -05:00
Marcel Bischoff
ffff98db78 Evaluation in Transformers (#218) 
* 2serious

* load/save

* fixing GPU

* added DEBUG

* needs BatchNorm or doesn't learn anything

* old file not needed

* added conv biases

* added extra/training.py and checkpoint

* assert in test only

* save

* padding

* num_classes

* checkpoint

* checkpoints for padding

* training was broken

* merge

* rotation augmentation

* more aug

* needs testing

* streamline augment, augment is fast thus bicubic

* tidying up

* transformer eval
 2020-12-28 09:24:51 -05:00
George Hotz
65b07d2f4f fix onehot embed 2020-12-27 18:50:38 -05:00
George Hotz
d864e1c71a transformer is training 2020-12-27 18:46:32 -05:00
George Hotz
a361ef6861 fixup training loop 2020-12-27 18:35:56 -05:00
George Hotz
f15bec6dbc make multidot work on CPU 2020-12-27 17:25:37 -05:00
George Hotz
131e04c90c cpu only decorator 2020-12-27 17:18:55 -05:00
George Hotz
2f1b2c0a3b add transpose, start on transformer 2020-12-27 16:59:12 -05:00
gamwe6
d379502c04 Cleaning (#211) 
* Cleaned

* Brought the lines into line

Co-authored-by: gamwe6 <gamwe6@users.noreply.github.com>
 2020-12-27 09:58:51 -05:00
George Hotz
8a335f03ad clock speed 32x32 2020-12-22 18:18:52 -05:00
George Hotz
aae2e35208 benchmarking 512x512 GEMM 2020-12-22 18:01:36 -05:00
George Hotz
bd18e03138 conv from weights works 2020-12-22 17:42:17 -05:00
George Hotz
b3cf53e39b more docs 2020-12-22 17:14:38 -05:00
George Hotz
4065eae0fb docs for tensor stride 2020-12-22 17:06:36 -05:00
George Hotz
6fb127d5c7 l2 cache note 2020-12-22 16:48:19 -05:00
George Hotz
78a06a1285 more readme 2020-12-22 16:23:08 -05:00
George Hotz
0ab951f21c better readme 2020-12-22 15:57:33 -05:00
George Hotz
6ca449afd2 sum works 2020-12-22 12:53:20 -05:00
George Hotz
ebc7f8305c 3x3 gemm in conv 2020-12-22 12:00:44 -05:00
George Hotz
ff52f832dc conv hwx 2020-12-22 11:49:54 -05:00
George Hotz
5b93d9c03a aneregs 2020-12-22 10:41:12 -05:00
George Hotz
ccfd03e401 parse ane struct 2020-12-22 10:17:06 -05:00
Nicklas Boman
06f359baa3 issue-193 - Move torch loader out of efficientnet code (#213) 2020-12-22 00:19:16 -05:00
George Hotz
3abda29648 add debug to ane lib 2020-12-21 18:49:33 -05:00
George Hotz
c48917fdf0 ane reg notes 2020-12-21 15:26:52 -05:00
George Hotz
451e944648 aneregs from debug 2020-12-21 13:37:02 -05:00
George Hotz
16f0cb7d80 fix sign python 2020-12-20 15:44:10 -05:00
George Hotz
e03759f01d channels concat 2020-12-19 09:43:21 -08:00
George Hotz
73ae6d7821 working on gemm 2020-12-19 09:30:29 -08:00
Liam
075cf4bc02 Update examples to new API (#205) 2020-12-19 09:04:12 -08:00
George Hotz
3c9150a9f4 lil note 2020-12-18 09:54:20 -08:00
George Hotz
be41e73ad9 ane docs continue 2020-12-18 09:49:13 -08:00
George Hotz
e5f2055776 ane refactor, fix examples 2020-12-18 09:19:06 -08:00
George Hotz
fbcd1912cf ane struct 2020-12-18 09:06:25 -08:00
iainwo
56d44637f3 fixed pylint, formatted python files iwth cblack on localhost (#204) 
* fixed pylint, formatted python files iwth cblack on localhost

* Revert "fixed pylint, formatted python files iwth cblack on localhost"

This reverts commit 07e2b88466.

* dedented 4-spaces added linter

Co-authored-by: Iain Wong <iainwong@outlook.com>
 2020-12-17 14:37:31 -08:00
iainwo
799ad5ba17 refactor/softmax (#201) 
* generalized logsoftmax and sigmoid with softmax

* reverted sigmoid impl

Co-authored-by: Iain Wong <iainwong@outlook.com>
 2020-12-17 09:53:28 -08:00