Commit Graph

1179 Commits

Author SHA1 Message Date
George Hotz
bdde95f16e CACHE_LAZYBUFFERS options + benchmark. only a couple x from torch 2022-06-24 22:33:53 -07:00
Edward Betts
a6203b5ecd Correct spelling mistakes (#314) 2022-04-05 05:22:18 -07:00
Ildar Kashaev
d065040df0 Fix yolo param name (#306) 2022-02-10 01:40:24 -07:00
George Hotz
2a10116bfa support drawing graphs 2022-01-16 10:45:58 -08:00
George Hotz
2cae2dfa07 don't crash the dataloader for imagenet 2022-01-16 08:41:26 -08:00
George Hotz
907ff7dbb6 imagenet is training 2022-01-15 23:27:31 -08:00
George Hotz
d1e082e0ef add imagenet training support 2022-01-15 23:16:38 -08:00
George Hotz
ade2af7ac0 data doesn't require grad 2022-01-15 22:41:27 -08:00
George Hotz
0973e54eb0 fix DEBUG for GPU 2022-01-15 22:14:28 -08:00
cjg91
7025c9bbeb Transfer learning for ResNet (#295)
* Transfer learning for ResNet

* moved ResNet depth specifics into the class
2022-01-15 23:22:10 -05:00
George Hotz
55d792b065 Revert "fixup resnet"
This reverts commit 4eabe677ed.
2022-01-15 20:22:01 -08:00
George Hotz
4eabe677ed fixup resnet 2022-01-15 20:21:02 -08:00
George Hotz
e0bef0bd01 training is False by default 2022-01-15 19:57:41 -08:00
George Hotz
8ec2341cca fix bn training 2022-01-15 19:47:01 -08:00
George Hotz
c0c2c0b041 support larger ViT models 2021-12-12 10:45:10 -08:00
George Hotz
4b7c31b5b7 break vit into it's own file 2021-11-30 11:19:22 -05:00
George Hotz
46bbbcf7f0 model touchups 2021-11-30 11:13:34 -05:00
George Hotz
835869974c clean up vit code 2021-11-30 10:58:03 -05:00
George Hotz
535f02cc64 use sequential 2021-11-30 00:25:39 -05:00
George Hotz
de938c2d9d vit is now tested 2021-11-30 00:23:06 -05:00
George Hotz
aff810e722 unify transformer block 2021-11-29 18:58:15 -05:00
George Hotz
58ed46963e fix broadcastdot 2021-11-29 18:54:57 -05:00
George Hotz
125e74293f promote layernorm to tensor op 2021-11-29 18:08:21 -05:00
George Hotz
dca076dbf1 remove dumb nn ops 2021-11-29 18:05:31 -05:00
George Hotz
70544e7e9f sum hook override 2021-11-29 17:14:24 -05:00
George Hotz
8097b8f7d6 vit works 2021-11-29 16:28:14 -05:00
George Hotz
7c07c5efdd plz fix vit 2021-11-29 15:45:19 -05:00
George Hotz
ca160504e1 affine is always the last dim 2021-11-29 15:22:49 -05:00
George Hotz
e86f7a4aa3 deterministic 2021-11-29 15:10:15 -05:00
George Hotz
f909ab194f gelu with broken test 2021-11-29 15:00:50 -05:00
George Hotz
9ce881f88c fix bug in getitem, drop int axis 2021-11-29 14:01:24 -05:00
George Hotz
99b6051467 add ff_dim to transformer 2021-11-29 12:40:52 -05:00
George Hotz
b0f14b4af8 move datasets into datasets 2021-10-30 19:55:50 -07:00
George Hotz
ba28761894 move yolo into examples/yolo 2021-10-30 19:46:00 -07:00
George Hotz
121d5a17ee use tinynn for Conv2d 2021-10-30 19:40:44 -07:00
Guglielmo Camporese
2b7589db64 Added ResNet-{18, 34, 50, 101, 152} (#271)
* added resnets

* fix minor

* fix minor

* resnet in models

* added resnet test

* added resnet train test

* added linear, conv2d nn tests

* fix minor in extra/training

* resnet in models

* fix minor

* fix tolerance for linear in nn test

* fix eval, this causes cpu and gpu UT failing

* revert transformer test

* fix minor for CPU test

* improved model get_params for sequential layer

* fix minor for params counting

* commented broken ops tests

* improved train for resnet
2021-06-21 09:37:24 -07:00
George Hotz
d3f169b267 move good models to models, add a training step test 2021-06-19 11:24:15 -07:00
Jacky Lee
611d81dcb4 Add asserts for non-zero indices (#264) 2021-06-13 21:14:46 -07:00
Josh Smith
ad756f6112 minor optimizations & cleaning (#257)
* use isinstance, some optimizations & whitespace removal

* revert whitespace changes

* revert more whitespace

* some more cleanup

* revert fstring (not a fan of the {{}})

* fix typo

* fix typo
2021-06-02 09:57:15 -07:00
George Hotz
b80cacb416 fix GPU efficientnet example 2021-05-26 17:29:35 -07:00
20kdc
2653d33292 vgg7 (image upscaling) implementation - not the best, but it works (#255)
* vgg7 implementation - not the best, but it works

* VGG7 implementation: Spread nansbane to deter NaNs, maybe improved training experience

* VGG7 implementation: Fix training, for real this time

Results actually attempt to approximate the input

* VGG7 implementation: Sample probability management
2021-05-12 23:48:51 -07:00
Skosh
81bf933a91 Improved __getitem__ (#254)
* Some progress on yolov3

* Removed some debugging comments… Also, the forward pass eats all RAM for some reason

* forward pass almost runs

* forward pass runs almost

* forward pass runs, now we gotta load the weights

* loading weights works

* fetches config and weights

* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done

* some changes

* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly

* Something is wrong with the forward pass, Conv2d tests added

* forward pass almost outputs correct values, gotta fix one more thign

* yolo works

* some final changes

* reverting changes

* removed dataloader

* fixed some indentation

* comment out failing test, somehow it fails CI even though it passes on my computer…

* fixed wrong probabilities

* added webcam option to YOLO, now just need to add bounding boxes and speed it up

* some progress towards adding bounding boxes

* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage

* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image

* removed some debugging print statements

* updated result image

* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…

* Improved __getitem__

* Updated

* Updated __getitem__

* Linebreaks

* Maybe this works?

* Added MNIST locally, tests run now
2021-05-05 22:15:22 -07:00
Skosh
78aa147b39 [WIP] YOLO working on tinygrad! (#245)
* Some progress on yolov3

* Removed some debugging comments… Also, the forward pass eats all RAM for some reason

* forward pass almost runs

* forward pass runs almost

* forward pass runs, now we gotta load the weights

* loading weights works

* fetches config and weights

* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done

* some changes

* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly

* Something is wrong with the forward pass, Conv2d tests added

* forward pass almost outputs correct values, gotta fix one more thign

* yolo works

* some final changes

* reverting changes

* removed dataloader

* fixed some indentation

* comment out failing test, somehow it fails CI even though it passes on my computer…

* fixed wrong probabilities

* added webcam option to YOLO, now just need to add bounding boxes and speed it up

* some progress towards adding bounding boxes

* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage

* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image

* removed some debugging print statements

* updated result image

* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…
2021-04-25 18:06:52 -07:00
George Hotz
ce77dda805 yolov5 v4 2021-01-05 07:56:17 -08:00
Asim
1c148f2fe4 fixed example broken after gpu refactor (#238) 2021-01-05 07:41:54 -08:00
George Hotz
6842ad9ec8 minor cleanups, yolo work 2021-01-03 08:14:16 -08:00
George Hotz
895d142503 start trying to load yolo v5 2021-01-02 12:51:55 -08:00
Marcel Bischoff
42b4761025 transformer >99.98% test accuracy in ~30s (#230)
* transformer

* BS might divide len(Y_test)

* outoput when accuracy is high

* more readeable

* fixed loss in serious_mnist for new API
2021-01-02 07:45:09 -08:00
George Hotz
f9170505b3 if you like your transformers twice as slow, use the GPU 2020-12-29 17:14:23 -05:00
George Hotz
3f8e137b6f extra/transformer 2020-12-29 14:14:00 -05:00