Commit Graph

45 Commits

Author SHA1 Message Date
George Hotz
c162e748f5 fix float64 warning on training 2021-10-30 20:07:31 -07:00
George Hotz
b0f14b4af8 move datasets into datasets 2021-10-30 19:55:50 -07:00
Skosh
81bf933a91 Improved __getitem__ (#254)
* Some progress on yolov3

* Removed some debugging comments… Also, the forward pass eats all RAM for some reason

* forward pass almost runs

* forward pass runs almost

* forward pass runs, now we gotta load the weights

* loading weights works

* fetches config and weights

* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done

* some changes

* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly

* Something is wrong with the forward pass, Conv2d tests added

* forward pass almost outputs correct values, gotta fix one more thign

* yolo works

* some final changes

* reverting changes

* removed dataloader

* fixed some indentation

* comment out failing test, somehow it fails CI even though it passes on my computer…

* fixed wrong probabilities

* added webcam option to YOLO, now just need to add bounding boxes and speed it up

* some progress towards adding bounding boxes

* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage

* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image

* removed some debugging print statements

* updated result image

* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…

* Improved __getitem__

* Updated

* Updated __getitem__

* Linebreaks

* Maybe this works?

* Added MNIST locally, tests run now
2021-05-05 22:15:22 -07:00
Liam
ebd72ff437 Test split (#231)
* Split tests

Split tests into "Test CPU" and "Test GPU".

Add test flag "TEST_DEVICES" which is a comma separated list of devices:
CPU,GPU,ANE

* Run tests based on provided TEST_DEVICES flag

By default will run all "CPU,GPU,ANE"

* fix bad quote

* Revert changes and use GPU=1

This is done through setting the default Tensor Device to Device.CPU of
GPU=1 is set.

Run GPU tests: GPU=1 pytest -s -v
2021-01-01 09:19:03 -05:00
George Hotz
a361ef6861 fixup training loop 2020-12-27 18:35:56 -05:00
George Hotz
2f1b2c0a3b add transpose, start on transformer 2020-12-27 16:59:12 -05:00
Liam
bcf1518309 All devices are equal! (#196)
* Update all devices to be tested

ANE, CPU and OCL all now support all tests.

However tests are not currently passing on GPU and I cannot test on CPU.

Failing GPU test are not an issue caused by this update. Tests have not
been passing due to a missing "six" required installation.

OpenCL Tests have not been run since commit: 1a1c63a08b

devices have 3 types and are handle by a new DeviceTypes enum. (The goal
is to revert to Tensor.<type>, but this current setup allows for keyword
argument defaults: `device=DeviceType.CPU`)

All references to Tensor.GPU/CPU/ANE as been converted to the
corresponding `DeviceTypes` enum.

Refactor of the conversion code to allow for any device to any device
conversion.

* Add six dependency in requirements.txt

* Resolve failure to run tests

Move six into gpu required installs. Remove six from standard
installation.

* Remove repeated data conversion

* Refactor method names

Also reduce code with .to and .to_

* Dynamic device handlers

* Refactor DeviceTypes -> Device

* Add mem copy profiling back

* test_backward_pass_diamond_model passing

* Resolve Sum issue on GPU

* Revert batchnorm2d tests

* Update README with upadated API

* ANE testing with

* Last minute line gains
2020-12-15 23:44:08 -08:00
Marcel Bischoff
da72a0eed4 Big MNIST model with PIL augmentation and load/save (#160)
* 2serious

* load/save

* fixing GPU

* added DEBUG

* needs BatchNorm or doesn't learn anything

* old file not needed

* added conv biases

* added extra/training.py and checkpoint

* assert in test only

* save

* padding

* num_classes

* checkpoint

* checkpoints for padding

* training was broken

* merge

* rotation augmentation

* more aug

* needs testing

* streamline augment, augment is fast thus bicubic

* tidying up
2020-12-13 20:45:55 -08:00
George Hotz
1d10559d1d tinygrad.utils -> extra.utils 2020-12-12 15:26:07 -08:00
Liam
89d0ff6989 Consistent testing (#137)
* Consistent GPU classes

Convert the existing GPU classes into one standard format.

Remove duplicated functions in `test_mnist` and create a TestMNISTGPU
class. This reduces line count and ensures consistency.

Use `@unittest.skipUnless(GPU, "Requires GPU")` instead of `if GPU:` to
skip GPU testing. This will ensure that skipped tests are displayed
accordingly in the pytest output.

* Optim Testing now supports GPU

* Tensor testing now supports GPU

jacobian and gradcheck auto skipped until GPU float64 support added.

* GPU support for custom constructor methods

* Remove GPU flag from Model constructors

It was requested that the `gpu` kwarg be removed from the model
constructor. GPU conversion is now handled in the train function.

This also required the conversion of Optimizer parameters as they are
constructed prior to execution of the `train` function and are dependant
on the model GPU state.

* Fix typo: float32->float64

* Clean `get_parameters` utility

Just a quick refactor w/ the new support for optimizers.

* Remove GPU kwarg from TinyNet

Remove `gpu` kwarg from tiny net to match test_mnist `train` function.
2020-12-09 02:25:27 -08:00
George Hotz
97fd9c1237 zero_grad there to match readme 2020-12-07 23:12:18 -08:00
George Hotz
3b982f2f7a get_parameters 2020-12-06 13:47:28 -08:00
George Hotz
102e6356e9 replace layer_init_uniform with .uniform 2020-12-06 13:44:31 -08:00
dustcollector12
ee99d016e9 tensor implementation for rmsprop and adam (#121)
* tensor implementation for rmsprop and adam

* test_mnist.py extended to cover sgd, rmsprop and adam on cpu and gpu

* number of steps reduced for adam from 1000 to 200
2020-11-16 15:07:49 -08:00
adamritter
5ea3d76dfb Topological sort, zero_grads (#119)
* Topological sort, zero_grads

* Bug fix, add test

* Add zero_grads

* Put deepwalk function in backward

* Move zero_grad to optim

* Fix gradcheck hack

Co-authored-by: holonomicjl <58403584+holonomicjl@users.noreply.github.com>
2020-11-15 20:25:29 -08:00
Ryan Neph
8827a536e0 GPU MaxPool2D.backward(); TinyConvNet train passes (#103)
* no trailing whitespace

* GPU MaxPool2D.backward(); TinyConvNet train passes!

* Fix GPU avgpool.forward() init_val

Doesn’t change result but is simpler.

* Fix MaxPool GPU init_val

Tests only cover random non-negative inputs. This fixes issues if negative inputs are fed to GPU MaxPool2D. Test update to follow.
2020-11-11 07:58:43 -08:00
George Hotz
52ee913c98 move the mnist loader out of tinygrad proper 2020-11-10 15:37:39 -08:00
George Hotz
58e703d099 fix tests 2020-11-10 09:49:19 -08:00
George Hotz
bc7758cc5b getting convs to work on gpu 2020-11-07 09:17:57 -08:00
George Hotz
1f544d6ece test mnist on GPU 2020-11-01 07:46:17 -08:00
George Hotz
9ae3e9daf3 shape has to be a kwarg now, idk why this didn't break before 2020-10-29 08:13:05 -07:00
George Hotz
f9788eba14 parameters, and start on efficientnet 2020-10-27 08:53:35 -07:00
Timothy Mc Alister
15e5988323 make default parameters work for functions 2020-10-26 12:43:36 +01:00
George Hotz
4c42676cb6 400 -> 200 2020-10-25 17:19:59 -07:00
George Hotz
567707a5f6 rename max_pool2d to match torch, remove more fast conv crap 2020-10-25 17:16:47 -07:00
George Hotz
ea41f5e1c1 seems more generic 2020-10-25 16:40:37 -07:00
George Hotz
2333c4dea7 no tqdm in actions 2020-10-25 16:40:08 -07:00
George Hotz
8fcada8071 faster and better convnet 2020-10-25 13:48:44 -07:00
George Hotz
bb98cdfef7 improve conv testing 2020-10-25 12:46:04 -07:00
George Hotz
5216a1d9f3 refactor into tensor and ops 2020-10-23 10:34:21 -07:00
George Hotz
5756115e57 anyone else let down by the fast conv? 2020-10-23 09:09:29 -07:00
George Hotz
bcb60e0b7c wow, you have to name them test 2020-10-23 06:33:18 -07:00
George Hotz
2259c9faa1 low lr improves rmsprop 2020-10-23 06:22:32 -07:00
George Hotz
eda29fa0e0 clean up test 2020-10-23 06:11:38 -07:00
George Hotz
373b4e341b Merge pull request #15 from f0ti/master
added RMSprop optim
2020-10-23 06:08:20 -07:00
f0ti
0b87aaca1e update rsmprop 2020-10-23 14:46:45 +02:00
f0ti
c5f726ec2e all three 2020-10-23 11:53:01 +02:00
f0ti
6a38ccb6b0 update rmsprop and readme 2020-10-23 11:49:43 +02:00
George Hotz
21ebb0b769 if you wait 24 seconds, that gets 98% 2020-10-22 21:49:14 -07:00
George Hotz
816f648161 chans doesn't need to be in self 2020-10-22 21:19:35 -07:00
George Hotz
77251cc6c3 7x7 conv = more accuracy 2020-10-22 21:10:27 -07:00
f0ti
7e1eddb0c5 added RMSprop optim 2020-10-23 02:50:02 +02:00
Adrian Garcia Badaracco
9a8be135a7 incorporate changes 2020-10-21 13:21:44 -05:00
Adrian Garcia Badaracco
02adb0ac3a Make test_mnist runnable by pytest and directly 2020-10-21 11:30:08 -05:00
Adrian Garcia Badaracco
5afe6b1f68 rename files 2020-10-21 11:28:03 -05:00