Commit Graph

66 Commits

Author SHA1 Message Date
Felix
97a6029cf7 Corrected a few misspelled words (#1435) 2023-08-04 16:51:08 -07:00
Jacob Pradels
b112edd2c3 Add pylint trailing whitespace rule (#1314) 2023-07-21 13:37:55 -04:00
Roelof van Dijk
8f2e2f5ee2 style: else-after-return (#1216)
Co-authored-by: Roelof van Dijk <roelof.van.dijk@vitestro.com>
2023-07-12 10:26:38 -07:00
Anselm Coogan
a22aad7d32 Use generators instead of lists in anys and alls (#1111)
* Use generators in any(..) instead of lists for better best-case

* Use generators in all(...) instead of lists

* enable R1729 in .pylintrc

* revert import sorting

---------

Co-authored-by: Anselm Coogan <anselm@scandit.com>
2023-07-03 16:06:06 -07:00
Rob Grossman
2533a992e7 remove unused imports in models (#1088) 2023-07-01 12:04:19 -07:00
Kunwar Raj Singh
5d3310ce56 MaskRCNN Inference (#884)
* MaskRCNN weights loading

* backbone maybe works

* backbone works, but resnet body atol 1e-3

* RPN Call, but veryy wrong output

* fixed topk

* RPN maybe works, not sure about nms

* Fix cursed modules

* add back editorconfig

* Full call, wrong output

* Full call works

* fix mask

* use NMS from retinanet

* Removing extra funcs

* refactor

* readable

* Add example to run model

* remove filter

* Fix split, batched inference is worse

* Fix image sizes

* Matching reference

* merge master

* add filter on top detections

* cuda backend fixed

* add model eval and spec

* convert images to rgb

* fix eval

* simplify examples code

* remove extra code

* meshgrid using tinygrad

* removing numpy

* roi align, floor, ceil

* remove numpy from level_mapper

* remove numpy from pooler

* Revert "Merge branch 'master' of github.com:kunwar31/tinygrad into mrcnn-inference"

This reverts commit 4b95a3cb49, reversing
changes made to 98f2b1fa2e.

* roi align gather

* fix master merge

* revert to old floor, ceil as ints present in domain

* use log2 op

* fix indexes

* weird bug with ints and gpu

* weird bug with ints and gpu

* refactors, add env var for gather

* floor with contiguous, where

* refactor topk, sort

* remove staticmethod

* refactor stride

* remove log2 mlop

* realize -> contiguous

* refactor forward

* remove num_classes, stride_in_1x1 from state

* refactor forward

* refactoring

* flake8

* removing numpy in anchor gen, use numpy for gather, nonzero, optimize topk

* keep using tinygrad for smaller gathers

* fix empty tensors

* comms

* move from tensor.py

* resnet test passing

* add coco dataset back

* fix spaces

* add test for log2

* no need to create Tensors

* no need to create Tensors

---------

Co-authored-by: Kunwar Raj Singh <kunwar31@pop-os.localdomain>
2023-06-25 15:37:51 -07:00
George Hotz
791530045d Refactor LoadOps (#910)
* test

* work

* upd test

* loadops

* cleanups

* real ones

* remove LazyNumpyArray

* fix assign test

* remove range

* np.require

* llama uses arange kernels

* no caching consts

* fix enet

* torch load support

* tests cleanup

* fix shufflenet

* fix image

* fix torch_load test
2023-06-03 09:40:43 -07:00
JudeDavis1
f3168ee69b default transformer dropout to 0 (#828)
* default mha dropout to 0

* simplify assert

* reform

* default to 0.1
2023-05-29 08:06:16 -07:00
Jacky Lee
5d212864b5 Add MLPerf UNet3D model (#775)
* Add ResNet inference test and cannon

* Test with ResNet50

* test_car works with resnet fix

* Add KiTS19 dataset

* KiTS19: Implement iterate

* No batch load for this dataset

* Save results on iterate

* Implement dice score

* Add data prep and eval functions

* Resolve shape issue

* Conversion works but wrong values

* Segfaults when load_from_pretrained is called

* Fix segfault and assign properly

* Final result generated, though very slow

* Store and load final result to save time

* Fix typo in finalize

* Score computes

* More bug fixes, dice score is very low

* Working broken code

* Assign output values to result

* Getting a much higher score now

* Fix dataset preprocessing

* Mean DICE score of 88.5

* Ugh, typo

* Attempt to reimplement model

* Rename layers

* Tiny model works, kinda

* Accuracy? gone

* Implement InstanceNorm and match torch

* Test instance norm 2d and 3d

* Combined input block with downsample block

* Tiny model works, support strided convtranspose

* Commands to download dataset

* Clean up a bit

* unet3d_v2 -> unet3d

* Remove duplicated code

* Oops, put tests back
2023-05-28 20:38:19 -07:00
Sohaib
65d09031f2 add retinanet with resnet backbone (#813)
* add retinanet with resnet backbone

* adds resnext to support loading retinanet pretrained on openimages
* object detection post processing with numpy
* data is downloaded and converted to coco format with fiftyone
* data loading and mAP evaluation with pycocotools

* remove fiftyone dep

* * eval freq

* fix model timing

* del jit for last batch

* faster accumulate
2023-05-28 20:20:16 -07:00
wozeparrot
67de3aa1de Add mlperf bert model (#803)
* feat: add mlperf bert model

* feat: switch to nn.Embedding

* clean+fix: fix formatting

* feat: add simple downloader

* feat: metrics

* feat: don't actually need exact match

* feat: doing a run

* feat: set eps on the layernorms

* clean+fix: cleaner impl + hopefully fixed

* feat: move dataset initialization into iterate

* feat: move tokenizer out of iterate

* clean+fix: cleaner + working

* clean: cleanup

* fix: fix metrics

* feat: need to use original bert gelu + download vocab

* feat: make directory if it doesn't exist yet

* feat: jit go brrr
2023-05-27 14:53:32 -07:00
wozeparrot
0dc333cfab Promote Embedding to nn (#798)
* feat: promote Embedding to nn

* fix: fix failing test

* feat: add test with jit

* feat: rewrite embedding to no longer need stacked for loops

* clean+fix: don't know how that happened
2023-05-25 18:39:45 -07:00
wozeparrot
01ae45a43c Add mlperf RNN-T model (#782)
* feat: initial rnn-t

* feat: working with BS>1

* feat: add lstm test

* feat: test passing hidden

* clean: cleanup

* feat: specify start

* feat: way faster lstm & model

* fix: default batch size

* feat: optimization

* fix: fix metrics

* fix: fix feature splicing

* feat: cleaner stacktime

* clean: remove unused import

* clean: remove extra prints

* fix: fix tests and happy llvm

* feat: have the librispeech dataset in its own dir

* clean: unused variable

* feat: no longer need numpy for the embedding + slightly more memory efficient lstm

* fix: forgot to remove something that broke tests

* feat: use relative paths

* feat: even faster

* feat: remove pointless transposes in StackTime

* fix: correct forward

* feat: switch to soundfile for loading and fix some leaks

* feat: add comment about initial dataset setup

* feat: jit more things

* feat: default batch size back to 1

larger than 1 is broken again :(
and even in the reference implementation it gives worse results
2023-05-25 00:41:21 -07:00
George Hotz
b705510d5c getting 77% on imagenet eval 2023-05-13 07:46:27 -07:00
George Hotz
810f03dafa conv3d + unet3d (#772)
* conv3d, needs test

* test passes, padding wrong on unet

* unet3d

* no conv3d on images
2023-05-12 13:54:07 -07:00
George Hotz
46d419060b start on mlperf models 2023-05-10 16:30:49 -07:00
Jacky Lee
d13629cb26 ResNet: match implementation with Nvidia and PyTorch (#770)
* Match ResNet implementation with pytorch and nvidia

* Reduce number of Epochs
2023-05-10 09:01:22 -07:00
Jacky Lee
06ed958abd Fix train_resnet example (#744)
* Fix ResNet example

* Scientific notation
2023-04-12 13:48:39 +05:30
George Hotz
d6f4219952 LayerNorm2d for 2 lines 2023-03-20 16:58:43 -07:00
George Hotz
b1206bcb18 third try at torch loading (#677)
* third try at torch loading

* numpy fixed

* fix enet compile

* load_single_weight supports empty weights

* oops, CPU wasn't the default

* so many bugs
2023-03-10 19:11:29 -08:00
George Hotz
1a039306d2 good changes from llama branch (#671)
* good changes from llama

* transpose behavior changed
2023-03-09 20:51:22 -08:00
George Hotz
b14d31d6db ConvNeXt + extras (#657)
* simple convnext implementation

* shorter function names

* need to realize the random functions now

* creating an optimizer realizes all params

* assign contiguous

* fix lazy lazy

* why was i doing that...add convnext to tests

* LazyNumpyArray

* enable assert + comment

* no two tiny
2023-03-06 22:10:56 -08:00
George Hotz
2e56a4793e rename log_softmax, support dim, fix onnx Softmax 2023-02-24 10:11:24 -08:00
Jacky Lee
cb679cd051 Fix weight initialization (#566)
* Fix weight initialization

* Use scaled_uniform in serious_mnist
2023-02-19 11:25:29 -08:00
Kirill
7944cfdadc Remove Tensor.data (#565) 2023-02-18 16:36:12 -08:00
Jacky Lee
e172f0087a BatchNorm2D -> BatchNorm2d (#558)
* BatchNorm2D -> BatchNorm2d

* Fix typo
2023-02-16 12:31:49 -08:00
Lucas Keller
56a06280c5 Testing/utils (#548)
* New unittest for utils.py

Unit test fetch in basic ways. Would have tested more fetches, but
downloading stuff for tests is annoying and mocking is more
dependencies.

* Remove unused imports
2023-02-10 12:08:20 -06:00
George Hotz
a0d169eb59 fix efficientnet 2022-09-28 14:23:01 -07:00
Comma Device
a734df98fa TEST_ENET for openpilot compiler 2022-08-31 13:23:36 -04:00
George Hotz
368c0ce2f6 NUM=-2 for ants 2022-07-02 15:47:10 -07:00
George Hotz
0cb99d72e9 NUM=-1 is a small efficientnet for small people 2022-07-02 15:11:51 -07:00
George Hotz
8cf1aed0f4 don't track_running_stats, parameters must require_grad 2022-07-02 14:38:45 -07:00
George Hotz
67ff6b52fd move padding to convs in enet 2022-06-26 23:14:31 -07:00
George Hotz
892ac661e1 enet readability 2022-06-07 10:23:05 -07:00
George Hotz
0ee21ba115 add ViT test and car 2022-06-05 17:12:43 -07:00
George Hotz
c8b569a8c7 cleaner comments 2022-05-14 21:28:39 -07:00
cjg91
7025c9bbeb Transfer learning for ResNet (#295)
* Transfer learning for ResNet

* moved ResNet depth specifics into the class
2022-01-15 23:22:10 -05:00
George Hotz
55d792b065 Revert "fixup resnet"
This reverts commit 4eabe677ed.
2022-01-15 20:22:01 -08:00
George Hotz
4eabe677ed fixup resnet 2022-01-15 20:21:02 -08:00
George Hotz
c0c2c0b041 support larger ViT models 2021-12-12 10:45:10 -08:00
George Hotz
e28cdfb0cf clean up resnet 2021-11-30 16:14:54 -05:00
George Hotz
8f5779eeaa very minor change 2021-11-30 15:54:03 -05:00
George Hotz
d31ef0ae48 make vit names match pytorch 2021-11-30 11:34:14 -05:00
George Hotz
4b7c31b5b7 break vit into it's own file 2021-11-30 11:19:22 -05:00
George Hotz
46bbbcf7f0 model touchups 2021-11-30 11:13:34 -05:00
George Hotz
835869974c clean up vit code 2021-11-30 10:58:03 -05:00
George Hotz
c39824bc62 oops, forgot some stars 2021-11-30 00:46:14 -05:00
George Hotz
908db3bdea support bias in conv like linear 2021-11-30 00:44:59 -05:00
George Hotz
bd21304e3c linear takes in weight and bias 2021-11-30 00:38:47 -05:00
George Hotz
535f02cc64 use sequential 2021-11-30 00:25:39 -05:00