Francis Lata
c166d129df
add train dataloader for unet3d
2024-06-04 18:54:23 +00:00
Francis Lata
6678ee692e
Merge branch 'master' into unet3d_mlperf
2024-06-04 01:35:04 +00:00
George Hotz
eecfdd2f6e
hotfix: fix dataset reading for new llm.c
2024-06-03 14:10:05 +02:00
Francis Lata
707099487a
Multiprocessing UNet3D dataloader ( #4801 )
...
* testing dataloader
* matching dataloader implementation for unet3d
* remove comments
* clean up dataloader
* add cookie and cleanup
* use shm_path when creating SharedMemory
* add support for testing resnet and unet3d dataloaders
* update dataset test to return preprocesed data directory in prep for dataloader testing
* pass preprocessed dataset directory properly
* update loader function for dataloader
* add shuffling on indices
* update shm name
* more cleanup for unet3d dataloader
* remove changes to tests
---------
Co-authored-by: chenyu <chenyu@fastmail.com >
2024-06-02 11:30:47 -04:00
Francis Lata
0284c314b6
Merge branch 'master' into unet3d_mlperf
2024-06-01 16:56:09 +00:00
wozeparrot
ed0a740fe4
greater chat api endpoint compat ( #4792 )
2024-05-30 22:47:31 -07:00
Francis Lata
fdadba33e5
Merge branch 'master' into unet3d_mlperf
2024-05-29 17:56:09 +00:00
Francis Lata
e06ba105a1
remove JIT on eval_step
2024-05-29 17:55:36 +00:00
chenyu
f2414c666f
fix train_gpt2.py ( #4771 )
...
added `with Tensor.train():`
2024-05-29 12:01:34 -04:00
chenyu
7624ad3ddd
add --timing and --profile to llama3 example ( #4767 )
2024-05-28 16:24:44 -04:00
chenyu
e614b7c696
docs: showcase remove mnist_gan and add conversation.py ( #4757 )
...
fixed both examples, and i think it's better to show conversation
2024-05-28 11:09:26 -04:00
chenyu
fd249422f5
minor cleanup example stable_diffusion ( #4753 )
2024-05-28 00:05:37 -04:00
Elias Wahl
c4b0acf095
Global norm + small changes ( #4749 )
...
* norm
* no empty
* default loss scaler in float
2024-05-27 18:35:27 -04:00
Francis Lata
c4c16d4272
Merge branch 'master' into unet3d_mlperf
2024-05-27 12:00:21 -04:00
chenyu
31358cbea5
change Tensor.stack to method ( #4719 )
2024-05-24 17:04:19 -04:00
chenyu
38bc38cdff
fix llama example quantize ( #4699 )
...
* fix llama example quantize
import quantize layers from new example llama3
add to mac benchmark
* fix that
* save the files
2024-05-23 15:35:26 -04:00
chenyu
792a494eb8
fix various examples ( #4691 )
...
* fix examples that used ax1 and ax2 for transpose
* fix that
* update those
2024-05-22 20:43:21 -04:00
Francis Lata
b87724a072
add JIT on eval step
2024-05-22 21:25:40 +00:00
Elias Wahl
acc0039cfc
Resume fix + scheduler for non weight decay params ( #4679 )
...
* move ckpt dir
* fix resume. Add scheduler group
2024-05-21 19:38:13 -04:00
Francis Lata
9be663c722
realize when reassigning LR on scheduler and update default number of epochs
2024-05-21 12:19:16 +00:00
chenyu
5e3fbbb33e
llama3 example add manual seed and log seed ( #4667 )
2024-05-20 19:09:57 -04:00
chenyu
704cb1d8a0
fix conversation.py quantize ( #4663 )
...
it used to be true for int8, not it's a string for int8 or nf4
2024-05-20 17:36:37 -04:00
chenyu
ae861325ce
update llama sample for mac 32 input buffer limit ( #4662 )
...
set default sampling params to function call to 0, and top k in llama3 to 25.
2024-05-20 17:23:39 -04:00
Elias Wahl
993091adfa
loss scaler + nan fixes ( #4661 )
2024-05-20 17:08:35 -04:00
wozeparrot
b144d4b460
new llama3 example ( #4576 )
2024-05-19 22:42:23 -07:00
Francis Lata
4801354ea8
add message during benchmarking mode
2024-05-16 19:53:47 +00:00
Francis Lata
4d79bf0f34
more cleanups on training script
2024-05-16 19:49:55 +00:00
Francis Lata
3b78aa7acc
cleanup decorators
2024-05-16 19:42:07 +00:00
George Hotz
5ba611787d
move image into tensor.py. delete features ( #4603 )
...
* move image into tensor.py
* change setup.py
* openpilot tests need pythonpath now
2024-05-15 10:50:25 -07:00
Francis Lata
0517cf740e
add data fetching time
2024-05-15 15:26:46 +00:00
Francis Lata
91a38bfaff
Merge branch 'master' into unet3d_mlperf
2024-05-15 15:01:00 +00:00
George Hotz
53d082a2aa
move memory into schedule ( #4597 )
2024-05-15 07:54:20 -07:00
George Hotz
ff64bcab69
move graph/search to engine ( #4596 )
2024-05-14 23:12:59 -07:00
George Hotz
fd02ab1e8b
move disassemblers and openpilot ( #4592 )
...
* move disassemblers and openpilot
* delete junk
* put that in pre-commit
* fixup readme
2024-05-14 19:30:02 -07:00
chenyu
2b0ee74bb6
lshift and rshift ( #4591 )
2024-05-14 19:16:31 -04:00
qazal
9aa5e02229
update llmc export ( #4584 )
...
* update example
* move train to optim
* rename
* b2
2024-05-14 21:18:38 +03:00
Francis Lata
deb7ef9acb
Merge branch 'master' into unet3d_mlperf
2024-05-14 12:11:25 +00:00
Francis Lata
d4fa2af4d4
start eval after benchmarking
2024-05-14 11:50:27 +00:00
wozeparrot
d7670f8141
quantized llama multilazybuffer fix ( #4557 )
2024-05-12 14:19:21 -07:00
chenyu
01a0c1a948
slightly faster nf4 llama ( #4542 )
2024-05-12 14:24:42 -04:00
wozeparrot
e07c7668b3
nf4 llama ( #4540 )
2024-05-11 22:22:34 -07:00
chenyu
bed70b130c
mlperf bert getenv-able EVAL_STEP_FREQ ( #4534 )
2024-05-11 14:36:56 -04:00
chenyu
04a4980a51
touchup bert script ( #4531 )
...
small adjustments, remove duplicated training setting and stop the script once target is hit
2024-05-11 13:02:02 -04:00
George Hotz
347a3acb37
add renderer class ( #4524 )
...
* add renderer class
* tests pass
* fix pylint
* fix tensor cores
2024-05-10 21:40:02 -07:00
chenyu
b00b6b16f0
fix TRAIN_BEAM and Tensor.training for mlperf bert ( #4525 )
...
also hard coded bert model config instead of looking up a file
2024-05-11 00:18:36 -04:00
Francis Lata
c964484c85
Merge branch 'master' into unet3d_mlperf
2024-05-10 18:42:14 +00:00
George Hotz
4eef1ee9bf
move renderer into options ( #4514 )
...
* move renderer into options
* fix tests
* renders are functions
2024-05-10 10:01:51 -07:00
George Hotz
7c630a9a53
hotfix: fix llama spacing + fix hcq
2024-05-10 15:10:13 +00:00
chenyu
b399d98e41
fix resnet eval ( #4507 )
2024-05-10 00:49:00 -04:00
wozeparrot
a602dc67d3
feat: more mlperf fixes ( #4505 )
2024-05-09 20:50:20 -07:00
