David Hou e5385eecfc UnsyncedBatchNorm with synced trainable weights for hlb cifar (#3472) ...

* UnsyncedBatchNorm with synced trainable weights for hlb cifar

* multitensor reshape tests

* test mlb assign change axis

* E501

* argfix axis

* don't import batchnorm from hlb_cifar in test_multitensor

* pass num_devices to UnsyncedBatchNorm in test, allow UnsyncedBatchNorm to be used with LB

* add backprop test for UnsyncedBatchNorm

* break out MLB assign and reshape changes

* manually shard running mean and running var

* don't shard unless syncbn=0

* replace nn.BatchNorm2d with UnsyncedBatchNorm

* don't increment num_batches_tracked if not tracking running stats

* update tests

* oops

* Revert "oops"

This reverts commit 5e8a67a535.

* Revert "update tests"

This reverts commit 7ebf65d89a.

* Revert "don't increment num_batches_tracked if not tracking running stats"

This reverts commit 78de0ea9ee.

* Revert "replace nn.BatchNorm2d with UnsyncedBatchNorm"

This reverts commit d03da53da7.

* don't increment num_batched_tracked if not tracking running stats

* oops

* test_batchnorm_axis

* compare against torch

* types

---------

Co-authored-by: chenyu <chenyu@fastmail.com>

2024-02-29 22:52:07 -05:00

..

conversation_data

Whisper + LLAMA + VITS (#2332)

2023-12-02 15:03:46 -08:00

mlperf

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

rl

more beautiful_cartpole with exposed hparams

2024-01-07 17:41:09 -08:00

sovits_helpers

move dtypes to dtype.py (#2964)

2024-01-01 14:58:48 -08:00

vgg7_helpers

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00

webgl/yolov8

webgl backend in extra (#3041)

2024-01-08 09:29:13 -08:00

webgpu/stable_diffusion

s/lazydata.realized/lazydata.base.realized/g (#2914)

2023-12-22 14:45:13 -05:00

__init__.py

failing llama test

2023-03-11 16:28:10 -08:00

beautiful_cartpole.py

more beautiful_cartpole with exposed hparams

2024-01-07 17:41:09 -08:00

beautiful_mnist.py

move sample inside jit for beautiful_mnist (#3115)

2024-01-14 01:36:30 -05:00

benchmark_train_efficientnet.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

coder.py

move gpt2/llama sampling inside the model call (#3013)

2024-01-04 17:01:50 -05:00

compile_efficientnet.py

webgl backend in extra (#3041)

2024-01-08 09:29:13 -08:00

compile_tensorflow.py

updating to work with new internal apis (#2755)

2023-12-13 21:54:47 -08:00

conversation.py

fix conversation: llama generates token not prob now (#3120)

2024-01-14 13:10:01 -05:00

efficientnet.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

f16_w_uint32.py

move dtypes to dtype.py (#2964)

2024-01-01 14:58:48 -08:00

gpt2.py

fix gpt2 with empty prompt take 2 (#3102)

2024-01-12 14:46:36 -05:00

handcode_resnet50_opt.py

move create schedule and delete old API (#3377)

2024-02-12 18:10:45 +01:00

hlb_cifar10.py

UnsyncedBatchNorm with synced trainable weights for hlb cifar (#3472)

2024-02-29 22:52:07 -05:00

index.html

Enable Multi-Output Export (#2179)

2023-10-30 18:42:26 -07:00

llama.py

event driven hip (#3160)

2024-01-18 14:35:18 -08:00

mask_rcnn.py

move things, clean up extra (#2292)

2023-11-13 20:18:40 -08:00

mixtral.py

flag to profile mixtral - 1.7 tok/s now (#3104)

2024-01-12 18:54:27 -05:00

mnist_gan.py

move state to nn/state (#1619)

2023-08-22 07:36:24 -07:00

serious_mnist.py

move state to nn/state (#1619)

2023-08-22 07:36:24 -07:00

simple_conv_bn.py

with Tensor.train() (#1935)

2023-09-28 18:02:31 -07:00

so_vits_svc.py

move dtypes to dtype.py (#2964)

2024-01-01 14:58:48 -08:00

stable_diffusion_seed0.png

validate stable diffusion for seed 0 (#2773)

2023-12-15 00:07:09 -05:00

stable_diffusion.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

train_efficientnet.py

move things, clean up extra (#2292)

2023-11-13 20:18:40 -08:00

train_resnet.py

move things, clean up extra (#2292)

2023-11-13 20:18:40 -08:00

transformer.py

fix onehot and jit in examples/transformer (#3073)

2024-01-10 02:22:41 -05:00

vgg7.py

waifu2x vgg7: testcase, auto-RGBA->RGB, function to grab pretrained models, training "fix" (#2117)

2023-10-19 22:07:15 -07:00

vit.py

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00

vits.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

whisper.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

yolov3.py

Update yolov3.py (#2680)

2023-12-08 12:59:38 -08:00

yolov8-onnx.py

[ready] Replacing os with pathlib (#1708)

2023-08-30 10:41:08 -07:00

yolov8.py

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00