chenyu 7afca52796 replace pow in LAMB by tracking b1**t and b2**t per step (#4582) ...

* replace pow in LAMB by tracking b1**t and b2**t per step

* remove t, add [self.b1_t, self.b2_t] to return

* adam has one less kernel

2024-05-14 13:08:22 -04:00

..

__init__.py

batchnorm d(var)/d(mean) = 0 (#4430)

2024-05-05 00:25:45 -04:00

datasets.py

datasets isn't a feature + filter docstrings (#4228)

2024-04-19 16:16:10 +04:00

optim.py

replace pow in LAMB by tracking b1**t and b2**t per step (#4582)

2024-05-14 13:08:22 -04:00

state.py

quantized llama multilazybuffer fix (#4557)

2024-05-12 14:19:21 -07:00