github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-01-09 15:08:02 -05:00
Code Issues Packages Projects Releases Wiki Activity

nn.RMSNorm (#5272)

Browse Source 
the norm itself has no significant value to add to Tensor method, but we would want Tensor.normalize
This commit is contained in:
chenyu
2024-07-02 21:39:01 -04:00
committed by GitHub
parent 9a2a82a77f
commit b2c3a28a5e
7 changed files with 74 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/nn.md
View File

@@ -10,6 +10,7 @@
::: tinygrad.nn.InstanceNorm
::: tinygrad.nn.LayerNorm
::: tinygrad.nn.LayerNorm2d
::: tinygrad.nn.RMSNorm
::: tinygrad.nn.Embedding
## Optimizers