mirror of
https://github.com/tinygrad/tinygrad.git
synced 2026-04-29 03:00:14 -04:00
4b95350c41
* fp16 resnet * cast running mean and var back to default float * extra cast * check symbolic no overflow * add linearizer failure * loss scaler after grad contig * oops * i think this works * don't loss scale fp32 * remove overflow test case * remove symbolic bounds check * loss scaler should be float * temporarily disable padto cuz bug shruggie * make running stats in batchnorm float32? * calculate lars stuff in fp32? * oops * remove most changes * move loss scaler out of optimizer * no more FP16 var * oops --------- Co-authored-by: chenyu <chenyu@fastmail.com>
Each model should be a clean single file. They are imported from the top level `models` directory It should be capable of loading weights from the reference imp. We will focus on these 5 models: # Resnet50-v1.5 (classic) -- 8.2 GOPS/input # Retinanet # 3D UNET (upconvs) # RNNT # BERT-large (transformer) They are used in both the training and inference benchmark: https://mlcommons.org/en/training-normal-21/ https://mlcommons.org/en/inference-edge-30/ And we will submit to both. NOTE: we are Edge since we don't have ECC RAM