chenyu 7391376528 update bert hparams (#6876) ...

4h32m with this https://wandb.ai/chenyuxyz/MLPerf-BERT/runs/q99frv1l/overview.

loss scaler 2**13->2**10. matched the closest submission, no nan for ~10 runs.

increased lr and total step a bit.

`PARALLEL=0` after setup, same as resnet.

2024-10-04 00:39:06 -04:00

41 KiB

Raw Blame History

View Raw