Elias Wahl 27613dd881 MLPerf BERT: Main training loop (#4288) ...

* BERT language modeling head + trunc normal initializers

* add train loop + helpers

* shuffle in dataloaders + slight changes in main loop

* beam change

* Minor changes

* random.shuffle

* HParam update

* Use deque for dataloader

* wandb bert project name

* half fixes

* BENCHMARK + remove epoch

* cast + print()

---------

Co-authored-by: chenyu <chenyu@fastmail.com>

2024-04-29 14:35:27 -04:00

..

create_pretraining_data.py

Wikipedia preprocessing script (#4229)

2024-04-23 10:28:01 -04:00

external_test_preprocessing_part.py

MLPerf BERT: Main training loop (#4288)

2024-04-29 14:35:27 -04:00

pick_eval_samples.py

Wikipedia preprocessing script (#4229)

2024-04-23 10:28:01 -04:00

tokenization.py

Wikipedia preprocessing script (#4229)

2024-04-23 10:28:01 -04:00