Elias Wahl 69341144ba Wikipedia preprocessing script (#4229) ...

* Preprocessing script

* short seq prob

* comments + env vars

* Add preprocessing reference. Add test

* lint fix + add eval test support

* whitespaces

* point to commit

* comment

* rename

* better comments

2024-04-23 10:28:01 -04:00

..

__init__.py

remove numpy from ops_torch (#3124)

2024-01-14 22:46:57 -05:00

.gitignore

mock mnist data for imagenet trainer (#4095)

2024-04-06 08:08:40 -07:00

coco.py

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00

fake_imagenet_from_mnist.py

mock mnist data for imagenet trainer (#4095)

2024-04-06 08:08:40 -07:00

imagenet_download.py

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00

imagenet.py

mock mnist data for imagenet trainer (#4095)

2024-04-06 08:08:40 -07:00

kits19.py

scipy.signal.gaussian -> scipy.signal.windows.gaussian (#4205)

2024-04-17 19:15:37 -04:00

librispeech.py

Update file paths (#1179)

2023-07-07 18:41:58 -07:00

openimages.py

Fix missing import. (#3666)

2024-03-09 14:55:23 -08:00

preprocess_imagenet.py

move dtypes to dtype.py (#2964)

2024-01-01 14:58:48 -08:00

sops.gz

update kernel dataset to exclude the disktensor ones (#3651)

2024-03-07 17:35:19 -05:00

squad.py

move to new cached fetch (#2493)

2023-11-28 17:36:55 -08:00

wikipedia_download.py

Wikipedia download script for MLPerf BERT training (#4202)

2024-04-17 16:34:57 -04:00

wikipedia.py

Wikipedia preprocessing script (#4229)

2024-04-23 10:28:01 -04:00