Add tokenizers to requirements.txt (#1790)

* Add tokenizers to requirements and pin version

* Update process_skipfiles.py
This commit is contained in:
Ean Garvey
2023-08-24 19:44:04 -05:00
committed by GitHub
parent 07f6f4a2f7
commit 450c231171
2 changed files with 16 additions and 6 deletions

View File

@@ -18,6 +18,7 @@ Pillow
parameterized
# Add transformers, diffusers and scipy since it most commonly used
tokenizers==0.13.3
transformers
diffusers
#accelerate is now required for diffusers import from ckpt.