add proper multi-language support to dataset generator

This commit is contained in:
Alex O'Connell
2024-04-13 22:15:37 -04:00
parent ad312091b6
commit 547d2d9989
38 changed files with 8873 additions and 775 deletions

View File

@@ -7,6 +7,7 @@ webcolors
pandas
# flash-attn
sentencepiece
deep-translator
homeassistant
hassil