Maiko Bossuyt 0dddc94bda Add file ingestion methode in file_operations.py ...

Add the following functions to ingest data into memory before Auto-GPT run.

- split_file: given a content, split it in chunks of max_length with (or without) a specified overlap

- ingest_file: read a file, use split_file to split it in chunks and load each chunk in memory

- ingest_directory: ingest all files in a directory in memory

2023-04-12 19:13:04 +02:00

5.0 KiB

Raw Blame History

View Raw