Merge branch 'master' into summary_memory

.env.template

@@ -49,14 +49,6 @@ OPENAI_API_KEY=your-openai-api-key
 # FAST_TOKEN_LIMIT=4000
 # SMART_TOKEN_LIMIT=8000
 
-### EMBEDDINGS
-## EMBEDDING_MODEL       - Model to use for creating embeddings
-## EMBEDDING_TOKENIZER   - Tokenizer to use for chunking large inputs
-## EMBEDDING_TOKEN_LIMIT - Chunk size limit for large inputs
-# EMBEDDING_MODEL=text-embedding-ada-002
-# EMBEDDING_TOKENIZER=cl100k_base
-# EMBEDDING_TOKEN_LIMIT=8191
-
 ################################################################################
 ### MEMORY
 ################################################################################

.github/workflows/ci.yml

@@ -4,7 +4,7 @@ on:
   push:
     branches: [ master ]
   pull_request:
-    branches: [ master, stable ]
+    branches: [ master ]
 
 concurrency:
   group: ${{ format('ci-{0}', github.head_ref && format('pr-{0}', github.event.pull_request.number) || github.sha) }}

.github/workflows/docker-ci.yml

@@ -4,7 +4,7 @@ on:
   push:
     branches: [ master ]
   pull_request:
-    branches: [ master, stable ]
+    branches: [ master ]
 
 concurrency:
   group: ${{ format('docker-ci-{0}', github.head_ref && format('pr-{0}', github.event.pull_request.number) || github.sha) }}

BULLETIN.md

@@ -1,24 +1,9 @@
-# Website and Documentation Site 📰📖
-Check out *https://agpt.co*, the official news & updates site for Auto-GPT!
-The documentation also has a place here, at *https://docs.agpt.co*
+Welcome to Auto-GPT! We'll keep you informed of the latest news and features by printing messages here.
+If you don't wish to see this message, you can run Auto-GPT with the --skip-news flag
 
-# 🚀 v0.3.0 Release 🚀
-Over a week and 275 pull requests have passed since v0.2.2, and we are happy to announce
-the release of v0.3.0! *From now on, we will be focusing on major improvements* rather
-than bugfixes, as we feel stability has reached a reasonable level. Most remaining
-issues relate to limitations in prompt generation and the memory system, which will be
-the focus of our efforts for the next release.
+# INCLUDED COMMAND 'send_tweet' IS DEPRICATED, AND WILL BE REMOVED IN THE NEXT STABLE RELEASE
+Base Twitter functionality (and more) is now covered by plugins: https://github.com/Significant-Gravitas/Auto-GPT-Plugins
 
-Highlights and notable changes in this release:
+## Changes to Docker configuration
+The workdir has been changed from /home/appuser to /app. Be sure to update any volume mounts accordingly.
 
-## Plugin support 🔌
-Auto-GPT now has support for plugins! With plugins, you can extend Auto-GPT's abilities,
-adding support for third-party services and more.
-See https://github.com/Significant-Gravitas/Auto-GPT-Plugins for instructions and available plugins.
-
-## Changes to Docker configuration 🐋
-The workdir has been changed from */home/appuser* to */app*.
-Be sure to update any volume mounts accordingly!
-
-# ⚠️ Command `send_tweet` is DEPRECATED, and will be removed in v0.4.0 ⚠️
-Twitter functionality (and more) is now covered by plugins, see [Plugin support 🔌]

CONTRIBUTING.md

@@ -8,7 +8,7 @@ This document provides guidelines and best practices to help you contribute effe
 
 By participating in this project, you agree to abide by our [Code of Conduct]. Please read it to understand the expectations we have for everyone who contributes to this project.
 
-[Code of Conduct]: https://docs.agpt.co/code-of-conduct/
+[Code of Conduct]: https://significant-gravitas.github.io/Auto-GPT/code-of-conduct.md
 
 ## 📢 A Quick Word
 Right now we will not be accepting any Contributions that add non-essential commands to Auto-GPT.
@@ -101,7 +101,7 @@ https://github.com/Significant-Gravitas/Auto-GPT/pulls?q=is%3Apr+is%3Aopen+-labe
 If you add or change code, make sure the updated code is covered by tests.
 To increase coverage if necessary, [write tests using pytest].
 
-For more info on running tests, please refer to ["Running tests"](https://docs.agpt.co/testing/).
+For more info on running tests, please refer to ["Running tests"](https://significant-gravitas.github.io/Auto-GPT/testing/).
 
 [write tests using pytest]: https://realpython.com/pytest-python-testing/
 

README.md

@@ -1,5 +1,4 @@
 # Auto-GPT: An Autonomous GPT-4 Experiment
-[![Official Website](https://img.shields.io/badge/Official%20Website-agpt.co-blue?style=flat&logo=world&logoColor=white)](https://agpt.co)
 [![Unit Tests](https://img.shields.io/github/actions/workflow/status/Significant-Gravitas/Auto-GPT/ci.yml?label=unit%20tests)](https://github.com/Significant-Gravitas/Auto-GPT/actions/workflows/ci.yml)
 [![Discord Follow](https://dcbadge.vercel.app/api/server/autogpt?style=flat)](https://discord.gg/autogpt)
 [![GitHub Repo stars](https://img.shields.io/github/stars/Significant-Gravitas/auto-gpt?style=social)](https://github.com/Significant-Gravitas/Auto-GPT/stargazers)
@@ -100,21 +99,21 @@ Your support is greatly appreciated. Development of this free, open-source proje
 
 Please see the [documentation][docs] for full setup instructions and configuration options.
 
-[docs]: https://docs.agpt.co/
+[docs]: https://significant-gravitas.github.io/Auto-GPT/
 
 ## 📖 Documentation
 * [⚙️ Setup][docs/setup]
 * [💻 Usage][docs/usage]
 * [🔌 Plugins][docs/plugins]
 * Configuration
-  * [🔍 Web Search](https://docs.agpt.co/configuration/search/)
-  * [🧠 Memory](https://docs.agpt.co/configuration/memory/)
-  * [🗣️ Voice (TTS)](https://docs.agpt.co/configuration/voice/)
-  * [🖼️ Image Generation](https://docs.agpt.co/configuration/imagegen/)
+  * [🔍 Web Search](https://significant-gravitas.github.io/Auto-GPT/configuration/search/)
+  * [🧠 Memory](https://significant-gravitas.github.io/Auto-GPT/configuration/memory/)
+  * [🗣️ Voice (TTS)](https://significant-gravitas.github.io/Auto-GPT/configuration/voice/)
+  * [🖼️ Image Generation](https://significant-gravitas.github.io/Auto-GPT/configuration/imagegen/)
 
-[docs/setup]: https://docs.agpt.co/setup/
-[docs/usage]: https://docs.agpt.co/usage/
-[docs/plugins]: https://docs.agpt.co/plugins/
+[docs/setup]: https://significant-gravitas.github.io/Auto-GPT/setup/
+[docs/usage]: https://significant-gravitas.github.io/Auto-GPT/usage/
+[docs/plugins]: https://significant-gravitas.github.io/Auto-GPT/plugins/
 
 ## ⚠️ Limitations
 

autogpt/agent/agent.py

@@ -5,7 +5,6 @@ from autogpt.config import Config
 from autogpt.json_utils.json_fix_llm import fix_json_using_multiple_techniques
 from autogpt.json_utils.utilities import LLM_DEFAULT_RESPONSE_FORMAT, validate_json
 from autogpt.llm import chat_with_ai, create_chat_completion, create_chat_message
-from autogpt.llm.token_counter import count_string_tokens
 from autogpt.logs import logger, print_assistant_thoughts
 from autogpt.speech import say_text
 from autogpt.spinner import Spinner
@@ -234,16 +233,6 @@ class Agent:
                 )
                 result = f"Command {command_name} returned: " f"{command_result}"
 
-                result_tlength = count_string_tokens(
-                    str(command_result), cfg.fast_llm_model
-                )
-                memory_tlength = count_string_tokens(
-                    str(self.summary_memory), cfg.fast_llm_model
-                )
-                if result_tlength + memory_tlength + 600 > cfg.fast_token_limit:
-                    result = f"Failure: command {command_name} returned too much output. \
-                        Do not execute this command again with the same arguments."
-
                 for plugin in cfg.plugins:
                     if not plugin.can_handle_post_command():
                         continue

autogpt/commands/file_operations.py

@@ -1,12 +1,10 @@
 """File operations for AutoGPT"""
 from __future__ import annotations
 
-import hashlib
 import os
 import os.path
-from typing import Dict, Generator, Literal, Tuple
+from typing import Generator
 
-import charset_normalizer
 import requests
 from colorama import Back, Fore
 from requests.adapters import HTTPAdapter, Retry
@@ -19,96 +17,31 @@ from autogpt.utils import readable_file_size
 
 CFG = Config()
 
-Operation = Literal["write", "append", "delete"]
 
-
-def text_checksum(text: str) -> str:
-    """Get the hex checksum for the given text."""
-    return hashlib.md5(text.encode("utf-8")).hexdigest()
-
-
-def operations_from_log(log_path: str) -> Generator[Tuple[Operation, str, str | None]]:
-    """Parse the file operations log and return a tuple containing the log entries"""
-    try:
-        log = open(log_path, "r", encoding="utf-8")
-    except FileNotFoundError:
-        return
-
-    for line in log:
-        line = line.replace("File Operation Logger", "").strip()
-        if not line:
-            continue
-        operation, tail = line.split(": ", maxsplit=1)
-        operation = operation.strip()
-        if operation in ("write", "append"):
-            try:
-                path, checksum = (x.strip() for x in tail.rsplit(" #", maxsplit=1))
-            except ValueError:
-                path, checksum = tail.strip(), None
-            yield (operation, path, checksum)
-        elif operation == "delete":
-            yield (operation, tail.strip(), None)
-
-    log.close()
-
-
-def file_operations_state(log_path: str) -> Dict:
-    """Iterates over the operations log and returns the expected state.
-
-    Parses a log file at CFG.file_logger_path to construct a dictionary that maps
-    each file path written or appended to its checksum. Deleted files are removed
-    from the dictionary.
-
-    Returns:
-        A dictionary mapping file paths to their checksums.
-
-    Raises:
-        FileNotFoundError: If CFG.file_logger_path is not found.
-        ValueError: If the log file content is not in the expected format.
-    """
-    state = {}
-    for operation, path, checksum in operations_from_log(log_path):
-        if operation in ("write", "append"):
-            state[path] = checksum
-        elif operation == "delete":
-            del state[path]
-    return state
-
-
-def is_duplicate_operation(
-    operation: Operation, filename: str, checksum: str | None = None
-) -> bool:
-    """Check if the operation has already been performed
+def check_duplicate_operation(operation: str, filename: str) -> bool:
+    """Check if the operation has already been performed on the given file
 
     Args:
-        operation: The operation to check for
-        filename: The name of the file to check for
-        checksum: The checksum of the contents to be written
+        operation (str): The operation to check for
+        filename (str): The name of the file to check for
 
     Returns:
-        True if the operation has already been performed on the file
+        bool: True if the operation has already been performed on the file
     """
-    state = file_operations_state(CFG.file_logger_path)
-    if operation == "delete" and filename not in state:
-        return True
-    if operation == "write" and state.get(filename) == checksum:
-        return True
-    return False
+    log_content = read_file(CFG.file_logger_path)
+    log_entry = f"{operation}: {filename}\n"
+    return log_entry in log_content
 
 
-def log_operation(operation: str, filename: str, checksum: str | None = None) -> None:
+def log_operation(operation: str, filename: str) -> None:
     """Log the file operation to the file_logger.txt
 
     Args:
-        operation: The operation to log
-        filename: The name of the file the operation was performed on
-        checksum: The checksum of the contents to be written
+        operation (str): The operation to log
+        filename (str): The name of the file the operation was performed on
     """
-    log_entry = f"{operation}: {filename}"
-    if checksum is not None:
-        log_entry += f" #{checksum}"
-    logger.debug(f"Logging file operation: {log_entry}")
-    append_to_file(CFG.file_logger_path, f"{log_entry}\n", should_log=False)
+    log_entry = f"{operation}: {filename}\n"
+    append_to_file(CFG.file_logger_path, log_entry, should_log=False)
 
 
 def split_file(
@@ -154,12 +87,11 @@ def read_file(filename: str) -> str:
         str: The contents of the file
     """
     try:
-        charset_match = charset_normalizer.from_path(filename).best()
-        encoding = charset_match.encoding
-        logger.debug(f"Read file '{filename}' with encoding '{encoding}'")
-        return str(charset_match)
-    except Exception as err:
-        return f"Error: {err}"
+        with open(filename, "r", encoding="utf-8") as f:
+            content = f.read()
+        return content
+    except Exception as e:
+        return f"Error: {str(e)}"
 
 
 def ingest_file(
@@ -192,8 +124,8 @@ def ingest_file(
             memory.add(memory_to_add)
 
         logger.info(f"Done ingesting {num_chunks} chunks from {filename}.")
-    except Exception as err:
-        logger.info(f"Error while ingesting file '{filename}': {err}")
+    except Exception as e:
+        logger.info(f"Error while ingesting file '{filename}': {str(e)}")
 
 
 @command("write_to_file", "Write to file", '"filename": "<filename>", "text": "<text>"')
@@ -207,18 +139,17 @@ def write_to_file(filename: str, text: str) -> str:
     Returns:
         str: A message indicating success or failure
     """
-    checksum = text_checksum(text)
-    if is_duplicate_operation("write", filename, checksum):
+    if check_duplicate_operation("write", filename):
         return "Error: File has already been updated."
     try:
         directory = os.path.dirname(filename)
         os.makedirs(directory, exist_ok=True)
         with open(filename, "w", encoding="utf-8") as f:
             f.write(text)
-        log_operation("write", filename, checksum)
+        log_operation("write", filename)
         return "File written to successfully."
-    except Exception as err:
-        return f"Error: {err}"
+    except Exception as e:
+        return f"Error: {str(e)}"
 
 
 @command(
@@ -238,17 +169,15 @@ def append_to_file(filename: str, text: str, should_log: bool = True) -> str:
     try:
         directory = os.path.dirname(filename)
         os.makedirs(directory, exist_ok=True)
-        with open(filename, "a", encoding="utf-8") as f:
+        with open(filename, "a") as f:
             f.write(text)
 
         if should_log:
-            with open(filename, "r", encoding="utf-8") as f:
-                checksum = text_checksum(f.read())
-            log_operation("append", filename, checksum=checksum)
+            log_operation("append", filename)
 
         return "Text appended successfully."
-    except Exception as err:
-        return f"Error: {err}"
+    except Exception as e:
+        return f"Error: {str(e)}"
 
 
 @command("delete_file", "Delete file", '"filename": "<filename>"')
@@ -261,19 +190,19 @@ def delete_file(filename: str) -> str:
     Returns:
         str: A message indicating success or failure
     """
-    if is_duplicate_operation("delete", filename):
+    if check_duplicate_operation("delete", filename):
         return "Error: File has already been deleted."
     try:
         os.remove(filename)
         log_operation("delete", filename)
         return "File deleted successfully."
-    except Exception as err:
-        return f"Error: {err}"
+    except Exception as e:
+        return f"Error: {str(e)}"
 
 
-@command("list_files", "List Files in Directory", '"directory": "<directory>"')
-def list_files(directory: str) -> list[str]:
-    """lists files in a directory recursively
+@command("search_files", "Search Files", '"directory": "<directory>"')
+def search_files(directory: str) -> list[str]:
+    """Search for files in a directory
 
     Args:
         directory (str): The directory to search in
@@ -337,7 +266,7 @@ def download_file(url, filename):
                         spinner.update_message(f"{message} {progress}")
 
             return f'Successfully downloaded and locally stored file: "{filename}"! (Size: {readable_file_size(downloaded_size)})'
-    except requests.HTTPError as err:
-        return f"Got an HTTP Error whilst trying to download file: {err}"
-    except Exception as err:
-        return f"Error: {err}"
+    except requests.HTTPError as e:
+        return f"Got an HTTP Error whilst trying to download file: {e}"
+    except Exception as e:
+        return "Error: " + str(e)

autogpt/commands/web_selenium.py

@@ -175,9 +175,4 @@ def add_header(driver: WebDriver) -> None:
     Returns:
         None
     """
-    try:
-        with open(f"{FILE_DIR}/js/overlay.js", "r") as overlay_file:
-            overlay_script = overlay_file.read()
-        driver.execute_script(overlay_script)
-    except Exception as e:
-        print(f"Error executing overlay.js: {e}")
+    driver.execute_script(open(f"{FILE_DIR}/js/overlay.js", "r").read())

autogpt/config/config.py

@@ -35,9 +35,6 @@ class Config(metaclass=Singleton):
         self.smart_llm_model = os.getenv("SMART_LLM_MODEL", "gpt-4")
         self.fast_token_limit = int(os.getenv("FAST_TOKEN_LIMIT", 4000))
         self.smart_token_limit = int(os.getenv("SMART_TOKEN_LIMIT", 8000))
-        self.embedding_model = os.getenv("EMBEDDING_MODEL", "text-embedding-ada-002")
-        self.embedding_tokenizer = os.getenv("EMBEDDING_TOKENIZER", "cl100k_base")
-        self.embedding_token_limit = int(os.getenv("EMBEDDING_TOKEN_LIMIT", 8191))
         self.browse_chunk_max_length = int(os.getenv("BROWSE_CHUNK_MAX_LENGTH", 3000))
         self.browse_spacy_language_model = os.getenv(
             "BROWSE_SPACY_LANGUAGE_MODEL", "en_core_web_sm"
@@ -219,18 +216,6 @@ class Config(metaclass=Singleton):
         """Set the smart token limit value."""
         self.smart_token_limit = value
 
-    def set_embedding_model(self, value: str) -> None:
-        """Set the model to use for creating embeddings."""
-        self.embedding_model = value
-
-    def set_embedding_tokenizer(self, value: str) -> None:
-        """Set the tokenizer to use when creating embeddings."""
-        self.embedding_tokenizer = value
-
-    def set_embedding_token_limit(self, value: int) -> None:
-        """Set the token limit for creating embeddings."""
-        self.embedding_token_limit = value
-
     def set_browse_chunk_max_length(self, value: int) -> None:
         """Set the browse_website command chunk max length value."""
         self.browse_chunk_max_length = value

autogpt/json_utils/utilities.py

@@ -1,6 +1,5 @@
 """Utilities for the json_fixes package."""
 import json
-import os.path
 import re
 
 from jsonschema import Draft7Validator
@@ -36,8 +35,7 @@ def validate_json(json_object: object, schema_name: str) -> dict | None:
     :param schema_name: str
     :type json_object: object
     """
-    scheme_file = os.path.join(os.path.dirname(__file__), f"{schema_name}.json")
-    with open(scheme_file, "r") as f:
+    with open(f"autogpt/json_utils/{schema_name}.json", "r") as f:
         schema = json.load(f)
     validator = Draft7Validator(schema)
 

autogpt/llm/__init__.py

@@ -11,7 +11,6 @@ from autogpt.llm.base import (
 from autogpt.llm.chat import chat_with_ai, create_chat_message, generate_context
 from autogpt.llm.llm_utils import (
     call_ai_function,
-    chunked_tokens,
     create_chat_completion,
     get_ada_embedding,
 )
@@ -33,7 +32,6 @@ __all__ = [
     "call_ai_function",
     "create_chat_completion",
     "get_ada_embedding",
-    "chunked_tokens",
     "COSTS",
     "count_message_tokens",
     "count_string_tokens",

autogpt/llm/llm_utils.py

@@ -2,12 +2,9 @@ from __future__ import annotations
 
 import functools
 import time
-from itertools import islice
 from typing import List, Optional
 
-import numpy as np
 import openai
-import tiktoken
 from colorama import Fore, Style
 from openai.error import APIError, RateLimitError, Timeout
 
@@ -33,7 +30,7 @@ def retry_openai_api(
     api_key_error_msg = (
         f"Please double check that you have setup a "
         f"{Fore.CYAN + Style.BRIGHT}PAID{Style.RESET_ALL} OpenAI API Account. You can "
-        f"read more here: {Fore.CYAN}https://docs.agpt.co/setup/#getting-an-api-key{Fore.RESET}"
+        f"read more here: {Fore.CYAN}https://significant-gravitas.github.io/Auto-GPT/setup/#getting-an-api-key{Fore.RESET}"
     )
     backoff_msg = (
         f"{Fore.RED}Error: API Bad gateway. Waiting {{backoff}} seconds...{Fore.RESET}"
@@ -177,7 +174,7 @@ def create_chat_completion(
             if not warned_user:
                 logger.double_check(
                     f"Please double check that you have setup a {Fore.CYAN + Style.BRIGHT}PAID{Style.RESET_ALL} OpenAI API Account. "
-                    + f"You can read more here: {Fore.CYAN}https://docs.agpt.co/setup/#getting-an-api-key{Fore.RESET}"
+                    + f"You can read more here: {Fore.CYAN}https://significant-gravitas.github.io/Auto-GPT/setup/#getting-an-api-key{Fore.RESET}"
                 )
                 warned_user = True
         except (APIError, Timeout) as e:
@@ -210,23 +207,6 @@ def create_chat_completion(
     return resp
 
 
-def batched(iterable, n):
-    """Batch data into tuples of length n. The last batch may be shorter."""
-    # batched('ABCDEFG', 3) --> ABC DEF G
-    if n < 1:
-        raise ValueError("n must be at least one")
-    it = iter(iterable)
-    while batch := tuple(islice(it, n)):
-        yield batch
-
-
-def chunked_tokens(text, tokenizer_name, chunk_length):
-    tokenizer = tiktoken.get_encoding(tokenizer_name)
-    tokens = tokenizer.encode(text)
-    chunks_iterator = batched(tokens, chunk_length)
-    yield from chunks_iterator
-
-
 def get_ada_embedding(text: str) -> List[float]:
     """Get an embedding from the ada model.
 
@@ -237,7 +217,7 @@ def get_ada_embedding(text: str) -> List[float]:
         List[float]: The embedding.
     """
     cfg = Config()
-    model = cfg.embedding_model
+    model = "text-embedding-ada-002"
     text = text.replace("\n", " ")
 
     if cfg.use_azure:
@@ -246,7 +226,13 @@ def get_ada_embedding(text: str) -> List[float]:
         kwargs = {"model": model}
 
     embedding = create_embedding(text, **kwargs)
-    return embedding
+    api_manager = ApiManager()
+    api_manager.update_cost(
+        prompt_tokens=embedding.usage.prompt_tokens,
+        completion_tokens=0,
+        model=model,
+    )
+    return embedding["data"][0]["embedding"]
 
 
 @retry_openai_api()
@@ -265,31 +251,8 @@ def create_embedding(
         openai.Embedding: The embedding object.
     """
     cfg = Config()
-    chunk_embeddings = []
-    chunk_lengths = []
-    for chunk in chunked_tokens(
-        text,
-        tokenizer_name=cfg.embedding_tokenizer,
-        chunk_length=cfg.embedding_token_limit,
-    ):
-        embedding = openai.Embedding.create(
-            input=[chunk],
-            api_key=cfg.openai_api_key,
-            **kwargs,
-        )
-        api_manager = ApiManager()
-        api_manager.update_cost(
-            prompt_tokens=embedding.usage.prompt_tokens,
-            completion_tokens=0,
-            model=cfg.embedding_model,
-        )
-        chunk_embeddings.append(embedding["data"][0]["embedding"])
-        chunk_lengths.append(len(chunk))
-
-    # do weighted avg
-    chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lengths)
-    chunk_embeddings = chunk_embeddings / np.linalg.norm(
-        chunk_embeddings
-    )  # normalize the length to one
-    chunk_embeddings = chunk_embeddings.tolist()
-    return chunk_embeddings
+    return openai.Embedding.create(
+        input=[text],
+        api_key=cfg.openai_api_key,
+        **kwargs,
+    )

autogpt/llm/modelsinfo.py

@@ -3,8 +3,5 @@ COSTS = {
     "gpt-3.5-turbo-0301": {"prompt": 0.002, "completion": 0.002},
     "gpt-4-0314": {"prompt": 0.03, "completion": 0.06},
     "gpt-4": {"prompt": 0.03, "completion": 0.06},
-    "gpt-4-0314": {"prompt": 0.03, "completion": 0.06},
-    "gpt-4-32k": {"prompt": 0.06, "completion": 0.12},
-    "gpt-4-32k-0314": {"prompt": 0.06, "completion": 0.12},
     "text-embedding-ada-002": {"prompt": 0.0004, "completion": 0.0},
 }

autogpt/logs.py

@@ -75,7 +75,6 @@ class Logger(metaclass=Singleton):
         self.logger.setLevel(logging.DEBUG)
 
         self.speak_mode = False
-        self.chat_plugins = []
 
     def typewriter_log(
         self, title="", title_color="", content="", speak_text=False, level=logging.INFO
@@ -83,9 +82,6 @@ class Logger(metaclass=Singleton):
         if speak_text and self.speak_mode:
             say_text(f"{title}. {content}")
 
-        for plugin in self.chat_plugins:
-            plugin.report(f"{title}. {content}")
-
         if content:
             if isinstance(content, list):
                 content = " ".join(content)

autogpt/main.py

@@ -3,7 +3,7 @@ import logging
 import sys
 from pathlib import Path
 
-from colorama import Fore, Style
+from colorama import Fore
 
 from autogpt.agent.agent import Agent
 from autogpt.commands.command import CommandRegistry
@@ -13,11 +13,7 @@ from autogpt.logs import logger
 from autogpt.memory import get_memory
 from autogpt.plugins import scan_plugins
 from autogpt.prompts.prompt import DEFAULT_TRIGGERING_PROMPT, construct_main_ai_config
-from autogpt.utils import (
-    get_current_git_branch,
-    get_latest_bulletin,
-    markdown_to_ansi_style,
-)
+from autogpt.utils import get_current_git_branch, get_latest_bulletin
 from autogpt.workspace import Workspace
 from scripts.install_plugin_deps import install_plugin_dependencies
 
@@ -61,19 +57,9 @@ def run_auto_gpt(
     )
 
     if not cfg.skip_news:
-        motd, is_new_motd = get_latest_bulletin()
+        motd = get_latest_bulletin()
         if motd:
-            motd = markdown_to_ansi_style(motd)
-            for motd_line in motd.split("\n"):
-                logger.info(motd_line, "NEWS:", Fore.GREEN)
-            if is_new_motd and not cfg.chat_messages_enabled:
-                input(
-                    Fore.MAGENTA
-                    + Style.BRIGHT
-                    + "NEWS: Bulletin was updated! Press Enter to continue..."
-                    + Style.RESET_ALL
-                )
-
+            logger.typewriter_log("NEWS: ", Fore.GREEN, motd)
         git_branch = get_current_git_branch()
         if git_branch and git_branch != "stable":
             logger.typewriter_log(
@@ -139,13 +125,6 @@ def run_auto_gpt(
     full_message_history = []
     next_action_count = 0
 
-    # add chat plugins capable of report to logger
-    if cfg.chat_messages_enabled:
-        for plugin in cfg.plugins:
-            if hasattr(plugin, "can_handle_report") and plugin.can_handle_report():
-                logger.info(f"Loaded plugin into logger: {plugin.__class__.__name__}")
-                logger.chat_plugins.append(plugin)
-
     # Initialize memory and make sure it is empty.
     # this is particularly important for indexing and referencing pinecone memory
     memory = get_memory(cfg, init=True)

autogpt/memory/pinecone.py

@@ -38,9 +38,6 @@ class PineconeMemory(MemoryProviderSingleton):
             exit(1)
 
         if table_name not in pinecone.list_indexes():
-            logger.typewriter_log(
-                "Connecting Pinecone. This may take some time...", Fore.MAGENTA, ""
-            )
             pinecone.create_index(
                 table_name, dimension=dimension, metric=metric, pod_type=pod_type
             )

autogpt/memory_management/summary_memory.py

@@ -1,4 +1,3 @@
-import copy
 import json
 from typing import Dict, List, Tuple
 
@@ -45,9 +44,7 @@ def get_newly_trimmed_messages(
     return new_messages_not_in_context, new_index
 
 
-def update_running_summary(
-    current_memory: str, new_events: List[Dict[str, str]]
-) -> str:
+def update_running_summary(current_memory: str, new_events: List[Dict]) -> str:
     """
     This function takes a list of dictionaries representing new events and combines them with the current summary,
     focusing on key and potentially important information to remember. The updated summary is returned in a message
@@ -64,23 +61,17 @@ def update_running_summary(
         update_running_summary(new_events)
         # Returns: "This reminds you of these events from your past: \nI entered the kitchen and found a scrawled note saying 7."
     """
-    # Create a copy of the new_events list to prevent modifying the original list
-    new_events = copy.deepcopy(new_events)
-
     # Replace "assistant" with "you". This produces much better first person past tense results.
     for event in new_events:
         if event["role"].lower() == "assistant":
             event["role"] = "you"
-
             # Remove "thoughts" dictionary from "content"
             content_dict = json.loads(event["content"])
             if "thoughts" in content_dict:
                 del content_dict["thoughts"]
             event["content"] = json.dumps(content_dict)
-
         elif event["role"].lower() == "system":
             event["role"] = "your computer"
-
         # Delete all user messages
         elif event["role"] == "user":
             new_events.remove(event)

autogpt/plugins.py

@@ -33,7 +33,7 @@ def inspect_zip_for_modules(zip_path: str, debug: bool = False) -> list[str]:
     result = []
     with zipfile.ZipFile(zip_path, "r") as zfile:
         for name in zfile.namelist():
-            if name.endswith("__init__.py") and not name.startswith("__MACOSX"):
+            if name.endswith("__init__.py"):
                 logger.debug(f"Found module '{name}' in the zipfile at: {name}")
                 result.append(name)
     if len(result) == 0:

autogpt/utils.py

@@ -1,9 +1,8 @@
 import os
-import re
 
 import requests
 import yaml
-from colorama import Fore, Style
+from colorama import Fore
 from git.repo import Repo
 
 from autogpt.logs import logger
@@ -108,46 +107,15 @@ def get_current_git_branch() -> str:
         return ""
 
 
-def get_latest_bulletin() -> tuple[str, bool]:
+def get_latest_bulletin() -> str:
     exists = os.path.exists("CURRENT_BULLETIN.md")
     current_bulletin = ""
     if exists:
         current_bulletin = open("CURRENT_BULLETIN.md", "r", encoding="utf-8").read()
     new_bulletin = get_bulletin_from_web()
-    is_new_news = new_bulletin != "" and new_bulletin != current_bulletin
-
-    news_header = Fore.YELLOW + "Welcome to Auto-GPT!\n"
-    if new_bulletin or current_bulletin:
-        news_header += (
-            "Below you'll find the latest Auto-GPT News and updates regarding features!\n"
-            "If you don't wish to see this message, you "
-            "can run Auto-GPT with the *--skip-news* flag.\n"
-        )
+    is_new_news = new_bulletin != current_bulletin
 
     if new_bulletin and is_new_news:
         open("CURRENT_BULLETIN.md", "w", encoding="utf-8").write(new_bulletin)
-        current_bulletin = f"{Fore.RED}::NEW BULLETIN::{Fore.RESET}\n\n{new_bulletin}"
-
-    return f"{news_header}\n{current_bulletin}", is_new_news
-
-
-def markdown_to_ansi_style(markdown: str):
-    ansi_lines: list[str] = []
-    for line in markdown.split("\n"):
-        line_style = ""
-
-        if line.startswith("# "):
-            line_style += Style.BRIGHT
-        else:
-            line = re.sub(
-                r"(?<!\*)\*(\*?[^*]+\*?)\*(?!\*)",
-                rf"{Style.BRIGHT}\1{Style.NORMAL}",
-                line,
-            )
-
-        if re.match(r"^#+ ", line) is not None:
-            line_style += Fore.CYAN
-            line = re.sub(r"^#+ ", "", line)
-
-        ansi_lines.append(f"{line_style}{line}{Style.RESET_ALL}")
-    return "\n".join(ansi_lines)
+        return f" {Fore.RED}::UPDATED:: {Fore.CYAN}{new_bulletin}{Fore.RESET}"
+    return current_bulletin

data_ingestion.py

@@ -1,7 +1,7 @@
 import argparse
 import logging
 
-from autogpt.commands.file_operations import ingest_file, list_files
+from autogpt.commands.file_operations import ingest_file, search_files
 from autogpt.config import Config
 from autogpt.memory import get_memory
 
@@ -10,11 +10,12 @@ cfg = Config()
 
 def configure_logging():
     logging.basicConfig(
+        filemode="a",
         format="%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s",
         datefmt="%H:%M:%S",
         level=logging.DEBUG,
         handlers=[
-            logging.FileHandler(filename="log-ingestion.txt", mode="a"),
+            logging.FileHandler(filename="log-ingestion.txt"),
             logging.StreamHandler(),
         ],
     )
@@ -30,7 +31,7 @@ def ingest_directory(directory, memory, args):
     """
     global logger
     try:
-        files = list_files(directory)
+        files = search_files(directory)
         for file in files:
             ingest_file(file, memory, args.max_length, args.overlap)
     except Exception as e:
@@ -67,6 +68,7 @@ def main() -> None:
         help="The max_length of each chunk when ingesting files (default: 4000)",
         default=4000,
     )
+
     args = parser.parse_args()
 
     # Initialize memory

docs/index.md

@@ -1,5 +1,5 @@
 # Auto-GPT
 
-Welcome to Auto-GPT.  Please follow the [Installation](/setup/) guide to get started.
+Welcome to Auto-GPT.  Please follow the [Installation](https://significant-gravitas.github.io/Auto-GPT/setup/) guide to get started.
 
 It is recommended to use a virtual machine for tasks that require high security measures to prevent any potential harm to the main computer's system and data.

mkdocs.yml

@@ -1,5 +1,5 @@
 site_name: Auto-GPT
-site_url: https://docs.agpt.co/
+site_url: https://significantgravitas.github.io/Auto-GPT/
 repo_url: https://github.com/Significant-Gravitas/Auto-GPT
 nav:
   - Home: index.md

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "agpt"
-version = "0.3.0"
+version = "0.2.2"
 authors = [
   { name="Torantulino", email="support@agpt.co" },
 ]

requirements.txt

@@ -14,14 +14,13 @@ duckduckgo-search
 google-api-python-client #(https://developers.google.com/custom-search/v1/overview)
 pinecone-client==2.2.1
 redis
-orjson==3.8.10
+orjson
 Pillow
 selenium==4.1.4
 webdriver-manager
 jsonschema
 tweepy
 click
-charset-normalizer>=3.1.0
 spacy>=3.0.0,<4.0.0
 en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.5.0/en_core_web_sm-3.5.0-py3-none-any.whl
 

tests/integration/cassettes/test_llm_utils/test_get_ada_embedding.yaml

@@ -1,168 +0,0 @@
-interactions:
-- request:
-    body: '{"input": [[1985]], "model": "text-embedding-ada-002", "encoding_format":
-      "base64"}'
-    headers:
-      Accept:
-      - '*/*'
-      Accept-Encoding:
-      - gzip, deflate
-      Connection:
-      - keep-alive
-      Content-Length:
-      - '83'
-      Content-Type:
-      - application/json
-    method: POST
-    uri: https://api.openai.com/v1/embeddings
-  response:
-    body:
-      string: !!binary |
-        H4sIAAAAAAAAA1SaSxO6Orvl5++n2LWn9FsiIgl7xl3kkiAgYldXlyAiKHJNgJw6371L/6dOd08c
-        QAqV5HnWWr/kP/71119/t1ld5NPf//z197sap7//x/fa/Tbd/v7nr//5r7/++uuv//h9/n8jiyYr
-        7vfqU/6G/25Wn3ux/P3PX/x/X/m/g/756++DcanIKGZXd/X5SZTuBpdQW6+lYQqkfQ5NFN8pSgIw
-        zBGFBLyF+0x9ZATRzFVcC+xx3aJPfw3cJaBFA2RnibBlW4EuBNKhlLJ5dbH/QRJYDniFELe0Ryw1
-        zy4l2M8lVX4IVLvccrDkD26WikG50PuxCQFdnkwEiSTcsFJvgcv2m6mB50DfUpu5FLAPV1ZQMZ8b
-        IvW7azZ//KsHo/R9nYA/OPXa3M0WXkbEI3Dus2z9lNsEjEngUv+4PdWrN6EKdpdVoMba1vqqHZkH
-        b+fNG4mbTQRmWGsFxIdzQM3Kfkbt1AUcBI3doc1xk9ZLJVoC3ClDSm3yBtl4AC8Bvk47CzunCbFl
-        aqkGQtGR0A74sT4HY8DDq8puaE3xHC1C9H7BD749CHUSFM03CxuwDd2YQD5eskks+B4idSLUZd1V
-        n+fOt2FzPT2pkptdNtu6EQD6Cg2sRFSr1/NdKvcmSu74EM8ioDerUyT9vHfIWElpvUwkz0F2c2e0
-        vXpNNvuXsQe81EdofRp6LfDVTYMNUUx6PtQEjL//y79kHiuu1mRzc3xWsDg6TwQepu6OOhkEae88
-        TKzIbykanVhZ5SOLI3zWZCmjogx6OCXDhAT9yuoulKYevMWdRl2+GOtu9W6a9CHZBxvBramHc8Up
-        MNsImCJ0jBgThDiA13A6U8UU5IxJ8keQGG8I2At5RecPahjD4yJBrGZgAl2aDgjGwQth47BVXR5l
-        eQyPQVnRMISPjMUvNgNJeRm4GC/BsMh0SsDUKRt6PB+LelHlIJCl9SYjYfewsjbe5Rb8MNvDONKd
-        gclhbEAMywgba6u762TdAkjVIkJrd3Pc9antBBgEgUIEXZPZdDzHL/hCTxEtOGkHtg5XBQiuvsUe
-        DR76a+oCCDe720rWtGncxT69JHi49BTbp/jo8vxe5mEjlx02733NVuNIb4BDao0PgNnDnOWLBoNH
-        YlB8ebb6ulyVUS5at8MW8c5DeVb2IdQv8RN7R9F02VEWRxBM2Yka77HVZzqLMbwdpBSJVyNzRymr
-        rY2zKwusPUQpIrNlFYC+AgOHmmCxbfbkS3mMeR47t0UB/ImJHvBP64C4GV/rebiFCKKw66h78mRA
-        twSKwOJ5DtvXng4ru62tlPsgx+63Hpep/Sgwm2cXH55jxNZzfL4BkkeYHPqNMVBkCSFkaL8isbq4
-        gPmml0AnmldsPYNbvYiZBwEcZhcHqV9my/bSWVDZRwOaUSyxWs4WT17k15keClcZmCRTHua8lmGn
-        2QTR9CDbEtabDUaSqLHoMsfUkgoo+WRNL5+oc6ruJqnc3qeeHIwRldNjBXb81aCHeE7ZOnKrAMs6
-        JeRZtn00qXIQwho5Cek+J21Y5/NJg547drRQwStaLreWg+M2irB7ZM96em4kDSiX65bsDjAeRv5o
-        FxDt+JEqUVmzWUkwhD23rthXls0wLRvTAWgnjGhV9/nQ77ZKLpUhxNTYmEs0LU5pyNKTK4gEBjmb
-        9N1DA/zhpmPNutz0JdZ8A3Kq4OOD8HnUtBma8M99fXOpaxLfHF56VUxFwtozfS10vYc5jhl671zD
-        7ayTOErIrC4EGB9xmPbVK4aBW2dorrdAXwN9HGExwJCikPu4zMx2BF6OmU6k0DCGuXWjXLKi/EVj
-        6cPYlD+EGSr704CPt/NnWBRijjBqDzLaXMIuYrvNkgIm5DYRWWXWs7XbOlC0e4/aBlcNzNn0PcyX
-        4YiA8UmHlRVVDBsOKvQU8Za+RmkqgcwZPIy2xgiWNNzOsBncC9Wt/D6Q7a0RwFsKrlhP44ixuK1z
-        mJ9QT93w2LqzFJocrO+tRA9G1NdLW18R7N2qIuLuRoalfTwrYMvjTOZvf1qorhBgHHuB6tK+cef9
-        dXFgBIUnqTt7x2b1/kkgi/oLavuDDKZdJdrSSqWeatz5zOa38EHAy7BPzYDYYAG9ToDs9yrhv/O7
-        3kQ7Br4WHLD71bdOeCgF5OhVRnJYTWDmzWCG7gXK2MxhmlH95s9gPrsaPvrPul6GduWhpZUdPp5a
-        N2PcoxThd/6pJrx1sN5EJYGnk8OhrdsLw7AUhgfdNssRF36WqOtJGUDH9I5Ym54LYx6gDjjipiH8
-        GzRgILEkwqnjJHqA2qGeuYfFwfO6a5Akth82c+RUwlZrZ3o+eHt3GRvZAiS7xPiwOw86I0WqASsq
-        XiiYHiqYL1AeAa34CTufxHKX61vUwIDHiR6bOM3owxV4MCXdhM3g4ej8RRs8OBlsR80VvrMlLdIW
-        3nfBgJXJ9tz3tXReEIKAYteXjtHylLwGevfXA59j+VMvt9vZhtKaythJKk0XnCFuwFefscdLJhtP
-        bPZkJd16JMolO2PGpChw674Q4QXqsDnzm1y69MuduvxWi3jyiRA8bz2Rekfxra/2W4JwSk0DR8G5
-        1ufr6Qkh+qQ+eX/1tputOYUtvzHJOnU2W80EImmTcBHa8Ks0LPYlTOC2O26o/yjs73qiyR9/5CVW
-        XM/LPJbgFFUB2QUiZatq6RUsJPVB5u/7FOIks+HhaliIPWSZjUXjJHB95C69U3DRp+zJV0BYbJ6c
-        0rAG7KffB6kosW/K7bCcmBVAk5Idtk/1h43h6CuSu79cUNTZF0DuYPSAWRkOmWt5dafuPqxwDdID
-        RuVRdZmcuCvciMlEXSW9slV1Zw74/rPCSF6O7uortgAJ6Z9oEndrNKLXksPMs2qqHhRNHxkALyhe
-        vC1GZr4b5lrqCYyO1pZaqcTrHXgVAbwcNiN267qsV2z2OZzsV0ldheJh1nrowBRr7/96fxetRvBU
-        azySmk2Qje1JEeHWbRD1YL9j1FPzFiaf2id9EunZ+pATGy59ZKNdIGLwez5wwnOGjbJ86Gy6lTG0
-        3vHtT723vJnOcCPGE3a//WnZVbMDY+7IYeWtGtkCWMtDn/VP9OaAoDMuSBLAKamAtdi26+UdVCv0
-        nxHGB0/P63arZ8p+P8aAjKfdFgxTIvYSSooQqy0NdRbWqgDezcajWno5RLO5X0W407WeOob7yGbX
-        OzcSnYH01e8SzHSeEyjPUkidgdcH5sqjBEs+7mgx4339HtpVgKfH1FBNSEyXP1qAg+vmiqj9XHYZ
-        M7o0gWP4qAhI1MWdw9HUYErUiXqqvrB5TIpv/QoT1qT7yBbnagRAfUgmthscg4UF6igp/aTR42ab
-        DvOYezYYNchh/bW/6qPJm7lUJCeGkjXjs7EWmAIN1/Cxsk+ygcwukCCIA4aNoHtHdMCnAOyPTwfj
-        G8uHpRh8Afh+XSGiOfXPn/HAj5CGBKa2jKJE4aExFh+qtJH4/X1eCEP9HpE9pYrO25u4grz+vmHn
-        5o4ZScclgGVVMNTZgLqtsWYljPJUo97xibOlPro3aXEDhUbcbR+tGZgVyNXeh6r5rEbjKh17OJ+P
-        GrX7gg0lW4sc7PjMoLb9GdksWoIEHTUryV5528OuhwUH6xzX1ASdWn/rPQbVkLwRj/kFLP6wjlB7
-        4hrJqDfA1m1OhmQJwZlG5XrK/ughamObmvrwZrPjDC0wN35GcQsUIDiLpsHwfD5SDfcO4B2nbqF0
-        0mqql+sSMRSebHghLwsHXVC79OfHp1ueYPu+J1EHl9oCxFsHqj9Tk9EPM3t4etAGH+73rT7LzeqA
-        0UjP3/mANT2MPJKm4XbCasRb7uI3fgjuZ/FEHfG0uuNmjGYQcy6HXpL6yKbaPdrwlzd1IWuHpZxp
-        DrFcINJ882XHrLEBh0tLv/mnGlbFPoYgDcQc65L7za9bQ4GOESJqQU8DW+tc3sCoPj/Y3Yk70FWH
-        aw4DeTeTfWVe69mSqwamD67E5tfv7OxeKCC+kopsK/uZLbOcIVDySUd21/eoz3TQJCg81itWlxxF
-        8x4VIQzoWiNxF16GOS9bAukzS4i8v1tgvVw3CkRvTsAq92n1dc3nVN5fNx15g04dhL0g3eDY2io1
-        Y/kzrKNRxrK6hCV2HU3Jfs8D4oGcsKY9Fn1BtL8BdFYAdeAxc8k3/8GnlgmoV/rRnRtzEX71RK83
-        q/35j1jKK/tNfdS/wHxvFQu8zWYlUXDW9bksHg7sbo6N3SRt2PrJsARffL9DrDcrdyBp38DPfCoQ
-        f38r2Swmai5/x1Nlu22jKaDFC7KovRDh5FTunBp3WwqCUKEqunLR3KVlKC+39YX4aUjBKD+NEX7X
-        P1r22UOfYzbeoDffTtS5uV7GErZW0h+9KQJDX0JBhkDfLQo1N09nmL/5Bhiu5VPjmqjD7tUYLSik
-        EpA6ZsqwO2VVDlKiT+h1KIaoS4ugBd/5or/1O3qBGcD0yV/oY5tifW0rGoKNadyRYFuBy0javwD4
-        NMo3f94zFtZH/ud/aELW/pvXRgluwfP0x68vh2tg/PG/1tpHOvXUuP/lETKpQslI5AkatGqgI5k/
-        3cGiawuE9VPSyfztZ+xZ1go8v8cYeyipavbzb/GjD6minXuXFLrbg7g01m8+7Fxat9oN4g+j1AIX
-        nH3rrwVj66gYJY9b1L8uLw1WtpqSLd6fM4YsLtj7fJNjJMIzWPL3ywMavozUupb20AfSoYILtDn6
-        84vELv0S5kpIkbjrpuwPfwmGjYQku8uBcFtvEiSkfRIQHludJa8PhEuyAei1trU708ER4SSHB3zU
-        W06fV9Ks8BxeD1SP0rSe57UP4VooPFbf1gHM6LUUcvp4GGT51gfbA1mE9f1YYLuRlWi9kgGB4C5O
-        9GAvtvu5yjoPz5pYY4zugst2+tOBumSH1LxHyJ0jyo9wauua7O3XNVtMO2tg7+V3IontAcxD/Jqh
-        ubwIvbc6ijrTqCXotpJIFc5FbHYvcw4vnNFj5z5u6mWWIw8qkt/Qg3cLszXnPR7IhVVS1XluMxI/
-        xBBAKeapW5ApY5301mBNDlck2h8PrMn7JIJFbs7UiJ8kG03B9cDxWdzx8aPVbOVUmUBouguxfbPP
-        mKjWNxhGYEWQj0/ZrJOa/62fnz7U6zPUUjgEbfdnfuf8VDgwy94HrMe+566L/+bhKPgIl7zI6U8h
-        mhrw3oHDL78z9s3nIqGvHDvGEjG25ftY+vYDrGsvvp6PKioBMFaMxML29UXfXRTw43nuLaBgvb6R
-        Bb/8B9FI74f5+RbKXx6jaJtVQ0f3Q7k/12cNB3nc6lRO1QoWAxeixi1BtPiNGUL+teGpbQvD0Pey
-        IoLp824JTO59NpNJ6SFVFEjawWh1VlbtDF/K3qUG94CMvqdRA6p7fP78l7twyyPYJ3ZlYr+ePjW5
-        vi0L1iVnozJc02hOHa2Rv36ILOUprpcj/tjw+rY+2Dm/d4xOJ92Tf/xJfTAB0N96JpxrUttQ02ht
-        xgDJ19gZ8c//rBN7QMDrJkedOVSzMWzPMdyOyoT2xn2pxy9fg21cNtP+vHPcHnGuBiEkFlW2mDHm
-        OfsZdpYAsMKLhT5eT08OjALpCfzsNLaquO1hcZgcwhwJuIvbXA1gb5wDko4XO1vEArbAvXAyteFh
-        cNnpLGvQng2dPkprqic+WlJZGe0r2fNmw358FBRJxMii6Zcf38x/+QXndfvSf/4QvJ6XHmvRvhtY
-        4D0k6W7A5Ntv+2H83oe2m2Y0g9D55rVAg/0oQiKxswB6Im0SiWxMheL9Z2LdMPYV+OWH42mjusTM
-        ux7MZltgMy0k1ke58gLf+aFq7nhgTjiQ/3nf2mVZh+HKGgIDeTtj7CQkI6qSWFAUP5i8u+d9oD1M
-        uP03X9JDFdpRb5XXGG4dZ0fN4NG7bMcXJehasqBebD9gXiW1lW9pvKO6ewrAt7+uwB1Ch2rlNAHW
-        8bsGfip0Ii/8Zjq7bo4N+PJq+s1/YHs+vnK4+YjoD+9dvFeWAF0cAT7l12Egx3qL4ErFnhpxiIf9
-        GV9FuM5Cg60mr4cF7oIS6s3WJFs327G1GVMEI3nOqbbfpdGojXsR7urHEYlfXracuM4B37z8zW9P
-        0FOtWIF9D05fvnZnzO65HIIXDfHXr331UirBTn1H1OCEsP7pgXQqh4rI85Vn6+/9LE5ikr3/rIcv
-        b7Z+/AEf4aq7c+ebKQDGjIkgVLk+7697B2qCpJGBuqCeOHbWwEswEI6mZpuRn14GhG/JJtKd+udX
-        4I9H+8dmZfP97At/+MzmxuAw4c/Og/sxAdTyzoQxJeBz8PE39pdv7/W3U3UpLDUlxt4NfMD4q9d3
-        lxT4y1N0iryBQLe95tRNPu9h0TYOAvBS6/T40XS2rHYfSycGKbVhcmRbAroQLgN5YG+gA+sfFkhg
-        2PYJEoT9AkZwfcdgE79V6pq+r/M6fwqg3qEL9peDEs1uqCtw95JK8tRufs1AUNnAOJxe1Do5mr47
-        inog/fR9zmPb/fILBRKXn7G3T/WMbu61DS89uyMAETeQjU57sG4yRIZZoe46n68KtNj1g621Z+54
-        VK0S2l6c4JsalD+9QeCb12mmcChbrdsg/vIZxpxfRayMCg++L0WLoDFCl11lXQDvs5lQnz/J7Mej
-        gZvUzc+/RuyRqSncZ1eFfv25y3/rWT6cT3eMkiCrd+mIE7BjoMX6+/OqWXWTX/CrF4QT4ZbNh+3m
-        9ocXWf1QZXMfbC3ocXBPvTPl2Ki3XAo43q7oob/OP55syGopqPSX10lTAQPoUS1SWynf7penoD9+
-        6dSdrXoXk1SERg71n37UfSY4DfzuT2C9I162Oz1CBxo599O/dlhF5hB4E5cEF8MCAG2WbIR5sb38
-        /HJEj2pbwN4r7ljV9B2YuQeCkmJtEsRzjxysxZ4UkJ6klSoXdacT64RmcNw8O3pQjIFR7TlDuM2s
-        FStqumEL3KUV7EygIMsbXwPtD3tJEh6BjP0xnPSfvsBbAkpSfCIuY0HoJ0AXLh2av3x6Fpw1he5B
-        1bC2zPMwvgXqwaOPY4pbfxvN8f4kyE8zeFFzIyqR8Ov/H1+2qdL2ZOgVWw3+jDfubyXaYWfiwIdc
-        P9SrpHR4PbeqAdqoGagenHWXLP7E//whPpyTKWNfvyh/88mf/YWV2h4C1s4LsDMgqtP4YSXQo/mZ
-        BhMph13Udx7cbZMNgS/ryabuo4jwiFyMcXRTsuWbp6BUPg/kQ1EdLQFNGrgxrTs2pE8E+tsaSvAy
-        te6Pp4AJLoMB9XiDsdtMhb5uu30KN9GxR2y6JMP84rcB1IVzh7oqx9my9ZAnhfvXif78ycKUjQPS
-        zb7+7S8w8kQ3AkVJawjvbIp6XuUlgbzURmQP19plV9aMsBssHZtA93Xy88O/vK+ViaiP0r0U5K8f
-        psahcKNZcKQbbFwUY6x2oT6/KBbAb/9HxyWuZ+mZVjByKo/65oV3R/zwNPjjZUyITsMo908e3LVD
-        S/g4pDWTkOj94Y+XdHOpZ16wbLgRhSOST/CarSMnCWDevp/426/An/xvc+1Av+uVzXIj2dI+Fy5Y
-        iY7IpZddj8D+qhK0+er7aqR7CAso+kRsmr6epQu7yTW2Ttj/wCaaP5z1gostEqpc0hHQIuo18OXZ
-        ZJGO/cDsXsjB8T4w0hqqGHV0k3I//4rV6D5n9P1CEjjs9SO27mrGGA6iFGZw80Q722U1WY6cLe2U
-        LsV2RdRhXvkcwfAyZjTcbnBGfzx4t26m335a9NvvBV8egY+k58A01GYKv/yY+rMj6N3LA9ZPn7AD
-        d80wbMWbAr/9AvuqoLBp3r1n+M1zROhj7+tPrQpu1BNP3nZaRUtyaEZ4xK+G/Pj1ck2mAv78uTek
-        gt5utrMo/fTLFHiSrVn/GeUfn4y/ej0XVdjCe5xLGD0kTWeKb8ywAsz4+jPObVM8a/CsSTVG/srp
-        0zBWJfyUq0Qkv/m4q+ynrz/1AGrPYaszxC8Q5I8EzdrZ0dmLditYrhcfW5X/YqOPDhW8XVf05XOb
-        aC3sxYITjD2qLM1xYCu1R2guDUFSv9tnP14n9/dbivFxkw5ffbaBwCqLKi/j5tIHbsr9ZVOpWHck
-        oLPrPnZk8zsT6BNx0dgdzyOA1tum+uodwWrdakn8+3cq4D//9ddf/+t3wqBp78X7ezBgKpbp3/99
-        VODft/vt3zwv/JsKf04ikPFWFn//81+HEP7uhrbppv89ta/iM/79z1/bP6cN/p7a6fb+fy7/6/td
-        //mv/wMAAP//AwDOXgQl4SAAAA==
-    headers:
-      CF-Cache-Status:
-      - DYNAMIC
-      CF-RAY:
-      - 7c09bf823fb50b70-AMS
-      Connection:
-      - keep-alive
-      Content-Encoding:
-      - gzip
-      Content-Type:
-      - application/json
-      Date:
-      - Mon, 01 May 2023 17:29:41 GMT
-      Server:
-      - cloudflare
-      access-control-allow-origin:
-      - '*'
-      alt-svc:
-      - h3=":443"; ma=86400, h3-29=":443"; ma=86400
-      openai-organization:
-      - user-kd1j0bcill5flig1m29wdaof
-      openai-processing-ms:
-      - '69'
-      openai-version:
-      - '2020-10-01'
-      strict-transport-security:
-      - max-age=15724800; includeSubDomains
-      x-ratelimit-limit-requests:
-      - '3000'
-      x-ratelimit-remaining-requests:
-      - '2999'
-      x-ratelimit-reset-requests:
-      - 20ms
-      x-request-id:
-      - 555d4ffdb6ceac9f62f60bb64d87170d
-    status:
-      code: 200
-      message: OK
-version: 1

tests/integration/test_git_commands.py

@@ -1,42 +0,0 @@
-import pytest
-from git.exc import GitCommandError
-from git.repo.base import Repo
-
-from autogpt.commands.git_operations import clone_repository
-
-
-@pytest.fixture
-def mock_clone_from(mocker):
-    return mocker.patch.object(Repo, "clone_from")
-
-
-def test_clone_auto_gpt_repository(workspace, mock_clone_from, config):
-    mock_clone_from.return_value = None
-
-    repo = "github.com/Significant-Gravitas/Auto-GPT.git"
-    scheme = "https://"
-    url = scheme + repo
-    clone_path = str(workspace.get_path("auto-gpt-repo"))
-
-    expected_output = f"Cloned {url} to {clone_path}"
-
-    clone_result = clone_repository(url=url, clone_path=clone_path)
-
-    assert clone_result == expected_output
-    mock_clone_from.assert_called_once_with(
-        url=f"{scheme}{config.github_username}:{config.github_api_key}@{repo}",
-        to_path=clone_path,
-    )
-
-
-def test_clone_repository_error(workspace, mock_clone_from):
-    url = "https://github.com/this-repository/does-not-exist.git"
-    clone_path = str(workspace.get_path("does-not-exist"))
-
-    mock_clone_from.side_effect = GitCommandError(
-        "clone", "fatal: repository not found", ""
-    )
-
-    result = clone_repository(url=url, clone_path=clone_path)
-
-    assert "Error: " in result

tests/integration/test_llm_utils.py

@@ -1,14 +1,9 @@
 import string
-from unittest.mock import MagicMock
 
 import pytest
 from numpy.random import RandomState
-from pytest_mock import MockerFixture
 
-from autogpt.config import Config
-from autogpt.llm import llm_utils
-from autogpt.llm.api_manager import ApiManager
-from autogpt.llm.modelsinfo import COSTS
+from autogpt.llm.llm_utils import get_ada_embedding
 from tests.utils import requires_api_key
 
 
@@ -21,42 +16,10 @@ def random_large_string():
     return "".join(random.choice(list(string.ascii_lowercase), size=n_characters))
 
 
-@pytest.fixture()
-def api_manager(mocker: MockerFixture):
-    api_manager = ApiManager()
-    mocker.patch.multiple(
-        api_manager,
-        total_prompt_tokens=0,
-        total_completion_tokens=0,
-        total_cost=0,
-    )
-    yield api_manager
-
-
-@pytest.fixture()
-def spy_create_embedding(mocker: MockerFixture):
-    return mocker.spy(llm_utils, "create_embedding")
-
-
-@pytest.mark.vcr
-@requires_api_key("OPENAI_API_KEY")
-def test_get_ada_embedding(
-    config: Config, api_manager: ApiManager, spy_create_embedding: MagicMock
-):
-    token_cost = COSTS[config.embedding_model]["prompt"]
-    llm_utils.get_ada_embedding("test")
-
-    spy_create_embedding.assert_called_once_with("test", model=config.embedding_model)
-
-    assert (prompt_tokens := api_manager.get_total_prompt_tokens()) == 1
-    assert api_manager.get_total_completion_tokens() == 0
-    assert api_manager.get_total_cost() == (prompt_tokens * token_cost) / 1000
-
-
-@pytest.mark.vcr
+@pytest.mark.xfail(reason="We have no mechanism for embedding large strings.")
 @requires_api_key("OPENAI_API_KEY")
 def test_get_ada_embedding_large_context(random_large_string):
     # This test should be able to mock the openai call after we have a fix.  We don't need
     # to hit the API to test the logic of the function (so not using vcr). This is a quick
     # regression test to document the issue.
-    llm_utils.get_ada_embedding(random_large_string)
+    get_ada_embedding(random_large_string)

tests/test_llm_utils.py

@@ -1,7 +1,8 @@
 import pytest
 from openai.error import APIError, RateLimitError
 
-from autogpt.llm import llm_utils
+from autogpt.llm import COSTS, get_ada_embedding
+from autogpt.llm.llm_utils import retry_openai_api
 
 
 @pytest.fixture(params=[RateLimitError, APIError])
@@ -12,12 +13,22 @@ def error(request):
         return request.param("Error")
 
 
+@pytest.fixture
+def mock_create_embedding(mocker):
+    mock_response = mocker.MagicMock()
+    mock_response.usage.prompt_tokens = 5
+    mock_response.__getitem__.side_effect = lambda key: [{"embedding": [0.1, 0.2, 0.3]}]
+    return mocker.patch(
+        "autogpt.llm.llm_utils.create_embedding", return_value=mock_response
+    )
+
+
 def error_factory(error_instance, error_count, retry_count, warn_user=True):
     class RaisesError:
         def __init__(self):
             self.count = 0
 
-        @llm_utils.retry_openai_api(
+        @retry_openai_api(
             num_retries=retry_count, backoff_base=0.001, warn_user=warn_user
         )
         def __call__(self):
@@ -30,7 +41,7 @@ def error_factory(error_instance, error_count, retry_count, warn_user=True):
 
 
 def test_retry_open_api_no_error(capsys):
-    @llm_utils.retry_openai_api()
+    @retry_openai_api()
     def f():
         return 1
 
@@ -103,31 +114,16 @@ def test_retry_openapi_other_api_error(capsys):
     assert output.out == ""
 
 
-def test_chunked_tokens():
-    text = "Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model"
-    expected_output = [
-        (
-            13556,
-            12279,
-            2898,
-            374,
-            459,
-            22772,
-            1825,
-            31874,
-            3851,
-            67908,
-            279,
-            17357,
-            315,
-            279,
-            480,
-            2898,
-            12,
-            19,
-            4221,
-            1646,
-        )
-    ]
-    output = list(llm_utils.chunked_tokens(text, "cl100k_base", 8191))
-    assert output == expected_output
+def test_get_ada_embedding(mock_create_embedding, api_manager):
+    model = "text-embedding-ada-002"
+    embedding = get_ada_embedding("test")
+    mock_create_embedding.assert_called_once_with(
+        "test", model="text-embedding-ada-002"
+    )
+
+    assert embedding == [0.1, 0.2, 0.3]
+
+    cost = COSTS[model]["prompt"]
+    assert api_manager.get_total_prompt_tokens() == 5
+    assert api_manager.get_total_completion_tokens() == 0
+    assert api_manager.get_total_cost() == (5 * cost) / 1000

tests/test_utils.py

@@ -56,13 +56,67 @@ def test_readable_file_size():
 
 @patch("requests.get")
 def test_get_bulletin_from_web_success(mock_get):
-    expected_content = "Test bulletin from web"
-
     mock_get.return_value.status_code = 200
-    mock_get.return_value.text = expected_content
+    mock_get.return_value.text = "Test bulletin"
     bulletin = get_bulletin_from_web()
 
-    assert expected_content in bulletin
+    assert bulletin == "Test bulletin"
+
+
+@patch("requests.get")
+def test_get_bulletin_from_web_failure(mock_get):
+    mock_get.return_value.status_code = 404
+    bulletin = get_bulletin_from_web()
+    print(bulletin)
+    assert bulletin == ""
+
+
+@skip_in_ci
+def test_get_current_git_branch():
+    branch_name = get_current_git_branch()
+
+    # Assuming that the branch name will be non-empty if the function is working correctly.
+    assert branch_name != ""
+
+
+def test_get_latest_bulletin_no_file():
+    if os.path.exists("CURRENT_BULLETIN.md"):
+        os.remove("CURRENT_BULLETIN.md")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value=""):
+        bulletin = get_latest_bulletin()
+        assert bulletin == ""
+
+
+def test_get_latest_bulletin_with_file():
+    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
+        f.write("Test bulletin")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value=""):
+        bulletin = get_latest_bulletin()
+        assert bulletin == "Test bulletin"
+
+    os.remove("CURRENT_BULLETIN.md")
+
+
+def test_get_latest_bulletin_with_new_bulletin():
+    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
+        f.write("Old bulletin")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value="New bulletin"):
+        bulletin = get_latest_bulletin()
+        assert "New bulletin" in bulletin
+
+    os.remove("CURRENT_BULLETIN.md")
+
+
+@patch("requests.get")
+def test_get_bulletin_from_web_success(mock_get):
+    mock_get.return_value.status_code = 200
+    mock_get.return_value.text = "Test bulletin"
+    bulletin = get_bulletin_from_web()
+
+    assert bulletin == "Test bulletin"
     mock_get.assert_called_with(
         "https://raw.githubusercontent.com/Significant-Gravitas/Auto-GPT/master/BULLETIN.md"
     )
@@ -84,62 +138,6 @@ def test_get_bulletin_from_web_exception(mock_get):
     assert bulletin == ""
 
 
-def test_get_latest_bulletin_no_file():
-    if os.path.exists("CURRENT_BULLETIN.md"):
-        os.remove("CURRENT_BULLETIN.md")
-
-    bulletin, is_new = get_latest_bulletin()
-    assert is_new
-
-
-def test_get_latest_bulletin_with_file():
-    expected_content = "Test bulletin"
-    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
-        f.write(expected_content)
-
-    with patch("autogpt.utils.get_bulletin_from_web", return_value=""):
-        bulletin, is_new = get_latest_bulletin()
-        assert expected_content in bulletin
-        assert is_new == False
-
-    os.remove("CURRENT_BULLETIN.md")
-
-
-def test_get_latest_bulletin_with_new_bulletin():
-    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
-        f.write("Old bulletin")
-
-    expected_content = "New bulletin from web"
-    with patch("autogpt.utils.get_bulletin_from_web", return_value=expected_content):
-        bulletin, is_new = get_latest_bulletin()
-        assert "::NEW BULLETIN::" in bulletin
-        assert expected_content in bulletin
-        assert is_new
-
-    os.remove("CURRENT_BULLETIN.md")
-
-
-def test_get_latest_bulletin_new_bulletin_same_as_old_bulletin():
-    expected_content = "Current bulletin"
-    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
-        f.write(expected_content)
-
-    with patch("autogpt.utils.get_bulletin_from_web", return_value=expected_content):
-        bulletin, is_new = get_latest_bulletin()
-        assert expected_content in bulletin
-        assert is_new == False
-
-    os.remove("CURRENT_BULLETIN.md")
-
-
-@skip_in_ci
-def test_get_current_git_branch():
-    branch_name = get_current_git_branch()
-
-    # Assuming that the branch name will be non-empty if the function is working correctly.
-    assert branch_name != ""
-
-
 @patch("autogpt.utils.Repo")
 def test_get_current_git_branch_success(mock_repo):
     mock_repo.return_value.active_branch.name = "test-branch"
@@ -156,5 +154,47 @@ def test_get_current_git_branch_failure(mock_repo):
     assert branch_name == ""
 
 
+def test_get_latest_bulletin_no_file():
+    if os.path.exists("CURRENT_BULLETIN.md"):
+        os.remove("CURRENT_BULLETIN.md")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value=""):
+        bulletin = get_latest_bulletin()
+        assert bulletin == ""
+
+
+def test_get_latest_bulletin_with_file():
+    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
+        f.write("Test bulletin")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value=""):
+        bulletin = get_latest_bulletin()
+        assert bulletin == "Test bulletin"
+
+    os.remove("CURRENT_BULLETIN.md")
+
+
+def test_get_latest_bulletin_with_new_bulletin():
+    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
+        f.write("Old bulletin")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value="New bulletin"):
+        bulletin = get_latest_bulletin()
+        assert f" {Fore.RED}::UPDATED:: {Fore.CYAN}New bulletin{Fore.RESET}" in bulletin
+
+    os.remove("CURRENT_BULLETIN.md")
+
+
+def test_get_latest_bulletin_new_bulletin_same_as_old_bulletin():
+    with open("CURRENT_BULLETIN.md", "w", encoding="utf-8") as f:
+        f.write("Test bulletin")
+
+    with patch("autogpt.utils.get_bulletin_from_web", return_value="Test bulletin"):
+        bulletin = get_latest_bulletin()
+        assert bulletin == "Test bulletin"
+
+    os.remove("CURRENT_BULLETIN.md")
+
+
 if __name__ == "__main__":
     pytest.main()

tests/unit/test_file_operations.py

@@ -2,19 +2,25 @@
 This set of unit tests is designed to test the file operations that autoGPT has access to.
 """
 
-import hashlib
 import os
-import re
-from io import TextIOWrapper
 from pathlib import Path
+from tempfile import gettempdir
 
 import pytest
-from pytest_mock import MockerFixture
 
-import autogpt.commands.file_operations as file_ops
+from autogpt.commands.file_operations import (
+    append_to_file,
+    check_duplicate_operation,
+    delete_file,
+    download_file,
+    log_operation,
+    read_file,
+    search_files,
+    split_file,
+    write_to_file,
+)
 from autogpt.config import Config
 from autogpt.utils import readable_file_size
-from autogpt.workspace import Workspace
 
 
 @pytest.fixture()
@@ -23,186 +29,66 @@ def file_content():
 
 
 @pytest.fixture()
-def test_file_path(config, workspace: Workspace):
-    return workspace.get_path("test_file.txt")
+def test_file(workspace, file_content):
+    test_file = str(workspace.get_path("test_file.txt"))
+    with open(test_file, "w") as f:
+        f.write(file_content)
+    return test_file
 
 
 @pytest.fixture()
-def test_file(test_file_path: Path):
-    file = open(test_file_path, "w")
-    yield file
-    if not file.closed:
-        file.close()
+def test_directory(workspace):
+    return str(workspace.get_path("test_directory"))
 
 
 @pytest.fixture()
-def test_file_with_content_path(test_file: TextIOWrapper, file_content):
-    test_file.write(file_content)
-    test_file.close()
-    file_ops.log_operation(
-        "write", test_file.name, file_ops.text_checksum(file_content)
-    )
-    return Path(test_file.name)
+def test_nested_file(workspace):
+    return str(workspace.get_path("nested/test_file.txt"))
 
 
-@pytest.fixture()
-def test_directory(config, workspace: Workspace):
-    return workspace.get_path("test_directory")
-
-
-@pytest.fixture()
-def test_nested_file(config, workspace: Workspace):
-    return workspace.get_path("nested/test_file.txt")
-
-
-def test_file_operations_log(test_file: TextIOWrapper):
-    log_file_content = (
-        "File Operation Logger\n"
-        "write: path/to/file1.txt #checksum1\n"
-        "write: path/to/file2.txt #checksum2\n"
-        "write: path/to/file3.txt #checksum3\n"
-        "append: path/to/file2.txt #checksum4\n"
-        "delete: path/to/file3.txt\n"
-    )
-    test_file.write(log_file_content)
-    test_file.close()
-
-    expected = [
-        ("write", "path/to/file1.txt", "checksum1"),
-        ("write", "path/to/file2.txt", "checksum2"),
-        ("write", "path/to/file3.txt", "checksum3"),
-        ("append", "path/to/file2.txt", "checksum4"),
-        ("delete", "path/to/file3.txt", None),
-    ]
-    assert list(file_ops.operations_from_log(test_file.name)) == expected
-
-
-def test_file_operations_state(test_file: TextIOWrapper):
-    # Prepare a fake log file
-    log_file_content = (
-        "File Operation Logger\n"
-        "write: path/to/file1.txt #checksum1\n"
-        "write: path/to/file2.txt #checksum2\n"
-        "write: path/to/file3.txt #checksum3\n"
-        "append: path/to/file2.txt #checksum4\n"
-        "delete: path/to/file3.txt\n"
-    )
-    test_file.write(log_file_content)
-    test_file.close()
-
-    # Call the function and check the returned dictionary
-    expected_state = {
-        "path/to/file1.txt": "checksum1",
-        "path/to/file2.txt": "checksum4",
-    }
-    assert file_ops.file_operations_state(test_file.name) == expected_state
-
-
-def test_is_duplicate_operation(config, mocker: MockerFixture):
-    # Prepare a fake state dictionary for the function to use
-    state = {
-        "path/to/file1.txt": "checksum1",
-        "path/to/file2.txt": "checksum2",
-    }
-    mocker.patch.object(file_ops, "file_operations_state", lambda _: state)
-
-    # Test cases with write operations
-    assert (
-        file_ops.is_duplicate_operation("write", "path/to/file1.txt", "checksum1")
-        is True
-    )
-    assert (
-        file_ops.is_duplicate_operation("write", "path/to/file1.txt", "checksum2")
-        is False
-    )
-    assert (
-        file_ops.is_duplicate_operation("write", "path/to/file3.txt", "checksum3")
-        is False
-    )
-    # Test cases with append operations
-    assert (
-        file_ops.is_duplicate_operation("append", "path/to/file1.txt", "checksum1")
-        is False
-    )
-    # Test cases with delete operations
-    assert file_ops.is_duplicate_operation("delete", "path/to/file1.txt") is False
-    assert file_ops.is_duplicate_operation("delete", "path/to/file3.txt") is True
+def test_check_duplicate_operation(config, test_file):
+    log_operation("write", test_file)
+    assert check_duplicate_operation("write", test_file) is True
 
 
 # Test logging a file operation
-def test_log_operation(config: Config):
-    file_ops.log_operation("log_test", "path/to/test")
-    with open(config.file_logger_path, "r", encoding="utf-8") as f:
+def test_log_operation(test_file, config):
+    file_logger_name = config.file_logger_path
+    if os.path.exists(file_logger_name):
+        os.remove(file_logger_name)
+
+    log_operation("log_test", test_file)
+    with open(config.file_logger_path, "r") as f:
         content = f.read()
-    assert f"log_test: path/to/test\n" in content
-
-
-def test_text_checksum(file_content: str):
-    checksum = file_ops.text_checksum(file_content)
-    different_checksum = file_ops.text_checksum("other content")
-    assert re.match(r"^[a-fA-F0-9]+$", checksum) is not None
-    assert checksum != different_checksum
-
-
-def test_log_operation_with_checksum(config: Config):
-    file_ops.log_operation("log_test", "path/to/test", checksum="ABCDEF")
-    with open(config.file_logger_path, "r", encoding="utf-8") as f:
-        content = f.read()
-    assert f"log_test: path/to/test #ABCDEF\n" in content
+    assert f"log_test: {test_file}" in content
 
 
 # Test splitting a file into chunks
 def test_split_file():
     content = "abcdefghij"
-    chunks = list(file_ops.split_file(content, max_length=4, overlap=1))
+    chunks = list(split_file(content, max_length=4, overlap=1))
     expected = ["abcd", "defg", "ghij"]
     assert chunks == expected
 
 
-def test_read_file(test_file_with_content_path: Path, file_content):
-    content = file_ops.read_file(test_file_with_content_path)
+def test_read_file(test_file, file_content):
+    content = read_file(test_file)
     assert content == file_content
 
 
-def test_write_to_file(test_file_path: Path):
+def test_write_to_file(config, test_nested_file):
     new_content = "This is new content.\n"
-    file_ops.write_to_file(str(test_file_path), new_content)
-    with open(test_file_path, "r", encoding="utf-8") as f:
+    write_to_file(test_nested_file, new_content)
+    with open(test_nested_file, "r") as f:
         content = f.read()
     assert content == new_content
 
 
-def test_write_file_logs_checksum(config: Config, test_file_path: Path):
-    new_content = "This is new content.\n"
-    new_checksum = file_ops.text_checksum(new_content)
-    file_ops.write_to_file(str(test_file_path), new_content)
-    with open(config.file_logger_path, "r", encoding="utf-8") as f:
-        log_entry = f.read()
-    assert log_entry == f"write: {test_file_path} #{new_checksum}\n"
-
-
-def test_write_file_fails_if_content_exists(test_file_path: Path):
-    new_content = "This is new content.\n"
-    file_ops.log_operation(
-        "write",
-        str(test_file_path),
-        checksum=file_ops.text_checksum(new_content),
-    )
-    result = file_ops.write_to_file(str(test_file_path), new_content)
-    assert result == "Error: File has already been updated."
-
-
-def test_write_file_succeeds_if_content_different(test_file_with_content_path: Path):
-    new_content = "This is different content.\n"
-    result = file_ops.write_to_file(str(test_file_with_content_path), new_content)
-    assert result == "File written to successfully."
-
-
-def test_append_to_file(test_nested_file: Path):
+def test_append_to_file(test_nested_file):
     append_text = "This is appended text.\n"
-    file_ops.write_to_file(test_nested_file, append_text)
+    write_to_file(test_nested_file, append_text)
 
-    file_ops.append_to_file(test_nested_file, append_text)
+    append_to_file(test_nested_file, append_text)
 
     with open(test_nested_file, "r") as f:
         content_after = f.read()
@@ -210,45 +96,24 @@ def test_append_to_file(test_nested_file: Path):
     assert content_after == append_text + append_text
 
 
-def test_append_to_file_uses_checksum_from_appended_file(
-    config: Config, test_file_path: Path
-):
-    append_text = "This is appended text.\n"
-    file_ops.append_to_file(test_file_path, append_text)
-    file_ops.append_to_file(test_file_path, append_text)
-    with open(config.file_logger_path, "r", encoding="utf-8") as f:
-        log_contents = f.read()
-
-    digest = hashlib.md5()
-    digest.update(append_text.encode("utf-8"))
-    checksum1 = digest.hexdigest()
-    digest.update(append_text.encode("utf-8"))
-    checksum2 = digest.hexdigest()
-    assert log_contents == (
-        f"append: {test_file_path} #{checksum1}\n"
-        f"append: {test_file_path} #{checksum2}\n"
-    )
+def test_delete_file(config, test_file):
+    delete_file(test_file)
+    assert os.path.exists(test_file) is False
+    assert delete_file(test_file) == "Error: File has already been deleted."
 
 
-def test_delete_file(test_file_with_content_path: Path):
-    result = file_ops.delete_file(str(test_file_with_content_path))
-    assert result == "File deleted successfully."
-    assert os.path.exists(test_file_with_content_path) is False
-
-
-def test_delete_missing_file(config):
-    filename = "path/to/file/which/does/not/exist"
-    # confuse the log
-    file_ops.log_operation("write", filename, checksum="fake")
+def test_delete_missing_file(test_file):
+    os.remove(test_file)
     try:
-        os.remove(filename)
-    except FileNotFoundError as err:
-        assert str(err) in file_ops.delete_file(filename)
+        os.remove(test_file)
+    except FileNotFoundError as e:
+        error_string = str(e)
+        assert error_string in delete_file(test_file)
         return
-    assert False, f"Failed to test delete_file; {filename} not expected to exist"
+    assert True, "Failed to test delete_file"
 
 
-def test_list_files(workspace: Workspace, test_directory: Path):
+def test_search_files(config, workspace, test_directory):
     # Case 1: Create files A and B, search for A, and ensure we don't return A and B
     file_a = workspace.get_path("file_a.txt")
     file_b = workspace.get_path("file_b.txt")
@@ -266,7 +131,7 @@ def test_list_files(workspace: Workspace, test_directory: Path):
     with open(os.path.join(test_directory, file_a.name), "w") as f:
         f.write("This is file A in the subdirectory.")
 
-    files = file_ops.list_files(str(workspace.root))
+    files = search_files(str(workspace.root))
     assert file_a.name in files
     assert file_b.name in files
     assert os.path.join(Path(test_directory).name, file_a.name) in files
@@ -279,28 +144,26 @@ def test_list_files(workspace: Workspace, test_directory: Path):
 
     # Case 2: Search for a file that does not exist and make sure we don't throw
     non_existent_file = "non_existent_file.txt"
-    files = file_ops.list_files("")
+    files = search_files("")
     assert non_existent_file not in files
 
 
-def test_download_file(config, workspace: Workspace):
+def test_download_file():
     url = "https://github.com/Significant-Gravitas/Auto-GPT/archive/refs/tags/v0.2.2.tar.gz"
-    local_name = workspace.get_path("auto-gpt.tar.gz")
+    local_name = os.path.join(gettempdir(), "auto-gpt.tar.gz")
     size = 365023
     readable_size = readable_file_size(size)
     assert (
-        file_ops.download_file(url, local_name)
+        download_file(url, local_name)
         == f'Successfully downloaded and locally stored file: "{local_name}"! (Size: {readable_size})'
     )
     assert os.path.isfile(local_name) is True
     assert os.path.getsize(local_name) == size
 
     url = "https://github.com/Significant-Gravitas/Auto-GPT/archive/refs/tags/v0.0.0.tar.gz"
-    assert "Got an HTTP Error whilst trying to download file" in file_ops.download_file(
+    assert "Got an HTTP Error whilst trying to download file" in download_file(
         url, local_name
     )
 
     url = "https://thiswebsiteiswrong.hmm/v0.0.0.tar.gz"
-    assert "Failed to establish a new connection:" in file_ops.download_file(
-        url, local_name
-    )
+    assert "Failed to establish a new connection:" in download_file(url, local_name)