From 7c3ea9fa084c6e08491131445e7e5329c0959398 Mon Sep 17 00:00:00 2001 From: Robert Brennan Date: Fri, 19 Apr 2024 10:47:54 -0400 Subject: [PATCH] Fix up documentation for specific LLMs (#1217) * Update LOCAL_LLM_GUIDE.md * Create WINDOWS_WSL_GUIDE.md * update docs * Update docs/Agents.md * update windows docs --------- Co-authored-by: Engel Nyst --- README.md | 23 ++-- docs/{documentation => }/Agents.md | 6 +- docs/documentation/LOCAL_LLM_GUIDE.md | 108 ------------------ .../AzureLLMs.md} | 0 docs/guides/LocalLLMs.md | 67 +++++++++++ docs/guides/Windows.md | 31 +++++ .../images/ollama.png | Bin 7 files changed, 117 insertions(+), 118 deletions(-) rename docs/{documentation => }/Agents.md (91%) delete mode 100644 docs/documentation/LOCAL_LLM_GUIDE.md rename docs/{documentation/AZURE_LLM_GUIDE.md => guides/AzureLLMs.md} (100%) create mode 100644 docs/guides/LocalLLMs.md create mode 100644 docs/guides/Windows.md rename docs/{documentation => guides}/images/ollama.png (100%) diff --git a/README.md b/README.md index 54f882354c..61ff16a465 100644 --- a/README.md +++ b/README.md @@ -117,7 +117,7 @@ After completing the MVP, the team will focus on research in various areas, incl * OpenDevin is still an alpha project. It is changing very quickly and is unstable. We are working on getting a stable release out in the coming weeks. * OpenDevin will issue many prompts to the LLM you configure. Most of these LLMs cost money--be sure to set spending limits and monitor usage. * OpenDevin runs `bash` commands within a Docker sandbox, so it should not affect your machine. But your workspace directory will be attached to that sandbox, and files in the directory may be modified or deleted. -* Our default Agent is currently the MonologueAgent, which has limited capabilities, but is fairly stable. We're working on other Agent implementations, including [SWE Agent](https://swe-agent.com/). You can [read about our current set of agents here](./docs/documentation/Agents.md). +* Our default Agent is currently the MonologueAgent, which has limited capabilities, but is fairly stable. We're working on other Agent implementations, including [SWE Agent](https://swe-agent.com/). You can [read about our current set of agents here](./docs/Agents.md). ## 🚀 Get Started The easiest way to run OpenDevin is inside a Docker container. @@ -144,6 +144,8 @@ If you want to use the (unstable!) bleeding edge, you can use `ghcr.io/opendevin See [Development.md](Development.md) for instructions on running OpenDevin without Docker. +If you're running on Windows and having trouble, check out our [guide for Windows users](./docs/guides/Windows.md) + ## 🤖 LLM Backends OpenDevin can work with any LLM backend. For a full list of the LM providers and models available, please consult the @@ -159,13 +161,20 @@ The following environment variables might be necessary for some LLMs: * `LLM_DEPLOYMENT_NAME` * `LLM_API_VERSION` -**Note on Alternative Models:** -Some alternative models may prove more challenging to tame than others. -Fear not, brave adventurer! We shall soon unveil LLM-specific documentation to guide you on your quest. -And if you've already mastered the art of wielding a model other than OpenAI's GPT, -we encourage you to [share your setup instructions with us](https://github.com/OpenDevin/OpenDevin/issues/417). +We have a few guides for running OpenDevin with specific model providers: +* [ollama](./docs/guides/LocalLLMs.md) +* [Azure](./docs/guides/AzureLLMs.md) + +If you're using another provider, we encourage you to open a PR to share your setup! + +**Note on Alternative Models:** +The best models are GPT-4 and Claude 3. Current local and open source models are +not nearly as powerful. When using an alternative model, +you may see long wait times between messages, +poor responses, or errors about malformed JSON. OpenDevin +can only be as powerful as the models driving it--fortunately folks on our team +are actively working on building better open source models! -There is also [documentation for running with local models using ollama](./docs/documentation/LOCAL_LLM_GUIDE.md). ## ⭐️ Research Strategy diff --git a/docs/documentation/Agents.md b/docs/Agents.md similarity index 91% rename from docs/documentation/Agents.md rename to docs/Agents.md index e0ed60fd15..b983a2291f 100644 --- a/docs/documentation/Agents.md +++ b/docs/Agents.md @@ -33,7 +33,7 @@ Short term memory is stored as a Monologue object and the model can condense it `_initialize`: Utilizes the `INITIAL_THOUGHTS` list to give the agent a context for its capabilities and how to navigate the `/workspace` -`step`: Modifies the current state by adding the most rescent actions and observations, then prompts the model to think about its next action to take. +`step`: Modifies the current state by adding the most recent actions and observations, then prompts the model to think about its next action to take. `search_memory`: Uses `VectorIndexRetriever` to find related memories within the long term memory. @@ -91,6 +91,6 @@ The Code Act Agent is a minimalist agent. The agent works by passing the model a ### Methods: `__init__`: Initializes an agent with `llm` and a list of messages `List[Mapping[str, str]]` -`step`: First, gets messages from state and then compiles them into a list for context. Next, pass the context list with the prompt to get the next command to execute. Finally, Execute command if valid, else return `AgentEchoAction(INVALID_INPUT_MESSAGE)` +`step`: First, gets messages from state and then compiles them into a list for context. Next, pass the context list with the prompt to get the next command to execute. Finally, Execute command if valid, else return `AgentEchoAction(INVALID_INPUT_MESSAGE)` -`search_memory`: Not yet implemented \ No newline at end of file +`search_memory`: Not yet implemented diff --git a/docs/documentation/LOCAL_LLM_GUIDE.md b/docs/documentation/LOCAL_LLM_GUIDE.md deleted file mode 100644 index c70526327a..0000000000 --- a/docs/documentation/LOCAL_LLM_GUIDE.md +++ /dev/null @@ -1,108 +0,0 @@ -# Local LLM Guide with Ollama server - -## 0. Install Ollama: -run the following command in a conda env with CUDA etc. - -Linux: -``` -curl -fsSL https://ollama.com/install.sh | sh -``` -Windows or macOS: - -- Download from [here](https://ollama.com/download/) - -## 1. Install Models: -Ollama model names can be found [here](https://ollama.com/library) (See example below) - -![alt text](images/ollama.png) - -Once you have found the model you want to use copy the command and run it in your conda env. - -Example of llama2 q4 quantized: -``` -conda activate -ollama run llama2:13b-chat-q4_K_M -``` - -you can check which models you have downloaded like this: -``` -~$ ollama list -NAME ID SIZE MODIFIED -llama2:latest 78e26419b446 3.8 GB 6 weeks ago -mistral:7b-instruct-v0.2-q4_K_M eb14864c7427 4.4 GB 2 weeks ago -starcoder2:latest f67ae0f64584 1.7 GB 19 hours ago -``` -## 2. Run Ollama in CLI: -This command starts up the ollama server that is on port `11434` - -This will show the requests in CLI -``` -conda activate -ollama serve -``` -or - -This will run with no output in the background -``` -sudo systemctl start ollama -``` - -If you see something like this: -``` -Error: listen tcp 127.0.0.1:11434: bind: address already in use -``` -This is not an error it just means the server is already running - -To stop the server use: -``` -sudo systemctl stop ollama -``` - -For more info go [here](https://github.com/ollama/ollama/blob/main/docs/faq.md) - -## 3. Start OpenDevin - -Use the instructions in [README.md](/README.md) to start OpenDevin using Docker. -When running `docker run`, add the following environment variables using `-e`: - -``` -LLM_API_KEY="ollama" -LLM_MODEL="ollama/" -LLM_EMBEDDING_MODEL="local" -LLM_BASE_URL="http://localhost:" -WORKSPACE_DIR="./workspace" -``` -Notes: -- The API key should be set to `"ollama"` -- The base url needs to be `localhost` -- By default ollama port is `11434` unless you set it -- `model_name` needs to be the entire model name - - Example: `LLM_MODEL="ollama/llama2:13b-chat-q4_K_M"` - -You should now be able to connect to `http://localhost:3001/` with your local model running! - - -## Additional Notes for WSL2 Users: - -1. If you encounter the following error during setup: `Exception: Failed to create opendevin user in sandbox: b'useradd: UID 0 is not unique\n'` -You can resolve it by running: - ``` - export SANDBOX_USER_ID=1000 - ``` - -2. If you face issues running Poetry even after installing it during the build process, you may need to add its binary path to your environment: - ``` - export PATH="$HOME/.local/bin:$PATH" - ``` - -3. If you experiencing issues related to networking, such as `NoneType object has no attribute 'request'` when executing `make run`, you may need to configure your WSL2 networking settings. Follow these steps: - - Open or create the `.wslconfig` file located at `C:\Users\%username%\.wslconfig` on your Windows host machine. - - Add the following configuration to the `.wslconfig` file: - ``` - [wsl2] - networkingMode=mirrored - localhostForwarding=true - ``` - - Save the `.wslconfig` file. - - Restart WSL2 completely by exiting any running WSL2 instances and executing the command `wsl --shutdown` in your command prompt or terminal. - - After restarting WSL, attempt to execute `make run` again. The networking issue should be resolved. diff --git a/docs/documentation/AZURE_LLM_GUIDE.md b/docs/guides/AzureLLMs.md similarity index 100% rename from docs/documentation/AZURE_LLM_GUIDE.md rename to docs/guides/AzureLLMs.md diff --git a/docs/guides/LocalLLMs.md b/docs/guides/LocalLLMs.md new file mode 100644 index 0000000000..4a91e132da --- /dev/null +++ b/docs/guides/LocalLLMs.md @@ -0,0 +1,67 @@ +# Local LLM Guide with Ollama server + +## 0. Install and Start ollama: +run the following command in a conda env with CUDA etc. + +Linux: +``` +curl -fsSL https://ollama.com/install.sh | sh +``` +Windows or macOS: + +- Download from [here](https://ollama.com/download/) + +Then run: +```bash +ollama serve +``` + +## 1. Install Models: +Ollama model names can be found [here](https://ollama.com/library). For a small example, you can use +the codellama:7b model. Bigger models will generally perform better. + +``` +ollama pull codellama:7b +``` + +you can check which models you have downloaded like this: +``` +~$ ollama list +NAME ID SIZE MODIFIED +llama2:latest 78e26419b446 3.8 GB 6 weeks ago +mistral:7b-instruct-v0.2-q4_K_M eb14864c7427 4.4 GB 2 weeks ago +starcoder2:latest f67ae0f64584 1.7 GB 19 hours ago +``` + +## 3. Start OpenDevin + +Use the instructions in [README.md](/README.md) to start OpenDevin using Docker. +When running `docker run`, add the following environment variables using `-e`: + +```bash +LLM_API_KEY="ollama" +LLM_BASE_URL="http://localhost:11434" +``` + +For example: +```bash +# The directory you want OpenDevin to modify. MUST be an absolute path! +export WORKSPACE_DIR=$(pwd)/workspace + +docker run \ + -e LLM_API_KEY="ollama" \ + -e LLM_BASE_URL="http://localhost:11434" + -e WORKSPACE_MOUNT_PATH=$WORKSPACE_DIR \ + -v $WORKSPACE_DIR:/opt/workspace_base \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -p 3000:3000 \ + ghcr.io/opendevin/opendevin:main +``` + +You should now be able to connect to `http://localhost:3001/` + +## 4. Select your Model +In the OpenDevin UI, click on the Settings wheel in the bottom-left corner. +Then in the `Model` input, enter `codellama:7b`, or the name of the model you pulled earlier, and click Save. + +And now you're ready to go! diff --git a/docs/guides/Windows.md b/docs/guides/Windows.md new file mode 100644 index 0000000000..60a31fea59 --- /dev/null +++ b/docs/guides/Windows.md @@ -0,0 +1,31 @@ +# Notes for Windows and WSL Users +OpenDevin only supports Windows via [WSL](https://learn.microsoft.com/en-us/windows/wsl/install). +Please be sure to run all commands inside your WSL terminal. + +## Troubleshooting + +### Failed to create opendevin user +If you encounter the following error during setup: `Exception: Failed to create opendevin user in sandbox: b'useradd: UID 0 is not unique\n'` +You can resolve it by running: + ``` + export SANDBOX_USER_ID=1000 + ``` + +### Poetry Installation +If you face issues running Poetry even after installing it during the build process, you may need to add its binary path to your environment: + ``` + export PATH="$HOME/.local/bin:$PATH" + ``` + +### NoneType object has no attribute 'request' +If you experiencing issues related to networking, such as `NoneType object has no attribute 'request'` when executing `make run`, you may need to configure your WSL2 networking settings. Follow these steps: +- Open or create the `.wslconfig` file located at `C:\Users\%username%\.wslconfig` on your Windows host machine. +- Add the following configuration to the `.wslconfig` file: +``` +[wsl2] +networkingMode=mirrored +localhostForwarding=true +``` +- Save the `.wslconfig` file. +- Restart WSL2 completely by exiting any running WSL2 instances and executing the command `wsl --shutdown` in your command prompt or terminal. +- After restarting WSL, attempt to execute `make run` again. The networking issue should be resolved. diff --git a/docs/documentation/images/ollama.png b/docs/guides/images/ollama.png similarity index 100% rename from docs/documentation/images/ollama.png rename to docs/guides/images/ollama.png