github/home-llm
1
1
Fork 0
mirror of https://github.com/acon96/home-llm.git synced 2026-01-09 21:58:00 -05:00
Code Issues Packages Projects Releases Wiki Activity

quickstart guide typos

Browse Source
This commit is contained in:
Alex O'Connell
2024-03-24 11:27:26 -04:00
parent ed622e7bbd
commit 19ae895f96
1 changed files with 7 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/Setup.md
View File

@@ -57,6 +57,7 @@ To ensure compatibility with your Home Assistant and Python versions, select the
- Example filenames:
- `llama_cpp_python-{version}-cp311-cp311-musllinux_1_2_x86_64.whl`
- `llama_cpp_python-{version}-cp312-cp312-musllinux_1_2_x86_64.whl`
Download the appropriate wheel and copy it to the `custom_components/llama_conversation/` directory.
After the wheel file has been copied to the correct folder.
@@ -70,10 +71,10 @@ This will trigger the installation of the wheel. If you ever need to update the
Once `llama-cpp-python` is installed, continue to the model selection.
### Step 2: Model Selection
The next step is to specify which model will be used by the integration. You may select any repository on HuggingFace that has a model in GGUF format in it. We will use `acon96/Home-3B-v3-GGUF` for this example. If you have less than 4GB of RAM then use ``acon96/Home-1B-v2-GGUF`.
The next step is to specify which model will be used by the integration. You may select any repository on HuggingFace that has a model in GGUF format in it. We will use `acon96/Home-3B-v3-GGUF` for this example. If you have less than 4GB of RAM then use `acon96/Home-1B-v2-GGUF`.
**Model Name**: Use either `acon96/Home-3B-v3-GGUF` or `acon96/Home-1B-v2-GGUF`
**Quantization Level**: The model will be downloaded in the selected quantization level from the HuggingFace repository. If unsure which level to choose, select `Q4_K_M`.
**Model Name**: Use either `acon96/Home-3B-v3-GGUF` or `acon96/Home-1B-v2-GGUF`
**Quantization Level**: The model will be downloaded in the selected quantization level from the HuggingFace repository. If unsure which level to choose, select `Q4_K_M`.
Pressing `Submit` will download the model from HuggingFace.
@@ -86,7 +87,7 @@ The model will be loaded into memory and should now be available to select as a
## Path 2: Using Mistral-Instruct-7B with Ollama Backend
### Overview
For those who have access to a GPU, you can also use the Mistral-Instruct-7B model to power your conversation agent. This path requires a separate machine that has a GPU that has [Ollama](https://ollama.com/) already installed on it. This path utilizes in-context learning examples, to prompt the model to produce the output that we expect.
For those who have access to a GPU, you can also use the Mistral-Instruct-7B model to power your conversation agent. This path requires a separate machine that has a GPU and has [Ollama](https://ollama.com/) already installed on it. This path utilizes in-context learning examples, to prompt the model to produce the output that we expect.
### Step 1: Downloading and serving the Model
Mistral can be easily set up and downloaded on the serving machine using the `ollama pull mistral` command.
@@ -113,6 +114,8 @@ This step allows you to configure how the model is "prompted". See [here](./Mode
For now, defaults for the model should have been populated and you can just scroll to the bottom and click `Submit`.
> NOTE: The key settings in this case are that our prompt references the `{{ response_examples }}` variable and the `Enable in context learning (ICL) examples` option is turned on.
## Configuring the Integration as a Conversation Agent
Now that the integration is configured and providing the conversation agent, we need to configure Home Assistant to use our conversation agent instead of the built in intent recognition system.