# TODO
- [x] ChatML format (actually need to add special tokens)  
- [x] Vicuna dataset merge (yahma/alpaca-cleaned)  
- [x] Phi-2 fine tuning  
- [x] Quantize /w llama.cpp  
- [x] Make custom component use llama.cpp + ChatML  
- [x] Continued synthetic dataset improvements (there are a bunch of TODOs in there)  
- [x] Licenses + Attributions  
- [x] Finish Readme/docs for initial release  
- [x] Function calling as JSON  
- [ ] multi-turn prompts; better instruct dataset like dolphin/wizardlm?  
- [x] Fine tune Phi-1.5 version  
- [x] make llama-cpp-python wheels for "llama-cpp-python>=0.2.24"  
- [ ] prime kv cache with current "state" so that requests are faster  
- [x] make a proper evaluation framework to run. not just loss. should test accuracy on the function calling  
- [x] add more remote backends  
    - LocalAI (openai compatible)  
    - Ollama  
    - support chat completions API (might fix Ollama + adds support for text-gen-ui characters)
- [x] more config options for prompt template (allow other than chatml)  
- [x] publish snapshot of dataset on HF  
- [ ] figure out DPO for refusals + fixing incorrect entity id  
- [ ] mixtral + prompting (no fine tuning)  
- [ ] use varied system prompts to add behaviors  
- [ ] setup github actions to build wheels that are optimized for RPIs

## more complicated ideas
- [ ] "context requests"  
    - basically just let the model decide what RAG/extra context it wants  
    - the model predicts special tokens as the first few tokens of its output  
    - the requested content is added to the context after the request tokens and then generation continues  
    - needs more complicated training b/c multi-turn + there will be some weird masking going on for training the responses properly  
- [ ] RAG for getting info for setting up new devices  
    - set up vectordb  
    - ingest home assistant docs  
    - "context request" from above to initiate a RAG search  
