{% extends "admin/base.html" %} {% block title %}Help & Credits{% endblock %} {% block header_title %}Help & Credits{% endblock %} {% block content %}

How to Use Ollama Proxy Fortress

This proxy server acts as a secure gateway to your Ollama instances. The workflow is simple:

  1. Create a User: From the Dashboard, create a virtual user. This helps you organize API keys (e.g., one user for each application).
  2. Generate an API Key: Click "Manage" next to a user to access the key management page.
  3. Copy the Key: The full API key is shown only once upon creation. Copy it and store it securely in your application's configuration.
  4. Make Secure API Calls: Use the proxy's URL (e.g., `http://127.0.0.1:8080`) instead of your direct Ollama URL. Provide the API key in the `Authorization` header as a Bearer token.

Advanced Key Management

Per-Key Rate Limiting

When creating a new API key, you can set a custom rate limit that applies only to that key. You do this by filling in the two rate limit fields:

  • Rate Limit Requests: The maximum number of requests this key can make within the time window.
  • Window (Minutes): The duration of the time window in minutes.

Example: Setting `Requests` to `100` and `Window` to `5` means this specific key can be used a maximum of 100 times every 5 minutes.

If you leave these fields blank, the key will automatically use the global rate limit settings defined in your server's `.env` file.

Key Status: Active, Disabled, and Revoked

  • Active: The key is fully functional.
  • Disabled: The key is temporarily blocked and cannot be used for API calls. It can be re-enabled at any time. This is useful for temporarily pausing access for an application without deleting the key.
  • Revoked: The key is permanently and irreversibly destroyed. It can never be used again. Use this action if a key has been compromised or is no longer needed.

Usage Examples

In all examples, replace `op_..._...` with your actual generated API key.

Example 1: Listing Available Models (cURL)

This command fetches the federated list of all models from all active backend servers.

curl http://127.0.0.1:8080/api/tags \ -H "Authorization: Bearer op_prefix_secret"

Example 2: Streaming Generation with Python (`requests`)

A standard way to stream a response using the popular `requests` library.

import requests import json API_KEY = "op_prefix_secret" # <-- Replace with your key PROXY_URL = "http://127.0.0.1:8080/api/generate" headers = { "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" } data = { "model": "llama3", "prompt": "Tell me a short story about a robot who discovers music.", "stream": True } try: with requests.post(PROXY_URL, headers=headers, json=data, stream=True) as response: response.raise_for_status() # Raise an exception for bad status codes print("Story: ", end="", flush=True) for chunk in response.iter_lines(): if chunk: decoded_chunk = json.loads(chunk.decode('utf-8')) print(decoded_chunk.get("response", ""), end="", flush=True) print() except requests.exceptions.RequestException as e: print(f"\nAn error occurred: {e}")

Example 3: Using with `lollms-client`

To use the proxy with the `lollms-client` library, configure the host address to point to the proxy and provide your key as the `service_key`.

from lollms_client import LollmsClient, MSG_TYPE # Configure the client to use the secure proxy lc = LollmsClient( llm_binding_name="ollama", llm_binding_config={ "host_address": "http://localhost:8080", # <-- Point to the proxy server "model_name": "llama3", # Or any model you have "service_key": "op_prefix_secret", # <-- Replace with your API key } ) def streaming_callback(data, msg_type): if msg_type == MSG_TYPE.MSG_TYPE_CHUNK: print(data, end="", flush=True) return True print("--- Streaming with lollms-client ---") lc.generate_text( prompt="What is the capital of France?", stream=True, streaming_callback=streaming_callback ) print("\n\n--- Listing models ---") models = lc.listModels() print(models)

Credits & Acknowledgements

This application was developed with passion by the open-source community. It stands on the shoulders of giants and wouldn't be possible without the following incredible projects:

  • FastAPI: For the high-performance, easy-to-use web framework.
  • SQLAlchemy: For providing a powerful and Pythonic SQL toolkit and ORM.
  • Jinja2: For fast and expressive server-side templating.
  • Chart.js: For the beautiful and interactive data visualizations.
  • Tailwind CSS: For the utility-first CSS framework that makes the UI beautiful.

A special thank you to **Saifeddine ALOUI (ParisNeo)** for creating and maintaining this project.

Visit the project on GitHub to contribute, report issues, or star the repository!

{% endblock %}