{% extends "admin/base.html" %} {% block title %}Server Management{% endblock %} {% block header_title %}Server Management{% endblock %} {% block content %}

Add New Backend Server

Deployment & Backend Guide

🦙 Ollama (Local/Cloud)

The industry standard for local LLMs.

  • Install: Use the Instance Manager or visit ollama.com.
  • Cloud: Use https://ollama.com/api with your Ollama Cloud API key.
  • Config: Hub handles the /api/tags and /api/chat endpoints automatically.

🚀 vLLM & OpenLLM

High-throughput serving for production.

  • vLLM: Optimized for PagedAttention. Install via Instance Manager.
  • OpenLLM: Built on BentoML. To install: pip install openllm or use our auto-installer.
  • Usage: openllm start llama3. Default port: 3000.

🌐 Gateways (Open WebUI)

Connecting to other AI frontends.

  • Purpose: Use the models already configured in your Open WebUI instance.
  • API: Ensure "API Key" is enabled in Open WebUI settings.
  • URL: Usually http://127.0.0.1:3000/api.

Configured Servers

{% for server in servers %} {% else %} {% endfor %}
Name URL Type API Key Status Models Actions
{{ server.name }} {{ server.url }} {{ server.server_type | upper }} {% if server.has_api_key %} Configured {% else %} - {% endif %} {% if not server.is_active %} Disabled {% elif server.last_error %} {% else %} Active {% endif %} {{ server.available_models|length if server.available_models is not none else 'N/A' }} model(s) {% if server.models_last_updated %} (updated {{ server.models_last_updated.strftime('%Y-%m-%d %H:%M') }}) {% endif %}
| Edit Manage Models
No servers configured.
{% endblock %}