Connect a custom endpoint

If your AI runs behind an OpenAI-compatible API that is not one of the built-in providers, you can connect it as a Custom (OpenAI-compatible endpoint). That covers servers such as vLLM, LiteLLM, LM Studio, llama.cpp and Ollama's compatibility layer, hosted services such as Together or Groq, and gateways run by your own company. Anything that answers the standard chat/completions request works, including tool calls, so custom tools keep working.

What you need#

• The base URL of the API, normally ending in /v1. SiteTrellis calls chat/completions and models below it.
• An API key, or the word none if the server needs no key (nothing is sent in that case).
• A public address with https. The endpoint must be reachable from the internet. Private networks, localhost and addresses such as 192.168.x.x are refused, both when you save the key and on every request, as a protection against attacks on internal systems. To use a model on your own machine, put it behind a tunnel or reverse proxy with a public https address and protect it with a key.

Connect it#

1. Open AI providers and choose Custom (OpenAI-compatible endpoint) under Add a key.
2. Give it a label, paste the key (or none) and enter the base URL.
3. Choose Save and test key. The test asks the server for its model list. A "not found" answer usually means the base URL is missing /v1 or has the wrong path.
4. In the agent's Model tab choose the key. The model list is loaded from the server; if it has none, choose Custom model id and type the model name your server expects.

How it behaves#

• It can be any step of a fallback chain, for example your own model first and a hosted model as backup. Errors, timeouts, rate limits and server errors fall through to the next step exactly like the built-in providers.
• The output limit is sent as max_tokens, the most widely supported name. If a server rejects tool definitions or JSON mode, SiteTrellis retries once without them, so smaller models still answer in plain text.
• There are no price estimates for custom models, so costs show as zero in Analytics.
• Extra custom request headers are not supported; use the API key for authentication.
• The key is stored encrypted and never shown again, like every provider key. Changing the base URL of an existing key is limited to the account owner, or requires re-entering the key, because the key is sent to that address.

For the built-in providers see the AI keys overview.