> For the complete documentation index, see [llms.txt](https://developer.emporix.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/configuration/self-hosted-models.md).

# AI Self-Hosted Models

Connect AI agents to self-hosted LLMs or to external APIs compatible with a supported provider, such as Mistral AI.

Use a self-hosted provider when the LLM you want is not on the agent's **Provider** list. The **Self-hosted Ollama** and **Self-hosted vLLM** providers connect an agent to a model endpoint that you specify. That endpoint can run in your own infrastructure, or it can be an external API that is compatible with one of the supported cloud providers.

For example, Mistral AI is not on the **Provider** list, but its API is compatible with the OpenAI API, so you connect it through **Self-hosted vLLM**. The steps below use it as the example.

## Prerequisites

Before you configure the agent, prepare the following items:

* The URL of the model endpoint and the cloud provider whose API format it follows
* The model identifier that the endpoint accepts
* An API key or other credential, if the endpoint requires authentication

For the Mistral example, you need an API key from an account with API access enabled in your subscription plan, and a model identifier such as `ministral-3b-latest`.

{% hint style="info" %}
For the available identifiers, see the [Mistral models overview](https://docs.mistral.ai/models).
{% endhint %}

## Connect an agent to a self-hosted model

{% stepper %}
{% step %}

#### Store the API key

In the Management Dashboard, go to **Agentic AI** -> **AI Tokens**, create a token, and enter the API key as its **Token Value**. Use the key exactly as your provider issued it, without adding a `Bearer` prefix. The agent references this token, so the key never appears in the agent definition. For details, see [AI Tokens](https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/configuration/tokens.md).
{% endstep %}

{% step %}

#### Open the agent's model settings

In the Management Dashboard, go to **Agentic AI** -> **AI Agents**, create a new agent or open an existing one, and go to the **Model** tab.

{% hint style="warning" %}
When you switch an existing agent to a different provider, the **Additional parameters** field is cleared. Copy its content first if you want to reuse it.
{% endhint %}
{% endstep %}

{% step %}

#### Point the agent to the model endpoint

On the **Model** tab, set the following values:

| Field                         | What to enter                                                                                                                                             | Mistral example             |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
| **Provider**                  | **Self-hosted vLLM** or **Self-hosted Ollama**, matching the server that hosts the model. For an external OpenAI-compatible API, use **Self-hosted vLLM** | **Self-hosted vLLM**        |
| **Custom Model Name**         | The model identifier that the endpoint accepts                                                                                                            | `ministral-3b-latest`       |
| **Base Provider**             | The cloud provider whose API format the endpoint follows, which is **OpenAI**, **Google**, or **Anthropic**                                               | **OpenAI**                  |
| **Self-hosted URL**           | The base URL of the model endpoint                                                                                                                        | `https://api.mistral.ai/v1` |
| **Authorization Header Name** | The header that carries the credential, if the endpoint requires one                                                                                      | `Authorization`             |
| **Authorization Token**       | The token you created in the first step, listed by its **Token Name**                                                                                     | The token with your API key |

<figure><img src="https://1530167654-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8GgoeZEZYjZrpjOU6w52%2Fuploads%2Fgit-blob-a213005b4d6c5c292e4d9362fa735273a4daf94c%2Fagentic_self_hosted_mistral.png?alt=media" alt="Model tab with Base Provider, Self-hosted URL, and authorization settings for Mistral"><figcaption><p>Example self-hosted model settings</p></figcaption></figure>

Leave **Use OAuth authentication** off when the endpoint authenticates with an API key. If your endpoint sits behind an OAuth-protected gateway, see [AI OAuth-Based Gateway](https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/configuration/oauth-gateway.md) instead.

Configure the remaining options, such as **Max Tokens** and **Temperature**, as for any other agent. For a description of each option, see [Configure AI model](https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/agents.md#configure-ai-model).
{% endstep %}

{% step %}

#### Save and test the agent

Select **Save**. Then send the agent a test request, for example through the chat endpoint described in [How to communicate with an agent](https://developer.emporix.io/api-documentation/api-guides/artificial-intelligence/ai-service/ai-tutorial.md#how-to-communicate-with-an-agent). If the request fails with an authentication error, check the API key stored in the token and the **Authorization Header Name**. If the endpoint reports an unknown model, check the **Custom Model Name**.
{% endstep %}
{% endstepper %}

{% hint style="success" %}
Test the agent with varied inputs before you use it in production, especially when you switch from another provider. For guidance, see [LLM Model recommendations and testing](https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/best-practices.md#llm-model-recommendations-and-testing).
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://developer.emporix.io/agentic-commerce-intelligence/agentic-intelligence/configuration/self-hosted-models.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
