Models and AI Gateway Overview
Agentbase has two separate project services: Models (LLM Gateway) connects model providers and invokes models; AI Gateway manages tool and agent access, policies, and credentials.
Two entry points
| Models (LLM Gateway) | AI Gateway | |
|---|---|---|
| Primary use | Connect providers and invoke available chat, embedding, and rerank models | Manage gateways, tool connections, MCP, A2A, and HTTP Agents |
| Project service switch | llmGateway | aiGateway |
| Credentials | Use credentials authorized for the model API | Use an abk_ credential authorized for the gateway |
The switches control separate services. Check the project and request path before changing either one. Available entries, permissions, and models depend on the deployment.
Connect a model
Connect a provider and store its credentials on the project's Models page.
Select a chat, embedding, or rerank model for your use case, and verify that the provider and model are available in your environment.
Follow model configuration and key management to make a call and check the response. Confirm the model types required by your knowledge base and memory configuration too.
Use AI Gateway
AI Gateway provides MCP, A2A, and HTTP Agent tool entry points. Configure the gateway, tools, policies, and credentials available in your deployment, then verify them with actual requests.
v2.12.0 boundary: AI Gateway model inference data-plane endpoints, logical-model and model-deployment management, model discovery, and standard-model management are unavailable. Use Models (LLM Gateway) for model calls; do not copy older AI Gateway model-routing examples. OPA policy dry-run can inspect a policy decision, but does not prove that live requests use the same policy.
Security and troubleshooting
Remove sensitive information before sending model or tool requests. Do not assume input and output are automatically redacted or that prompt injection is universally blocked. Semantic response caching is unavailable in this version.
If a request fails, check the target project's service switch, endpoint, credential, resource authorization, and upstream service separately. Disabling aiGateway affects tool calls governed by that service switch; llmGateway is a separate switch.