LLM Gateway API
Project-scoped LLM gateway data plane and management plane: OpenAI-compatible chat/completions and embeddings, Anthropic Messages-compatible messages, and rerank; upstream provider (openai / anthropic / dashscope / theta) connections, model catalogs, health checks, gateway sub-keys, and platform trial provider bindings.
Service and resource boundaries
The LLM Gateway API (/v1/llm-gateway/*) is the project-wide unified data plane and management plane interface for managed hosting of LLM provider credentials (e.g., displaying the project's configured Alibaba Cloud Bailian dashscope-prod) and for standardized model inference:
Multi-protocol inference data plane: natively provides OpenAI-compatible
/chat/completions(e.g., invokingqwen-plus,qwen-turbo,qwen-vl-max) and/embeddings(e.g., invoking 1024-dimensiontext-embedding-v3), Anthropic-compatible/messages, and RAG-retrieval-specific/rerank(e.g., invokingqwen3.7-text-rerank).Provider hosting and preset catalog: built-in preset catalog from 24 major model vendors (
/provider-presets), automatic discovery and classification of DashScope available models (/provider-model-discoveries), model-level health probes, gateway sub-key (Keys) quota control, and platform trial model bindings.
Resources and Base Paths
Base URL:
https://<your-endpoint>/v1, path prefix/llm-gateway.Resources:
provider({providerId}, uniqueconnectionNamewithin the project),key({keyId}, gateway sub-key),trialProviderBinding({bindingId}).Coexists with ai-gateway: this service targets LLM inference/Embedding/rerank; AI Gateway provides MCP, A2A, and HTTP Agent tool entry points.
Authentication and Permissions
See Common conventions for authentication and request header conventions. Four scope types:
llm-gateway.execute: data plane (chat / messages / embeddings / rerank / models);llm-gateway.providers.read/llm-gateway.providers.write: provider, health, model discovery, and trial binding management;llm-gateway.keys.read/llm-gateway.keys.write: gateway sub-key management.
Data plane rate limiting is independent of the ordinary management plane: chat / messages / rerank 60 requests per minute, embeddings 120 requests per minute. When a provider enables allowByok, callers may bring their own upstream API key via the X-Provider-Key header (BYOK).
Endpoint overview
The LLM Gateway provides 26 endpoints in total, divided into: model inference and vector/rerank data plane (6), provider credential CRUD and health checks/model discovery/preset catalog (10), gateway sub-key management (4), and platform trial provider binding management (6).
Endpoint List
| Method + path | Purpose |
|---|---|
POST /llm-gateway/chat/completions | Chat Completions (OpenAI-compatible) |
POST /llm-gateway/messages | Messages (Anthropic-compatible) |
POST /llm-gateway/embeddings | Text embeddings |
POST /llm-gateway/rerank | Document reranking |
GET /llm-gateway/models / GET .../models/{modelId} | Available model list/details |
GET|POST /llm-gateway/providers、GET|PATCH|DELETE .../providers/{providerId} | provider CRUD |
GET .../providers/{providerId}/health | Provider health |
POST .../providers/{providerId}/health-checks | Validate all configured provider models |
GET .../providers/{providerId}/models | Upstream actually available models |
POST .../providers/cache/refresh | Refresh provider cache |
POST /llm-gateway/provider-model-discoveries | Discover and classify models available to DashScope credentials |
GET /llm-gateway/provider-presets | Vendor preset model catalog (2.9.0: 24 vendors / 313 models, including Base URL and per-model parameters) |
GET|POST /llm-gateway/keys, PATCH|DELETE .../keys/{keyId} | Gateway sub-key management |
GET|POST /llm-gateway/trial-provider-bindings, GET|DELETE .../trial-provider-bindings/{bindingId}, POST .../{bindingId}/health-checks | Platform trial provider bindings and binding health checks |
GET /llm-gateway/trial-providers | List of bindable platform trial providers |
Endpoint details
Internal service linkage identifier: the project's knowledge base (Embedding / Rerank / Wiki / GraphRAG), memory system (extraction / profiling / distillation), and agent workbench all reference model connections hosted by this module via
project:<providerId>(e.g.,project:dashscope-prod) ortrial:<bindingId>.Streaming and non-streaming dual mode:
/chat/completionsand/messagessupportstream: truereturning standardtext/event-streamSSE incremental chunks, and automatically aggregate input/output tokens at the project, model, and sub-key three-level metering ledger when the stream ends.
Requests and Responses
Complete specifications are given per endpoint below: input (field-level tables for path / query / body), output (return model and key fields), call examples (copyable curl) and response examples. Endpoints marked with "generalized schema any" return the JSON document of the corresponding resource; refer to Server SDK type definitions and the actual resource for specific fields.
Data plane
POST /v1/llm-gateway/chat/completions
Create a chat completion in OpenAI Chat Completions-compatible format: the gateway routes to the active Provider based on the request body's model, passes native message fields through, and records usage at the project / model / gateway sub-key three levels.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
model | body | string | Yes | — | Model ID used for the completion; must exactly match a model in an active Provider's model catalog (the provider/model prefix is not supported). |
messages | body | object[] | Yes | — | Upstream native conversation message array; fields are passed through as-is: text, multimodal (content as an array of segments), and tool-calling (role: tool, tool_calls) are all supported. |
temperature | body | number | No | — | Sampling temperature, 0–2; defaults to the upstream default value if omitted. |
maxTokens | body | integer | No | — | Maximum number of tokens to generate. |
stream | body | boolean | No | — | Whether to return as an SSE stream; when true, the response consists of text/event-stream chunks (chat.completion.chunk). |
tools | body | object[] | No | — | Array of OpenAI-compatible tool definitions (type: function + function.{name,parameters}), passed through to the upstream. |
tool_choice | body | object | No | — | Tool choice policy: "auto" / "none" or {"type":"function","function":{"name":"..."}} to force a specific tool. |
Output: Returns the upstream native completion (passed through by the gateway): for non-streaming, a complete chat.completion JSON (including choices[].message and usage usage fields); when stream=true, a sequence of SSE chunks. Usage is aggregated per UTC day at the project / model / sub-key levels.
When the Provider is not ready or the model has no available route, returns
404+llm_gateway_model_not_found/llm_gateway_provider_not_found; when all upstreams fail, the gateway returns the upstream error with provider attribution.Data-plane rate limiting is independent of the management plane: this endpoint allows 60 requests/minute; exceeding the limit returns
429+llm_gateway_rate_limit_exceeded.When authenticating with a gateway sub-key and
allowedModelsdoes not contain the requested model, returns403+GENERAL_ACCESS_FORBIDDEN; with sub-key authentication,X-Appwrite-Projectis not required (the project is embedded in the key).When the Provider enables
allowByok,-H "X-Provider-Key: <上游密钥>"may be added to supply your own upstream API Key (BYOK).
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/chat/completions" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{ "role": "system", "content": "你是一个简洁的中文助手。" },
{ "role": "user", "content": "用一句话介绍 TablesDB。" }
],
"temperature": 0.7,
"maxTokens": 1024,
"stream": false
}'
# Streaming: set stream to true; the response is SSE (data: {"choices":[{"delta":{...}}]} chunks, ending with data: [DONE])Response example:
{
"id": "chatcmpl-8f2a9c1d",
"object": "chat.completion",
"created": 1757491200,
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "TablesDB 是 Agentbase 提供的结构化表格存储服务,支持行级权限与事务。"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 28,
"completion_tokens": 33,
"total_tokens": 61
}
}POST /v1/llm-gateway/messages
Creates messages in the Anthropic Messages compatible format: routing prefers anthropic-type Providers, falls back to general model routing when necessary, and returns an Anthropic Messages structure.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
model | body | string | Yes | — | Target model ID; anthropic models are routed preferentially to anthropic Providers. |
messages | body | object[] | Yes | — | Anthropic native message array (role: user|assistant, where content is a string or an array of segments), passed through as-is. |
maxTokens | body | integer | Yes | — | Maximum number of tokens to generate, required; when omitted, the gateway treats it as 1024. |
system | body | string | No | — | System prompt; the gateway prepends it as a system message before passing it through. |
temperature | body | number | No | — | Sampling temperature, 0–1. |
stream | body | boolean | No | — | Whether to return chunks as an SSE stream. |
Output: Returns an Anthropic Messages compatible structure: content[] (segments such as type: text), stop_reason, and usage (input_tokens / output_tokens); for streaming, SSE chunks.
Shares the same routing, BYOK (
X-Provider-Key), and sub-keyallowedModelsvalidation as chat/completions; rate limit of 60 requests/minute.Returns
403+GENERAL_ACCESS_FORBIDDENwhen the gateway sub-key does not include the requested model.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/messages" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-3-5-sonnet-20241022",
"maxTokens": 1024,
"system": "你是客服质检助手,只输出 JSON。",
"messages": [
{ "role": "user", "content": "检查这条回复是否包含承诺赔付的表述。" }
],
"temperature": 0.5,
"stream": false
}'
# Gateway Key auth: Authorization: Bearer abk_... (X-Appwrite-Project may be omitted)Response example:
{
"id": "msg_01XFDUDYJgAACzvnptvVoaEL",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20241022",
"content": [
{
"type": "text",
"text": "{\"contains_compensation\": false, \"confidence\": 0.92}"
}
],
"stop_reason": "end_turn",
"usage": {
"input_tokens": 42,
"output_tokens": 28
}
}POST /v1/llm-gateway/embeddings
Generates vectors for a batch of texts: routes to an active Provider by embedding model ID and returns an OpenAI-compatible vector array with token usage. Streaming is not supported.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
model | body | string | Yes | — | embedding model ID, must be in the Provider's embeddingModels. |
input | body | string[] | Yes | — | Array of input texts; each element generates one vector; result data[].index corresponds to the array order. |
Output: Returns llmGatewayEmbeddingResponse:
| Field | Type | Description |
|---|---|---|
object | string | Object type, always list (OpenAI-compatible field). |
model | string | The model that actually generated the vectors. |
data | object[] | Array of vector objects, each containing object, index, and a embedding float array, in one-to-one correspondence with input. |
usage | object | Token usage: prompt_tokens / total_tokens. |
Rate limit of 120 requests/minute (higher than chat/messages/rerank's 60 requests/minute).
When the upstream model supports it,
dimensionsmay be included to control dimensions, but it must match the downstream VectorsDB collection dimension; otherwise vectors will fail to match on write.For models referenced by Memory / knowledge base embedding configuration, check
memoryEmbeddingReferenced/immutableFieldsbefore modifying the Provider.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/embeddings" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"model": "text-embedding-3-small",
"input": [
"Agentbase 是一个 BaaS 平台",
"TablesDB 提供结构化数据存储"
]
}'Response example:
{
"object": "list",
"model": "text-embedding-3-small",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [
0.0023,
-0.0091,
0.0154
]
},
{
"object": "embedding",
"index": 1,
"embedding": [
0.0031,
-0.0087,
0.0149
]
}
],
"usage": {
"prompt_tokens": 24,
"total_tokens": 24
}
}POST /v1/llm-gateway/rerank
Reranks candidate documents by query relevance: returns input indices and scores in descending relevance order, optionally with the document text. Streaming is not supported.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
model | body | string | Yes | — | rerank model ID, must be in the Provider's rerankModels. |
query | body | string | Yes | — | Query text used for relevance judgment. |
documents | body | string[] | Yes | — | Array of candidate document texts; the index in the results points to indices of this array. |
topN | body | integer | No | — | Return only the top N results. |
returnDocuments | body | boolean | No | — | Whether to include the document text (document field) in the results. |
Output: Returns llmGatewayRerankResponse:
| Field | Type | Description |
|---|---|---|
model | string | The model that performed the rerank. |
results | object[] | Array of results in descending relevance order, each containing index (index into the input documents) and relevance_score (0–1); when returnDocuments=true, the document text is included. |
usage | object | Token usage: total_tokens. |
Knowledge base retrieval (e.g., rerankProviderRef) reuses this capability; platform trial Providers participate in routing as
trial:<bindingId>.Rate limit of 60 requests/minute; BYOK and sub-key
allowedModelsvalidation are the same as chat.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/rerank" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"model": "gte-rerank",
"query": "如何创建 Agentbase 用户",
"documents": [
"使用 Account API 创建邮箱密码用户。",
"TablesDB 提供结构化数据存储。"
],
"topN": 2,
"returnDocuments": false
}'Response example:
{
"model": "gte-rerank",
"results": [
{
"index": 0,
"relevance_score": 0.97
},
{
"index": 1,
"relevance_score": 0.31
}
],
"usage": {
"total_tokens": 45
}
}GET /v1/llm-gateway/models
Lists all currently routable models in the project: aggregates and deduplicates chatModels / embeddingModels / rerankModels / vlmModels across active Providers.
Input: no path/query/body parameters.
Output: Returns llmGatewayModelList (total + models[]), where array elements are llmGatewayModel:
| Field | Type | Description |
|---|---|---|
$id | string | Model ID (the value that must be matched exactly by the request body's model). |
object | string | Object type, always model (OpenAI-compatible field). |
created | integer | Upstream model creation time (Unix seconds). |
ownedBy | string | Owning Provider (type or connection identifier). |
capabilities | string[] | Supported capabilities: chat / embedding / rerank / vlm. |
active | boolean | Whether currently routable (the Provider must be active). |
source | string, optional | Model source: project (project Provider) / platformTrial (platform trial) / mixed. |
sources | object[], optional | List of sanitized source projections. |
trialQuota | object, optional | When a platform trial source exists, the trial quota at the account-model dimension. |
Data-plane endpoint, scope
llm-gateway.execute; for Provider management–style model discovery, useGET .../providers/{providerId}/models(providers.read).When
source=platformTrial, pay attention totrialQuota: trial quota is tied to the account-level quota.
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/models" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"total": 2,
"models": [
{
"$id": "gpt-4o-mini",
"object": "model",
"created": 1721174400,
"ownedBy": "openai",
"capabilities": [
"chat"
],
"active": true,
"source": "project",
"sources": [
"project"
]
},
{
"$id": "gte-rerank",
"object": "model",
"created": 1719043200,
"ownedBy": "dashscope",
"capabilities": [
"rerank"
],
"active": true,
"source": "platformTrial",
"sources": [
"platformTrial"
],
"trialQuota": {
"remainingTokens": 480000
}
}
]
}GET /v1/llm-gateway/models/{modelId}
Reads details of a single available model; the model must be explicitly declared by an active Provider or matched via wildcard routing.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
modelId | path | string | Yes | — | Model ID, identical to the $id returned by the list endpoint. |
Output: Returns llmGatewayModel:
| Field | Type | Description |
|---|---|---|
$id | string | Model ID (the value that must be matched exactly by the request body's model). |
object | string | Object type, always model (OpenAI-compatible field). |
created | integer | Upstream model creation time (Unix seconds). |
ownedBy | string | Owning Provider (type or connection identifier). |
capabilities | string[] | Supported capabilities: chat / embedding / rerank / vlm. |
active | boolean | Whether currently routable (the Provider must be active). |
source | string, optional | Model source: project (project Provider) / platformTrial (platform trial) / mixed. |
sources | object[], optional | List of sanitized source projections. |
trialQuota | object, optional | When a platform trial source exists, the trial quota at the account-model dimension. |
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/models/gpt-4o-mini" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"$id": "gpt-4o-mini",
"object": "model",
"created": 1721174400,
"ownedBy": "openai",
"capabilities": [
"chat"
],
"active": true,
"source": "project",
"sources": [
"project"
]
}Provider management
GET /v1/llm-gateway/providers
Lists all Provider connections in the project: sorted by routing priority, used to inspect configured upstreams, model inventories, status, and health.
Input: no path/query/body parameters.
Output: Returns llmGatewayProviderList (total + providers[]), with elements of llmGatewayProvider:
| Field | Type | Description |
|---|---|---|
$id | string | Provider ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
connectionName | string | Connection name, unique within the project. |
providerType | string | Upstream type: openai / anthropic / dashscope / theta. |
baseUrl | string | Upstream API base URL. |
apiKeySet | boolean | Whether an upstream key has been configured; responses never echo the apiKey itself. |
defaultModel | string | Default model for this connection. |
allowedCapabilities | string[] | Allowed capabilities: chat / embedding / rerank / vlm (derived from each model inventory). |
supportedModels | string[] | Compatibility field: aggregated list of all supported model IDs (server-derived). |
chatModels | string[] | Model IDs available for chat completions. |
embeddingModels | string[] | Model IDs available for embeddings. |
rerankModels | string[] | Model IDs available for rerank. |
vlmModels | string[] | Model IDs available for vision understanding (reuses the chat completions endpoint). |
description | string | Connection description. |
status | string | Runtime status: active / paused. |
isDefault | boolean | Whether this is the default Provider for its type. |
priority | integer | Failover priority; lower values take precedence. |
timeoutMs | integer | Upstream request timeout (milliseconds). |
maxRetries | integer | Maximum number of retries. |
maxInputTokens | integer | Maximum input tokens; null means unlimited. |
allowByok | boolean | Whether callers may supply their own upstream key via the X-Provider-Key header (BYOK). |
healthStatus | string | Health status: HEALTHY / UNHEALTHY / UNKNOWN. |
healthMessage | string | Summary of the most recent model validation failure. |
lastValidatedAt | string | Time of the most recent health check. |
lastUsedAt | string | Time of the most recent routing use. |
memoryEmbeddingReferenced | boolean | Whether it is already referenced by the project's Memory embedding configuration; null means the constraint was not evaluated in this response. |
immutableFields | string[] | List of fields that cannot be modified while referenced by Memory. |
deleteAllowed | boolean | Whether deletion is currently allowed; null means not evaluated. |
credentialPolicy | string | Credential rotation policy: mutable / rotate-only; empty means not evaluated. |
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/providers" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"total": 1,
"providers": [
{
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "HEALTHY",
"healthMessage": "",
"lastValidatedAt": "2026-08-01T03:13:02.884+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
}
]
}POST /v1/llm-gateway/providers
Create a Provider connection: it is recommended to declare the available models for each capability using the four capability model lists; the server derives allowedCapabilities and the compatibility field supportedModels, and verifies all models one by one before saving.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | body | string | Yes | — | Provider ID, custom or ID.unique(). |
connectionName | body | string | Yes | — | Connection name, unique within the project; duplicate creation returns 409. |
providerType | body | string | Yes | Enum openai, anthropic, dashscope, theta | Upstream type enum: openai / anthropic / dashscope / theta. |
baseUrl | body | string | Yes | — | Upstream API base URL, must be a valid HTTP(S) address. |
apiKey | body | string | No | — | Upstream key or token; stored encrypted, any read endpoint only echoes apiKeySet and never returns the original. |
defaultModel | body | string | No | — | Default model for this connection. |
allowedCapabilities | body | string[] | No | — | Allowed capability array; usually omitted, derived from the four model lists. |
supportedModels | body | string[] | No | — | Field for backward compatibility with old clients; when passed together with the four lists, the list-derived result takes precedence. The * wildcard is not accepted. |
chatModels | body | string[] | No | — | chat capability model list; VLM reuses the chat completions endpoint, vision models are registered in vlmModels. |
embeddingModels | body | string[] | No | — | embedding capability model list. |
rerankModels | body | string[] | No | — | rerank capability model list. |
vlmModels | body | string[] | No | — | Vision-language model list. |
chatCompletionsPath | body | string | No | — | Upstream chat completions path override, default /v1/chat/completions. |
embeddingsPath | body | string | No | — | Upstream embeddings path override, default /v1/embeddings. |
rerankPath | body | string | No | — | Upstream rerank path override; empty means the Provider does not support rerank. |
description | body | string | No | — | Provider connection description. |
isDefault | body | boolean | No | — | Set as the default Provider of its type. |
isActive | body | boolean | No | — | Whether the connection participates in routing immediately after creation. |
timeoutMs | body | integer | No | — | Upstream request timeout (milliseconds). |
maxRetries | body | integer | No | — | Maximum number of retries. |
priority | body | integer | No | — | Failover priority, lower is tried first; tried in ascending order when the primary Provider fails. |
maxInputTokens | body | integer | No | — | Maximum input tokens; null means unlimited. |
allowByok | body | boolean | No | — | Allow callers to supply their own upstream key via the X-Provider-Key header. |
output : returns 201 + llmGatewayProvider:
| Field | Type | Description |
|---|---|---|
$id | string | Provider ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
connectionName | string | Connection name, unique within the project. |
providerType | string | Upstream type: openai / anthropic / dashscope / theta. |
baseUrl | string | Upstream API base URL. |
apiKeySet | boolean | Whether an upstream key has been configured; responses never echo the apiKey itself. |
defaultModel | string | Default model for this connection. |
allowedCapabilities | string[] | Allowed capabilities: chat / embedding / rerank / vlm (derived from each model inventory). |
supportedModels | string[] | Compatibility field: aggregated list of all supported model IDs (server-derived). |
chatModels | string[] | Model IDs available for chat completions. |
embeddingModels | string[] | Model IDs available for embeddings. |
rerankModels | string[] | Model IDs available for rerank. |
vlmModels | string[] | Model IDs available for vision understanding (reuses the chat completions endpoint). |
description | string | Connection description. |
status | string | Runtime status: active / paused. |
isDefault | boolean | Whether this is the default Provider for its type. |
priority | integer | Failover priority; lower values take precedence. |
timeoutMs | integer | Upstream request timeout (milliseconds). |
maxRetries | integer | Maximum number of retries. |
maxInputTokens | integer | Maximum input tokens; null means unlimited. |
allowByok | boolean | Whether callers may supply their own upstream key via the X-Provider-Key header (BYOK). |
healthStatus | string | Health status: HEALTHY / UNHEALTHY / UNKNOWN. |
healthMessage | string | Summary of the most recent model validation failure. |
lastValidatedAt | string | Time of the most recent health check. |
lastUsedAt | string | Time of the most recent routing use. |
memoryEmbeddingReferenced | boolean | Whether it is already referenced by the project's Memory embedding configuration; null means the constraint was not evaluated in this response. |
immutableFields | string[] | List of fields that cannot be modified while referenced by Memory. |
deleteAllowed | boolean | Whether deletion is currently allowed; null means not evaluated. |
credentialPolicy | string | Credential rotation policy: mutable / rotate-only; empty means not evaluated. |
The model lists must be enumerable, concrete model IDs: each is verified by a live test before saving; wildcards or nonexistent models cause creation to fail.
After creation, call
POST .../providers/cache/refreshto make the current Worker's routing cache take effect immediately.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/providers" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"providerId": "openai-prod",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKey": "sk-xxxxxxxxxxxxxxxx",
"defaultModel": "gpt-4o-mini",
"chatModels": ["gpt-4o-mini"],
"embeddingModels": ["text-embedding-3-small"],
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"isActive": true,
"description": "生产 OpenAI 连接"
}'Response example:
{
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "HEALTHY",
"healthMessage": "",
"lastValidatedAt": "2026-08-01T03:13:02.884+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
}GET /v1/llm-gateway/providers/{providerId}
Read a single Provider by ID: returns the routing configuration and health metadata, never exposing the upstream key in plaintext.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
Output: Returns llmGatewayProvider:
| Field | Type | Description |
|---|---|---|
$id | string | Provider ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
connectionName | string | Connection name, unique within the project. |
providerType | string | Upstream type: openai / anthropic / dashscope / theta. |
baseUrl | string | Upstream API base URL. |
apiKeySet | boolean | Whether an upstream key has been configured; responses never echo the apiKey itself. |
defaultModel | string | Default model for this connection. |
allowedCapabilities | string[] | Allowed capabilities: chat / embedding / rerank / vlm (derived from each model inventory). |
supportedModels | string[] | Compatibility field: aggregated list of all supported model IDs (server-derived). |
chatModels | string[] | Model IDs available for chat completions. |
embeddingModels | string[] | Model IDs available for embeddings. |
rerankModels | string[] | Model IDs available for rerank. |
vlmModels | string[] | Model IDs available for vision understanding (reuses the chat completions endpoint). |
description | string | Connection description. |
status | string | Runtime status: active / paused. |
isDefault | boolean | Whether this is the default Provider for its type. |
priority | integer | Failover priority; lower values take precedence. |
timeoutMs | integer | Upstream request timeout (milliseconds). |
maxRetries | integer | Maximum number of retries. |
maxInputTokens | integer | Maximum input tokens; null means unlimited. |
allowByok | boolean | Whether callers may supply their own upstream key via the X-Provider-Key header (BYOK). |
healthStatus | string | Health status: HEALTHY / UNHEALTHY / UNKNOWN. |
healthMessage | string | Summary of the most recent model validation failure. |
lastValidatedAt | string | Time of the most recent health check. |
lastUsedAt | string | Time of the most recent routing use. |
memoryEmbeddingReferenced | boolean | Whether it is already referenced by the project's Memory embedding configuration; null means the constraint was not evaluated in this response. |
immutableFields | string[] | List of fields that cannot be modified while referenced by Memory. |
deleteAllowed | boolean | Whether deletion is currently allowed; null means not evaluated. |
credentialPolicy | string | Credential rotation policy: mutable / rotate-only; empty means not evaluated. |
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "HEALTHY",
"healthMessage": "",
"lastValidatedAt": "2026-08-01T03:13:02.884+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
}PATCH /v1/llm-gateway/providers/{providerId}
Update a Provider: only explicitly passed fields are modified; capability model lists can be updated independently, and the server re-derives allowedCapabilities and supportedModels.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
connectionName | body | string | No | — | Connection name (uniqueness check same as creation). |
providerType | body | string | No | Enum openai, anthropic, dashscope, theta | Upstream type; when only the type is changed, explicit custom paths are kept, and if rerankPath equals the old type's built-in default it migrates to the new type's default. |
baseUrl | body | string | No | — | Upstream base URL. |
apiKey | body | string | No | — | Pass an empty string to clear the stored key; omitting the field keeps the existing key unchanged. |
defaultModel | body | string | No | — | Default model. |
allowedCapabilities | body | string[] | No | — | Compatibility field, generally not passed directly. |
supportedModels | body | string[] | No | — | When only the legacy field is passed, legacy compatibility semantics apply. |
chatModels | body | string[] | No | — | When updating the chat list alone, the other three capability lists remain unchanged. |
embeddingModels | body | string[] | No | — | Update the embedding list alone. |
rerankModels | body | string[] | No | — | Update the rerank list alone. |
vlmModels | body | string[] | No | — | Update the VLM list alone. |
chatCompletionsPath | body | string | No | — | chat completions path override. |
embeddingsPath | body | string | No | — | embeddings path override. |
rerankPath | body | string | No | — | rerank path override. |
description | body | string | No | — | Connection description. |
isDefault | body | boolean | No | — | Set as the default of its type. |
isActive | body | boolean | No | — | false removes the Provider from routing. |
timeoutMs | body | integer | No | — | Upstream timeout (milliseconds). |
maxRetries | body | integer | No | — | Maximum number of retries. |
priority | body | integer | No | — | Failover priority. |
maxInputTokens | body | integer | No | — | Maximum input tokens. |
allowByok | body | boolean | No | — | Whether BYOK is allowed. |
Output: Returns the updated llmGatewayProvider:
| Field | Type | Description |
|---|---|---|
$id | string | Provider ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
connectionName | string | Connection name, unique within the project. |
providerType | string | Upstream type: openai / anthropic / dashscope / theta. |
baseUrl | string | Upstream API base URL. |
apiKeySet | boolean | Whether an upstream key has been configured; responses never echo the apiKey itself. |
defaultModel | string | Default model for this connection. |
allowedCapabilities | string[] | Allowed capabilities: chat / embedding / rerank / vlm (derived from each model inventory). |
supportedModels | string[] | Compatibility field: aggregated list of all supported model IDs (server-derived). |
chatModels | string[] | Model IDs available for chat completions. |
embeddingModels | string[] | Model IDs available for embeddings. |
rerankModels | string[] | Model IDs available for rerank. |
vlmModels | string[] | Model IDs available for vision understanding (reuses the chat completions endpoint). |
description | string | Connection description. |
status | string | Runtime status: active / paused. |
isDefault | boolean | Whether this is the default Provider for its type. |
priority | integer | Failover priority; lower values take precedence. |
timeoutMs | integer | Upstream request timeout (milliseconds). |
maxRetries | integer | Maximum number of retries. |
maxInputTokens | integer | Maximum input tokens; null means unlimited. |
allowByok | boolean | Whether callers may supply their own upstream key via the X-Provider-Key header (BYOK). |
healthStatus | string | Health status: HEALTHY / UNHEALTHY / UNKNOWN. |
healthMessage | string | Summary of the most recent model validation failure. |
lastValidatedAt | string | Time of the most recent health check. |
lastUsedAt | string | Time of the most recent routing use. |
memoryEmbeddingReferenced | boolean | Whether it is already referenced by the project's Memory embedding configuration; null means the constraint was not evaluated in this response. |
immutableFields | string[] | List of fields that cannot be modified while referenced by Memory. |
deleteAllowed | boolean | Whether deletion is currently allowed; null means not evaluated. |
credentialPolicy | string | Credential rotation policy: mutable / rotate-only; empty means not evaluated. |
When referenced by a Memory embedding configuration, subject to
immutableFields; whencredentialPolicy=rotate-only, only credential rotation is allowed, anddeleteAllowedis checked before deletion.After batch changes, call
POST .../providers/cache/refreshto refresh the routing cache immediately.
Call example:
curl -X PATCH "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"priority": 10,
"timeoutMs": 30000,
"maxRetries": 2
}'Response example:
{
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "HEALTHY",
"healthMessage": "",
"lastValidatedAt": "2026-08-01T03:13:02.884+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
}DELETE /v1/llm-gateway/providers/{providerId}
Delete a Provider connection: removes it from routing, after which it no longer serves model requests for the project.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
Output: 204 No Content, no response body.
Provider referenced in a Memory embedding configuration: when
deleteAllowed=false, adjust the Memory configuration before deleting.
Call example:
curl -X DELETE "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" response example : 204 No Content: no response body (deletion succeeded).
GET /v1/llm-gateway/providers/{providerId}/health
Run live validation on all (能力, 模型) combinations of the Provider and update health status; compatibility interface, still returns the latest Provider object.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
output : returns llmGatewayProvider carrying the latest health status (see healthStatus / healthMessage / lastValidatedAt):
| Field | Type | Description |
|---|---|---|
$id | string | Provider ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
connectionName | string | Connection name, unique within the project. |
providerType | string | Upstream type: openai / anthropic / dashscope / theta. |
baseUrl | string | Upstream API base URL. |
apiKeySet | boolean | Whether an upstream key has been configured; responses never echo the apiKey itself. |
defaultModel | string | Default model for this connection. |
allowedCapabilities | string[] | Allowed capabilities: chat / embedding / rerank / vlm (derived from each model inventory). |
supportedModels | string[] | Compatibility field: aggregated list of all supported model IDs (server-derived). |
chatModels | string[] | Model IDs available for chat completions. |
embeddingModels | string[] | Model IDs available for embeddings. |
rerankModels | string[] | Model IDs available for rerank. |
vlmModels | string[] | Model IDs available for vision understanding (reuses the chat completions endpoint). |
description | string | Connection description. |
status | string | Runtime status: active / paused. |
isDefault | boolean | Whether this is the default Provider for its type. |
priority | integer | Failover priority; lower values take precedence. |
timeoutMs | integer | Upstream request timeout (milliseconds). |
maxRetries | integer | Maximum number of retries. |
maxInputTokens | integer | Maximum input tokens; null means unlimited. |
allowByok | boolean | Whether callers may supply their own upstream key via the X-Provider-Key header (BYOK). |
healthStatus | string | Health status: HEALTHY / UNHEALTHY / UNKNOWN. |
healthMessage | string | Summary of the most recent model validation failure. |
lastValidatedAt | string | Time of the most recent health check. |
lastUsedAt | string | Time of the most recent routing use. |
memoryEmbeddingReferenced | boolean | Whether it is already referenced by the project's Memory embedding configuration; null means the constraint was not evaluated in this response. |
immutableFields | string[] | List of fields that cannot be modified while referenced by Memory. |
deleteAllowed | boolean | Whether deletion is currently allowed; null means not evaluated. |
credentialPolicy | string | Credential rotation policy: mutable / rotate-only; empty means not evaluated. |
Actually accesses the external upstream: single-model probing waits at most 10 seconds, the whole batch at most 30 seconds;
HEALTHYis set only if all models pass validation.Use
POST .../providers/{providerId}/health-checksinstead when per-model results are needed.
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod/health" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "UNHEALTHY",
"healthMessage": "Model validation failed: 1 invalid, 0 unavailable, 0 skipped.",
"lastValidatedAt": "2026-09-14T02:00:11.400+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
}POST /v1/llm-gateway/providers/{providerId}/health-checks
Run a unified validity check on all configured models of the Provider, returning the Provider, summary statistics, and per-model results; health fields are written back via sparse update.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
Output: Returns llmGatewayProviderHealthCheck:
| Field | Type | Description |
|---|---|---|
provider | object | Provider object carrying the latest persisted health status (fields same as llmGatewayProvider, see the Provider management section, not repeated here). |
summary | object | Validation summary (fields below). |
summary.total | integer | Total number of validation results. |
summary.valid | integer | Number of validations passed. |
summary.invalid | integer | Number of validation failures (invalid configuration). |
summary.unavailable | integer | Number of models whose upstream is temporarily unavailable. |
summary.skipped | integer | Number skipped. |
summary.durationMs | integer | Total batch validation time (milliseconds). |
results | llmGatewayProviderValidationResult[] | Validation result per (capability, model) (fields below). |
results[].capability | string | The capability this validation targets. |
results[].model | string | The model ID being validated. |
results[].status | string | Result: valid / invalid / unavailable / skipped. |
results[].code | string | Stable machine-readable result code (e.g. ok). |
results[].retryable | boolean | Whether retrying later without configuration changes could succeed. |
results[].latencyMs | integer | Duration of a single probe (milliseconds). |
The response contains no upstream body or credentials;
results[].retryable=truemeans retrying later may succeed without changing the configuration.Fields written back:
healthStatus,healthMessage,lastHealthCheck.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod/health-checks" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"provider": {
"$id": "openai-prod",
"$createdAt": "2026-08-01T03:12:45.231+00:00",
"$updatedAt": "2026-08-01T03:12:45.231+00:00",
"connectionName": "openai-production",
"providerType": "openai",
"baseUrl": "https://api.openai.com",
"apiKeySet": true,
"defaultModel": "gpt-4o-mini",
"allowedCapabilities": [
"chat",
"embedding"
],
"supportedModels": [
"gpt-4o-mini",
"text-embedding-3-small"
],
"chatModels": [
"gpt-4o-mini"
],
"embeddingModels": [
"text-embedding-3-small"
],
"rerankModels": [],
"vlmModels": [],
"description": "生产 OpenAI 连接",
"status": "active",
"isDefault": true,
"priority": 0,
"timeoutMs": 60000,
"maxRetries": 3,
"maxInputTokens": 128000,
"allowByok": false,
"healthStatus": "HEALTHY",
"healthMessage": "",
"lastValidatedAt": "2026-08-01T03:13:02.884+00:00",
"lastUsedAt": "2026-09-01T08:00:11.102+00:00",
"memoryEmbeddingReferenced": false,
"immutableFields": [],
"deleteAllowed": true,
"credentialPolicy": "mutable"
},
"summary": {
"total": 2,
"valid": 1,
"invalid": 1,
"unavailable": 0,
"skipped": 0,
"durationMs": 256
},
"results": [
{
"capability": "chat",
"model": "gpt-4o-mini",
"status": "valid",
"code": "ok",
"retryable": false,
"latencyMs": 128
},
{
"capability": "embedding",
"model": "text-embedding-3-small",
"status": "invalid",
"code": "model_not_allowed",
"retryable": false,
"latencyMs": 94
}
]
}GET /v1/llm-gateway/providers/{providerId}/models
Pull the actually available model list directly from the upstream (model discovery): currently for the dashscope Provider; other compatible types require manual maintenance of the model list.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | path | string | Yes | — | Provider ID. |
Output: returns plain-ID-shaped { "models": string[], "total": number } (not an array of llmGatewayModel objects).
The scope is
llm-gateway.providers.read, narrower than the data plane'sllm-gateway.execute.The discovery result can serve as a lightweight probe in addition to
POST /llm-gateway/provider-model-discoveries; neither automatically rewrites the Provider configuration.
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/providers/openai-prod/models" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"models": [
"qwen-plus",
"qwen-max",
"text-embedding-v3"
],
"total": 3
}POST /v1/llm-gateway/providers/cache/refresh
Force-reload the current project's in-memory Provider routing cache: after batch Provider changes, make the new routing snapshot take effect immediately on the current Worker.
Input: no path/query/body parameters.
Output: 204 No Content, no response body.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/providers/cache/refresh" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: 204 No Content: no response body (cache refreshed).
POST /v1/llm-gateway/provider-model-discoveries
Use Bailian DashScope credentials to pull the account's available models, and pre-classify them via the built-in model catalog as Chat / Embedding / Rerank / VLM for use when creating or editing a Provider.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | body | string | No | — | Existing Provider ID; used in the edit context to reuse saved credentials. |
providerType | body | string | No | Enum openai, anthropic, dashscope, theta | Used in the create context (required when providerId is absent); the enum matches the Provider. |
baseUrl | body | string | No | — | When non-empty, overrides the saved Provider's base URL. |
apiKey | body | string | No | — | When non-empty, overrides the saved Provider's key. |
Output: returns the discovery result: a model list pre-classified by chat / embedding / rerank / vlm plus an unclassified preview.
Unclassified models are not automatically assigned to Chat; image, audio/video, speech, and multimodal embedding models not yet supported are shown only in the preview and are not written to the Provider automatically.
providerIdandproviderTypeare mutually exclusive: use the former to edit an existing Provider, the latter in the create context.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/provider-model-discoveries" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"providerType": "dashscope",
"baseUrl": "https://dashscope.aliyuncs.com/compatible-mode",
"apiKey": "sk-xxxxxxxxxxxxxxxx"
}'Response example: the response is a discovery-result JSON (model list pre-classified by capability + unclassified preview; fields are subject to the actual response): for preview only; nothing is written to any Provider automatically.
GET /v1/llm-gateway/provider-presets (added in 2.9.0)
Vendor preset model catalog: returns the built-in vendor preset list (24 vendors, 313 preset models) and per-model parameters (context window, max output, Embedding dimensions and input limit, tool calling, multimodal Embedding) for each model, for selecting vendor presets and importing model lists when creating / editing a Provider.
Input: no path/query/body parameters.
Output: Returns llmGatewayProviderPresetList (total + presets[]), with elements of llmGatewayProviderPreset:
| Field | Type | Description |
|---|---|---|
providerType | string | The Provider type the preset corresponds to (e.g. volcengine, openai-official, openai-compatible, ollama, etc.). |
name / description | string | Vendor display name and description. |
baseUrl | string | The upstream Base URL fixed by the preset. |
models | array | The preset model list (llmGatewayProviderPresetModel: model ID, capability annotations, and per-model parameters metadata). |
For manual configuration, only the protocol type is still specified; after selecting a preset, the Base URL and model list can be carried directly into
POST /llm-gateway/providers.
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/provider-presets" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"total": 24,
"presets": [
{
"providerType": "volcengine",
"name": "火山方舟(豆包)",
"baseUrl": "https://ark.cn-beijing.volces.com/api/v3",
"models": [
{
"model": "doubao-seed-2-1-pro-260915",
"capabilities": ["chat", "vlm"],
"metadata": { "contextWindow": 262144 }
}
]
}
]
}Gateway Sub-keys
GET /v1/llm-gateway/keys
List the gateway sub-keys of the current project: plaintext is never returned; keys are identified by keyPrefix, name, enabled/disabled state, expiry time, and last used time.
Input: no path/query/body parameters.
Output: returns llmGatewayKeyList (total + keys[]), whose elements are llmGatewayKey (where key plaintext is always empty / not echoed back):
| Field | Type | Description |
|---|---|---|
$id | string | Key ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
key | string | The full plaintext key (abk_ prefix), appearing only once in the creation response; afterwards no read endpoint returns it again. |
keyPrefix | string | The key's display prefix (non-sensitive), used to identify the key in lists. |
name | string | Key name. |
description | string | Key description. |
allowedModels | string[] | Allowlist of model IDs permitted for calls; ["*"] allows all models. |
monthlyTokenBudget | integer | Monthly token budget; null means unlimited. Rolled up cumulatively from tiered usage data aggregated by UTC day; once exhausted, requests with this key are rejected. |
monthlyTokensUsed | integer | Tokens consumed in the current billing cycle. |
budgetResetAt | string | Next monthly budget reset time. |
rateLimit | integer | Per-minute request limit; null uses the project-level default rate limit. |
isActive | boolean | Whether enabled; when false, the key becomes immediately unavailable. |
expiresAt | string | Expiry time (ISO 8601); null means never expires. |
lastUsedAt | string | Last used time. |
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/keys" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example:
{
"total": 1,
"keys": [
{
"$id": "key_mobile_chat",
"$createdAt": "2026-08-12T09:20:00.000+00:00",
"$updatedAt": "2026-08-12T09:20:00.000+00:00",
"key": "",
"keyPrefix": "abk_proj6789_aB3xK9",
"name": "mobile-app-chat",
"description": "移动端聊天功能专用",
"allowedModels": [
"gpt-4o-mini"
],
"monthlyTokenBudget": 1000000,
"monthlyTokensUsed": 45230,
"budgetResetAt": "2026-09-01T00:00:00.000+00:00",
"rateLimit": 60,
"isActive": true,
"expiresAt": null,
"lastUsedAt": "2026-09-10T14:02:33.000+00:00"
}
]
}POST /v1/llm-gateway/keys
Issue a gateway sub-key: a restricted credential for downstream applications that can only call data-plane endpoints, constrained by allowedModels / budget / rate limit / validity period.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
keyId | body | string | Yes | — | Sub-key ID, custom or ID.unique(). |
name | body | string | Yes | — | Key name. |
description | body | string | No | — | Key description. |
allowedModels | body | string[] | No | — | Model allowlist; ["*"] allows all. If the requested model is not in the allowlist, the data plane returns 403. |
monthlyTokenBudget | body | integer | No | — | Monthly token budget; null unlimited. Once exhausted, requests with this key are rejected until the next reset. |
rateLimit | body | integer | No | — | Per-minute request limit; null uses the global default. |
expiresAt | body | string | No | — | Expiry time (ISO 8601); null never expires. |
Output: returns 201 + llmGatewayKey, where only this response contains the plaintext key (abk_ prefix); afterwards only keyPrefix is visible:
| Field | Type | Description |
|---|---|---|
$id | string | Key ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
key | string | The full plaintext key (abk_ prefix), appearing only once in the creation response; afterwards no read endpoint returns it again. |
keyPrefix | string | The key's display prefix (non-sensitive), used to identify the key in lists. |
name | string | Key name. |
description | string | Key description. |
allowedModels | string[] | Allowlist of model IDs permitted for calls; ["*"] allows all models. |
monthlyTokenBudget | integer | Monthly token budget; null means unlimited. Rolled up cumulatively from tiered usage data aggregated by UTC day; once exhausted, requests with this key are rejected. |
monthlyTokensUsed | integer | Tokens consumed in the current billing cycle. |
budgetResetAt | string | Next monthly budget reset time. |
rateLimit | integer | Per-minute request limit; null uses the project-level default rate limit. |
isActive | boolean | Whether enabled; when false, the key becomes immediately unavailable. |
expiresAt | string | Expiry time (ISO 8601); null means never expires. |
lastUsedAt | string | Last used time. |
Sub-keys cannot manage Providers / keys / usage statistics; they can only access execution endpoints.
When calling the data plane with sub-key authentication,
X-Appwrite-Projectis not needed (the project ID is embedded in the key).
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/keys" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"keyId": "key_mobile_chat",
"name": "mobile-app-chat",
"description": "移动端聊天功能专用",
"allowedModels": ["gpt-4o-mini"],
"monthlyTokenBudget": 1000000,
"rateLimit": 60,
"expiresAt": null
}'Response example:
{
"$id": "key_mobile_chat",
"$createdAt": "2026-08-12T09:20:00.000+00:00",
"$updatedAt": "2026-08-12T09:20:00.000+00:00",
"key": "abk_proj6789_aB3xK9mN2pQ7wR4tY8vF1jH5cE0iU6oL3sA9bD2gX7zW4nM",
"keyPrefix": "abk_proj6789_aB3xK9",
"name": "mobile-app-chat",
"description": "移动端聊天功能专用",
"allowedModels": [
"gpt-4o-mini"
],
"monthlyTokenBudget": 1000000,
"monthlyTokensUsed": 45230,
"budgetResetAt": "2026-09-01T00:00:00.000+00:00",
"rateLimit": 60,
"isActive": true,
"expiresAt": null,
"lastUsedAt": "2026-09-10T14:02:33.000+00:00"
}PATCH /v1/llm-gateway/keys/{keyId}
Update a sub-key: rename, adjust the model allowlist, rate limit, budget, enabled state, or expiry time; only explicitly passed fields are modified.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
keyId | path | string | Yes | — | Sub-key ID. |
name | body | string | No | — | Key name. |
description | body | string | No | — | Key description. |
allowedModels | body | string[] | No | — | Model allowlist. |
monthlyTokenBudget | body | integer | No | — | Monthly token budget. |
rateLimit | body | integer | No | — | Per-minute request limit. |
isActive | body | boolean | No | — | false disables immediately. |
expiresAt | body | string | No | — | Expiry time (ISO 8601). |
Output: Returns the updated llmGatewayKey:
| Field | Type | Description |
|---|---|---|
$id | string | Key ID。 |
$createdAt | string | Creation time (ISO 8601). |
$updatedAt | string | Last update time (ISO 8601). |
key | string | The full plaintext key (abk_ prefix), appearing only once in the creation response; afterwards no read endpoint returns it again. |
keyPrefix | string | The key's display prefix (non-sensitive), used to identify the key in lists. |
name | string | Key name. |
description | string | Key description. |
allowedModels | string[] | Allowlist of model IDs permitted for calls; ["*"] allows all models. |
monthlyTokenBudget | integer | Monthly token budget; null means unlimited. Rolled up cumulatively from tiered usage data aggregated by UTC day; once exhausted, requests with this key are rejected. |
monthlyTokensUsed | integer | Tokens consumed in the current billing cycle. |
budgetResetAt | string | Next monthly budget reset time. |
rateLimit | integer | Per-minute request limit; null uses the project-level default rate limit. |
isActive | boolean | Whether enabled; when false, the key becomes immediately unavailable. |
expiresAt | string | Expiry time (ISO 8601); null means never expires. |
lastUsedAt | string | Last used time. |
Once
rateLimit,monthlyTokenBudget, andexpiresAthave been set, they cannot be cleared back to unset: passingnullor omitting them both keep the current value.
Call example:
curl -X PATCH "https://<your-endpoint>/v1/llm-gateway/keys/key_mobile_chat" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"rateLimit": 30,
"isActive": false
}'Response example:
{
"$id": "key_mobile_chat",
"$createdAt": "2026-08-12T09:20:00.000+00:00",
"$updatedAt": "2026-09-14T02:10:00.000+00:00",
"key": "abk_proj6789_aB3xK9mN2pQ7wR4tY8vF1jH5cE0iU6oL3sA9bD2gX7zW4nM",
"keyPrefix": "abk_proj6789_aB3xK9",
"name": "mobile-app-chat",
"description": "移动端聊天功能专用",
"allowedModels": [
"gpt-4o-mini"
],
"monthlyTokenBudget": 1000000,
"monthlyTokensUsed": 45230,
"budgetResetAt": "2026-09-01T00:00:00.000+00:00",
"rateLimit": 30,
"isActive": false,
"expiresAt": null,
"lastUsedAt": "2026-09-10T14:02:33.000+00:00"
}DELETE /v1/llm-gateway/keys/{keyId}
Permanently delete a sub-key (hard delete, not via soft revocation): the document is removed immediately and the key immediately becomes unable to call execution endpoints.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
keyId | path | string | Yes | — | Sub-key ID. |
Output: 204 No Content, no response body.
Call example:
curl -X DELETE "https://<your-endpoint>/v1/llm-gateway/keys/key_mobile_chat" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: 204 No Content: no response body (key revoked).
Trial Provider Bindings
GET /v1/llm-gateway/trial-providers
List the bindable trial Providers offered by the platform: before binding, confirm the platform provider ID and trial quota here.
Input: no path/query/body parameters.
Output: returns the trial provider list (generalized schema any: elements contain the platform provider ID, capabilities, and trial quota information; fields are subject to the actual response).
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/trial-providers" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: the response is a trial provider array document (generalized schema; fields are subject to the actual response); use the platform provider ID within it to create a binding.
GET /v1/llm-gateway/trial-provider-bindings
List the platform trial Provider bindings currently in effect for this project.
Input: no path/query/body parameters.
Output: returns the binding list (generalized schema any: each entry contains bindingId, platform provider and project associations, quota status, and other information; fields are subject to the actual response).
A bound Provider appears in model routing in the form
trial:<bindingId>(e.g.rerankProviderReffor knowledge base rerank).
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/trial-provider-bindings" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: the response is a binding array document (generalized schema; fields are subject to the actual response).
POST /v1/llm-gateway/trial-provider-bindings
Bind a platform trial Provider to a project under your account: after a successful binding (201), it participates in model routing as trial:<bindingId>.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
providerId | body | string | Yes | — | Platform provider ID (from GET /llm-gateway/trial-providers). |
Output: returns 201 + the binding object (generalized schema any; fields are subject to the actual response).
Trial quota is linked to account-level quotas; an account's quota is shared across projects.
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/trial-provider-bindings" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{ "providerId": "trial-dashscope-gte-rerank" }'Response example: 201: the response is a binding object JSON (generalized schema; fields are subject to the actual response); note the bindingId for use by subsequent endpoints.
GET /v1/llm-gateway/trial-provider-bindings/{bindingId}
Read the details and quota status of a single trial Provider binding.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
bindingId | path | string | Yes | — | Trial binding ID. |
Output: returns the binding object (generalized schema any; fields are subject to the actual response).
Call example:
curl "https://<your-endpoint>/v1/llm-gateway/trial-provider-bindings/<BINDING_ID>" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: the response is a binding object JSON (generalized schema; fields are subject to the actual response).
DELETE /v1/llm-gateway/trial-provider-bindings/{bindingId}
Unbind the trial Provider binding: trial:<bindingId> routing immediately becomes invalid.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
bindingId | path | string | Yes | — | Trial binding ID. |
Output: 204 No Content, no response body.
Call example:
curl -X DELETE "https://<your-endpoint>/v1/llm-gateway/trial-provider-bindings/<BINDING_ID>" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: 204 No Content: no response body (unbound).
POST /v1/llm-gateway/trial-provider-bindings/{bindingId}/health-checks
Live-test the availability of the bound platform trial Provider: verify credentials, network, and model connectivity.
Input:
| Parameter | Location | Type | Required | Default/enum | Description |
|---|---|---|---|---|---|
bindingId | path | string | Yes | — | Trial binding ID. |
Output: returns the live test result (generalized schema any: contains health status and per-model verification information; fields are subject to the actual response).
Call example:
curl -X POST "https://<your-endpoint>/v1/llm-gateway/trial-provider-bindings/<BINDING_ID>/health-checks" \
-H "X-Appwrite-Project: <YOUR_PROJECT_ID>" \
-H "X-Appwrite-Key: <YOUR_API_KEY>"Response example: the response is a health check result JSON (generalized schema; fields are subject to the actual response).
Errors and boundaries
Model does not exist or no available provider:
404+llm_gateway_model_not_found/llm_gateway_provider_not_found; when all upstreams fail, the gateway returns the upstream error with provider attribution.Data-plane rate limits are independent (chat/messages/rerank 60 requests/min, embeddings 120 requests/min): exceeding the limit returns
429+llm_gateway_rate_limit_exceeded; subkeys additionally have their ownrateLimit.Subkey
monthlyTokenBudgetexhausted: requests are rejected and no longer allowed within the key; budgets roll up cumulatively based on three-tier usage data collected by UTC day.allowedModelsdoes not include the requested model: requests are rejected even if the upstream is available;supportedModelsdoes not accept wildcards and is validated item by item before saving.apiKeyis only written at creation/update and is never echoed by any read endpoint; BYOK (X-Provider-Key) only takes effect whenallowByok=true.Failover is attempted in ascending order of
priority; providers withisActive=falsedo not participate in routing, andisDefaultonly takes effect within the same type.
Correspondence with the Server SDK
| service.method | Endpoint |
|---|---|
LLMGateway.createChatCompletion | POST /llm-gateway/chat/completions |
LLMGateway.createMessage | POST /llm-gateway/messages |
LLMGateway.createEmbeddings | POST /llm-gateway/embeddings |
LLMGateway.createRerank | POST /llm-gateway/rerank |
LLMGateway.listModels / getModel | /llm-gateway/models* |
LLMGateway.listProviders / createProvider / getProvider / updateProvider / deleteProvider | /llm-gateway/providers* |
LLMGateway.getProviderHealth / createProviderHealthCheck / listProviderModels / refreshProviderCache | .../health*、.../models、.../cache/refresh |
LLMGateway.createProviderModelDiscovery | POST /llm-gateway/provider-model-discoveries |
LLMGateway.listKeys / createKey / updateKey / deleteKey | /llm-gateway/keys* |
LLMGateway.listTrialProviders / listTrialProviderBindings / createTrialProviderBinding / getTrialProviderBinding / deleteTrialProviderBinding / createTrialProviderBindingHealthCheck | /llm-gateway/trial-provider* |
2.12.0 endpoints and compatibility
Paths below are relative to /v1. Server requests carry X-Appwrite-Project and X-Appwrite-Key; keep the key on the server. Scopes below are necessary; dataset access, ownership and resource state may add checks. HTTP 202 means accepted; use the returned operation identifier to read its terminal result.
| HTTP | Scope | SDK |
|---|---|---|
POST /llm-gateway/rerank | llm-gateway.execute | llmGateway.createRerank |
POST /llm-gateway/rerank
Rerank documents
| Parameter | Location | Required | Type | Constraints / default | Meaning |
|---|---|---|---|---|---|
model | body | yes | string | — | Rerank model ID. |
query | body | yes | string | — | Query text to rank documents against. |
documents | body | yes | array[string] | — | Documents to rerank. |
topN | body | no | integer | format=int32 | Return only the top N results. |
returnDocuments | body | no | boolean | default=false | Include document text in the response. |
instruct | body | no | string | default="" | Optional native sorting instruction for supported DashScope rerank models. Empty omits the instruction. |
Success response: HTTP 200 — llmGatewayRerankResponse.
Response fields: model (string), results (array), usage (object).
Errors and retries
400/422: check types, enumerations and rules. 401/403: verify identity and scopes; dataset analyses additionally require trace and dataset read access. 404: check that the resource exists in the current project. 409: read current state before deciding to retry. 429: honor Retry-After. After 5xx or network loss, inspect operation state first. Preserve an idempotency key across retries; change it only for a deliberate new operation. Confirm deletion and clearing targets before writing.