AI Gateway (Unified AI Data Plane)
Overview and use cases
AI Gateway is the unified AI data plane and control plane introduced in 2.6.0 (services/ai-gateway), hosting Target onboarding, Tool/MCP, A2A, HTTP Agent, identity (Consumer/Key), policy (Policy), budgets, caching, and auditing. Resource model: Gateway / Target / Tool / Consumer / Policy / Snapshot — Targets and Tools are project-level assets that enter one or more Gateway Snapshots via mount relationships.
Coexists with the existing LLM Gateway: it does not migrate legacy resources, does not take over legacy production paths, and does not provide compatibility aliases without gatewayId. As long as you use existing Provider/Gateway Key/Chat/Embeddings/Rerank/VLM, continue with llm-gateway.md; use this page only when Gateway/Target/Tool/Consumer/Policy/MCP/A2A is involved. The two systems' names, credentials, and paths are not interchangeable.
Prerequisites
Enablement gating: Docker Compose requires explicitly enabling the
ai-gatewayprofile; aggregated Helm defaults toaiGateway.enabled=false; currently only OceanBase Shared V2 projects are supported. The project control tables are initialized only when the project service is explicitly enabled for the first time.Public listeners use an abk_ Key issued by the target Gateway: verifies the consistency of Key + Gateway + Consumer + published Snapshot. LLM Gateway Keys, project API Keys, and upstream Provider Keys cannot be used as abk_ keys.
Internal services use a separate internal listener + signed runtime context (no credentials are sent); the internal listener must not be exposed to the Ingress, and the public listener must reject internal contexts.
Usage workflow
Data plane entry points (HTTP, added in 2.6.0; SDKs do not wrap these paths):
Model inference is unavailable in AI Gateway in v2.12.0; use LLM Gateway for model calls.
MCP:
/ai-gateway/g/{gatewayId}/mcp(Tool listing/search/calling share the Agent allowlist and OPA authorization; side effects are protected by the idempotency gate).A2A:
/ai-gateway/g/{gatewayId}/a2a/{targetSlug}(Task/Context are isolated by caller + Target Revision).HTTP Agent:
/ai-gateway/g/{gatewayId}/http/{targetSlug}[/{path...}](only forwards methods/relative paths/query/body allowed by the Target, does not replay caller authentication headers, and credentials are resolved from the controlled Credential).
Control plane (SDK AiGateway service):
Create a Gateway and manage policies:
createPolicyEngine/listPolicyEngines/getPolicyEngine;updateGatewayPolicy({ gatewayId, ... }); policy bundlescreateRevision/publishPolicyBundle/listPolicyBundles/getPolicyBundle; validate withdryRunPolicybefore going live; release semantics use Snapshots:sealSnapshot/getSnapshotStatus.Onboard Targets:
createResource/listResources/getResource/replaceResource(kindis the resource category path parameter); revision managementcreateRevision/getRevision/transitionRevision(/resources/{kind}/{id}/revisions/{revisionId}/{action}); health checkscheckTargetHealth; mountinglistTargetMounts/replaceTargetMounts.Model inference, model deployments, and standard-model management are unavailable in v2.12.0.
Tool/MCP connections:
createToolConnection/discoverToolConnection/updateToolConnection/deleteToolConnection.Consumer credentials:
createKey/listKeys/getKey/updateKeyStatus(issuance and enable/disable of abk_).Observability and auditing:
getObservabilityOverview/listInvocations/getInvocation.
2.7.0 additions (appbase release-2.7.0 PR 526):
Credential governance:
createKeyadds an optionallimits— an array of hard rate-limit/quota rules, with elements{ scope, resourceId, rate, quota }(resourceId,rate,quotaall optional):scopetakescredential/model/tool/agent(non-credentialrequires specifyingresourceId);rateisrequestsPerSecond+burst(burstdefaults torequestsPerSecond);quotalimitsrequests/tokensperminute/hour/day/monthperiod (tokensonly forcredential/modelscope). The gateway data plane enforces rules fail-closed.Tool approval closed loop: The Tool's static approval floor (
none/always/policy) combined with the policy engine decision composes a fail-closed runtime gate; hittingREQUIRE_APPROVALgenerates an approval request (ID with aapr_prefix, TTL 15 minutes, replayed calls correlated via theX-AppBase-Approval-Request-Idheader); policy-enforced denials cannot be elevated to allow by human approval. The approval surface adds no new public SDK methods/endpoints.Managed tool templates:
ragprovides managed RAG; the standard-model management API required bystandard-modelsis disabled in v2.12.0.
2.8.0 additions:
Tool connection connectivity test:
testToolConnection(id, toolId)(toolIdoptional) (POST /ai-gateway/tool-connections/{id}/test, 2.8.0) — performs connectivity testing and diagnostics on tool connections (target reachability, credentials, and per-tool checks, same source as the Console connection diagnostics panel); optionaltoolIdtests only a single tool.Logical-model mounts: logical-model configuration is unavailable in v2.12.0; historical SDK methods do not establish availability.
Key parameters and returns
For resource-class methods,
kindis a path parameter (resource category) used together with the resource ID to locate revisions and mounts; available categories are subject to the target instance's specs/responses; do not guess category names.Snapshot: the landing point of release gating — changes to Target/Consumer/Tool allowlist/Policy only take effect for public traffic on a Snapshot that is published and consistent.
Policy bundles are validated via dry-run before publish;
transitionRevisionperforms the revision state transition actions.
Errors and boundaries
Outbound access is subject to SSRF, redirect, and size limits. Private addresses are denied by default; deployments may allow specific private endpoints with an exact-host allowlist, while protected addresses such as cloud metadata remain blocked.
Tool invocation details can retain limited, truncated previews of arguments and results. Limited field redaction does not guarantee that business content is removed. Lists, CSV, OTel, and Langfuse do not contain those previews. Sanitize sensitive data before calling.
A2A/HTTP methods, outbound addresses, and redirects all pass through allowlists; unauthorized access is denied.
Project services
aiGatewayandllmGatewayhave separate switches; the tool data plane checksaiGateway.
SDK Category Differences
Server SDK (server-sdk/nodejs.md) and Client SDK (client-sdk/web.md):
AiGatewayThe service plane is consistent (full control plane); MCP/A2A/HTTP use HTTP entry points; model inference uses LLM Gateway.Console SDK (console-sdk/nodejs.md): 2.6.0 does not include the AI Gateway service.
Both the Java Server / Console SDK have
AiGatewayservices.
Examples by language
The following provides examples by language, item by item along the usage workflow; each card can switch languages (synced across the page).
Tool data plane entry points (MCP / A2A / HTTP Agent)
MCP / A2A / HTTP Agent are HTTP data plane entry points; SDKs in all languages (including Web) do not wrap these paths, so this card only has a REST tab. Public listeners use the abk_ Key issued by the target Gateway (carried via Authorization: Bearer); the LLM Gateway Key, project API Key, and upstream Provider Key cannot be used as abk_.
- REST (curl)
# MCP: /ai-gateway/g/<GATEWAY_ID>/mcp (tool list/search/call share the Agent allowlist and OPA authorization; side effects are protected by an idempotency gate)
# A2A: /ai-gateway/g/<GATEWAY_ID>/a2a/<targetSlug> (Task/Context isolated per caller + Target Revision)
# HTTP Agent: /ai-gateway/g/<GATEWAY_ID>/http/<targetSlug>[/{path...}] (only forwards methods/relative paths/query/body allowed by the Target,
# Caller authentication Headers are not replayed; credentials are resolved from the managed Credential)Create a Gateway and manage policies (Policy engine / Policy bundle / Snapshot)
After the policy engine is created, validate the policy bundle via dry-run, then publish it and attach it to the Gateway via updateGatewayPolicy; sealSnapshot seals the current configuration (asynchronously). Changes to Target / Consumer / Tool allowlist / Policy only take effect for public traffic on Snapshots that are published and consistent.
- Node.js (Server)
- Web (user session)
- Java
- Python
- Go
- REST (curl)
await aiGateway.listPolicyEngines(); // Policy engine list
await aiGateway.updateGatewayPolicy({ gatewayId: '<gatewayId>' }); // Mount strategy (fields see SDK type definitions)
await aiGateway.sealSnapshot({ gatewayId: '<gatewayId>' }); // Archived as a Snapshot (asynchronous)
await aiGateway.getSnapshotStatus({ gatewayId: '<gatewayId>' }); // Poll archive results
// Policy engine create/detail: createPolicyEngine / getPolicyEngine
// Policy bundles: createRevision / publishPolicyBundle / listPolicyBundles / getPolicyBundle; validate with dryRunPolicy before rollout (evaluates only, does not execute)// Web SDK has the same AiGateway control plane
await aiGateway.sealSnapshot({ gatewayId: '<gatewayId>' });
await aiGateway.getSnapshotStatus({ gatewayId: '<gatewayId>' });aiGateway.listPolicyEngines(); // Policy engine list
aiGateway.updateGatewayPolicy("<gatewayId>"); // Mount strategy
aiGateway.sealSnapshot("<gatewayId>"); // Archived as a Snapshot (asynchronous)
aiGateway.getSnapshotStatus("<gatewayId>"); // Poll archive results
// Policy engine create/detail: createPolicyEngine / getPolicyEngine
// Policy bundles: createRevision / publishPolicyBundle / listPolicyBundles / getPolicyBundle; validate with dryRunPolicy before rollout (evaluates only, does not execute)# server-python@release-2.6.0 has no AiGateway service; the control plane is currently accessed via REST (see below) or the nodejs SDK// server-go@release-2.6.0 has no AiGateway service; the control plane is currently called via REST (see below) or the Node.js SDKcurl -sS -X POST "$APPBASE_ENDPOINT/ai-gateway/gateways/<GATEWAY_ID>/snapshots" \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" -H "X-Appwrite-Key: $APPBASE_API_KEY"
# Asynchronous archival: poll GET .../gateways/<GATEWAY_ID>/snapshots/status until completion
# Policy plane (body fields per the API reference / SDK type definitions): PUT .../gateways/<GATEWAY_ID>/policy;
# GET|POST /ai-gateway/policy-engines; POST .../policy-engines/{policyEngineId}/bundles (publish);
# POST .../policy-engines/{policyEngineId}/dry-run (dry run before going live)Onboard Targets (Resources / Revisions / Mounting)
Targets are onboarded via the unified resource endpoint (kind as a path parameter) and brought online through the revision state machine. Health checks and full replacement of mounts are resource-specific operations.
- Node.js (Server)
- Web (user session)
- Java
- Python
- Go
- REST (curl)
// Unified resource surface: listResources / createResource / getResource / replaceResource / deleteResource
// kind is the resource category path parameter; the request body is a generalized schema, subject to SDK type definitions
// Revisions: createRevision / getRevision / transitionRevision (resource go-live uses the revision state machine)
// Target operations: checkTargetHealth; mounting: listTargetMounts / replaceTargetMounts// Web SDK has the same AiGateway control plane: resource / revision / mount method names are the same as the Node tab// Java side does not expose a generalized schema request body; methods only take locator parameters (kind is the resource-category path parameter):
aiGateway.createResource("targets"); // Create resource
aiGateway.createRevision("targets", "<id>"); // Create a revision
aiGateway.transitionRevision("targets", "<id>", "<revisionId>", "publish"); // Revision state machine goes live
aiGateway.checkTargetHealth("<id>"); // Health probe
aiGateway.replaceTargetMounts("<id>"); // Mount performs full replacement# server-python@release-2.6.0 has no AiGateway service// server-go@release-2.6.0 has no AiGateway service(
set +x
set -o pipefail
: "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
# Resource creation (kind=targets) and full replacement: POST|PUT /ai-gateway/resources/{kind}[/{id}]
# (request body is a generalized schema; see API reference / SDK type definitions for fields)
# Go live via the revision state machine: POST .../resources/{kind}/{id}/revisions → POST .../revisions/{revisionId}/{action}
builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -sS -X POST "$APPBASE_ENDPOINT/ai-gateway/resources/targets/<ID>/revisions/<REVISION_ID>/publish" \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID"
# Health check: POST .../resources/targets/<ID>/health-check
# Mount full replacement: PUT .../resources/targets/<ID>/mounts
)Model deployments and standard models
AI Gateway model deployments, discovery, and standard-model management are disabled in v2.12.0; use LLM Gateway for models.
Tool/MCP connections
A Tool connection is the upstream onboarding for Tool/MCP data plane calls: after creation, you can trigger tool discovery, test connectivity, update configuration, and delete it with a region parameter after disabling an active connection; verify the actual result of associated tool calls. Connectivity testing is supported starting from 2.8.0: testToolConnection(id, toolId) (toolId optional) (POST /ai-gateway/tool-connections/{id}/test), which returns diagnostic results (target reachability, credential checks, and per-tool status); the Console connection diagnostics panel uses the same source.
- Node.js (Server)
- Web (user session)
- Java
- Python
- Go
- REST (curl)
// createToolConnection / discoverToolConnection / updateToolConnection / deleteToolConnection
// 2.8.0 Connectivity test: aiGateway.testToolConnection('<id>'); the optional second parameter toolId tests a single tool only
// payload is defined by the target resource; see the SDK type definitions for fields// Web SDK has the same AiGateway control plane: tool connection method names are the same as the Node tab// Check the API reference for tool connection request bodies and region requirements;
// verify the signatures of the Java SDK in use.# server-python@release-2.6.0 has no AiGateway service// server-go@release-2.6.0 has no AiGateway service(
set +x
set -o pipefail
: "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
# Create / discover available tools: POST /ai-gateway/tool-connections, POST .../tool-connections/discover
# (payload is defined by the target resource; see SDK type definitions for fields)
builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -sS -X POST "$APPBASE_ENDPOINT/ai-gateway/tool-connections/discover" \
-H 'Content-Type: application/json' \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" \
-d '{"endpoint":"https://tools.example.com/mcp"}'
# Update: PUT .../tool-connections/<ID> (complete body in API reference); delete: DELETE .../tool-connections/<ID>?region=<REGION> (disable active connections first)
# Connectivity test (2.8.0): POST .../tool-connections/<ID>/test (optional body {"toolId":"<TOOL_ID>"} tests a single tool only)
)Issuing and enabling/disabling Consumer credentials (abk_ Keys)
createKey issues abk_ credentials; use gatewayIds to scope which gateways can be called, and allowedModels / allowedTools / allowedAgents to scope models, tools, and Agent Targets. Disabling attempts to invalidate authentication caches; if that fails, an old cached credential can remain valid briefly and in-flight requests may continue. Verify that calls are rejected rather than relying on a successful response alone.
Omitting allowedModels, allowedTools, or allowedAgents defaults that field to ["*"]; an empty array denies that type. The example allows only the specified tool and denies models and agents, subject to snapshot and policy restrictions.
- Node.js (Server)
- Web (user session)
- Java
- Python
- Go
- REST (curl)
import { AiGateway } from 'agentbase-node-sdk';
const aiGateway = new AiGateway(client); // Control plane: aiGateway.read / aiGateway.write
await aiGateway.listKeys(); // abk_ metadata view
await aiGateway.createKey({
name: 'agent-consumer',
gatewayIds: ['<gatewayId>'],
allowedModels: [], allowedTools: ['<TOOL_ID>'], allowedAgents: [] // For the allowlist and principal fields, see the SDK type definitions
});
// Enable/disable: aiGateway.updateKeyStatus(...); verify actual calls after disablingimport { AiGateway } from 'agentbase-web-sdk'; // Web SDK's AiGateway service surface matches the Server's (full control plane)
const aiGateway = new AiGateway(client);
await aiGateway.listKeys();
await aiGateway.createKey({ name: 'agent-consumer', gatewayIds: ['<gatewayId>'], allowedModels: [], allowedTools: ['<TOOL_ID>'], allowedAgents: [] });import io.agentbase.services.AiGateway;
AiGateway aiGateway = new AiGateway(client); // Java Server SDK has the AiGateway service
aiGateway.listKeys();
aiGateway.createKey("agent-consumer", List.of("<gatewayId>"),
null, List.of(), List.of("<TOOL_ID>"), List.of(), null, null); // region / allowedModels / allowedTools / allowedAgents / defaultLogicalModel / principal# server-python@release-2.6.0 has no AiGateway service; abk_ issuance/enable-disable is currently done via REST (see below) or the nodejs SDK// server-go@release-2.6.0 has no AiGateway service;
// abk_ Key issuance/enabling and disabling is currently done via REST (see below) or the Node.js SDK(
set +x
set -o pipefail
: "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -sS -X POST "$APPBASE_ENDPOINT/ai-gateway/keys" \
-H 'Content-Type: application/json' \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" \
-d '{"name":"agent-consumer","gatewayIds":["<gatewayId>"],"allowedModels":[],"allowedTools":["<TOOL_ID>"],"allowedAgents":[]}'
# allowlist scoping: allowedModels / allowedTools / allowedAgents; principal carries roles and flat claims (fields see API reference)
# List keys GET /ai-gateway/keys; retrieve plaintext GET .../keys/{keyId} (recoverable semantics); enable/disable PUT .../keys/{keyId}/status
)Observability and audit
Call records and the observability overview are queried by aiGateway.read scope, used for troubleshooting and auditing gateway calls.
- Node.js (Server)
- Web (user session)
- Java
- Python
- Go
- REST (curl)
await aiGateway.getObservabilityOverview(); // Observability overview
await aiGateway.listInvocations(); // Invocation record list (filter parameters see SDK type definitions)
// Single invocation record: getInvocation// Web SDK has the same AiGateway control plane
await aiGateway.getObservabilityOverview();
await aiGateway.listInvocations();aiGateway.getObservabilityOverview(); // Observability overview
aiGateway.listInvocations(); // Invocation record list (filter parameters see SDK type definitions)
// Single invocation record: getInvocation# server-python@release-2.6.0 has no AiGateway service// server-go@release-2.6.0 has no AiGateway service;
// Observability and audit currently go through REST (see below) or the nodejs SDKcurl -sS "$APPBASE_ENDPOINT/ai-gateway/observability/overview" \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" -H "X-Appwrite-Key: $APPBASE_API_KEY"
# Invocation records: GET /ai-gateway/invocations; single record: GET /ai-gateway/invocations/<INVOCATION_ID>Example Guide
JavaScript (Node control plane calls + data plane HTTP form): examples/javascript.md
Java:examples/java.md
2.12.0 — Approvals, initialization and tool embeddings
New methods include listApprovalRequests/getApprovalRequest/approveApprovalRequest/rejectApprovalRequest, getInitializationStatus and listToolEmbeddings/createToolEmbeddings. Reads and writes require aiGateway.read and aiGateway.write respectively. Inspect the tool, invocation and current request state before approving or rejecting; retrying a read must not repeat approval. Tool embedding can invoke a model. Reading initialization status does not prove all resources are ready.