Docs
Contents

AI Gateway (Unified AI Data Plane)

Overview and use cases

AI Gateway is the unified AI data plane and control plane introduced in 2.6.0 (services/ai-gateway), hosting Target onboarding, Tool/MCP, A2A, HTTP Agent, identity (Consumer/Key), policy (Policy), budgets, caching, and auditing. Resource model: Gateway / Target / Tool / Consumer / Policy / Snapshot — Targets and Tools are project-level assets that enter one or more Gateway Snapshots via mount relationships.

Coexists with the existing LLM Gateway: it does not migrate legacy resources, does not take over legacy production paths, and does not provide compatibility aliases without gatewayId. As long as you use existing Provider/Gateway Key/Chat/Embeddings/Rerank/VLM, continue with llm-gateway.md; use this page only when Gateway/Target/Tool/Consumer/Policy/MCP/A2A is involved. The two systems' names, credentials, and paths are not interchangeable.

Prerequisites

Usage workflow

Data plane entry points (HTTP, added in 2.6.0; SDKs do not wrap these paths):

Control plane (SDK AiGateway service):

  1. Create a Gateway and manage policies: createPolicyEngine / listPolicyEngines / getPolicyEngine; updateGatewayPolicy({ gatewayId, ... }); policy bundles createRevision/publishPolicyBundle/listPolicyBundles/getPolicyBundle; validate with dryRunPolicy before going live; release semantics use Snapshots: sealSnapshot / getSnapshotStatus.

  2. Onboard Targets: createResource / listResources / getResource / replaceResource (kind is the resource category path parameter); revision management createRevision / getRevision / transitionRevision (/resources/{kind}/{id}/revisions/{revisionId}/{action}); health checks checkTargetHealth; mounting listTargetMounts / replaceTargetMounts.

  3. Model inference, model deployments, and standard-model management are unavailable in v2.12.0.

  4. Tool/MCP connections: createToolConnection / discoverToolConnection / updateToolConnection / deleteToolConnection.

  5. Consumer credentials: createKey / listKeys / getKey / updateKeyStatus (issuance and enable/disable of abk_).

  6. Observability and auditing: getObservabilityOverview / listInvocations / getInvocation.

2.7.0 additions (appbase release-2.7.0 PR 526):

2.8.0 additions:

Key parameters and returns

Errors and boundaries

SDK Category Differences

Examples by language

The following provides examples by language, item by item along the usage workflow; each card can switch languages (synced across the page).

Tool data plane entry points (MCP / A2A / HTTP Agent)

MCP / A2A / HTTP Agent are HTTP data plane entry points; SDKs in all languages (including Web) do not wrap these paths, so this card only has a REST tab. Public listeners use the abk_ Key issued by the target Gateway (carried via Authorization: Bearer); the LLM Gateway Key, project API Key, and upstream Provider Key cannot be used as abk_.

# MCP: /ai-gateway/g/<GATEWAY_ID>/mcp (tool list/search/call share the Agent allowlist and OPA authorization; side effects are protected by an idempotency gate)
# A2A: /ai-gateway/g/<GATEWAY_ID>/a2a/<targetSlug> (Task/Context isolated per caller + Target Revision)
# HTTP Agent: /ai-gateway/g/<GATEWAY_ID>/http/<targetSlug>[/{path...}] (only forwards methods/relative paths/query/body allowed by the Target,
#   Caller authentication Headers are not replayed; credentials are resolved from the managed Credential)

Create a Gateway and manage policies (Policy engine / Policy bundle / Snapshot)

After the policy engine is created, validate the policy bundle via dry-run, then publish it and attach it to the Gateway via updateGatewayPolicy; sealSnapshot seals the current configuration (asynchronously). Changes to Target / Consumer / Tool allowlist / Policy only take effect for public traffic on Snapshots that are published and consistent.

await aiGateway.listPolicyEngines();                             // Policy engine list
await aiGateway.updateGatewayPolicy({ gatewayId: '<gatewayId>' }); // Mount strategy (fields see SDK type definitions)
await aiGateway.sealSnapshot({ gatewayId: '<gatewayId>' });      // Archived as a Snapshot (asynchronous)
await aiGateway.getSnapshotStatus({ gatewayId: '<gatewayId>' }); // Poll archive results
// Policy engine create/detail: createPolicyEngine / getPolicyEngine
// Policy bundles: createRevision / publishPolicyBundle / listPolicyBundles / getPolicyBundle; validate with dryRunPolicy before rollout (evaluates only, does not execute)

Onboard Targets (Resources / Revisions / Mounting)

Targets are onboarded via the unified resource endpoint (kind as a path parameter) and brought online through the revision state machine. Health checks and full replacement of mounts are resource-specific operations.

// Unified resource surface: listResources / createResource / getResource / replaceResource / deleteResource
//   kind is the resource category path parameter; the request body is a generalized schema, subject to SDK type definitions
// Revisions: createRevision / getRevision / transitionRevision (resource go-live uses the revision state machine)
// Target operations: checkTargetHealth; mounting: listTargetMounts / replaceTargetMounts

Model deployments and standard models

AI Gateway model deployments, discovery, and standard-model management are disabled in v2.12.0; use LLM Gateway for models.

Tool/MCP connections

A Tool connection is the upstream onboarding for Tool/MCP data plane calls: after creation, you can trigger tool discovery, test connectivity, update configuration, and delete it with a region parameter after disabling an active connection; verify the actual result of associated tool calls. Connectivity testing is supported starting from 2.8.0: testToolConnection(id, toolId) (toolId optional) (POST /ai-gateway/tool-connections/{id}/test), which returns diagnostic results (target reachability, credential checks, and per-tool status); the Console connection diagnostics panel uses the same source.

// createToolConnection / discoverToolConnection / updateToolConnection / deleteToolConnection
// 2.8.0 Connectivity test: aiGateway.testToolConnection('<id>'); the optional second parameter toolId tests a single tool only
// payload is defined by the target resource; see the SDK type definitions for fields

Issuing and enabling/disabling Consumer credentials (abk_ Keys)

createKey issues abk_ credentials; use gatewayIds to scope which gateways can be called, and allowedModels / allowedTools / allowedAgents to scope models, tools, and Agent Targets. Disabling attempts to invalidate authentication caches; if that fails, an old cached credential can remain valid briefly and in-flight requests may continue. Verify that calls are rejected rather than relying on a successful response alone.

Omitting allowedModels, allowedTools, or allowedAgents defaults that field to ["*"]; an empty array denies that type. The example allows only the specified tool and denies models and agents, subject to snapshot and policy restrictions.

import { AiGateway } from 'agentbase-node-sdk';

const aiGateway = new AiGateway(client);   // Control plane: aiGateway.read / aiGateway.write

await aiGateway.listKeys();                // abk_ metadata view
await aiGateway.createKey({
    name: 'agent-consumer',
    gatewayIds: ['<gatewayId>'],
    allowedModels: [], allowedTools: ['<TOOL_ID>'], allowedAgents: [] // For the allowlist and principal fields, see the SDK type definitions
});
// Enable/disable: aiGateway.updateKeyStatus(...); verify actual calls after disabling

Observability and audit

Call records and the observability overview are queried by aiGateway.read scope, used for troubleshooting and auditing gateway calls.

await aiGateway.getObservabilityOverview();   // Observability overview
await aiGateway.listInvocations();            // Invocation record list (filter parameters see SDK type definitions)
// Single invocation record: getInvocation

Example Guide

2.12.0 — Approvals, initialization and tool embeddings

New methods include listApprovalRequests/getApprovalRequest/approveApprovalRequest/rejectApprovalRequest, getInitializationStatus and listToolEmbeddings/createToolEmbeddings. Reads and writes require aiGateway.read and aiGateway.write respectively. Inspect the tool, invocation and current request state before approving or rejecting; retrying a read must not repeat approval. Tool embedding can invoke a model. Reading initialization status does not prove all resources are ready.