Docs
Contents

Memory Audit Logs and Trace Localization

Feature overview and core scenarios

Memory Audit & Trace (accessed through the audit tab and trace entry) is the core observability system for enterprise security and compliance, change tracking, and production performance troubleshooting in Agentbase Memory. In multi-team environments with frequent agent interactions, operations and architecture teams need clear answers to three types of questions:

  1. Configuration change history tracing: who modified which Memory configuration namespace or Pipeline, when, and for what reason (reason), and what are the corresponding lastEventId and ETag?

  2. Business data-plane operation audit: whether a memory write (POST /v1/memory/memories), batch delete (DELETE), search retrieval (search / recall), or experience distillation (distillations) executed successfully, which async task ID it was associated with (jobId), and how many milliseconds it took?

  3. End-to-end call chain tracing (Trace): how much time and how many Tokens were consumed at each stage of an async write or hybrid retrieval — API acceptance, queue waiting, OceanBase vector query, and dashscope-prod model calls (qwen-plus / text-embedding-v3 / qwen3.7-text-rerank)?

In the reference project 企业智能客服与知识中台 (enterprise-ai-hub-210) on Agentbase 企业智能中台 (69c6d1aa00062e6bb60), request auditing (policy.requestAuditEnabled: true) is enabled by default. The X-AppBase-Trace-Id response header links memory operations to traces in Observability / AgentOps.

Audit list showing test memory write operations

The test-project audit entries have request, job, and user identifiers masked.


Core Concepts and Architecture

Agentbase Memory's audit and observability architecture consists of three complementary layers: control-plane configuration version history, data-plane business request auditing (memory_events), and OpenTelemetry / Langfuse distributed tracing (Traces).

Comparison of responsibilities and boundaries of the three types of observability data

Observability dimensionQuery entry / interfaceRecorded scope and core fieldsContent not recorded and design boundaries
Configuration change history (Config History)Configuration → Config Versions or GET /v1/projects/{projectId}/memory/config/{namespace}/historyRecords the version number (version), event ID (lastEventId), etag, operator (actor), change reason (reason), and before/after Diff for every config save and rollback.Does not record unsaved page form inputs or sample trial-run raw text.
Business request audit (Request Audit)Memory → Audit or GET /v1/memory/auditsRecords memory creation, update, single/batch delete, search, unified retrieval, feedback/observation, import/export submissions, profile extraction, and context/knowledge governance operations; includes requestId, jobId, userId, HTTP status code, and elapsed time.Does not record ordinary single-item reads (getMemory), list browsing (listMemories), statistics queries, or async task status polling (getJob) one by one, to prevent polling storms from drowning out critical business change audits.
Memory TraceMemory traces button (opens Agent Ops → Tracing, with tag=memory)Records API acceptance, background queue waiting, staged execution (fact extraction, vectorization, conflict detection, Rerank), model Token consumption and gateway error codes (gatewayStatus, errorCode).Controlled by the platform-wide switch agentops.enabled and sampling rate (policy.observability.successSampleRate); not being sampled does not mean the business request was not executed.

Feature Entrypoints and Permission Matrix

Console entry points and permission matrix

In project enterprise-ai-hub-210, open the Memory → Audit tab. Use the separate Memory traces button to open tracing.

Function actionConsole entryREST API endpoint / routeRequired permission Scope
Query the business request audit listMemory → Audit tab search barGET /v1/memory/auditsA privileged Console session, or the current project API key with memories.read; ordinary end users cannot query project audits
Toggle project request auditingMemory → Configuration → Basic ConfigurationPUT /v1/projects/{projectId}/memory/config/policy (requestAuditEnabled)projects.write
View configuration change history and ETagMemory → Configuration → Expand Config VersionsGET /v1/projects/{projectId}/memory/config/{namespace}/historyprojects.read
Open Memory TraceMemory traces buttonRoute to the project trace page; initial filter is tag=memoryobservability.traces.read

Request-level Trace ID propagation and correlation mechanism

  1. X-AppBase-Trace-Id response header: All successful responses passing through the Agentbase /v1/memory/* facade (HTTP 2xx) return X-AppBase-Trace-Id in the HTTP response header (a 32-character non-all-zero lowercase hexadecimal correlation ID). The server first adopts a valid W3C traceparent request header passed by the caller, then a valid X-Trace-ID; if neither is provided, a new ID is generated automatically.

  2. Trace tag filtering: Opening tracing from the Memory page navigates to the observability trace list. The initial filter is tag=memory; adjust it on the trace page as needed.

  3. Content capture dual gate (captureContent): Memory content capture depends on platform agentops.memoryCaptureContentEnabled and project policy.observability.captureContent. Ordinary diagnostic Generations omit model input/output; a separate evaluation Trace may contain raw content when capture is enabled. Sanitize before reporting and do not assume automatic replacement with [REDACTED].


Feature Details and Operation Guide

1. Verify write, retrieval, and deletion results with "Request Audit" in the Console

  1. Log in to the Agentbase Console, enter the project 企业智能客服与知识中台 (enterprise-ai-hub-210), and open the Memory (Memory) → Audit tab.

  2. Enter filter conditions in the top filter bar:

    • You can search by Request ID (requestId), Target User ID (userId, e.g. user_zhangwei_001), Resource ID (e.g. memoryId), or Operation Name (operation);

    • If investigating a specific asynchronous intelligent write, import/export, or distillation task, you can enter the exact jobId in the task filter box;

    • Quick filtering by result status (success / failure) is supported.

  3. Click to expand the target audit record and verify the following key troubleshooting information:

    • Request metadata: HTTP method, request path, HTTP status code, operation principal (actor), target user (userId), and associated resource ID;

    • Request audit payload: audit processing recursively handles recognizable credential key names and truncates oversized values; it cannot identify all sensitive content in arbitrary text. Do not put secrets or unsanitized business data in metadata or request bodies.

    • Async task terminal state and three-stage elapsed time: for async write tasks, cross-check GET /v1/memory/jobs/{jobId} to compare queueing time (queueDurationMs), actual execution time (executionDurationMs), and end-to-end total time (totalDurationMs), distinguishing "API accepted and enqueued (HTTP 202)" from "background model finally completed execution (completed / partial_failed / failed)".

2. Audit and configuration history API parameters explained with call examples

(1) Query business request audit (GET /v1/memory/audits)

Query parameter / response fieldTypeDescription and usage constraints
limit (query parameter)integerNumber of items returned per page, value 1~100 (default 25).
cursor (query parameter)stringPagination cursor; pass back the nextCursor from the previous page's response as-is.
jobId (query parameter)stringPrecisely filter associated audit records by a specific asynchronous task ID.
userId / operationstringFilter by target business user ID (e.g. user_zhangwei_001) or operation type.
audits (response field)arrayList of matched audit events, stored in the project-specific memory_events table, subject to the retention.auditEventDays retention period.
scanned / truncatedinteger / boolNumber of underlying events scanned by this query and whether the scan limit truncation was triggered.
(
  set +x
  set -o pipefail
  : "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
  : "${APPBASE_ENDPOINT:?Set APPBASE_ENDPOINT to the target API endpoint including /v1}"
  APPBASE_ENDPOINT="${APPBASE_ENDPOINT%/}"
  : "${APPBASE_PROJECT_ID:?Set APPBASE_PROJECT_ID for the target environment}"
  : "${APPBASE_API_KEY:?Set APPBASE_API_KEY for the target environment}"

  # 1. Query recent memory writes and retrieval audit logs for user_zhangwei_001
  builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -fsS --get "${APPBASE_ENDPOINT}/memory/audits" \
    -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
    --data-urlencode "userId=user_zhangwei_001" \
    --data-urlencode "limit=20" | jq .

  # 2. Verify audit results precisely by the jobId of the async write task
  builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -fsS --get "${APPBASE_ENDPOINT}/memory/audits" \
    -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
    --data-urlencode "jobId=job_01hxyz9876543210" | jq .
)

(2) Review configuration change history (lastEventId, etag, actor, reason)

When you need to audit when and by whom a configuration item (such as policy, provider, or evolution) was modified, call the configuration history endpoint:

(
  set +x
  set -e -o pipefail
  : "${APPBASE_CONSOLE_SESSION:?Set APPBASE_CONSOLE_SESSION}"
# Query the configuration change audit history of the policy namespace
builtin printf 'Cookie: %s\n' "$APPBASE_CONSOLE_SESSION" | curl --disable --header @- -fsS "${APPBASE_ENDPOINT}/projects/${APPBASE_PROJECT_ID}/memory/config/policy/history" \
  -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
  -H "X-Appwrite-Mode: admin" | jq .
)

The typical response structure contains complete governance metadata for each change:

{
  "component": "memory",
  "scopeType": "project",
  "namespace": "policy",
  "currentVersion": 3,
  "lastEventId": "cfg_evt_01j9k2m4n6p8q0r2",
  "etag": "\"v3-8f9e1d2c3b4a\"",
  "items": [
    {
      "version": 3,
      "eventId": "cfg_evt_01j9k2m4n6p8q0r2",
      "etag": "\"v3-8f9e1d2c3b4a\"",
      "actor": "admin:69c6d1aa00062e6bb60",
      "reason": "开启搜索重排 rerankEnabled 并设置慢查询追踪阈值 slowThresholdMs=800",
      "createdAt": "2026-10-22T09:30:00.000Z"
    }
  ]
}

3. Jump from Memory to Trace to locate slow requests and model call anomalies

When you find in the Audit tab that a request took unusually long or an async task shows retries, use the Memory traces button to open the trace view (the initial filter is tag=memory):

  1. Verify the time window and associated IDs: enter the X-AppBase-Trace-Id returned in the response header into the trace list search box (if _APP_OBSERVABILITY_YUNTU_TRACE_LINK_ENABLED=enabled is enabled in the Jiuzhou Yuntu environment, you can also directly search the SOFA-TraceId corresponding to metadata.cloudAtlasTraceId).

  2. Analyze the stage waterfall (Timeline) and model calls:

    • For async intelligent writing (infer=true): multiple retries of the same jobId reuse the same root Trace ID, and each attempt generates an independent child Span; expanding the memory.model.call node lets you view the precise time and Prompt/Completion Token consumption of calls to qwen-plus or text-embedding-v3;

    • For stage failure diagnosis: if a stage fails, the memory.ingest.stage_failed event directly records errorClass, errorCode (e.g. LLM_RATE_LIMITED, LLM_TIMEOUT, LLM_INVALID_JSON), retryable, and gatewayStatus, enabling root cause identification without logging into servers to dig through container logs;

    • For online memory retrieval (search): if the project has neither profile rewriting (profileRewriteEnabled=false) nor reranking (rerankEnabled=false) enabled, the retrieval chain only contains Embedding vectorization and OceanBase deterministic retrieval Spans, with no Chat Generation node — this is a normal high-performance deterministic retrieval path.


FAQ (Frequently Asked Questions and Troubleshooting)

1. Why can't I find the corresponding record in the "Audit" tab after calling GET /v1/memory/memories to query the list or polling GET /v1/memory/jobs/{jobId}?

2. If an audit record is missing or a successful memory operation cannot be found in the Trace list, does that mean the operation did not take effect?

3. How to make the “Metrics analysis” in Agent Operations only count model Tokens and latency produced by the Memory pipeline?