Docs
Contents

Knowledge Distillation of Experiences and Skills

Feature overview and core scenarios

In enterprise-grade agent production systems, traditional declarative memory (e.g., single facts fact and user preferences preference) can only answer “who the user is and what the system's current configuration is”, but cannot answer “when encountering a certain type of complex production failure, which methodology and steps should be followed to invoke tools and resolve the problem”. As customer support and operations agents handle hundreds to thousands of multi-turn support ticket conversations and tool execution trajectories every day, if successful troubleshooting operation trajectories cannot be consolidated into procedural knowledge, the agent must still trial-and-error from scratch the next time it encounters a similar failure.

Agentbase Memory provides native Experiences and Skills knowledge distillation capability. Through an orchestrable knowledge distillation pipeline (distillation Pipeline), the system uses project-configured models to process session messages (messages) and tool invocation event trajectories (toolEvents) into experience and skill candidates. Review the actual results against their evidence:

In the example project 企业智能客服与知识中台 (project ID: enterprise-ai-hub-210, based on the OceanBase Shared V2 storage mode) of Agentbase 企业智能中台 (organization ID: 69c6d1aa00062e6bb60), the operations team submitted the conversations and tool trajectories of VIP architects user_zhangwei_001 and frontline engineers user_lina_002 handling example support tickets such as OceanBase 向量内存不足 (-4013) and AI 网关 429 限流自动故障转移 for distillation (e.g., example distillation tasks dst_3e5b09f456c5d5c332507ec1dbe3eb6d and dst_adc2cfda21c940fb5d5e4a3263ae56f9), achieving automatic cross-session, cross-user reuse of troubleshooting wisdom.


Core Concepts and Architecture

The knowledge distillation engine of Agentbase Memory chains evidence reception, skill extraction and merging, experience extraction and association, safety and quality validation, and storage projection through a seven-step standard pipeline (7-Step Distillation Pipeline).

1. In-depth comparison of the Experience vs Skill structured models

Although Experience and Skill both belong to the L1 procedural knowledge layer (sharing the knowledgeId naming scheme and immutable revision versions revisionId), they have clearly distinct responsibilities in cognitive positioning, structured Schema, and consumption methods:

Comparison dimensionTroubleshooting Experience (Experience)Standardized Skill (Skill)
Core cognitive positioningContextualized cases and pitfall-avoidance guides (Case-Based Reasoning): answers “under what business background and failure symptoms, why a certain approach worked or failed, and what lessons were learned”.Executable Standard Operating Procedure (SOP): answers “when a certain condition is triggered, what to check in step one, which tool to invoke in step two, how to fill in parameters, and how to verify recovery”.
Core structured fieldsThe outer knowledge object has knowledgeId, title, version, scope, and experience; experience fields include situation, decision, action, constraints, evidence, outcome, and skillItemIds.The outer knowledge object has knowledgeId, title, version, scope, and skill; skill fields include description, inputs, applicability, preconditions, procedure, and expectedOutcome.
Cross-referencesAn Experience can be associated with and reference the Skill produced in this run or previously (when a dependent Skill has not yet been approved and activated, skill_dependency_not_approved protection is triggered).Skill In the distillation pipeline, executes before Experience extraction (Step 2 & 3), so that the subsequent Experience (Step 4) can directly reference the already standardized skill identifiers.
Prompt injection timingInjected when the agent performs root cause analysis, solution selection, or answers complex queries, providing references from similar historical cases.Injected when the agent plans tool calls (Tool Planning) or executes automated operations SOPs, guiding precise multi-step operations.

2. Node-by-node breakdown of the seven-step knowledge distillation pipeline (distillation Pipeline)

In Memory → Configuration → Experiences & Skills Pipeline, each distillation pipeline always consists of the following 7 sequential nodes:

Step numberNode identifier (step)RequiredCore processing logic, model invocation, and tunable parameters
Step 1receiveRequiredReceive and perform bounded evidence validation: Validates the input messages conversation turns and toolEvents tool execution traces (using maxToolEvents to limit the maximum number of tool events per run, preventing excessively long loop logs from blowing up the context), and freezes an immutable L0 raw evidence record in the OceanBase sources table.
Step 2skill_extractionOptional (at least one of Step 4 and this step must be enabled)Extract reusable SOP skill candidates: Invokes the configured model to analyze the successful tool call sequences in the conversation and toolEvents, distilling candidate skills containing title, preconditions, procedure, inputs, and expectedOutcome. Supports custom skillDistillationPrompt.
Step 3skill_mergeLinked with Step 2Retrieve similar existing skills and decide on merging: Uses the project instance’s configured embedding model and dimension to retrieve the Top maxCandidates (value 1~20) existing skills within the same governance scope, and combines the cosine similarity threshold mergeThreshold (value (0, 1]) with skillMergePrompt to let the model decide: create a new independent skill (ADD), merge and supplement into an existing skill and generate a new revised version (MERGE), or skip duplicates (SKIP).
Step 4experience_extractionOptional (at least one of Step 2 and this step must be enabled)Extract troubleshooting experiences and associate skill references: Invokes the configured model to distill contextual cases from the full task picture (situation, decision, action, outcome, evidence), and automatically binds the relevant skills produced by Step 2/3 as knowledge references. Supports custom experienceDistillationPrompt.
Step 5experience_mergeLinked with Step 4Retrieve similar existing experiences and decide on merging: Retrieves the Top maxCandidates (1~20) similar existing experiences, and performs cross-session merging based on mergeThreshold and experienceMergePrompt (accumulating active source identifiers sourceIds, which do not guarantee distinct sessions, helping reach the minIndependentSessionsForAgent activation threshold) or creates a new entry.
Step 6validationRequiredQuality and safety gate validation: An independently enforced validation node (supports customization via evolutionValidationPrompt). Checks candidate consistency, evidence, and confidence (confidence); low-confidence knowledge can be retained in shadow rather than always discarded. Safety checks on input and generated text cover limited patterns: redact evidence before submission and do not rely on the system to catch all sensitive information.
Step 7knowledge_operationRequiredKnowledge persistence and status routing: Persists the knowledge master record, immutable version snapshots (Revision), and L0 source associations. A vector projection task may be queued separately afterwards according to source policy; creation of the main record does not mean the projection is ready. Knowledge status depends on validation, scope, configuration, and other conditions; active statuses are user-active or agent-active, while other records can remain candidate, shadow, and so on.

Feature Entrypoints and Permission Matrix

The Console entry point is located at: Project enterprise-ai-hub-210 → Memory → Evolution tab, with the accompanying pipeline configuration located at Memory → Configuration → Experiences & Skills Pipeline.

Functional actionConsole UI entry pointREST API endpointServer SDK method (agentbase-node-sdk@2.12.0)Required RBAC / API Key ScopeCross-module dependent association
Submit a knowledge distillation taskEvolution → Manual distillation → SubmitPOST /v1/memory/memories/distillationsmemory.submitDistillationmemories.writeUses the active settings in Model routing and the Configuration Center
Query the distillation task list and detailsEvolution → Distillation task listGET /v1/memory/memories/distillations, GET .../{distillationId}memory.listDistillations / memory.getDistillationmemories.readAssociated with Memory audit and Memory tracing
Replay failed distillation tasksAt the end of a distillation task row ReplayPOST /v1/memory/memories/distillations/{distillationId}/replaymemory.replayDistillationmemories.write and knowledge-management permissionReplays tasks in terminal failed or dead state using their original runtime snapshot
List and semantically search ExperiencesEvolution → Experiences list and search boxGET /v1/memory/memories/experiences, POST .../experiences/searchmemory.listExperiences / memory.searchExperiencesmemories.readProvides contextual experiences for the agent workbench and recall
List and semantically search SkillsEvolution → Skills list and search boxGET /v1/memory/memories/skills, POST .../skills/searchmemory.listSkills / memory.searchSkillsmemories.readProvides multi-step SOPs for the agent workbench and recall
Trace the L0 raw source of an experience/skillExpand experience or skill details → Source evidenceGET /v1/memory/memories/experiences/{knowledgeId}/sources, /skills/{knowledgeId}/sourcesmemory.listExperienceSources / memory.listSkillSourcesmemories.readAssociated with the L0 raw evidence layer in Multi-layer memory partitioning
Update or forget (delete) an experience/skillEdit / Forget button at the end of an experience/skill rowPATCH /v1/memory/memories/knowledge/{knowledgeId}, DELETE .../knowledge/{knowledgeId} (or /experiences/{id}, /skills/{id})memory.updateKnowledge / memory.forgetKnowledgememories.write / memories.deleteA forget marker excludes the knowledge from retrieval; physical cleanup is handled by a later task

Feature Details and Operation Guide

Step One: Submit execution evidence distillation by scope in the Console and retrieve the output

  1. Log in to the Agentbase Console, select the target project, click Memory in the left navigation bar, and switch to the Evolution tab. Before submitting distillation, check that evolution.enabled is enabled; the platform default is false, though the effective setting can differ.

  2. In the Scope configuration bar at the top, select the target scope:

    • Scope (scope): Supports user (personal level), agent (agent level; agentId such as enterprise-support-agent must also be filled in), and project (project-level global sharing);

    • User ID (userId): Enter user_zhangwei_001 (or user_lina_002) and click Apply scope.

  3. Click Manual distillation, enter conversation evidence one line at a time in the dialog, using user:, assistant:, system:, and tool: prefixes to preserve roles. Select the default or a published custom pipeline and click Confirm.

  4. After submission, you can view the execution status of the distillation jobs (such as dst_3e5b09f456c5d5c332507ec1dbe3eb6d and dst_adc2cfda21c940fb5d5e4a3263ae56f9) (queued -> running -> succeeded) and their output statistics (created, merged, skipped, blocked) in real time in the Distillation task list below.

  5. Switch to the Experiences or Skills tab below, query results as needed, and inspect their details and sources.

Step Two: Submit a distillation task containing toolEvents via the REST API and replay to verify

In a real agent runtime, the backend service can submit the full conversation messages together with the structured tool traces toolEvents to POST /v1/memory/memories/distillations after a ticket is closed or a task executes successfully:

Set a unique DISTILLATION_IDEMPOTENCY_KEY for each new task. Reuse the original key and identical request content only when retrying the same submission, to avoid reusing an old task or causing a conflict.

(
  set +x
  set -o pipefail
  : "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
  : "${APPBASE_ENDPOINT:?Set APPBASE_ENDPOINT to the target API endpoint including /v1}"
  APPBASE_ENDPOINT="${APPBASE_ENDPOINT%/}"
  : "${APPBASE_PROJECT_ID:?Set APPBASE_PROJECT_ID for the target environment}"
  : "${APPBASE_API_KEY:?Set APPBASE_API_KEY for the target environment}"
  : "${DISTILLATION_IDEMPOTENCY_KEY:?Set a unique key for this new task}"

  # 1. Submit a knowledge distillation task with messages and toolEvents (mode="wait", waits up to 20s for the final state)
  builtin printf 'X-Appwrite-Key: %s\n' "${APPBASE_API_KEY:?Set APPBASE_API_KEY}" | curl --disable --header @- -fsS -X POST "${APPBASE_ENDPOINT}/memory/memories/distillations" \
    -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
    -H "Idempotency-Key: ${DISTILLATION_IDEMPOTENCY_KEY}" \
    -H "Content-Type: application/json" \
    --data '{
      "userId": "user_zhangwei_001",
      "scope": "user",
      "sessionId": "sess_ticket_TK_20261022_001",
      "runId": "run_tk_20261022_001",
      "mode": "wait",
      "messages": [
        {
          "role": "user",
          "content": "生产告警:enterprise-ai-hub-210 项目在批量构建 1024 维 HNSW 向量索引时报错 OB_ALLOCATE_MEMORY_FAILED (-4013),请排查并恢复。"
        },
        {
          "role": "assistant",
          "content": "这是合成示例:已记录并检查测试工单,处理结果已记录。真实内存错误需结合数据库版本、日志和资源指标排查。"
        }
      ],
      "toolEvents": [
        {
          "eventId": "event-ob-sql-1",
          "name": "check_ticket",
          "status": "succeeded",
          "arguments": {
            "ticketId": "TK-20261022-001"
          },
          "result": {
            "status": "open"
          }
        },
        {
          "eventId": "event-ob-parameter-2",
          "name": "record_ticket",
          "status": "succeeded",
          "arguments": {
            "ticketId": "TK-20261022-001"
          },
          "result": {
            "status": "resolved"
          }
        }
      ]
    }'
)

Take distillationId from the submit response and set DISTILLATION_ID. Check the same user and scope, then query the task status and output separately.

(
  set +x
  set -o pipefail
  : "${APPBASE_ENDPOINT:?Set endpoint including /v1}"
  : "${APPBASE_PROJECT_ID:?Set project ID}"
  : "${APPBASE_API_KEY:?Set API key}"
  : "${DISTILLATION_ID:?Set distillation ID from the submitted task}"
  builtin printf 'X-Appwrite-Key: %s\n' "$APPBASE_API_KEY" | curl --disable --header @- -fsS "${APPBASE_ENDPOINT%/}/memory/memories/distillations/${DISTILLATION_ID}?scope=user&userId=user_zhangwei_001" \
    -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" | jq .
)

Run this command separately only after confirming the task is in terminal failed or dead state, replay is needed, and you have knowledge-management permission. If the response is lost or the outcome is unknown, query the task and verify execution first; do not retry blindly. The replay endpoint does not provide an idempotency-key guarantee.

(
  set +x
  set -o pipefail
  : "${APPBASE_ENDPOINT:?Set endpoint including /v1}"
  : "${APPBASE_PROJECT_ID:?Set project ID}"
  : "${APPBASE_API_KEY:?Set API key}"
  : "${DISTILLATION_ID:?Set distillation ID from the submitted task}"
  builtin printf 'X-Appwrite-Key: %s\n' "$APPBASE_API_KEY" | curl --disable --header @- -fsS -X POST "${APPBASE_ENDPOINT%/}/memory/memories/distillations/${DISTILLATION_ID}/replay?scope=user&userId=user_zhangwei_001" \
    -H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
    -H "Content-Type: application/json" \
    --data '{}' | jq .
)

Replay does not adopt a new pipeline or model configuration; submit a new task to use new settings.

Step Three: Use the Server SDK to retrieve experiences and skills, trace sources, and phase out outdated knowledge

In a Node.js business backend, you can call searchExperiences, searchSkills, listSkillSources, and forgetKnowledge via agentbase-node-sdk@2.12.0 respectively, completing a full closed loop from knowledge retrieval and source auditing to phase-out cleanup:

Before running, replace <APPBASE_ENDPOINT_WITH_V1> and <APPBASE_PROJECT_ID> with the confirmed target API endpoint and project ID. Until replaced, the SDK rejects the invalid endpoint before making a request.

// Use the official server SDK: agentbase-node-sdk@2.12.0
import { Client, Memory } from 'agentbase-node-sdk';

const client = new Client()
    .setEndpoint('<APPBASE_ENDPOINT_WITH_V1>')
    .setProject('<APPBASE_PROJECT_ID>')
    .setKey(process.env.APPBASE_API_KEY);

const memory = new Memory(client);

// 1. Semantic search for matching troubleshooting experiences (POST /v1/memory/memories/experiences/search)
const experiences = await memory.searchExperiences({
    userId: 'user_zhangwei_001',
    scope: 'user',
    query: 'OceanBase 向量索引构建报 -4013 内存不足如何解决',
    limit: 5
});

for (const item of experiences.items || []) {
    const exp = item.knowledge;
    console.log('[Experience]', exp.knowledgeId, exp.title, 'score:', item.score);
}

// 2. Semantic search for matching standardized SOP skills (POST /v1/memory/memories/skills/search)
const skills = await memory.searchSkills({
    userId: 'user_zhangwei_001',
    scope: 'user',
    query: '测试工单排查与处理记录',
    limit: 5
});

if (skills.items?.length > 0) {
    const topSkill = skills.items[0].knowledge;
    console.log('[Skill SOP]', topSkill.knowledgeId, topSkill.title, 'steps:', topSkill.skill?.procedure?.length);

    // 3. Trace the L0 raw session and tool trajectory sources supporting that skill (GET .../skills/{knowledgeId}/sources)
    const sources = await memory.listSkillSources({
        knowledgeId: topSkill.knowledgeId,
        userId: 'user_zhangwei_001',
        scope: 'user',
        limit: 10
    });
    console.log('关联的不可变 L0 Source 数量:', sources.total);
}

// 4. When an old experience is no longer applicable due to a system architecture upgrade, call the forgetting endpoint for that experience (DELETE)
async function retireObsoleteKnowledge(knowledge) {
    await memory.forgetKnowledge({
        knowledgeId: knowledge.knowledgeId,
        expectedVersion: knowledge.version,
        userId: 'user_zhangwei_001',
        scope: 'user',
        reason: 'OceanBase 集群已升级至自动弹性向量内存管理,旧手动阈值经验退役'
    });
}

FAQ (Frequently Asked Questions and Troubleshooting)

For forgetting, use the version from the retrieved knowledge object. On a version conflict, reload the object and confirm before trying again.

1. Why can't the skill_extraction and experience_extraction nodes both be disabled when configuring the "Experiences & Skills Pipeline"?

2. When submitting a distillation request with mode: "wait", if an HTTP 202 with "waitTimedOut": true is returned, has the task failed? Does it need to be resubmitted?

3. Why does activating or approving an experience (Experience) that has already completed extraction prompt skill_dependency_not_approved?