Docs
Contents

Experience and Skill Distillation Evolution

Feature overview and core scenarios

Experience and skill evolution (Memory Evolution; the Console tab is Experience & Skills) is the core capability by which Agentbase Memory (v2.10.0) enables agents to “self-learn from real-world practice and accumulate knowledge.” In enterprise customer service and ops troubleshooting scenarios, agents perform large volumes of multi-turn conversations and tool executions every day. Ordinary memories focus on recording “objective facts and preferences about the user,” whereas experience and skill distillation extracts cross-session reusable troubleshooting experience (Experience) and standardized operational skills (Skill) from real task execution traces and troubleshooting conversation evidence.

In the flagship project 企业智能客服与知识中台 (enterprise-ai-hub-210) of Agentbase 企业智能中台 (69c6d1aa00062e6bb60), the system completed two typical production distillation tasks (dst_3e5b09f456c5d5c332507ec1dbe3eb6d and dst_adc2cfda21c940fb5d5e4a3263ae56f9) via the distillation pipeline (calling dashscope-prod's qwen-plus and text-embedding-v3):


Core Concepts and Architecture

Experience and skill distillation extracts candidate knowledge from raw conversations or tool traces, then goes through semantic deduplication and merging with existing knowledge, model self-consistency validation, and activation threshold checks (or secondary human confirmation), finally building vector indexes for online retrieval and unified recall.

Experience and Skill Distillation Evolution: workflow diagram, not a runtime screenshot

Flow diagram, not a runtime screenshot; read the relationships between steps or states along the arrows.

View the flow diagram in full size

Core Differences and Attribute Table of Experience vs. Skill

Comparison dimensionExperienceSkill
Positioning and formReusable situational insights and pitfall-avoidance experience: records “what problem was encountered in which scenario, which approaches worked, and which approaches lead to failure.”Reusable standardized operating steps (SOP): records “to achieve a specific goal, which checks, commands, or tool calls should be executed in what order.”
Core structured fieldsTitle (title), summary (summary), applicable context (context), key insights/recommendations (insights / lesson), confidence (confidence), tags (tags).Name (name), description (description), trigger conditions (triggers), ordered step list (steps, including action descriptions and inputs parameter templates), verification criteria (verification).
Scope of ownership (scope)Supports user (user level), agent (agent level, requires specifying agentId), and project (project global level).Supports user, agent, project; agent-level skills require reproduction validation across multiple independent sessions (minIndependentSessionsForAgent) before automatic activation.
Lifecycle statesshadow (candidate/observation state, not yet meeting the activation threshold or awaiting human confirmation) -> active (activated and in effect) -> forgotten (forgotten, removed from retrieval).shadow -> active -> forgotten; additionally, when an experience depends on an unapproved Skill version, skill_dependency_not_approved blocking protection is triggered.
Retrieval and provenance APIsGET /v1/memory/memories/experiences, POST .../experiences/search, GET .../experiences/{knowledgeId}/sourcesGET /v1/memory/memories/skills, POST .../skills/search, GET .../skills/{knowledgeId}/sources

Feature Entrypoints and Permission Matrix

Console Entry Points and API Permission Matrix

The Console entry point is located at: Project enterprise-ai-hub-210 → Memory → Experience & Skills tab.

Function moduleConsole operation entryREST API pathRequired permissions and authentication requirements
Submit distillation taskManual distillation area → Submit distillationPOST /v1/memory/memories/distillationsmemories.write
Query and replay distillation tasksDistillation task listGET /v1/memory/memories/distillations, GET .../{distillationId}, POST .../{distillationId}/replaymemories.read / memories.write
Query and retrieve experiences/skillsExperience / Skill lists and search boxGET/POST /v1/memory/memories/experiences(/search), GET/POST /v1/memory/memories/skills(/search)memories.read
Query knowledge source evidenceExpand experience/skill details → Source evidenceGET /v1/memory/memories/experiences/{knowledgeId}/sources, /skills/{knowledgeId}/sourcesmemories.read
Version comparison and manual secondary confirmationVersions and Confirmations sub-tabGET/PATCH /v1/memory/memories/knowledge/reviews/{reviewId}, POST .../approve, POST .../reject, POST .../{knowledgeId}/rollbackMust use real user credentials with project management permissions (Console session or administrator JWT) + memories.write; an ordinary subject-less API Key cannot replace a human approver
Stop reuse (forget knowledge)End of Experience/Skill row Forget buttonDELETE /v1/memory/memories/experiences/{knowledgeId}, /skills/{knowledgeId}memories.delete

Feature Details and Operation Guide

1. Select the ownership scope and submit execution evidence for distillation (POST /v1/memory/memories/distillations)

  1. Log in to the Agentbase Console, enter project 企业智能客服与知识中台 (enterprise-ai-hub-210), and open the Memory (Memory) → Experiences and Skills tab.

  2. Set the knowledge scope in the ownership scope bar at the top of the page:

    • Select a scope (user, agent, or project);

    • Enter the target user ID (e.g., user_zhangwei_001 or user_lina_002); if the agent scope is selected, you must also fill in the target agentId (e.g., enterprise-support-agent), then click apply scope.

Experiences and Skills evolution view (reusing the registered screenshot)

Interface reference: reuses previously registered historical screenshots; demo data may differ from the current project; no new screenshots were taken this time.

  1. In the manual distillation area, enter the desensitized customer support troubleshooting conversation or tool invocation evidence (you may use the user:, assistant:, system:, tool: prefixes to mark roles), select use default Pipeline (or specify a published Experiences and Skills Pipeline), and click submit.

  2. API invocation and mode=async | wait dual-mode explanation: When submitting distillation via API, two waiting semantics are supported:

    • mode: "async": immediately returns HTTP 202 after the task is persisted and enqueued; the response includes distillationId (e.g., dst_3e5b09f456c5d5c332507ec1dbe3eb6d) and status: "queued";

    • mode: "wait": the server holds the connection and waits for up to 20 seconds. If the task reaches a terminal state (succeeded, partial_failed, failed, or dead) within the 20-second window, HTTP 200 and complete output references are returned directly; if more than 20 seconds pass and it is still queued or running, HTTP 202 is returned along with "waitTimedOut": true and the Location status query URL (the waiting timeout only ends the current HTTP blocking and never cancels the distillation task already running in the background).

# Submit the troubleshooting trajectory of support ticket TK-20261022-001 for experience and skill distillation (mode=wait)
curl -fsS -X POST "https://appbuild.oceanbase.com/v1/memory/memories/distillations" \
  -H "X-Appwrite-Project: enterprise-ai-hub-210" \
  -H "X-Appwrite-Key: <APPBASE_API_KEY>" \
  -H "Idempotency-Key: distill-ticket-20261022-001" \
  -H "Content-Type: application/json" \
  --data '{
    "userId": "user_zhangwei_001",
    "agentId": "enterprise-support-agent",
    "scope": "user",
    "sessionId": "sess_ticket_TK_20261022_001",
    "runId": "run_9f8e7d6c5b4a",
    "mode": "wait",
    "verifiedSuccess": true,
    "messages": [
      {
        "role": "user",
        "content": "工单 TK-20261022-001:OceanBase Shared V2 表在写入 1024 维向量时报错 OB_ALLOCATE_MEMORY_FAILED (错误码 -4013),如何快速恢复?"
      },
      {
        "role": "tool",
        "content": "{\"tool\":\"sql_diagnostics\",\"output\":\"tenant_vector_memory_limit_percentage=10%, current usage=98.7%\"}"
      },
      {
        "role": "assistant",
        "content": "经排查为租户向量内存池配额不足。标准处置步骤:1. 查询 GV$OB_VECTOR_MEMORY_INFO 核实当前租户向量内存水位;2. 将 ob_vector_memory_limit_percentage 从 10 调升至 30;3. 重新触发增量向量索引构建并核验 schema.status=ready。已执行验证恢复成功。"
      }
    ]
  }' | jq .
  1. Check task progress in the distillation task list (e.g., dst_3e5b09f456c5d5c332507ec1dbe3eb6d and dst_adc2cfda21c940fb5d5e4a3263ae56f9 in the benchmark environment) and verify the output statistics:

    • created: the number of newly created Experiences or Skills;

    • merged: the number of items highly semantically overlapping with existing knowledge whose evidence was merged and versions advanced;

    • skipped: the number of items skipped because the model determined the conversation contained no reusable methodology;

    • blocked: the number of items not activated because validation failed or a governance policy blocked them.

2. Activation policy configuration (evolution.activation) and the shadow -> active lifecycle

Whether newly distilled knowledge immediately becomes active by default is controlled by the evolution namespace in Memory → Configuration → Basic Configuration:

Configuration field pathTypeDefault value / value rangeFunction and activation determination logic
evolution.enabledbooleantrueWhether to enable project-level Experiences and Skills evolution capability.
evolution.autoDistillEnabledbooleanfalseWhether to automatically trigger background knowledge distillation on regular session writes.
evolution.requireConfirmationbooleanfalseSelf-evolution secondary confirmation switch. When enabled, all newly added or merged knowledge candidates remain in shadow status and generate pending approval confirmation records (pending); they must be manually approved by a project administrator before transitioning to active.
evolution.allowSingleVerifiedUserActivationbooleantrueUnder the user scope, whether a single verified successful task is allowed to directly activate personal Experiences and Skills.
evolution.activation.minConfidencenumber0.0 ~ 1.0 (e.g., 0.8)The confidence score given during the model validation stage must be no lower than this threshold; otherwise the item stays in shadow status.
evolution.activation.minIndependentSessionsForAgentintegerPositive integer (e.g., 2 or 3)For agent scope knowledge, promotion from shadow to active is allowed only after it has been repeatedly distilled and confirmed in at least N distinct independent sessionId, preventing a single accidental conversation from polluting the global Agent skill library.
evolution.activation.requireVerifiedSuccessbooleantrueWhether the source task submitted for distillation must carry an explicit verification success marker (verifiedSuccess: true).

3. Version comparison, manual secondary confirmation, and rollback (版本与确认)

When self-evolution secondary confirmation (evolution.requireConfirmation: true) is enabled or an administrator proactively initiates a knowledge revision:

  1. In the Experiences and Skills tab, switch to the Versions and Confirmations sub-view and filter pending (pending confirmation) records.

  2. Expand the target confirmation record; the page fully displays:

    • candidate body and structured steps, source evidence list (Provenance, showing at most 100 entries plus the total count), reuse scope;

    • Version Diff: a field-by-field comparison of additions, modifications, or deletions between the candidate version (candidateRevisionId) and the currently effective online baseline (baseActiveRevisionId).

  3. Edit the candidate online (optional): if the administrator finds that candidate step wording needs minor adjustment, click edit candidate, modify the content, fill in the reason, and save. The backend starts an asynchronous edit validation task (editJob), revalidating all source governance policies and recomputing vector projections.

  4. Approve or Reject:

    • After validation is ready (canApprove: true, with no blocking reasons of projections_not_ready, verified_success_required, skill_dependency_not_approved, or source_set_changed), fill in the approval reason and click approve; the candidate version atomically switches to the current active version;

    • If reject is clicked, the candidate is discarded and the previously effective version remains online.

  5. Historical version rollback (Rollback): select any historical revision in the historical versions list that was previously approved and whose underlying sources are still valid, fill in the reason, and safely roll back.

4. Retrieving Experiences and Skills and provenance queries (agentbase-node-sdk@2.10.0)

Applications can retrieve matching active Experiences and Skills before agent reasoning via the dedicated search interface or unified retrieval (recall), and verify their source evidence at any time:

// Use the official server SDK: agentbase-node-sdk@2.10.0
import { Client, Memory } from 'agentbase-node-sdk';

const client = new Client()
    .setEndpoint('https://appbuild.oceanbase.com/v1')
    .setProject('enterprise-ai-hub-210')
    .setKey(process.env.APPBASE_API_KEY);

const memory = new Memory(client);

// 1. Semantic retrieval of activated troubleshooting experiences (Experiences)
const expResults = await memory.searchExperiences({
    userId: 'user_zhangwei_001',
    scope: 'user',
    query: 'OceanBase 向量索引构建内存不足 -4013 排障经验',
    limit: 5
});

// 2. Semantic retrieval of activated standardized skills (Skills)
const skillResults = await memory.searchSkills({
    userId: 'user_zhangwei_001',
    scope: 'user',
    query: 'AI 网关 429 限流与故障转移配置步骤',
    limit: 5
});

// 3. Query the immutable provenance source chain of a skill (Provenance Sources)
if (skillResults.items?.length > 0) {
    const provenance = await memory.listSkillSources({
        knowledgeId: skillResults.items[0].knowledgeId,
        limit: 20
    });
    console.log('支撑该技能的原始会话来源数:', provenance.total);
}

FAQ (Frequently Asked Questions and Troubleshooting)

1. Why does the distillation task status show succeeded, but no new records appear in the active Experiences and Skills list?

2. When clicking Approve in “Versions and Confirmations”, why is the button disabled with the prompt projections_not_ready or source_set_changed?

3. After turning off the “self-evolution secondary confirmation (evolution.requireConfirmation)” switch, will previously backlogged pending candidates be published automatically?