Experience and Skill Distillation Evolution
Feature overview and core scenarios
Experience and skill evolution (Memory Evolution; the Console tab is Evolution) is the core capability by which Agentbase Memory enables agents to “self-learn from real-world practice and accumulate knowledge.” In enterprise customer service and ops troubleshooting scenarios, agents perform large volumes of multi-turn conversations and tool executions every day. Ordinary memories focus on recording “objective facts and preferences about the user,” whereas experience and skill distillation extracts cross-session reusable troubleshooting experience (Experience) and standardized operational skills (Skill) from real task execution traces and troubleshooting conversation evidence.
In the example project 企业智能客服与知识中台 (enterprise-ai-hub-210) of Agentbase 企业智能中台 (69c6d1aa00062e6bb60), the system completed two typical example distillation tasks (dst_3e5b09f456c5d5c332507ec1dbe3eb6d and dst_adc2cfda21c940fb5d5e4a3263ae56f9) via the distillation pipeline (calling dashscope-prod's qwen-plus and text-embedding-v3):
Distilling the successful troubleshooting traces for ticket
TK-20261022-001(OceanBase vector index build timeout and out-of-memory) andTK-20261022-002(AI gateway rate-limit 429 failover) into structured experience (Experience) and multi-step skills (Skill);Supporting candidate review and activation governance, confidence- and source-count-based automatic activation policies (
evolution.activation), and, for high-compliance scenarios, self-evolution secondary human confirmation (evolution.requireConfirmation), ensuring that every experience and skill entering online retrieval carries complete, traceable provenance.
Core Concepts and Architecture
Experience and skill distillation extracts candidate knowledge from conversations or tool traces and checks it for duplication and validity. Verification alone does not guarantee activation: user-scoped knowledge is also subject to publication policy and can remain in shadow. Check the actual status.
Core Differences and Attribute Table of Experience vs. Skill
| Comparison dimension | Experience | Skill |
|---|---|---|
| Positioning and form | Reusable situational insights and pitfall-avoidance experience: records “what problem was encountered in which scenario, which approaches worked, and which approaches lead to failure.” | Reusable standardized operating steps (SOP): records “to achieve a specific goal, which checks, commands, or tool calls should be executed in what order.” |
| Core structured fields | The knowledge object contains knowledgeId, title, version, and experience; experience fields include situation, decision, action, evidence, outcome, and skillItemIds. | The knowledge object contains knowledgeId, title, version, and skill; skill fields include inputs, applicability, preconditions, procedure, and expectedOutcome. |
Scope of ownership (scope) | Supports user (user level), agent (agent level, requires specifying agentId), and project (project global level). | Supports user, agent, project; agent-level skills require validation across multiple active sources (minIndependentSessionsForAgent) before automatic activation. |
| Lifecycle states | Knowledge statuses include candidate, shadow, user-active, agent-active, quarantine, and deprecated; user-active and agent-active are active. A forget marker controls retrieval and is not a separate status enum. | The same status set applies to skills; skill_dependency_not_approved can block an experience that depends on an unapproved skill version. |
| Retrieval and provenance APIs | GET /v1/memory/memories/experiences, POST .../experiences/search, GET .../experiences/{knowledgeId}/sources | GET /v1/memory/memories/skills, POST .../skills/search, GET .../skills/{knowledgeId}/sources |
Feature Entrypoints and Permission Matrix
Console Entry Points and API Permission Matrix
The Console entry point is located at: Project enterprise-ai-hub-210 → Memory → Evolution tab.
| Function module | Console operation entry | REST API path | Required permissions and authentication requirements |
|---|---|---|---|
| Submit distillation task | Manual distillation area → Submit distillation | POST /v1/memory/memories/distillations | memories.write |
| Query and replay distillation tasks | Distillation task list | GET /v1/memory/memories/distillations, GET .../{distillationId}, POST .../{distillationId}/replay | memories.read / memories.write |
| Query and retrieve experiences/skills | Experience / Skill lists and search box | GET/POST /v1/memory/memories/experiences(/search), GET/POST /v1/memory/memories/skills(/search) | memories.read |
| Query knowledge source evidence | Expand experience/skill details → Source evidence | GET /v1/memory/memories/experiences/{knowledgeId}/sources, /skills/{knowledgeId}/sources | memories.read |
| Version comparison and manual secondary confirmation | Reviews sub-tab | GET/PATCH /v1/memory/memories/knowledge/reviews/{reviewId}, POST .../approve, POST .../reject, POST .../{knowledgeId}/rollback | Reads require memories.read; editing, approving, rejecting, and rollback require memories.write and a real user principal with project management permissions (Console session or administrator JWT). A subject-less API Key cannot replace an approver. |
| Stop reuse (forget knowledge) | End of Experience/Skill row Forget button | DELETE /v1/memory/memories/experiences/{knowledgeId}, /skills/{knowledgeId} | memories.delete |
Feature Details and Operation Guide
1. Select the ownership scope and submit execution evidence for distillation (POST /v1/memory/memories/distillations)
Log in to the Agentbase Console, enter project
企业智能客服与知识中台(enterprise-ai-hub-210), and open the Memory → Evolution tab. Before submission, check and enableevolution.enabledin Memory → Configuration → Basic Configuration; the platform default isfalse, though an environment may override it. Automatic distillation also requiresevolution.autoDistillEnabled.Set the knowledge scope in the ownership scope bar at the top of the page:
Select a scope (
user,agent, orproject);Enter the target user ID (e.g.,
user_zhangwei_001oruser_lina_002); if theagentscope is selected, you must also fill in the targetagentId(e.g.,enterprise-support-agent), then click apply scope.
Click Manual distillation. In the dialog, enter desensitized customer support troubleshooting conversation or tool invocation evidence (you may use the
user:,assistant:,system:,tool:prefixes to mark roles), select the default pipeline (or specify a published experience and skill pipeline), and click Confirm.API invocation and
mode=async | waitdual-mode explanation: When submitting distillation via API, two waiting semantics are supported:mode: "async": returns HTTP 202 after the first persistence and enqueue; an idempotent replay may return HTTP 200. The response includesdistillationIdand the currentstatus(queuedis typical on first enqueue; replay can return another status);mode: "wait": the server holds the connection and waits for up to 20 seconds. If the task reaches a terminal state (succeeded,partial_failed,failed, ordead) within the 20-second window, HTTP 200 returns a receipt with task identifiers and status; usedistillationIdto query task details and actual results; if more than 20 seconds pass and it is stillqueuedorrunning, HTTP 202 is returned along with"waitTimedOut": trueand theLocationstatus query URL (the waiting timeout only ends the current HTTP blocking and never cancels the distillation task already running in the background).
Set a unique DISTILLATION_IDEMPOTENCY_KEY for each new task; reuse the original key and identical request content only when retrying the same submission.
(
set +x
set -o pipefail
: "${APPBASE_API_KEY:?Set APPBASE_API_KEY}"
: "${DISTILLATION_IDEMPOTENCY_KEY:?Set a unique key for this new task}"
: "${APPBASE_ENDPOINT:?Set APPBASE_ENDPOINT to the target API endpoint including /v1}"
APPBASE_ENDPOINT="${APPBASE_ENDPOINT%/}"
: "${APPBASE_PROJECT_ID:?Set APPBASE_PROJECT_ID for the target environment}"
# Submit the troubleshooting trajectory of support ticket TK-20261022-001 for experience and skill distillation (mode=wait)
builtin printf 'X-Appwrite-Key: %s\n' "$APPBASE_API_KEY" | curl --disable --header @- -fsS -X POST "${APPBASE_ENDPOINT}/memory/memories/distillations" \
-H "X-Appwrite-Project: ${APPBASE_PROJECT_ID}" \
-H "Idempotency-Key: ${DISTILLATION_IDEMPOTENCY_KEY}" \
-H "Content-Type: application/json" \
--data '{
"userId": "user_zhangwei_001",
"scope": "user",
"sessionId": "sess_ticket_TK_20261022_001",
"runId": "run_9f8e7d6c5b4a",
"mode": "wait",
"messages": [
{
"role": "user",
"content": "工单 TK-20261022-001:OceanBase Shared V2 表在写入 1024 维向量时报错 OB_ALLOCATE_MEMORY_FAILED (错误码 -4013),如何快速恢复?"
},
{
"role": "tool",
"content": "{\"tool\":\"check_ticket\",\"output\":\"示例工单已记录,等待排查\"}"
},
{
"role": "assistant",
"content": "这是一条合成示例:已记录工单,后续应结合数据库版本、日志和资源指标定位原因,并依据对应版本的官方文档评估处理方案。"
}
]
}' | jq .
)Check task progress in the distillation task list (e.g.,
dst_3e5b09f456c5d5c332507ec1dbe3eb6danddst_adc2cfda21c940fb5d5e4a3263ae56f9in the benchmark environment) and verify the output statistics:created: the number of newly created Experiences or Skills;merged: the number of items highly semantically overlapping with existing knowledge whose evidence was merged and versions advanced;skipped: the number of items skipped because the model determined the conversation contained no reusable methodology;blocked: the number of items not activated because validation failed or a governance policy blocked them.
2. Activation policy configuration (evolution.activation) and the knowledge lifecycle
Whether newly distilled knowledge becomes active depends on validation, scope, sources, and configuration; the relevant configuration is in the evolution namespace in Memory → Configuration → Basic Configuration:
| Configuration field path | Type | Default value / value range | Function and activation determination logic |
|---|---|---|---|
evolution.enabled | boolean | false (platform default) | Whether project-level Experiences and Skills evolution is enabled; an environment can override the default, and distillation cannot be submitted while disabled. |
evolution.autoDistillEnabled | boolean | false | Whether to automatically trigger background knowledge distillation on regular session writes. |
evolution.requireConfirmation | boolean | false | Self-evolution secondary confirmation switch. When enabled, all newly added or merged knowledge candidates remain in shadow status and generate pending approval confirmation records (pending); they must be manually approved by a project administrator before activation. |
evolution.allowSingleVerifiedUserActivation | boolean | true | Under the user scope, whether a single verified successful task is allowed to directly activate personal Experiences and Skills. |
evolution.activation.minConfidence | number | 0.0 ~ 1.0 (e.g., 0.8) | The confidence score given during the model validation stage must be no lower than this threshold; otherwise the item stays in shadow status. |
evolution.activation.minIndependentSessionsForAgent | integer | Positive integer (e.g., 2 or 3) | For agent scope knowledge, promotion from shadow to agent-active depends on the number of distinct active source identifiers. Distinct sources do not guarantee distinct sessionId values; multiple inline submissions from one session may create multiple sources. |
evolution.activation.requireVerifiedSuccess | boolean | true | Whether verified-success evidence from a trusted source is required. A declaration in the public request body is not treated as trusted verification; use the project configuration and review outcome to determine activation. |
3. Version comparison, manual secondary confirmation, and rollback (Reviews)
When self-evolution secondary confirmation (evolution.requireConfirmation: true) is enabled or an administrator proactively initiates a knowledge revision:
In the Evolution tab, switch to the Reviews sub-view and filter
pending(pending confirmation) records.Expand the target confirmation record; the page fully displays:
candidate body and structured steps, source evidence list (Provenance, showing at most 100 entries plus the total count), reuse scope;
Version Diff: a field-by-field comparison of additions, modifications, or deletions between the candidate version (
candidateRevisionId) and the currently effective online baseline (baseActiveRevisionId).
Edit the candidate online (optional): if the administrator finds that candidate step wording needs minor adjustment, click edit candidate, modify the content, fill in the reason, and save. The backend starts an asynchronous edit validation task (
editJob), revalidating all source governance policies and recomputing vector projections.Approve or Reject:
After validation is ready (
canApprove: true, with no blocking reasons ofprojections_not_ready,verified_success_required,skill_dependency_not_approved, orsource_set_changed), fill in the approval reason and click approve; the candidate version atomically switches to the current effective version;If reject is clicked, the candidate is discarded and the previously effective version remains online.
Historical version rollback (Rollback): select any historical revision in the historical versions list that was previously approved and whose underlying sources are still valid, fill in the reason, and safely roll back.
4. Retrieving Experiences and Skills and provenance queries (agentbase-node-sdk@2.12.0)
Applications can retrieve matching active Experiences and Skills (user-active / agent-active) before agent reasoning via the dedicated search interface or unified retrieval (recall), and verify their source evidence at any time:
Before running, replace <APPBASE_ENDPOINT_WITH_V1> and <APPBASE_PROJECT_ID> with the confirmed target API endpoint and project ID. Until replaced, the SDK rejects the invalid endpoint before making a request.
// Use the official server SDK: agentbase-node-sdk@2.12.0
import { Client, Memory } from 'agentbase-node-sdk';
const client = new Client()
.setEndpoint('<APPBASE_ENDPOINT_WITH_V1>')
.setProject('<APPBASE_PROJECT_ID>')
.setKey(process.env.APPBASE_API_KEY);
const memory = new Memory(client);
// 1. Semantic retrieval of activated troubleshooting experiences (Experiences)
const expResults = await memory.searchExperiences({
userId: 'user_zhangwei_001',
scope: 'user',
query: 'OceanBase 向量索引构建内存不足 -4013 排障经验',
limit: 5
});
// 2. Semantic retrieval of activated standardized skills (Skills)
const skillResults = await memory.searchSkills({
userId: 'user_zhangwei_001',
scope: 'user',
query: 'AI 网关 429 限流与故障转移配置步骤',
limit: 5
});
// 3. Query the immutable provenance source chain of a skill (Provenance Sources)
if (skillResults.items?.length > 0) {
const provenance = await memory.listSkillSources({
knowledgeId: skillResults.items[0].knowledge.knowledgeId,
userId: 'user_zhangwei_001',
scope: 'user',
limit: 20
});
console.log('支撑该技能的原始会话来源数:', provenance.total);
}FAQ (Frequently Asked Questions and Troubleshooting)
1. Why does the distillation task status show succeeded, but no new records appear in the active Experiences and Skills list?
Cause analysis: there may be three reasons: ① the conversation content is too brief or contains only greetings and conclusions, and the model determines it has no general reuse value (
skipped); ② the project has enabledevolution.requireConfirmation: true, or the knowledge belongs to theagentscope and has not yet accumulatedminIndependentSessionsForAgentdistinct active sources, so the generated record is inshadow(candidate status); ③ the extracted confidence is belowevolution.activation.minConfidence.Solution: switch the status in the list filter to
shadow, or open the Reviews sub-tab to check. If it is pending confirmation, the administrator verifies the Diff and clicks approve to activate and go live.
2. When clicking Approve in “Reviews”, why is the button disabled with the prompt projections_not_ready or source_set_changed?
Root cause analysis:
projections_not_ready: the candidate version (or the newly manually edited revision) is computing vector projections using the project instance configuration in the background and is not yet ready in storage;source_set_changed: between opening the detail page and clicking Approve, new session evidence was merged into the same knowledge item in the background, advancingcandidateRevisionId, or a user deleted an underlying source, changing the evidence set.
Solution: refresh the current confirmation record detail page. If it is index preparation, waiting a few seconds and refreshing suffices; if it is version advancement, re-review the latest Diff and source evidence before submitting the approval.
3. After turning off the “self-evolution secondary confirmation (evolution.requireConfirmation)” switch, will previously backlogged pending candidates be published automatically?
Cause analysis: for production safety boundary reasons, turning off the
requireConfirmationswitchonly restores automatic activation rules for distillation tasks newly produced afterward; it will never silently batch-publish pending confirmation records that historically remained inpendingstatus.Solution: for pending confirmation records generated before the switch was turned off, the administrator must still review each item in the Reviews page and click approve or reject.