Retrieval and Source Tracing
Overview and use cases
The Retrieval and Source Tracing module covers the data-plane query loop for knowledge bases: single-KB retrieval (createSearch), cross-KB federated retrieval (createMultiSearch, one query spanning 1–10 knowledge bases with RRF fusion and global reranking returning TopK), grounded answer synthesis (answer: true), enhanced-index cooperative retrieval (joint retrieval over GraphRAG knowledge graph context and LLM Wiki pages), and protected source download for matched chunks (getDocumentSource, supporting HTTP Range range reads of 206 Partial Content).
In enterprise customer service and technical support scenarios (e.g., knowledge base kb_enterprise_support under demo project enterprise-ai-hub-210), server-side applications or agent runtimes use this module to perform hybrid (vector + full-text) hybrid retrieval: if the knowledge base has an enabled lexicon binding (e.g., lex_enterprise_synonyms), the server automatically applies synonym and expansion term rewriting to query before retrieval; after fusing text-embedding-v3 vector retrieval with full-text search, it calls qwen3.7-text-rerank to rerank and locates the source document section offset (locator).
Prerequisites
Client initialization and authentication: available only on the server and management planes; you must initialize the Server SDK
Client(setEndpoint,setProject,setKey), and the API Key must have at least theknowledgebases.readscope (source download viagetDocumentSourcelikewise only requiresknowledgebases.read, with no need to requestfiles.readseparately).Document and index status: the target knowledge base must contain at least
status=readydocuments whoseactiveGenerationis active; ifuseKnowledgeGraphoruseWikiis enabled, the corresponding capability must be enabled in the knowledge base index governance and have a published successful build artifact (see wiki.md).Model gateway readiness: the LLM Gateway Provider of the embedding model and the optional rerank model (e.g.,
qwen3.7-text-rerank) and the answer-synthesis Chat model (e.g.,qwen-plus) must be active and available.
Usage workflow
Initialize the client and services: construct
ClientwithsetEndpoint('https://appbuild-sit.oceanbase.com/v1'),setProject('enterprise-ai-hub-210'), andsetKey('<APPBASE_API_KEY>'), then instantiateKnowledgeBases(client).Single knowledge base hybrid retrieval (
createSearch): pass inknowledgeBaseId,query,searchMode: 'hybrid'(note the parameter name issearchModeand must not be written asmode),limit,similarityThreshold,vectorWeightandrerank: true. Optionally adddocumentTagorsubProductIdto narrow the document scope, or passanswer: trueto obtain a grounded answer synthesized from the retrieved chunks.Cross-knowledge-base joint retrieval (
createMultiSearch): pass inknowledgeBaseIds(an array of 1–10 unique knowledge base IDs),query,searchMode,limit,offset, and ametadataFilterkeyed by knowledge base ID (a specific library ID's configuration overrides the"*"fallback; the two are not merged). The server first retrieves candidate sets from each knowledge base and applies RRF (Reciprocal Rank Fusion); when reranking is enabled, a single global Rerank is then performed on the merged pool.Parse retrieved hits and trace back to read the source (
getDocumentSource): iterate over the chunks in the responseresults[](includingcontent,score,sourceId,sourceandlocator). For ordinary document chunks, usesourceIdasdocumentIdto callgetDocumentSourceand fetch the original file binary byte stream (supportsRangeheader partial downloads), for front-end citation previews or compliance audit archiving.
Key parameters and returns
Retrieval request parameters (createSearch / createMultiSearch)
| Parameter | Applicable method | Type | Required | Default / constraint | Description |
|---|---|---|---|---|---|
knowledgeBaseId | createSearch | string | Yes | Non-empty string | Target single knowledge base ID (e.g. kb_enterprise_support). |
knowledgeBaseIds | createMultiSearch | string[] | Yes | 1–10 unique IDs | List of target knowledge base IDs for cross-knowledge-base retrieval. |
query | Both | string | Yes | Non-empty query text | The user's natural language question or retrieval keywords. |
searchMode | Both | string | No | Default hybrid; enum: vector / fulltext / hybrid | Retrieval mode (must not be miswritten as mode). |
limit | Both | integer | No | Default 10 (range 1–100) | Maximum number of chunks returned (global TopK). |
offset | createMultiSearch | integer | No | Default 0 (0–5000) | Pagination offset after cross-knowledge-base fusion ranking. |
recallLimit | Both | integer | No | Default min(100, max(limit * 4, 20)) when reranking is enabled; explicitly 1–100 and not less than limit | Upper limit of initial candidate retrieval before reranking. |
similarityThreshold | Both | number | No | 0–1 (defaults to the KB retrievalDefaults) | Applies to the vector retrieval branch of vector and hybrid modes; does not filter the full-text branch. |
vectorWeight | Both | number | No | 0–1 (commonly 0.7) | Vector score weight in hybrid mode; the full-text weight is 1 - vectorWeight. |
rerank | Both | boolean | No | Default false | Whether to perform reranking via the LLM Gateway. |
rerankModel | Both | string | No | e.g. qwen3.7-text-rerank | Explicitly specify the reranking model; if omitted and rerank=true, the platform selects automatically. |
rerankProviderRef | Both | string | No | project:<id> / trial:<id> | Provider reference that pins the reranking model. |
documentTag | Both | string | No | {subProductId}_{sectionCode}_{fileKey} | Added in 2.10.0: exact match by document locator; the first segment contains only letters or digits, the second segment only digits, and either segment may be empty; the final segment must be a 32-character hexadecimal string. |
subProductId | Both | string | No | 1–64 letters or digits | Added in 2.10.0: filter by sub-product identifier (AND relation with documentTag). |
metadataFilter | Both | object | No | JSON filter object | For single knowledge base, pass logic and conditions (each item contains name / comparison / value); for cross-knowledge-base, key by knowledge base ID, with "*" as the fallback for libraries without individual configuration. |
useKnowledgeGraph | createSearch | boolean | No | Default false | Whether to jointly retrieve GraphRAG entity, relationship and community report context. |
useWiki | createSearch | boolean | No | Default false | Whether to jointly retrieve published LLM Wiki page chunks. |
allowDegraded | createSearch | boolean | No | Default true | Controls degradation-eligible errors of augmented retrieval; see capabilities / degradedCapabilities for statuses; does not control reranking model failures. |
answer | createSearch | boolean | No | Default false | Added in 2.9.0: whether to call a Chat model on the retrieved Top evidence (up to 12 items) to generate a grounded answer (≤1024 tokens). |
Retrieval response structure
Single knowledge base returns KnowledgeBaseSearchResultList; cross-knowledge-base returns KnowledgeBaseMultiSearchResultList. Do not apply single-knowledge-base augmentation parameters or response fields directly to cross-knowledge-base calls.
| Field | Applicable method | Description |
|---|---|---|
total | Both | Number of results returned this time. |
results | Both | Results array; each item uses $id, sourceId, content, score, source, locator, metadata. Cross-knowledge-base items additionally carry knowledgeBaseId, identifying the knowledge base the result belongs to. |
reranked | whether both | completed reranking successfully; false when automatic model selection fails and falls back. |
resolvedVersionId | createSearch | Knowledge base version resolved for this request; null on legacy read paths. |
capabilities | createSearch | RAPTOR / GraphRAG / Wiki status, result generation, elapsed time and sanitized errors. |
degradedCapabilities | createSearch | Augmented capabilities where degradation explicitly occurred. |
answer | createSearch | The { status, text, model, latencyMs, error } when answer: true was requested; otherwise null. |
Single-knowledge-base results also distinguish ordinary chunks from augmented results via indexSource, resultType and resultGeneration. source is the source description object and locator is the location object; fields depend on the respective result type.
The answer model reads indexingDefaults.chatModel / chatProviderRef. answer.status is not_configured (no model configured), no_evidence (no valid evidence), succeeded (generated text) or failed (call error or empty text). Even on a successful answer, verify the sources in results.
Errors and boundaries
Parameter misuse (
400 general_argument_invalid): passingsearchModeinstead ofmode, orcreateMultiSearchwith aknowledgeBaseIdsoutside the 1–10 range or containing duplicate knowledge base IDs.Distinguishing reranking model failure semantics: when
rerankModelorrerankProviderRefis explicitly passed, upstream call errors return an error; when neither is specified and only the automatic reranking ofrerank: trueis enabled, it may fall back to the original ordering and returnreranked: false.allowDegradedonly controls degradation-eligible errors of single-knowledge-base augmented retrieval.Augmented index not ready (
409/ degradation): ifuseKnowledgeGraph: trueoruseWiki: trueis requested withallowDegraded: falseand the corresponding index has not yet been built, a conflict error is returned; in production, it is recommended to keepallowDegraded: trueand checkcapabilitiesanddegradedCapabilities.Handle binary data carefully when reading source files:
getDocumentSourcedirectly returns the original file byte stream (not a JSON wrapper); SDK callers should receive it perArrayBuffer/byte[]/bytesand write to disk or forward as a stream, and must not callJSON.parse().
SDK Category Differences
| SDK category | Coverage | Authentication method | Typical scenarios |
|---|---|---|---|
| Server SDK (Node.js / Java / Python / Go) | Full support createSearch, createMultiSearch, getDocumentSource | Project API Key (knowledgebases.read) | Backend RAG services, agent tool retrieval, source-traceability download |
| Console SDK (Node.js / Java / Python / Go) | The Java / Python / Go Console SDKs provide the full KnowledgeBases retrieval surface; the JS Console SDK uses client.call for calls | Management-plane session / short-lived JWT / API Key | Console retrieval test bench, automated inspection, and evaluation |
| Client / Web SDK (Web) | Does not support knowledge base retrieval and source file download (only includes Yuque OAuth helper methods) | User Session | Browsers must invoke retrieval APIs via the business backend or a Function proxy |
Examples by language
Each card allows switching languages (synchronized across the full page).
Retrieval (single knowledge base / cross knowledge base)
createSearch single knowledge base; createMultiSearch spans 1–10 knowledge bases (cross-knowledge-base first merges via RRF, then performs one rerank on the merged pool); result chunks carry content / source / locator. The Web surface has no retrieval capability.
- Node.js (Server)
- Java
- Python
- Go
- REST (curl)
const found = await kb.createSearch({
knowledgeBaseId: base.$id,
query: '退款政策是什么?',
searchMode: 'hybrid', // Parameter name is searchMode; vector / fulltext / hybrid
limit: 10,
rerank: true // Expand retrieval first, then rerank via the Gateway; automatic selection falls back and returns reranked: false on failure
});
// Cross knowledge base: kb.createMultiSearch({ knowledgeBaseIds: [...], query, limit }) (1–10 knowledge bases, global TopK)var found = kb.createSearch(base.getId(), "退款政策是什么?",
10L, // limit (note: located before searchMode)
io.agentbase.enums.SearchMode.HYBRID, // vector / fulltext / hybrid
true, // rerank: first expands retrieval, then reranks via the Gateway; automatically falls back on selection failure and returns reranked: false
null, null, null, null, null); // recallLimit / rerankModel / rerankProviderRef / similarityThreshold / vectorWeight
// Cross knowledge base: kb.createMultiSearch(...) (1–10 knowledge bases, global TopK)found = kb.create_search(
knowledge_base_id=base['$id'],
query='退款政策是什么?',
search_mode='hybrid', # The parameter name is search_mode; vector / fulltext / hybrid
limit=10,
rerank=True) # Expand retrieval first, then rerank via the Gateway; automatic selection falls back and returns reranked: false on failure
# Cross-knowledge-base: kb.create_multi_search(knowledge_base_ids=[...], query=..., limit=...) (1–10 knowledge bases, global TopK)found, err := kb.CreateSearch(base.Id, "退款政策是什么?",
kb.WithCreateSearchSearchMode("hybrid"), // vector / fulltext / hybrid
kb.WithCreateSearchLimit(10),
kb.WithCreateSearchRerank(true)) // Falls back on auto-selection failure and returns reranked: false
// Cross knowledge base: kb.CreateMultiSearch(...) (1–10 knowledge bases, global TopK)curl -sS -X POST "$APPBASE_ENDPOINT/knowledgebases/kb_product_docs/search" \
-H 'Content-Type: application/json' \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" -H "X-Appwrite-Key: $APPBASE_API_KEY" \
-d '{"query":"早餐机首次使用要注意什么","limit":5,"searchMode":"hybrid","similarityThreshold":0.5,"vectorWeight":0.7}'
# Cross-knowledge-base: POST /knowledgebases/search, add "knowledgeBaseIds":["kb_a","kb_b"] to the body (1–10 knowledge bases, no duplicates)Traceability and reading source files
Retrieved chunks carry content, source (protected source URI) and locator (section / text offsets); getDocumentSource reads back the original file (binary stream, supports Range partial responses 206), requiring only knowledgebases.read and not additionally files.read. The Web surface has no such capability.
- Node.js (Server)
- Java
- Python
- Go
- REST (curl)
// The sourceId of a hit chunk is the original document ID; read back the original for manual verification or archiving
const raw = await kb.getDocumentSource({ knowledgeBaseId: base.$id, documentId: '<documentId>' });byte[] raw = kb.getDocumentSource(base.getId(), "<documentId>"); // Binary stream, do not parse as JSON; only knowledgebases.read requiredraw: bytes = kb.get_document_source(knowledge_base_id=base['$id'], document_id='<documentId>')
# Binary stream, do not parse as JSON; only knowledgebases.read requiredraw, err := kb.GetDocumentSource(base.Id, "<documentId>") // *[]byte: raw file binary, do not parse as JSONcurl -sS "$APPBASE_ENDPOINT/knowledgebases/kb_product_docs/documents/<documentId>/source" \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" -H "X-Appwrite-Key: $APPBASE_API_KEY" \
-o source.bin # Binary stream, do not parse as JSON; returns segments 206 when a Range header is presentAdvanced retrieval (tag filtering, answer synthesis and cross-knowledge-base metadata filtering)
The ob / aigw and document locators in the examples must be replaced with actual document values; subProductId does not accept hyphens, underscores or spaces, and documentTag must not be filled with a title or hierarchy tag.
Single knowledge base supports answer: true grounded answer synthesis, documentTag / subProductId document scope filtering, and per-library metadataFilter and offset pagination of createMultiSearch.
- Node.js (Server)
- Java
- Python
- Go
- REST (curl)
// 1. Single-library advanced retrieval: filter by subProductId / documentTag + GraphRAG/Wiki enhancement + answer synthesis
const singleResult = await kb.createSearch({
knowledgeBaseId: 'kb_enterprise_support',
query: 'OceanBase 分布式事务锁冲突与向量索引构建超时的排障步骤是什么?',
searchMode: 'hybrid',
limit: 6,
similarityThreshold: 0.45,
vectorWeight: 0.7,
rerank: true,
rerankModel: 'qwen3.7-text-rerank',
subProductId: 'ob',
documentTag: 'ob_12_4a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d',
useKnowledgeGraph: true,
useWiki: true,
allowDegraded: true,
answer: true,
});
console.log('合成答案:', singleResult.answer?.text);
// 2. Cross-database joint retrieval: pagination with metadataFilter + offset specified per database
const multiResult = await kb.createMultiSearch({
knowledgeBaseIds: ['kb_enterprise_support', 'kb_product_docs'],
query: 'AI 网关语义缓存与限流计费策略怎么配置?',
searchMode: 'hybrid',
limit: 10,
offset: 0,
rerank: true,
subProductId: 'aigw',
metadataFilter: {
'*': { logic: 'and', conditions: [{ name: 'env', comparison: 'is', value: 'production' }] },
kb_enterprise_support: { logic: 'and', conditions: [{ name: 'category', comparison: 'is', value: 'sop' }] },
},
});// Java Server SDK passes documentTag / subProductId / answer via client.call or createSearch with extension parameters; cross-library createMultiSearch does not support answer
java.util.Map<String, Object> searchBody = new java.util.HashMap<>();
searchBody.put("query", "OceanBase 分布式事务锁冲突与向量索引构建超时的排障步骤是什么?");
searchBody.put("searchMode", "hybrid");
searchBody.put("limit", 6);
searchBody.put("similarityThreshold", 0.45);
searchBody.put("vectorWeight", 0.7);
searchBody.put("rerank", true);
searchBody.put("rerankModel", "qwen3.7-text-rerank");
searchBody.put("subProductId", "ob");
searchBody.put("documentTag", "ob_12_4a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d");
searchBody.put("useKnowledgeGraph", true);
searchBody.put("useWiki", true);
searchBody.put("allowDegraded", true);
searchBody.put("answer", true);
// POST /knowledgebases/kb_enterprise_support/search# Single-knowledge-index retrieval: enable answer synthesis (answer=True) and document tag filtering (sub_product_id / document_tag)
single_result = kb.create_search(
knowledge_base_id='kb_enterprise_support',
query='OceanBase 分布式事务锁冲突与向量索引构建超时的排障步骤是什么?',
search_mode='hybrid',
limit=6,
similarity_threshold=0.45,
vector_weight=0.7,
rerank=True,
rerank_model='qwen3.7-text-rerank',
sub_product_id='ob',
document_tag='ob_12_4a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d',
use_knowledge_graph=True,
use_wiki=True,
allow_degraded=True,
answer=True,
)
# Cross-knowledge-base federated retrieval: metadata_filter + offset pagination per knowledge base
multi_result = kb.create_multi_search(
knowledge_base_ids=['kb_enterprise_support', 'kb_product_docs'],
query='AI 网关语义缓存与限流计费策略怎么配置?',
search_mode='hybrid',
limit=10,
offset=0,
rerank=True,
sub_product_id='aigw',
metadata_filter={
'*': {'logic': 'and', 'conditions': [{'name': 'env', 'comparison': 'is', 'value': 'production'}]},
'kb_enterprise_support': {'logic': 'and', 'conditions': [{'name': 'category', 'comparison': 'is', 'value': 'sop'}]},
},
)singleResult, err := kb.CreateSearch("kb_enterprise_support",
"OceanBase 分布式事务锁冲突与向量索引构建超时的排障步骤是什么?",
kb.WithCreateSearchSearchMode("hybrid"),
kb.WithCreateSearchLimit(6),
kb.WithCreateSearchSimilarityThreshold(0.45),
kb.WithCreateSearchVectorWeight(0.7),
kb.WithCreateSearchRerank(true),
kb.WithCreateSearchRerankModel("qwen3.7-text-rerank"),
kb.WithCreateSearchSubProductId("ob"),
kb.WithCreateSearchDocumentTag("ob_12_4a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d"),
kb.WithCreateSearchUseKnowledgeGraph(true),
kb.WithCreateSearchUseWiki(true),
kb.WithCreateSearchAllowDegraded(true),
kb.WithCreateSearchAnswer(true),
)
_ = singleResult
_ = err# 1. Advanced retrieval in a single knowledge base (with tag filtering, enhanced index and answer synthesis)
curl -sS -X POST "$APPBASE_ENDPOINT/knowledgebases/kb_enterprise_support/search" \
-H 'Content-Type: application/json' \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" \
-H "X-Appwrite-Key: $APPBASE_API_KEY" \
-d '{
"query": "OceanBase 分布式事务锁冲突与向量索引构建超时的排障步骤是什么?",
"searchMode": "hybrid",
"limit": 6,
"similarityThreshold": 0.45,
"vectorWeight": 0.7,
"rerank": true,
"rerankModel": "qwen3.7-text-rerank",
"subProductId": "ob",
"documentTag": "ob_12_4a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d",
"useKnowledgeGraph": true,
"useWiki": true,
"allowDegraded": true,
"answer": true
}'
# 2. Cross-knowledge-base federated retrieval (with metadataFilter and offset pagination)
curl -sS -X POST "$APPBASE_ENDPOINT/knowledgebases/search" \
-H 'Content-Type: application/json' \
-H "X-Appwrite-Project: $APPBASE_PROJECT_ID" \
-H "X-Appwrite-Key: $APPBASE_API_KEY" \
-d '{
"knowledgeBaseIds": ["kb_enterprise_support", "kb_product_docs"],
"query": "AI 网关语义缓存与限流计费策略怎么配置?",
"searchMode": "hybrid",
"limit": 10,
"offset": 0,
"rerank": true,
"subProductId": "aigw",
"metadataFilter": {
"*": {"logic": "and", "conditions": [{"name": "env", "comparison": "is", "value": "production"}]},
"kb_enterprise_support": {"logic": "and", "conditions": [{"name": "category", "comparison": "is", "value": "sop"}]}
}
}'Example Guide
The model names, project IDs and query contents in this document are configuration examples; you still need to complete real model call verification in your own environment.
JavaScript (full Node.js RAG ingestion and retrieval pipeline): examples/javascript.md
Python (Server SDK knowledge base retrieval and traceability script): examples/python.md
Java (Server / Console SDK call examples): examples/java.md
Go (Server SDK retrieval and options pattern): examples/go.md