Docs
Contents

Content Moderation

Content Moderation is Agentbase's 2.10.0 multimodal compliance risk control engine for the write path, built on Alibaba Cloud Content Security (Green MultiModalGuard). On the critical paths of business data persistence and file storage, it performs automated content security review on string fields of database tables and image objects (image/*) uploaded to storage buckets, supporting synchronous pre-moderation (pre), asynchronous post-moderation (post), and mixed long/short text split moderation (hybrid), and provides platform-level/project-level dual-layer credential fallback, storage-side Fail-Closed cleanup, read isolation of unpublished content, and 30-day moderation call statistics with full audit logs.

Content Moderation page (reusing a registered screenshot)

Interface reference: reuses previously registered historical screenshots; demo data may differ from the current project; no new screenshots were taken this time.

Feature Overview

When an enterprise externally provides services such as customer support ticket submission, user community comments, or knowledge base image attachment uploads (e.g., the ticket table customer_tickets and multimedia storage bucket agent_multimedia_bucket in the production sample project 企业智能客服与知识中台 enterprise-ai-hub-210), preventing policy-violating content involving politics, pornography, violence and terrorism, prohibited items, or abuse from entering the production database and object storage is a rigid bottom line of enterprise compliance.

Agentbase Moderation frees business developers from manually writing third-party content security SDK call code in every endpoint, implementing a transparent gate directly on the underlying data-plane gateway and storage engine:

  1. Precise dual-domain coverage (database text + storage bucket images):

    • Database text moderation: for tables with moderation enabled, text content in all string-type fields is automatically extracted and submitted for review upon single-row insertion or update.

    • Storage image moderation: for buckets with moderation enabled, uploading an image file with MIME type image/* automatically generates a time-limited signed URL for review, and strictly enforces the Fail-Closed policy—once content is judged violating or the moderation service is abnormal, the uploaded image bytes are immediately physically deleted.

  2. Three-mode moderation strategies and Phase-Two asynchronous state machine (2.9.0 / 2.10.0):

    • pre (synchronous pre-mode): synchronous moderation is completed first; only after passing (pass) may data be persisted; violating content (block / review) is directly intercepted and rejected from writing.

    • post (asynchronous post-mode) and hybrid (mixed long/short text split mode): writes are persisted first and marked as under moderation (checking), with moderation completed asynchronously in the background; before formal publication (published), strict read isolation is enforced for non-creators, and supports escalation of violations to manual review (quarantined) or private withholding (private_hold) with periodic task self-healing.

  3. Dual-layer credential priority and model gateway decoupling: supports configuring the enterprise's own Alibaba Cloud Green AK/SK and endpoint within a project (encrypted and never echoed back); when not configured, it automatically falls back in pairs to the platform-level unified credentials; and it is completely physically isolated from the LLM Gateway's large model inference credentials.

Core feature viewConsole route pathCoverage capabilities and configuration itemsRequired permissions / role boundaries
Moderation home page (overview and credentials)/console/project-{region}-{project}/moderationView total call volume over the last 30 days (split by platform credentials vs. project credential calls); switch credential mode, configure Alibaba Cloud AK/SK, timeout (1–10s), log retention days (1–365 days), and single-image size limit (1–20 MB)View: any project member; modify project credentials: projects.write (owner/admin/developer/editor)
Moderation global logs page/console/project-{region}-{project}/moderation/logs · .../logs/{log}Search by resource type (table/bucket), resource ID, moderation result (pass/blocked/review/error), and time range; view hit risk labels (Labels), confidence, and suggested actionsReadable by any project member (logs only record restricted file names and ≤2000-character text previews; no raw image bytes are stored)
Table-level moderation configuration and logs/databases/database-{db}/table-{table}/moderationEnable text moderation for a specified table, choose the mode (pre / post / hybrid) and the violation-to-manual-review switch, and view the table's dedicated moderation logsDatabase table write permission
Bucket-level moderation configuration and logs/storage/bucket-{bucket}/moderationEnable image (image) moderation for a specified bucket and view the bucket's dedicated moderation logsStorage bucket write permission

Core Concepts and Architecture

The following diagram shows the credential resolution order, synchronous pre-moderation (pre) gate, storage-side Fail-Closed physical cleanup, and the asynchronous post-moderation (post / hybrid) state machine with read isolation architecture when database text writes and storage bucket image uploads pass through the Agentbase content moderation engine:

Content Moderation: workflow diagram, not a runtime screenshot

Flow diagram, not a runtime screenshot; read the relationships between steps or states along the arrows.

View the flow diagram in full size

1. Comparison table of the four single-run moderation verdicts and system actions

Regardless of the detailed classification returned by the underlying Alibaba Cloud Green MultiModalGuard call, Agentbase normalizes it into the following four standard verdicts and fully records the hit risk labels (Labels), confidence (Confidence), and model-suggested action (Suggestion: pass / review / block / mask) in the moderation logs:

Moderation verdict identifierMeaningpre Database write behavior in synchronous pre-modeStorage bucket (Storage image/*) upload behavior (Fail-Closed)
pass (pass)No violation risk detected or confidence below the interception thresholdAllow as normal: the data row is successfully written to the OceanBase tableKeep as normal: the image file completes upload and is persisted to the bucket
block (block)Explicitly hits high-risk political, pornographic, violent/terrorist, or prohibited termsReject write: the API directly errors and intercepts; the data is not persistedDelete and clean up: the request is rejected and the uploaded image file copy is immediately physically deleted
review (manual review)The moderation model judges suspected violation and recommends manual reviewTreated as block: under the conservative policy, suspected violating content is not allowed through (in Phase-Two asynchronous mode it enters the pending-review/isolated state)Delete and clean up: treated as block; uploaded images are immediately physically deleted
error (error)Invalid credentials, network timeout (> the configured seconds), or the image exceeds the single-image size limitConservative block: security first; unreviewed writes are not allowed through when the moderation service is abnormalDelete and clean up: the Fail-Closed mechanism forcibly deletes upload objects whose review was not completed

2. Moderation Phase Two: asynchronous post-moderation (post), mixed split (hybrid), and the document moderation state machine

In environments where the platform administrator has enabled the Phase-Two moderation switch (if the platform has not enabled the Phase-Two switch, resources configured as post or hybrid will automatically and safely fall back to synchronous pre-moderation equivalent to pre), table text moderation supports asynchronous state transitions:

Core attribute or state valueEnum values / default valuePrecise business semantics and read/write visibility rules
Moderation mode (mode)pre · post · hybrid• pre: synchronous check before saving;
• post: persist the write first, then submit for review asynchronously in the background;
• hybrid: automatically split according to the text length threshold configured for the platform—short text stays on the synchronous path for immediate return, while overly long text moves to the asynchronous path to avoid blocking the write response.
Violation-to-human-review switchrequireManualReviewOnViolation (default true)• When enabled (true): violation records enter the quarantined (Blocked) quarantine state awaiting review;
• When disabled (false): violation records enter the private_hold (Review Exception) state, held privately and visible only to the creator.
moderationStatus Status transitionschecking → pending → published / quarantined / private_hold / action_errorThe data table displays Chinese status badges inline for intuitive viewing (Reviewing / Pending Review / Blocked / Review Exception / Review Failed); for records stuck in checking due to occasional network jitter, the platform's built-in periodic inspection task automatically re-evaluates and self-heals according to the current configuration semantics, with no manual intervention required.
Read isolation and deferred events for unpublished contentFor all records not in published statusdata rows in checking, pending, quarantined, or private_hold states are completely hidden from non-creators (query lists and total counts are correspondingly reduced); only after the status transitions to published (Published) do they become visible to other authorized users, and Realtime, Functions events and Webhooks are also delivered only after the publication passes.

Roles, Entry Points, and Relationships

1. Resource-Level Configuration Status Semantics and Scope Boundaries

When enabling content moderation for a specific data table or storage bucket, the resource configuration object (/v1/moderation-configs) has the following three runtime states:

Clear scope boundaries:

2. Strict Distinction Between Content Moderation Credentials and Model Provider Credentials

During enterprise configuration, never confuse Moderation (content moderation credentials) with Models (LLM Gateway model inference credentials). The cloud services they connect to, the authentication protocols, and the configuration entry points are completely independent:

Comparison dimensionContent moderation credentials (Moderation Credentials)Model Provider credentials (LLM Gateway Providers)
Console configuration entryProject sidebar Moderation (Content Moderation) homepage configuration formProject sidebar Models (/llm-gateway) → Providers
Target cloud serviceAlibaba Cloud Content Moderation (Green MultiModalGuard)Large model inference services (e.g., Alibaba Cloud Model Studio DashScope dashscope-prod, OpenAI, Anthropic, etc.)
Required credential fieldsAlibaba Cloud Content Moderation HTTPS review endpoint (Endpoint) + AccessKey ID / AccessKey Secret (AK/SK pair)Model inference Base URL + API Key (e.g., <DASHSCOPE_API_KEY>) + model capability catalog
Isolation principleDedicated to data table write paths and storage bucket image upload risk control, must not be used for LLM chat or embeddingDedicated to Chat, Embedding, Rerank, and Vision inference, must not be used for content moderation

Feature Details and Operation Guide

1. Configuring Project-Level Content Moderation Credentials and Advanced Risk Control Parameters

  1. Log in to the Console and enter project 企业智能客服与知识中台 (enterprise-ai-hub-210), then click Moderation (Content Moderation) in the left sidebar (path /console/project-default-enterprise-ai-hub-210/moderation, as shown in Figure 1).

  2. Review the usage cards for the last 30 days at the top:

    • The top of the page displays three key statistics: total moderation calls over the last 30 days, platform credential call volume, and project custom credential call volume, along with a badge indicating the currently active credential source.

  3. Select the credential mode and configure parameters (corresponding to PATCH /v1/projects/{projectId}/moderation under the hood):

    • In the configuration form, you can switch between "Use platform credentials" and "Use custom project credentials".

    • Whole-pair parsing and fallback rule: When project custom credentials are configured, the project's own credentials take priority; when the project has no own credentials configured (or they are actively cleared and reset), the system automatically falls back to the whole pair of platform credentials, never cross-mixing the project's AK with the platform's Endpoint across levels.

    • If you select Use custom project credentials, fill in the following fields according to the table:

Configuration form fieldValue range / constraintsDefault value and production configuration recommendation
Endpoint (moderation service endpoint)A valid https:// Alibaba Cloud Content Moderation access addressFor example https://green-cip.cn-shanghai.aliyuncs.com (validated by the platform's SSRF guard; internal network addresses are forbidden).
AccessKey ID & SecretAn Alibaba Cloud RAM sub-account AK/SK with Green Content Moderation permissionsAfter saving, encrypted server-side storage, one-way masked in the UI without echo display; you can click clear at any time and re-enter a new key pair.
Timeout (moderation timeout)1 ~ 10 seconds (integer)Recommended to set to 3 ~ 5 seconds to prevent external network jitter from blocking database writes or image uploads for long periods.
Log retention (moderation log retention days)1 ~ 365 days (integer)Recommended to set to 90 or 180 days in production environments to meet compliance audit traceability requirements.
Max image size (per-image size limit)1 ~ 20 MB (integer)Sets the maximum size of a single image allowed for moderation (e.g., 10 MB); image uploads exceeding this limit will be directly intercepted and cleaned up under the Fail-Closed mechanism.

2. Enabling Moderation Switches Resource by Resource for Data Tables and Storage Buckets

After confirming the project-level credentials, you need to go to the specific data table to be protected (e.g., the customer support ticket table customer_tickets) or storage bucket (e.g., agent_multimedia_bucket) to enable resource-level moderation configuration (corresponding to the /v1/moderation-configs series of endpoints under the hood):

A. Enabling Text Moderation for Data Tables (Database Table)

  1. Click Databases in the left sidebar → enter the database enterprise_biz_db → click the data table customer_tickets → switch to the Moderation tab at the top of the table details (path .../table-customer_tickets/moderation).

  2. Turn on the table's text moderation switch (content type is fixed to text).

  3. Select the moderation mode:

    • The default selection is pre (synchronous check before save): any ticket submission containing violating words will be directly intercepted at write time.

    • If the platform has enabled Phase Two moderation capabilities, you can optionally select post (asynchronous post-hoc moderation) or hybrid (mixed routing for long and short texts), and check whether to enable human review on violation (requireManualReviewOnViolation).

  4. After saving, you can intuitively see the moderation status badge for each record in the table's Rows list, and view the moderation interception logs belonging only to that table under the table's Moderation tab.

B. Enabling Image Moderation for Storage Buckets

  1. Click Storage in the left sidebar → click to enter the target storage bucket (e.g., agent_multimedia_bucket) → switch to the Moderation tab at the top of the bucket details (path /storage/bucket-agent_multimedia_bucket/moderation).

  2. Turn on the bucket's image moderation switch (content type is fixed to image) and save.

  3. Verify Fail-Closed behavior: From then on, any image/* image uploaded to the bucket is submitted to MultiModalGuard for detection via a short-lived signed URL; if the image contains violating content, exceeds the configured Max image size (1–20 MB), or the moderation call times out, the system not only returns an upload failure error but also automatically deletes the already-stored file object, ensuring zero violation residue in the bucket.

3. Querying and Auditing Content Moderation Logs

When end users report "ticket submission rejected" or "image upload failed", administrators can locate the interception reason within 1 minutes via the global or resource-level moderation logs:

  1. Go to the project sidebar Moderation → Logs tab (path /console/project-default-enterprise-ai-hub-210/moderation/logs, corresponding to the API /v1/moderation-logs).

  2. Use the top combined filter bar to narrow the scope:

    • Resource type: select Table or Bucket.

    • Resource ID: enter e.g. customer_tickets or agent_multimedia_bucket.

    • Result: filter by pass (pass), blocked (blocked), review (blocked on review), or error (call exception).

    • Time range: frame the time period in which the interception occurred.

  3. Open any interception record to enter the log detail page (/moderation/logs/{log}) and verify the following key forensic fields:

    • Basic context: timestamp, associated resource type and ID, content type (text / image), underlying Provider, and Alibaba Cloud request ID (Request ID).

    • Privacy-safe content preview: text moderation displays a truncated text preview of no more than 2000 characters; image moderation only records a restricted file name preview, never storing raw image bytes, Base64 encoding, or full internal object paths in the logs, strictly protecting user data privacy.

    • Decision basis: the matched risk Labels, Confidence (confidence score), Suggestion (model-recommended action: pass / review / block / mask), and the specific error message.

FAQ and Troubleshooting

Q1: Why does uploading a PDF document or MP4 video after enabling image moderation on a storage bucket not trigger a moderation record, while uploading a 15 MB high-resolution PNG image fails outright and the file cannot be found in the bucket?

Cause and troubleshooting steps:

  1. PDF / videos do not trigger moderation: Agentbase Storage content moderation strictly targets only image files whose MIME type matches image/*; PDF documents, Markdown files, audio, and videos are not covered by image moderation and are uploaded via the normal file flow.

  2. 15 MB image upload fails and the file disappears: check the Moderation configuration form in the project for Max image size (range 1–20 MB). If the current limit is set to the default 10 MB, uploading a 15 MB image will trigger an over-limit rejection; per the storage-side Fail-Closed semantics, any image that is intercepted, over-limit, or fails moderation will have its already-uploaded object physically deleted immediately — no unreviewed copies are kept in the bucket. Simply raise the max image size to 20 MB (or compress the image) and re-upload.

Q2:After enabling Phase Two moderation post (asynchronous post-moderation) mode, why can user A see their just-submitted comment after refreshing, but user B cannot see it, and no Webhook is triggered either?

Cause and troubleshooting steps: This is the expected security behavior of the Phase Two moderation read isolation for unpublished content :

Q3:I have already configured the Alibaba Cloud DashScope API Key on the “Models (LLM Gateway)” page, so why does enabling Moderation content moderation still report invalid credentials?

Cause and troubleshooting steps: Content moderation credentials and model gateway credentials are two completely independent configurations that are never reused: