Docs
Contents

Content Moderation

Content moderation can review database text and bucket images. Availability depends on environment switches, resource configuration, and the moderation service. Database text supports synchronous and, when enabled, asynchronous review; storage images are reviewed after upload according to configuration.

Moderation configuration in a test project

Interface reference: platform moderation credentials are not configured in the test project; configuration varies by environment.

Feature Overview

When an enterprise externally provides services such as customer support ticket submission, user community comments, or knowledge base image attachment uploads (e.g., the ticket table customer_tickets and multimedia storage bucket agent_multimedia_bucket in the example project 企业智能客服与知识中台 enterprise-ai-hub-210), preventing policy-violating content involving politics, pornography, violence and terrorism, prohibited items, or abuse from entering the production database and object storage is a rigid bottom line of enterprise compliance.

Agentbase Moderation frees business developers from manually writing third-party content security SDK call code in every endpoint, implementing a transparent gate directly on the underlying data-plane gateway and storage engine:

  1. Precise dual-domain coverage (database text + storage bucket images):

    • Database text moderation: for tables with moderation enabled, text content in all string-type fields is automatically extracted and submitted for review upon single-row insertion or update.

    • Storage image moderation: when platform, project, and bucket image moderation are enabled, the upload path reads image bytes and submits them using the Base64 adapter. The device has already received the file; rejected or unavailable moderation fails the request. When the corresponding cleanup step is reached, deletion is attempted; verify its outcome.

  2. Three-mode moderation strategies and Phase-Two asynchronous state machine (2.9.0 / 2.10.0):

    • pre (synchronous pre-mode): database text is reviewed before writing. Storage images are first uploaded to the device, then reviewed according to configuration. A rejection or unavailable moderation fails the request; this does not guarantee that no bytes were written or cleanup succeeded.

    • post (asynchronous post-mode) and hybrid (mixed long/short text split mode): writes are persisted first and marked as under moderation (checking), with moderation completed asynchronously in the background; before publication (published), ordinary reads are subject to the moderation-state check, and supports escalation of violations to manual review (quarantined) or private withholding (private_hold) with periodic task self-healing.

  3. Credential priority and model gateway separation: a complete project AK/SK pair takes priority; otherwise the platform pair is used. Endpoint selection is independent: the project endpoint takes priority, with fallback to the platform endpoint when empty. Moderation and model inference credentials are configured separately.

Core feature viewConsole route pathCoverage capabilities and configuration itemsRequired permissions / role boundaries
Moderation home page (overview and credentials)/console/project-{region}-{project}/moderationView total call volume over the last 30 days (split by platform credentials vs. project credential calls); switch credential mode, configure Alibaba Cloud AK/SK, timeout (1–10s), log retention days (1–365 days), and single-image size limit (1–20 MB)View: any project member; modify project credentials: projects.write (owner/admin/developer/editor)
Moderation global logs page/console/project-{region}-{project}/moderation/logs · .../logs/{log}Search by resource type (table/bucket), resource ID, moderation result (pass/blocked/review/error), and time range; view hit risk labels (Labels), confidence, and suggested actionsReadable by any project member (logs only record restricted file names and ≤2000-character text previews; no raw image bytes are stored)
Table-level moderation configuration and logs/databases/database-{db}/table-{table}/moderationEnable text moderation for a specified table, choose the mode (pre / post / hybrid) and the violation-to-manual-review switch, and view the table's dedicated moderation logsDatabase table write permission
Bucket-level moderation configuration and logs/storage/bucket-{bucket}/moderationEnable image (image) moderation for a specified bucket and view the bucket's dedicated moderation logsStorage bucket write permission

Core Concepts and Architecture

1. Comparison table of the four single-run moderation verdicts and system actions

Regardless of the detailed classification returned by the underlying Alibaba Cloud Green MultiModalGuard call, Agentbase normalizes it into the following four standard verdicts and fully records the hit risk labels (Labels), confidence (Confidence), and model-suggested action (Suggestion: pass / review / block / mask) in the moderation logs:

Moderation verdict identifierMeaningpre Database write behavior in synchronous pre-modeStorage bucket (Storage image/*) upload behavior (Fail-Closed)
pass (pass)No violation risk detected or confidence below the interception thresholdAllow as normal: the data row is successfully written to the OceanBase tableKeep as normal: the image file completes upload and is persisted to the bucket
block (block)Explicitly detected high-risk contentReject writeReject upload: attempt deletion when the cleanup step is reached; verify its outcome
review (manual review)The model recommends manual reviewDepends on configuration: synchronous mode blocks; asynchronous mode may enter reviewReject upload: attempt deletion when the cleanup step is reached; verify its outcome
error (error)Credentials, network, reading, or a size limit prevent reviewDepends on moderation configurationUpload fails: attempt cleanup when the corresponding step is reached; check actual state if an earlier step or cleanup fails

2. Moderation Phase Two: asynchronous post-moderation (post), mixed split (hybrid), and the document moderation state machine

In environments where the platform administrator has enabled the Phase-Two moderation switch (when Phase Two is disabled, create or update requests cannot select post or hybrid; execution of an existing configuration falls back to synchronous pre-moderation), table text moderation supports asynchronous state transitions:

Core attribute or state valueEnum values / default valuePrecise business semantics and read/write visibility rules
Moderation mode (mode)pre · post · hybrid• pre: synchronous check before saving;
• post: persist the write first, then submit for review asynchronously in the background;
• hybrid: automatically split according to the text length threshold configured for the platform—short text stays on the synchronous path for immediate return, while overly long text moves to the asynchronous path to avoid blocking the write response.
Violation-to-human-review switchrequireManualReviewOnViolation (default true)• When enabled (true): violation records enter the quarantined (Blocked) quarantine state awaiting review;
• When disabled (false): violation records enter the private_hold (Review Exception) state, held privately, with visibility determined by the rules and existing permissions below.
moderationStatus Status transitionschecking → pending → published / quarantined / private_hold / action_errorThe data table displays Chinese status badges inline for intuitive viewing (Reviewing / Pending Review / Blocked / Review Exception / Review Failed); for records stuck in checking due to occasional network jitter, the platform's built-in periodic inspection task automatically re-evaluates and self-heals according to the current configuration semantics, with no manual intervention required.
Read isolation and deferred events for unpublished contentFor all records not in published statusFor rows in checking, pending, quarantined, private_hold, or action_error, the moderation-state check allows the creator or last modifier; API key and privileged project-management identities are exempt from that check. All reads remain subject to applicable authentication and database permissions. For queries with attribute filters, total subtracts hidden rows only on the current page and may overcount; deferred events are handled after publication according to configuration.

Roles, Entry Points, and Relationships

1. Resource-Level Configuration Status Semantics and Scope Boundaries

When enabling content moderation for a specific data table or storage bucket, the resource configuration object (/v1/moderation-configs) has the following three runtime states:

Clear scope boundaries:

2. Strict Distinction Between Content Moderation Credentials and Model Provider Credentials

During enterprise configuration, never confuse Moderation (content moderation credentials) with Models (LLM Gateway model inference credentials). The cloud services they connect to, the authentication protocols, and the configuration entry points are completely independent:

Comparison dimensionContent moderation credentials (Moderation Credentials)Model Provider credentials (LLM Gateway Providers)
Console configuration entryProject sidebar Moderation (Content Moderation) homepage configuration formProject sidebar Models (/llm-gateway) → Providers
Target cloud serviceAlibaba Cloud Content Moderation (Green MultiModalGuard)Large model inference services (e.g., Alibaba Cloud Model Studio DashScope dashscope-prod, OpenAI, Anthropic, etc.)
Required credential fieldsAlibaba Cloud Content Moderation HTTPS review endpoint (Endpoint) + AccessKey ID / AccessKey Secret (AK/SK pair)Model inference Base URL + API Key (e.g., <DASHSCOPE_API_KEY>) + model capability catalog
Isolation principleDedicated to data table write paths and storage bucket image upload risk control, must not be used for LLM chat or embeddingDedicated to Chat, Embedding, Rerank, and Vision inference, must not be used for content moderation

Feature Details and Operation Guide

1. Configuring Project-Level Content Moderation Credentials and Advanced Risk Control Parameters

  1. Log in to the Console and enter project 企业智能客服与知识中台 (enterprise-ai-hub-210), then click Moderation (Content Moderation) in the left sidebar (path /console/project-default-enterprise-ai-hub-210/moderation, as shown in Figure 1).

  2. Review the usage cards for the last 30 days at the top:

    • The top of the page displays three key statistics: total moderation calls over the last 30 days, platform credential call volume, and project custom credential call volume, along with a badge indicating the currently active credential source.

  3. Select the credential mode and configure parameters (corresponding to PATCH /v1/projects/{projectId}/moderation under the hood):

    • In the configuration form, you can switch between "Use platform credentials" and "Use custom project credentials".

    • Credential and endpoint fallback: a complete project AK/SK pair is used; otherwise the platform pair is used. The endpoint is resolved separately and falls back to the platform endpoint when the project endpoint is empty. Project credentials can therefore be combined with a platform endpoint; check both the effective credential source and destination.

    • If you select Use custom project credentials, fill in the following fields according to the table:

Configuration form fieldValue range / constraintsDefault value and production configuration recommendation
Endpoint (moderation service endpoint)A valid https:// Alibaba Cloud Content Moderation access addressFor example https://green-cip.cn-shanghai.aliyuncs.com. Saving validates the HTTPS URL format; use a trusted endpoint and verify destination access against deployment network policies.
AccessKey ID & SecretAn Alibaba Cloud RAM sub-account AK/SK with Green Content Moderation permissionsAfter saving, encrypted server-side storage, one-way masked in the UI without echo display; you can click clear at any time and re-enter a new key pair.
Timeout (moderation timeout)1 ~ 10 seconds (integer)Recommended to set to 3 ~ 5 seconds to prevent external network jitter from blocking database writes or image uploads for long periods.
Log retention (moderation log retention days)1 ~ 365 days (integer)Recommended to set to 90 or 180 days in production environments to meet compliance audit traceability requirements.
Max image size (per-image size limit)1 ~ 20 MB (integer)Sets the maximum size of a single image for moderation; an over-limit upload fails. Deletion is attempted when the cleanup step is reached; verify its outcome.

2. Enabling Moderation Switches Resource by Resource for Data Tables and Storage Buckets

After confirming the project-level credentials, you need to go to the specific data table to be protected (e.g., the customer support ticket table customer_tickets) or storage bucket (e.g., agent_multimedia_bucket) to enable resource-level moderation configuration (corresponding to the /v1/moderation-configs series of endpoints under the hood):

A. Enabling Text Moderation for Data Tables (Database Table)

  1. Click Databases in the left sidebar → enter the database enterprise_biz_db → click the data table customer_tickets → switch to the Moderation tab at the top of the table details (path .../table-customer_tickets/moderation).

  2. Turn on the table's text moderation switch (content type is fixed to text).

  3. Select the moderation mode:

    • The default selection is pre (synchronous check before save): any ticket submission containing violating words will be directly intercepted at write time.

    • If the platform has enabled Phase Two moderation capabilities, you can optionally select post (asynchronous post-hoc moderation) or hybrid (mixed routing for long and short texts), and check whether to enable human review on violation (requireManualReviewOnViolation).

  4. After saving, you can intuitively see the moderation status badge for each record in the table's Rows list, and view the moderation interception logs belonging only to that table under the table's Moderation tab.

B. Enabling Image Moderation for Storage Buckets

  1. Click Storage in the left sidebar → click to enter the target storage bucket (e.g., agent_multimedia_bucket) → switch to the Moderation tab at the top of the bucket details (path /storage/bucket-agent_multimedia_bucket/moderation).

  2. Turn on the bucket's image moderation switch (content type is fixed to image) and save.

  3. Verify image moderation: confirm platform, project, and bucket image moderation are enabled. This upload path reads image bytes and submits them as Base64; violations, an exceeded moderation limit, or an unavailable service cause upload failure. Deletion is attempted when the corresponding cleanup step is reached; verify its outcome after errors.

3. Querying and Auditing Content Moderation Logs

When end users report "ticket submission rejected" or "image upload failed", administrators can locate the interception reason within 1 minutes via the global or resource-level moderation logs:

  1. Go to the project sidebar Moderation → Logs tab (path /console/project-default-enterprise-ai-hub-210/moderation/logs, corresponding to the API /v1/moderation-logs).

  2. Use the top combined filter bar to narrow the scope:

    • Resource type: select Table or Bucket.

    • Resource ID: enter e.g. customer_tickets or agent_multimedia_bucket.

    • Result: filter by pass (pass), blocked (blocked), review (blocked on review), or error (call exception).

    • Time range: frame the time period in which the interception occurred.

  3. Open any interception record to enter the log detail page (/moderation/logs/{log}) and verify the following key forensic fields:

    • Basic context: timestamp, associated resource type and ID, content type (text / image), underlying Provider, and Alibaba Cloud request ID (Request ID).

    • Privacy-safe content preview: text moderation displays a truncated text preview of no more than 2000 characters; image moderation only records a restricted file name preview, never storing raw image bytes, Base64 encoding, or full internal object paths in the logs, strictly protecting user data privacy.

    • Decision basis: the matched risk Labels, Confidence (confidence score), Suggestion (model-recommended action: pass / review / block / mask), and the specific error message.

FAQ and Troubleshooting

Q1: Why does uploading a PDF document or MP4 video after enabling image moderation on a storage bucket not trigger a moderation record, while uploading a 15 MB high-resolution PNG image fails outright and the file cannot be found in the bucket?

Cause and troubleshooting steps:

  1. PDF / videos do not trigger moderation: Agentbase Storage content moderation strictly targets only image files whose MIME type matches image/*; PDF documents, Markdown files, audio, and videos are not covered by image moderation and are uploaded via the normal file flow.

  2. A 15 MB image upload fails: check the project moderation Max image size (1–20 MB). A 15 MB image exceeds the default 10 MB limit; adjust the limit or compress the image before uploading again. After an error, verify cleanup of the uploaded object and its record.

Q2:After enabling Phase Two moderation post (asynchronous post-moderation) mode, why can user A see their just-submitted comment after refreshing, but user B cannot see it, and no Webhook is triggered either?

Cause and troubleshooting steps: This is the expected security behavior of the Phase Two moderation read isolation for unpublished content :

Q3:I have already configured the Alibaba Cloud DashScope API Key on the “Models (LLM Gateway)” page, so why does enabling Moderation content moderation still report invalid credentials?

Cause and troubleshooting steps: Content moderation credentials and model gateway credentials are two completely independent configurations that are never reused: