Integration / 调用指南

接入页先解决“怎么开始”

这页不讲空泛概念,只讲怎么把第一条请求、宿主适配、结构化返回和多模态入口真正接起来。 先把最小链路跑通,再逐步加同步、反馈和媒体网关。

最短落地路径

先把一条真实业务链跑通

  1. 先用 query 跑通第一条真实请求,验证 Token、softwareId、tenantId 和页面上下文都能带进来。
  2. 优先消费结构化返回,不只看 answer,也要让宿主继续使用 intent、taskState、nextAction、followups 和 cards。
  3. 把菜单、权限、用户画像通过 sync 或适配器映射接进来,避免只靠自然语言猜上下文。
  4. 最后再接媒体上传、ASR、OCR 和反馈闭环,不要一开始就把所有能力混在一起。

Entry Points

先确定你属于哪种接入方式

不同系统入口不同,但都不应该绕开同一套结构化协议和标准交付物。

前端助手接入

适合 Web、H5、App 和小程序直接给最终用户用。先接 Query,再决定是否做悬浮入口或助手面板。

媒体与多模态接入

适合截图报错、语音提问、文档问答场景。官网语音实时链路已接 OmniModal,任务型 ASR 仍可走本机 bridge 兜底。

First Request

第一条请求应该长什么样

先用一条带真实 Token 和页面上下文的 Query 证明协议可用,再继续往宿主系统里嵌。
Request
POST /api/ziin/query
curl -X POST https://ziin.shenliu.cc/api/ziin/query \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001' \
  -d '{
    "question": "为什么入库单提交失败?",
    "mode": "embedded-agent",
    "module": "inventory",
    "context": {
      "route": "/inventory/inbound/create",
      "pageTitle": "入库管理",
      "selectedMenu": "仓储中心 / 入库管理",
      "platform": "web"
    },
    "inputs": [
      {
        "type": "error-dialog",
        "content": "报错弹窗:无权限提交入库单"
      }
    ]
  }'
Response
Structured JSON
{
  "traceId": "q_20260415_0001",
  "conversationId": "conv_20260415_0001",
  "intent": "ship_create",
  "confidence": 0.91,
  "taskState": "ship_drafting",
  "riskLevel": "draft",
  "routingVerdict": {
    "mode": "next_step",
    "confidence": 0.91,
    "source": "strong_agent",
    "reason": "ship_create_draft_guidance",
    "nextActionPolicy": "ask_clarification"
  },
  "answer": "可以,已先按成都方向整理成订单草稿。还需要确认收货地址和件重体。",
  "nextAction": {
    "type": "create_order_draft",
    "payload": {
      "destinationCity": "成都",
      "cargoName": "日用品",
      "weight": "300公斤"
    }
  },
  "followups": [
    "直接帮我生成草稿",
    "我只是先咨询",
    "补一下收货地址"
  ],
  "cards": [
    {
      "type": "draft",
      "title": "订单草稿建议",
      "data": {
        "destinationCity": "成都",
        "cargoName": "日用品",
        "weight": "300公斤"
      }
    }
  ],
  "permissions": [
    "inventory.inbound.create",
    "inventory.inbound.read"
  ],
  "steps": [
    "进入入库管理页面",
    "先完成必填字段并保存单据",
    "联系具备审批权限的角色继续提交"
  ],
  "warnings": [
    "当前结果受宿主权限上下文限制"
  ],
  "sources": [
    {
      "assetCode": "ka_001",
      "title": "入库管理操作手册",
      "type": "manual"
    }
  ],
  "suggestedActions": [
    "如无提交按钮,请联系管理员开通提交权限"
  ],
  "actionPlans": [
    {
      "id": "explain-permission-gap",
      "label": "查看权限缺口",
      "description": "先确认当前角色缺的是哪个动作权限。",
      "action": "explain_permission",
      "payload": {
        "permissionKey": "inventory.write",
        "roleCode": "warehouse_clerk"
      }
    },
    {
      "id": "focus-follow-up-field",
      "label": "继续追问",
      "description": "继续补充报错或字段上下文。",
      "action": "focus_field",
      "payload": {
        "fieldKey": "followUpQuestion",
        "target": "[data-ziin-field=\"follow-up-question\"]",
        "behavior": "focus"
      }
    }
  ]
}
Action Plans
SDK payload-first contract
[
  {
    "action": "navigate",
    "payload": {
      "route": "/orders/list",
      "openInNewTab": false
    }
  },
  {
    "action": "scroll_to",
    "payload": {
      "target": "[data-page-id=\"orders_list\"]",
      "behavior": "smooth"
    }
  },
  {
    "action": "fill_field",
    "payload": {
      "fieldKey": "customerName",
      "target": "[name=\"customerName\"]",
      "behavior": "fill",
      "fieldValue": "请补充 customerName"
    }
  }
]
结构化协议优先级

新宿主不要从 answer 里反推动作。助手任务推进优先读 routingVerdicttaskStatenextActionfollowupscards;SDK 或宿主页面动作计划继续读 actionPlans。两层动作都必须由宿主白名单 executor 执行。

question

保留用户原始口语,不要先过度改写。系统怎么问,智引就先吃什么。

module + route

最好由宿主直接传模块和当前路由,避免只靠自然语言去猜当前业务位置。

permissionCodes / Token

权限边界最好来自宿主可信上下文,而不是由模型自由推断。

inputs[]

截图、语音、文档、报错弹窗都统一挂在 inputs 里,再进入识别和问答链路。

最小字段

对方系统至少要准备什么

必须提供

softwareId、tenantId、userId、question、当前 route 或页面标识。没有这些字段,智引只能退化成普通 FAQ。

强烈建议提供

pageTitle、selectedMenu、platform、module、当前角色或权限码。它们决定回答能否贴近当前业务页面。

可以后补

菜单树、权限点、用户画像、知识资产、错误库和反馈数据。先跑通第一条链路,再逐步同步。

没有权限体系时

先用 viewer/editor/admin 这类粗粒度角色降级接入,并在 warnings 里明确结果不是最终权限判断。

Minimum Body
最小可跑示例
{
  "question": "这个页面怎么提交?",
  "mode": "ui-assistant",
  "softwareId": "erp-suite",
  "tenantId": "tenant-east-cn",
  "userId": "u-1001",
  "module": "inventory",
  "context": {
    "route": "/inventory/inbound/create",
    "pageTitle": "入库管理",
    "selectedMenu": "仓储中心 / 入库管理",
    "platform": "web"
  }
}

公开 Query 状态

需 API Key。官网悬浮助手和在线控制台可走站内演示通道,正式对外接口调用则应按商用 Key 管理。

控制接口状态

控制接口已收口。知识导入、同步、运营、供应商配置和审计接口不建议长期公开。

资料下载状态

需商务申请。公开资料可用于评估和联调,但不等于生产商用授权。

Host Types

按宿主类型看最小接法

所有示例都保留,但不再一屏堆满。需要时展开看,保证页面先可读,再可抄。
Web / Node

先在服务端或页面助手里接通 Query

这是最常见的第一步。适合已有 Web 业务系统、BFF 或 Node 服务的项目。现在 Web 侧已经不止是裸 query,也可以直接从公网 ESM 路径挂最小 widget。

查看 Node 示例
import { ZiinClient } from "@ziin/sdk-js";

const client = new ZiinClient({
  baseUrl: "https://ziin.shenliu.cc",
  token: "editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
});

const result = await client.query({
  question: "怎么做入库单?",
  mode: "api-engine",
  module: "inventory",
  context: {
    route: "/inventory/inbound/create",
    platform: "web",
  },
});

console.log(result.answer);
查看浏览器直连示例
const response = await fetch("https://ziin.shenliu.cc/api/ziin/query", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Bearer editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
  },
  body: JSON.stringify({
    question: "这个模块是干嘛的?",
    mode: "ui-assistant",
    module: "inventory",
    context: {
      route: "/inventory/inbound/create",
      pageTitle: "入库管理",
      selectedMenu: "仓储中心 / 入库管理",
      platform: "web",
    },
  }),
});

const result = await response.json();
assistantPanel.render({
  answer: result.answer,
  intent: result.intent,
  routingVerdict: result.routingVerdict,
  taskState: result.taskState,
  nextAction: result.nextAction,
  followups: result.followups,
  cards: result.cards,
});
查看公网 ESM Widget 示例
import {
  createHostRuntimeAdapter,
  createActionPlanExecutor,
  createZiinWidget,
} from "https://ziin.shenliu.cc/sdk/js/index.js";

const runtimeAdapter = createHostRuntimeAdapter({
  adapterConfig: {
    clientConfig: {
      baseUrl: "https://ziin.shenliu.cc",
      token: "editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
    },
    defaults: {
      mode: "ui-assistant",
      module: "inventory",
      softwareId: "erp-suite",
      tenantId: "tenant-east-cn",
      userId: "u-1001",
    },
  },
  getContext: () => ({
    route: window.location.pathname,
    pageTitle: document.title,
    selectedMenu: readSelectedMenuFromHost(),
    platform: "web",
    pageId: readStablePageId(),
    availableActions: ["navigate", "focus_field", "scroll_to", "open_dialog"],
  }),
});

createZiinWidget({
  runtimeAdapter,
  actionExecutor: createActionPlanExecutor({
    allowActions: ["navigate", "focus_field", "scroll_to", "open_dialog"],
    handlers: {
      navigate: (plan) => router.push(plan.route || "/"),
      focusField: (plan) => focusHostField(plan.target || ""),
      scrollTo: (plan) => document.querySelector(plan.target || "")?.scrollIntoView({ behavior: "smooth", block: "center" }),
      openDialog: (plan) => openDialog(plan.target),
    },
  }),
  defaultOpen: true,
  title: "智引操作内核",
  subtitle: "权限感知式操作指引",
  buttonLabel: "打开使用助手",
  contextLabel: "当前页面",
  theme: {
    accentColor: "#c55b2d",
    panelWidth: 420,
    borderRadius: 26,
  },
  events: {
    onQuerySuccess: ({ response }) => console.log("ziin.trace", response.traceId),
    onActionExecuted: ({ plan }) => console.log("ziin.action", plan.id),
  },
});

await widget.refreshContext();
小程序 / App

重点是上下文和 Token 一起带上来

端的 UI 不同,但协议不应该分裂。小程序、App 和 H5 都应共用同一套 Query 结构。

查看小程序示例
wx.request({
  url: "https://ziin.shenliu.cc/api/ziin/query",
  method: "POST",
  header: {
    "Content-Type": "application/json",
    Authorization: "Bearer editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
  },
  data: {
    question: "报错是什么意思?",
    mode: "mini-program",
    module: "inventory",
    context: {
      route: "/pages/inventory/inbound/create",
      pageTitle: "入库管理",
      selectedMenu: "仓储中心 / 入库管理",
      platform: "mini-program",
    },
  },
});
Token 约定

推荐 Token 组织成 role|softwareId:...|tenantId:...|userId:...,当前项目已支持从这个格式中解析 `softwareId`、`tenantId` 和 `userId`。

宿主智能体

把 Ziin 当软件操作内核,而不是第二个对话页

宿主 AI 保留自己的语气、品牌和会话层,Ziin 只负责提供软件知识、权限判断和结构化答案。

Visible Loop

如果你想先让客户直观看到“结构化 Agent 不是纸面字段”,先打开 /embed。当前已经能直接演示intenttaskStatenextActioncards 和宿主 executor 日志如何一起工作。

查看 Host Adapter 示例
Host Adapter
import { createHostAdapter } from "@ziin/sdk-js";

const hostAdapter = createHostAdapter({
  clientConfig: {
    baseUrl: "https://ziin.shenliu.cc",
    token: "editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
  },
  defaults: {
    mode: "embedded-agent",
    softwareId: "erp-suite",
    tenantId: "tenant-east-cn",
    userId: "u-1001",
    context: {
      platform: "web",
      selectedMenu: "仓储中心 / 入库管理",
    },
  },
});

const result = await hostAdapter.query({
  question: "为什么入库单提交失败?",
  module: "inventory",
  context: {
    route: "/inventory/inbound/create",
    pageTitle: "入库管理",
    pageId: "inventory_inbound_create",
    recordId: "IN-20260417-001",
    permissionCodes: ["inventory.write"],
    errors: [{ code: "NO_PERMISSION", message: "当前用户无提交权限" }],
  },
});
查看 Runtime Adapter + 动作执行器示例
Runtime Adapter
import {
  createHostRuntimeAdapter,
  createActionPlanExecutor,
} from "https://ziin.shenliu.cc/sdk/js/index.js";
import { executeFloatingAssistantNextAction } from "./floating-ziin-assistant-action-executor";

const runtimeAdapter = createHostRuntimeAdapter({
  adapterConfig: {
    clientConfig: {
      baseUrl: "https://ziin.shenliu.cc",
      token: "editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
    },
    defaults: {
      mode: "ui-assistant",
      module: "inventory",
      softwareId: "erp-suite",
      tenantId: "tenant-east-cn",
      userId: "u-1001",
    },
  },
  getContext: () => ({
    route: window.location.pathname,
    pageTitle: document.title,
    selectedMenu: readSelectedMenuFromHost(),
    platform: "web",
    pageId: readStablePageId(),
    availableActions: ["navigate", "focus_field", "scroll_to", "open_dialog"],
  }),
});

const executor = createActionPlanExecutor({
  allowActions: ["navigate", "focus_field", "scroll_to", "open_dialog"],
  handlers: {
    navigate: (plan) => router.push(plan.payload?.route || plan.route || "/"),
    focusField: (plan) =>
      focusHostField(plan.payload && "target" in plan.payload ? plan.payload.target || "" : plan.target || ""),
    scrollTo: (plan) =>
      document
        .querySelector(plan.payload && "target" in plan.payload ? plan.payload.target || "" : plan.target || "")
        ?.scrollIntoView({ behavior: "smooth", block: "center" }),
    openDialog: (plan) =>
      openDialog(plan.payload && "dialogId" in plan.payload ? plan.payload.dialogId : plan.target),
  },
});

const result = await runtimeAdapter.query({
  question: "这个页面下一步怎么做?",
});

if (result.nextAction) {
  executeFloatingAssistantNextAction({
    action: result.nextAction,
    response: { answer: result.answer },
    context: {
      routerPush: (route) => router.push(route),
      setQuestion,
      setTextMode,
      setStatus,
      focusInput: () => inputRef.current?.focus(),
      submitQuestion: (question) => runtimeAdapter.query({ question }),
    },
  });
}

if (result.actionPlans?.[0]) {
  await executor(result.actionPlans[0]);
}
查看内置调用示例
Host Agent Example
const ziin = await fetch("https://ziin.shenliu.cc/api/ziin/query", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Bearer editor|softwareId:erp-suite|tenantId:tenant-east-cn|userId:u-1001",
  },
  body: JSON.stringify({
    question: userMessage,
    mode: "embedded-agent",
    module: currentModule,
    context: currentPageContext,
    inputs: multimodalInputs,
  }),
}).then((r) => r.json());

return llm.rewriteForUser({
  answer: ziin.answer,
  intent: ziin.intent,
  routingVerdict: ziin.routingVerdict,
  taskState: ziin.taskState,
  nextAction: ziin.nextAction,
  followups: ziin.followups,
  cards: ziin.cards,
  warnings: ziin.warnings,
  actions: ziin.actionPlans,
});
同步与适配器

真接业务系统时,优先同步菜单、权限和用户画像

RuoYi-Cloud 只是一个宿主,不是产品中心。以后接任何系统,都应只做 adapter,不要重写 core。

查看同步示例
curl -X POST https://ziin.shenliu.cc/api/ziin/sync \
  -H 'Content-Type: application/json' \
  -d '{
    "softwareId": "erp-suite",
    "tenant": {
      "id": "tenant-east-cn",
      "softwareId": "erp-suite",
      "name": "华东制造集团"
    },
    "menuNodes": [
      {
        "id": "mn-sync-001",
        "module": "inventory",
        "name": "入库管理",
        "route": "/inventory/inbound/create",
        "permissionKey": "inventory.write"
      }
    ],
    "permissionPoints": [
      {
        "roleCode": "warehouse_clerk",
        "permissionKey": "inventory.write"
      }
    ]
  }'
查看适配器映射约定
adapter:
  softwareId: erp-demo
  tenantMode: single
  defaultPlatform: web

mapping:
  roles:
    "采购员": purchaser
    "仓库管理员": warehouse_admin

  permissions:
    "inventory:inbound:create": "inventory.inbound.create"
    "inventory:inbound:read": "inventory.inbound.read"

Multimodal

多模态先统一协议,再逐步加能力

当前最务实的路线是:图像先 OCR、语音先 ASR、文档先切片,再回到 Query 做权限判断、知识召回和步骤生成。
当前真实状态

不是 Mock 演示,是可联调边界

  • 官网助手语音已改成 `omnimodal realtime`,外部程序可以按同协议直连实时语音会话。
  • OmniModal 官网和开发者入口已经独立上线,语音链路合同以 `https://omnimodal.shenliu.cc/developers/` 为准。
  • 本机 faster-whisper bridge 仍保留给任务型 ASR 和本地兜底链路。
  • 文档解析已支持 txt / doc / docx / pdf 的文本层预处理。
  • 扫描版 PDF 和复杂图片仍建议接外部 OCR,别把产品边界说满。
商用建议

上传、轮询、审计和配额应该一起做

真正商用不是“接个 OCR/ASR 接口”就结束,而是要建设媒体解析网关和统一任务链路。

Upload / Task
查看上传与发起任务示例
import { ZiinClient } from "@ziin/sdk-js";

const client = new ZiinClient({
  baseUrl: "https://ziin.shenliu.cc",
});

const created = await client.uploadAndCreateAsrTask({
  tenantId: "tenant-east-cn",
  userId: "u-1001",
  source: "app",
  scene: "voice-ticket",
  language: "zh",
  file: {
    fileName: file.name,
    mimeType: file.type,
    bytes: file,
  },
});
查看任务轮询示例
const detail = await client.waitForTask(created.task.taskId, {
  intervalMs: 2000,
  maxAttempts: 45,
});

if (detail.status !== "success") {
  console.error(detail.error);
}

console.log(detail.result);
inputs[] 规范
查看多模态输入示例
[
  { "type": "text", "content": "为什么提交失败?" },
  { "type": "error-dialog", "content": "提交失败:无权限提交入库单" },
  { "type": "ocr-text", "content": "OCR 结果:单据状态不允许过账" },
  { "type": "image", "content": "https://cdn.example.com/error.png", "mimeType": "image/png" },
  { "type": "document", "content": "https://cdn.example.com/manual.pdf", "mimeType": "application/pdf" }
]
Entry Skill

统一输入入口建议单独稳定下来

不要每个业务页各做一套语音、拍照、文本和附件逻辑。更稳的方式是先抽一层统一输入技能,再把 payload 路由给各业务技能。

查看多模态入口协议
type MultimodalEntryPayload = {
  scene: "assistant" | "order_create" | "order_detail" | string
  intent?: "lookup" | "shipping_consult" | "exception_judge" | string
  mode: "voice" | "text" | "image" | "file" | "mixed"
  text?: string
  audio?: { fileId?: string; durationMs?: number; transcript?: string }
  images?: Array<{ fileId?: string; name?: string }>
  files?: Array<{ fileId?: string; name?: string; mimeType?: string }>
  businessContext?: {
    route?: string
    pageTitle?: string
    selectedMenu?: string
    platform?: string
  }
  createdAt: string
}
OmniModal Realtime

官网语音入口现在直接复用 OmniModal 实时语音底座

  • OmniModal 开发者入口已公开:`https://omnimodal.shenliu.cc/developers/`。
  • 公开 OpenAPI 可直接看:`https://omnimodal.shenliu.cc/api/v1/public/developer/openapi.json`。
  • 官网助手语音现在走 OmniModal realtime,会话返回里已经带 `asrRouting / runtimeConfig / resume / expiresAt`。
  • 如果你是正式宿主,建议后端代调 OmniModal,浏览器不要长期持有真实租户 key。
查看 OmniModal realtime session 示例
curl -X POST https://omnimodal.shenliu.cc/api/v1/realtime/sessions \
  -H 'Content-Type: application/json' \
  -H 'x-omnimodal-api-key: YOUR_OMNIMODAL_KEY' \
  -d '{
    "tenantId": "YOUR_TENANT_ID",
    "scene": "ziin-public-site-voice",
    "mode": "voice",
    "audioConfig": {
      "codec": "pcm16",
      "sampleRate": 16000,
      "channelCount": 1
    },
    "businessContext": {
      "sourceSystem": "ziin",
      "channel": "floating-assistant"
    }
  }'

Delivery Standard

产品化交付的四个固定件

客户不应该自己猜建什么库、传什么字段、看哪份资料。对外必须把标准件公开出来。

标准接口

一期对外固定 query、feedback、sync、schema、health。协议先稳,后面才能持续扩能力。

打开资料

标准库表

建议独立 ziin_agent 库或独立 schema,不把知识、反馈和审计直接混进宿主核心业务表。

打开资料

标准模板

菜单映射、权限映射、知识导入模板已经公开,不要求客户自己猜字段。

打开资料

标准实施资料

客户接入清单、2 天接入流程、Media 模块模板都已经可公开下载。

打开资料