mirror of
https://github.com/hansjone/dsh-im-ops.git
synced 2026-10-09 06:30:45 +08:00
feat: support image prompts across IM channels
This commit is contained in:
xmanrui
parent
99a877c640
commit
65c842c045
40 changed files with 3806 additions and 274 deletions
|
|
@ -38,6 +38,8 @@ Connect IM bots to DeepSeek Harness by scanning a QR code, using an App Manifest
|
|||
|
||||
Other IM platforms can be added through the same channel-adapter structure.
|
||||
|
||||
All nine built-in channels can send JPEG, PNG, and WebP images, plus GIFs sent as image files, with optional captions to Harness. Each image is limited to 5 MB, and images in one message are limited to 20 MB in total.
|
||||
|
||||
## Installation
|
||||
|
||||
```sh
|
||||
|
|
@ -56,19 +58,19 @@ Feishu, QQ, DingTalk, and WeCom each provide two entry points. The blue **QR acc
|
|||
|
||||
Telegram and Discord do not provide an official QR flow for creating bots, so their pages expose only the key-marked **Manual access** action and request a Bot Token. Generate the Telegram token with BotFather; an existing webhook must be removed by its current service before Bot API long polling can receive updates. Generate the Discord token on the Developer Portal's Bot page, invite the bot to the target server, and grant View Channel, Send Messages, and Read Message History. The plugin reads DMs and server messages that explicitly mention the bot, so it does not request the privileged Message Content intent.
|
||||
|
||||
Slack provides Manifest-assisted creation with dual-Token access. Choose **Start setup**, copy the bundled App Manifest, open Slack's create page, and select **From a manifest**. Under **Basic Information → App-Level Tokens**, generate an App Token with `connections:write`; then install the app to the workspace under **OAuth & Permissions** to obtain the Bot Token. The plugin validates both Tokens before opening Socket Mode. Slack has no official QR-based bot-creation flow. Both Tokens are sent only to the local Harness Host and stored through its protected credential provider; status responses and bot lists never return them.
|
||||
Slack provides Manifest-assisted creation with dual-Token access. Choose **Start setup**, copy the bundled App Manifest, open Slack's create page, and select **From a manifest**. Under **Basic Information → App-Level Tokens**, generate an App Token with `connections:write`; then install the app to the workspace under **OAuth & Permissions** to obtain the Bot Token. The plugin validates both Tokens before opening Socket Mode. Slack has no official QR-based bot-creation flow. Image access uses the Manifest's `files:read` scope; Slack Apps installed before this upgrade must be reinstalled or reauthorized to receive it. Both Tokens are sent only to the local Harness Host and stored through its protected credential provider; status responses and bot lists never return them.
|
||||
|
||||
WhatsApp exposes only **QR access**. On the phone, open **WhatsApp → Settings → Linked devices → Link a device**, then scan the QR code shown by Harness. No Meta console, Cloud API, Webhook, Phone Number ID, or Access Token is required. Linked-device state stays under `~/.dsh/integrations/dsh-whatsapp/auth`; the browser receives only the one-time QR code and redacted account status. Personal accounts can use WhatsApp's **Message yourself** chat directly; the plugin suppresses only its own exact reply message IDs to prevent reply loops.
|
||||
|
||||
Use a dedicated WhatsApp number for the bot when possible. Linking a personal account makes DMs sent to that account eligible Harness input; group messages trigger only when they mention or reply to the linked account. Limit the number to trusted contacts, and remove the device from both Harness and the phone's **Linked devices** list when it is no longer used.
|
||||
|
||||
For DingTalk QR binding, scan with an account that belongs to an enterprise or organization and can create bots, then choose **Create a new bot** on the authorization page. If DingTalk reports that the account has not joined an organization, create one or switch to an account that has, then scan again. There is no second local sender-approval flow: the bot's DingTalk visibility is its inbound access scope, so restrict it to trusted organizations, groups, or members.
|
||||
For DingTalk QR binding, scan with an account that belongs to an enterprise or organization and can create bots, then choose **Create a new bot** on the authorization page. If DingTalk reports that the account has not joined an organization, create one or switch to an account that has, then scan again. There is no second local sender-approval flow: the bot's DingTalk visibility is its inbound access scope, so restrict it to trusted organizations, groups, or members. Image downloads add no separate scope, but they depend on the bot's existing **企业内机器人发送消息权限** permission. A manually bound app without that permission can receive an image callback but cannot exchange it for a temporary download URL.
|
||||
|
||||
For WeCom QR binding, scan with an account that belongs to an enterprise and can create or manage bots, then confirm creation of the intelligent bot in the mobile app. This creates a WeCom intelligent bot; it does not sign the plugin into a personal WeChat account. For both QR and credential binding, restrict the bot's WeCom visibility to trusted enterprise members and group chats.
|
||||
|
||||
QQ QR binding uses Tencent's official QQBot v2 flow. Tencent's default authorization page labels the integration as a third-party bot. Scanning creates a QQ Open Platform bot; it does not give the plugin direct control of a personal QQ account. QR binding accepts only the scanner's messages. Manual credentials cannot identify a scanner, so the bot's QQ Open Platform visibility becomes its inbound access scope.
|
||||
|
||||
Feishu QR binding records the scanner as an allowed user. Manual credentials cannot identify a scanner, so the Feishu application's visibility becomes its inbound access scope. Restrict the application to trusted tenants, groups, or members.
|
||||
Feishu QR binding records the scanner as an allowed user. Manual credentials cannot identify a scanner, so the Feishu application's visibility becomes its inbound access scope. Restrict the application to trusted tenants, groups, or members. Reading user-sent images requires the tenant scope `im:message:readonly`. Newly QR-created apps request it automatically; apps created before this upgrade must add the scope in the Feishu developer console, publish a version, and complete any required administrator approval.
|
||||
|
||||
Each bot maintains an independent Harness workspace. A newly connected bot records the Harness Host process's current working directory (`process.cwd()`) as its default; the path is persisted and does not change when the Host is later restarted from another directory. Every bot card shows the current path and lets it be edited.
|
||||
|
||||
|
|
|
|||
|
|
@ -41,6 +41,8 @@ Connect IM bots to DeepSeek Harness by scanning a QR code, using an App Manifest
|
|||
|
||||
其他 IM 平台可继续按同一渠道适配器结构接入。
|
||||
|
||||
九个内置渠道均支持把 JPEG、PNG、WebP 图片，以及以图片文件方式发送的 GIF，连同可选文字说明发送给 Harness；单张图片上限为 5 MB，单条消息中的图片总大小上限为 20 MB。
|
||||
|
||||
## 安装
|
||||
|
||||
```sh
|
||||
|
|
@ -59,19 +61,19 @@ dsh plugin --profile web add @xmanrui/dsh-im
|
|||
|
||||
Telegram 和 Discord 没有官方扫码创建机器人流程，因此页面只显示带钥匙图标的「手动接入」入口，并只要求 Bot Token。Telegram Token 由 @BotFather 生成；若该机器人已经配置 Webhook，需要先由原服务移除 Webhook，Bot API 长轮询才能接管消息。Discord Token 来自 Developer Portal 的 Bot 页面；还需把机器人邀请到目标服务器，并授予查看频道、发送消息和读取历史消息权限。本插件只读取私信和明确提及机器人的服务器消息，因此不要求 Message Content 特权 Intent。
|
||||
|
||||
Slack 页面提供 Manifest 辅助创建与双 Token 接入。点击「开始接入」，复制页面提供的 App Manifest，再打开 Slack 创建页并选择 **From a manifest**；创建后在 **Basic Information → App-Level Tokens** 生成包含 `connections:write` 的 App Token，并在 **OAuth & Permissions** 将应用安装到工作区以取得 Bot Token。插件会验证两个 Token，再通过 Socket Mode 建立连接；Slack 没有官方扫码创建机器人流程。两个 Token 只提交到本机 Harness Host 并写入受保护的凭据存储，状态接口和机器人列表不会回传 Token。
|
||||
Slack 页面提供 Manifest 辅助创建与双 Token 接入。点击「开始接入」，复制页面提供的 App Manifest，再打开 Slack 创建页并选择 **From a manifest**；创建后在 **Basic Information → App-Level Tokens** 生成包含 `connections:write` 的 App Token，并在 **OAuth & Permissions** 将应用安装到工作区以取得 Bot Token。插件会验证两个 Token，再通过 Socket Mode 建立连接；Slack 没有官方扫码创建机器人流程。图片读取使用 Manifest 中的 `files:read`；升级前已安装的 Slack App 需要重新安装或重新授权，才能获得该权限。两个 Token 只提交到本机 Harness Host 并写入受保护的凭据存储，状态接口和机器人列表不会回传 Token。
|
||||
|
||||
WhatsApp 页面只显示「扫码接入机器人」。打开手机 WhatsApp 的「设置 → 已关联设备 → 关联设备」，扫描 Harness 页面中的二维码即可，不需要 Meta 控制台、Cloud API、Webhook、Phone Number ID 或 Access Token。关联设备状态只保存在本机 `~/.dsh/integrations/dsh-whatsapp/auth`，浏览器只会收到一次性二维码和脱敏后的账号状态。个人账号可在 WhatsApp 的「给自己发消息」会话中直接使用；插件按消息 ID 过滤自己的回复，避免形成回复循环。
|
||||
|
||||
建议为机器人准备独立 WhatsApp 号码。关联个人常用账号会让发给该账号的私聊消息成为 Harness 输入；群聊只有明确提及该账号或回复该账号消息时才会触发。请只把机器人号码开放给可信联系人，并在不再使用时同时从 Harness 和手机「已关联设备」中移除。
|
||||
|
||||
钉钉扫码接入时，请使用已加入企业/组织且有权创建机器人的钉钉账号扫描页面二维码，再在钉钉授权页点击「一键创建新机器人」。若提示“该账号还未加入组织”，请先创建组织或换用已加入组织的账号后重新扫码。插件不设置本机二次批准流程，钉钉中的机器人可见范围就是入站访问范围，请只开放给信任的组织、群或成员。
|
||||
钉钉扫码接入时，请使用已加入企业/组织且有权创建机器人的钉钉账号扫描页面二维码，再在钉钉授权页点击「一键创建新机器人」。若提示“该账号还未加入组织”，请先创建组织或换用已加入组织的账号后重新扫码。插件不设置本机二次批准流程，钉钉中的机器人可见范围就是入站访问范围，请只开放给信任的组织、群或成员。图片下载不会新增独立权限，但依赖机器人已有的“企业内机器人发送消息权限”；手动绑定的已有应用若未开启该权限，可以收到图片回调，但无法换取临时下载地址。
|
||||
|
||||
企业微信扫码接入时，请使用已加入企业且具有机器人创建或管理权限的企业微信账号，并在手机端确认创建智能机器人。扫码创建的是企业微信智能机器人，不是让插件直接登录个人微信账号。无论扫码还是凭据绑定，企业微信中的机器人可见范围就是入站访问范围，请只开放给信任的企业成员和群聊。
|
||||
|
||||
QQ 扫码接入使用腾讯 QQBot v2 官方流程。默认腾讯授权页会把接入方显示为“第三方机器人”；扫码成功后创建的是 QQ 开放平台机器人，并不是让插件直接控制个人 QQ 账号。扫码绑定只接受扫码者的消息；手动凭据无法识别扫码人，因此使用 QQ 开放平台中的机器人可见范围作为入站访问范围。
|
||||
|
||||
飞书扫码绑定会把扫码者作为允许使用者；手动凭据同样无法识别扫码人，因此使用飞书应用的可见范围作为入站访问范围。请在飞书开放平台中只向信任的租户、群或成员开放应用。
|
||||
飞书扫码绑定会把扫码者作为允许使用者；手动凭据同样无法识别扫码人，因此使用飞书应用的可见范围作为入站访问范围。请在飞书开放平台中只向信任的租户、群或成员开放应用。读取用户发送的图片需要租户权限 `im:message:readonly`；新扫码创建的应用会申请该权限，升级前已存在的应用需要在飞书开放平台手动添加权限、发布版本并完成必要的管理员审批。
|
||||
|
||||
每个机器人维护独立的 Harness 工作区。新接入机器人会把 Harness Host 进程当时的工作目录（`process.cwd()`）记录为默认值；该路径会持久化，不会因为以后从其他目录重启 Host 而改变。设置页的机器人卡片会显示当前路径，并可直接修改。
|
||||
|
||||
|
|
|
|||
|
|
@ -5329,6 +5329,7 @@ oauth_config:
|
|||
bot:
|
||||
- app_mentions:read
|
||||
- chat:write
|
||||
- files:read
|
||||
- im:history
|
||||
settings:
|
||||
event_subscriptions:
|
||||
|
|
|
|||
240
lib/index.js
240
lib/index.js
File diff suppressed because one or more lines are too long
|
|
@ -1,5 +1,7 @@
|
|||
import { randomUUID } from 'node:crypto';
|
||||
|
||||
import { fetchImageBuffer, ImagePromptError } from '../shared/image-prompt.mjs';
|
||||
|
||||
export const DINGTALK_REGISTRATION_BASE_URL = 'https://oapi.dingtalk.com/';
|
||||
export const DINGTALK_API_BASE_URL = 'https://api.dingtalk.com/';
|
||||
export const DINGTALK_REGISTRATION_SOURCE = 'DING_DWS_CLAW';
|
||||
|
|
@ -14,6 +16,7 @@ export class DingtalkApiError extends Error {
|
|||
this.name = 'DingtalkApiError';
|
||||
this.code = code;
|
||||
this.status = options.status;
|
||||
this.providerCode = options.providerCode;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -21,6 +24,25 @@ function nonEmptyString(value) {
|
|||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function safeProviderCode(value) {
|
||||
const code = nonEmptyString(value);
|
||||
return code && /^[A-Za-z0-9_.:-]{1,160}$/.test(code) ? code : undefined;
|
||||
}
|
||||
|
||||
function secureDingtalkDownloadUrl(value) {
|
||||
let url;
|
||||
try {
|
||||
url = new URL(value);
|
||||
} catch (error) {
|
||||
throw new DingtalkApiError('invalid-image-download', '钉钉服务返回了无效的图片下载地址。', { cause: error });
|
||||
}
|
||||
if (url.protocol === 'http:' && (!url.port || url.port === '80')) {
|
||||
url.protocol = 'https:';
|
||||
url.port = '';
|
||||
}
|
||||
return url;
|
||||
}
|
||||
|
||||
function isDingtalkHost(hostname) {
|
||||
const normalized = hostname.toLowerCase().replace(/\.$/, '');
|
||||
return normalized === 'dingtalk.com' || normalized.endsWith('.dingtalk.com');
|
||||
|
|
@ -123,10 +145,17 @@ async function requestJson(fetchImpl, url, {
|
|||
signal: controller.signal,
|
||||
});
|
||||
if (!response.ok) {
|
||||
let providerCode;
|
||||
try {
|
||||
const errorBody = await response.json();
|
||||
providerCode = safeProviderCode(errorBody?.code ?? errorBody?.errcode);
|
||||
} catch {
|
||||
// DingTalk occasionally returns an empty or non-JSON error body.
|
||||
}
|
||||
throw new DingtalkApiError(
|
||||
'http-error',
|
||||
`钉钉服务请求失败（HTTP ${response.status}）。`,
|
||||
{ status: response.status },
|
||||
{ status: response.status, providerCode },
|
||||
);
|
||||
}
|
||||
try {
|
||||
|
|
@ -403,6 +432,58 @@ export function createDingtalkApi({
|
|||
|
||||
accessToken,
|
||||
|
||||
async downloadImage({
|
||||
clientId,
|
||||
clientSecret,
|
||||
robotCode,
|
||||
downloadCode,
|
||||
signal,
|
||||
maxBytes,
|
||||
}) {
|
||||
const botCode = nonEmptyString(robotCode);
|
||||
const fileCode = nonEmptyString(downloadCode);
|
||||
if (!botCode || !fileCode) throw new TypeError('robotCode and downloadCode are required');
|
||||
const token = await accessToken({ clientId, clientSecret, signal });
|
||||
let response;
|
||||
try {
|
||||
response = await requestJson(
|
||||
fetchImpl,
|
||||
endpoint(apiBase, 'v1.0/robot/messageFiles/download'),
|
||||
{
|
||||
body: { downloadCode: fileCode, robotCode: botCode },
|
||||
headers: { 'x-acs-dingtalk-access-token': token },
|
||||
signal,
|
||||
action: '图片下载地址',
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
if (signal?.aborted) throw error;
|
||||
throw new DingtalkApiError(
|
||||
'image-download-address-failed',
|
||||
'钉钉图片下载地址获取失败。',
|
||||
{ cause: error, status: error?.status, providerCode: error?.providerCode },
|
||||
);
|
||||
}
|
||||
const downloadUrl = nonEmptyString(response?.downloadUrl ?? response?.download_url);
|
||||
if (!downloadUrl) {
|
||||
throw new DingtalkApiError('invalid-image-download', '钉钉服务没有返回图片下载地址。');
|
||||
}
|
||||
try {
|
||||
return await fetchImageBuffer(secureDingtalkDownloadUrl(downloadUrl), {
|
||||
fetchImpl,
|
||||
signal,
|
||||
maxBytes,
|
||||
});
|
||||
} catch (error) {
|
||||
if (signal?.aborted || error instanceof ImagePromptError) throw error;
|
||||
throw new DingtalkApiError(
|
||||
'image-content-download-failed',
|
||||
'钉钉图片内容下载失败。',
|
||||
{ cause: error },
|
||||
);
|
||||
}
|
||||
},
|
||||
|
||||
async createAiCard({ clientId, clientSecret, target, initialText, signal }) {
|
||||
const appKey = nonEmptyString(clientId);
|
||||
const appSecret = nonEmptyString(clientSecret);
|
||||
|
|
|
|||
|
|
@ -12,6 +12,11 @@ import { HarnessApprovalQueue } from '../shared/harness-approval.mjs';
|
|||
import { runCompactCommand } from '../shared/compact-command.mjs';
|
||||
import { runWorkspaceCommand } from '../shared/workspace-command.mjs';
|
||||
import { askInWorkspaceSession } from '../shared/workspace-session.mjs';
|
||||
import {
|
||||
hasInboundImages,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from '../shared/image-prompt.mjs';
|
||||
|
||||
const CARD_INITIAL_TEXT = '已连接 DeepSeek Harness，正在思考…';
|
||||
const CARD_ERROR_TEXT = '消息处理失败，请稍后重试。';
|
||||
|
|
@ -20,7 +25,7 @@ const INTERACTION_RESOLVED_TEXT = '这个问题已在其他客户端处理，无
|
|||
const HELP_TEXT = [
|
||||
'钉钉机器人已连接 DeepSeek Harness。',
|
||||
'',
|
||||
'直接发送文字即可继续当前会话。',
|
||||
'直接发送文字或图片即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -35,10 +40,123 @@ function nonEmptyString(value) {
|
|||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function safeErrorDiagnostic(error) {
|
||||
const chain = [];
|
||||
const seen = new Set();
|
||||
let current = error;
|
||||
while (current && typeof current === 'object' && chain.length < 3 && !seen.has(current)) {
|
||||
seen.add(current);
|
||||
const name = nonEmptyString(current.name)?.slice(0, 80);
|
||||
const code = nonEmptyString(current.code)?.slice(0, 80);
|
||||
const providerCode = nonEmptyString(current.providerCode)?.slice(0, 160);
|
||||
const status = Number.isInteger(current.status) ? current.status : undefined;
|
||||
chain.push({
|
||||
...(name ? { name } : {}),
|
||||
...(code ? { code } : {}),
|
||||
...(providerCode ? { providerCode } : {}),
|
||||
...(status ? { status } : {}),
|
||||
});
|
||||
current = current.cause;
|
||||
}
|
||||
return chain;
|
||||
}
|
||||
|
||||
function dingtalkImageErrorUserMessage(error) {
|
||||
let current = error;
|
||||
const seen = new Set();
|
||||
while (current && typeof current === 'object' && !seen.has(current)) {
|
||||
seen.add(current);
|
||||
if (current.code === 'image-download-address-failed') {
|
||||
return '钉钉未能换取图片下载地址，请重新发送；若持续失败，请检查机器人的“企业内机器人发送消息权限”。';
|
||||
}
|
||||
if (current.code === 'invalid-image-download') {
|
||||
return '钉钉没有返回图片下载地址，请重新发送。';
|
||||
}
|
||||
if (current.code === 'image-content-download-failed') {
|
||||
return '钉钉返回的图片临时地址无法读取，请重新发送。';
|
||||
}
|
||||
current = current.cause;
|
||||
}
|
||||
return imagePromptUserMessage(error);
|
||||
}
|
||||
|
||||
function senderStaffId(message) {
|
||||
return nonEmptyString(message?.senderStaffId) ?? nonEmptyString(message?.senderId);
|
||||
}
|
||||
|
||||
function parsedMessageContent(message) {
|
||||
if (message?.content && typeof message.content === 'object') return message.content;
|
||||
if (typeof message?.content !== 'string') return null;
|
||||
try {
|
||||
const parsed = JSON.parse(message.content);
|
||||
return parsed && typeof parsed === 'object' ? parsed : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function richTextEntries(content) {
|
||||
const entries = content?.richText ?? content?.rich_text;
|
||||
return Array.isArray(entries) ? entries : [];
|
||||
}
|
||||
|
||||
function richTextEntryText(entry) {
|
||||
if (typeof entry?.text === 'string') return nonEmptyString(entry.text);
|
||||
if (typeof entry?.text?.content === 'string') return nonEmptyString(entry.text.content);
|
||||
if (String(entry?.type).toLowerCase() === 'text' && typeof entry?.content === 'string') {
|
||||
return nonEmptyString(entry.content);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function downloadCodeFor(value) {
|
||||
return nonEmptyString(value?.downloadCode) ?? nonEmptyString(value?.pictureDownloadCode);
|
||||
}
|
||||
|
||||
/** Normalize DingTalk picture and richText callbacks into lazy image references. */
|
||||
export function dingtalkInboundMessage(message, {
|
||||
api,
|
||||
clientId,
|
||||
clientSecret,
|
||||
} = {}) {
|
||||
const msgtype = String(message?.msgtype ?? '').toLowerCase();
|
||||
const content = parsedMessageContent(message);
|
||||
const richEntries = msgtype === 'richtext' ? richTextEntries(content) : [];
|
||||
const text = msgtype === 'text'
|
||||
? nonEmptyString(message?.text?.content) ?? ''
|
||||
: richEntries.map(richTextEntryText).filter(Boolean).join('\n');
|
||||
const imageCodes = [];
|
||||
if (msgtype === 'picture') {
|
||||
const code = downloadCodeFor(content);
|
||||
if (code) imageCodes.push(code);
|
||||
} else if (msgtype === 'richtext') {
|
||||
for (const entry of richEntries) {
|
||||
if (String(entry?.type ?? '').toLowerCase() !== 'picture') continue;
|
||||
const code = downloadCodeFor(entry);
|
||||
if (code) imageCodes.push(code);
|
||||
}
|
||||
}
|
||||
return {
|
||||
content: text,
|
||||
images: imageCodes.map((downloadCode, index) => ({
|
||||
name: index === 0 ? 'image' : `image-${index + 1}`,
|
||||
load: ({ signal, maxBytes }) => {
|
||||
if (typeof api?.downloadImage !== 'function') {
|
||||
throw new Error('DingTalk API does not support image downloads');
|
||||
}
|
||||
return api.downloadImage({
|
||||
clientId,
|
||||
clientSecret,
|
||||
robotCode: message?.robotCode,
|
||||
downloadCode,
|
||||
signal,
|
||||
maxBytes,
|
||||
});
|
||||
},
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
function conversationKey(message, sender) {
|
||||
if (String(message?.conversationType) === '2') {
|
||||
const conversationId = nonEmptyString(message?.conversationId);
|
||||
|
|
@ -317,49 +435,63 @@ export class DingtalkHarnessBridge {
|
|||
return;
|
||||
}
|
||||
|
||||
const text = message?.msgtype === 'text' ? nonEmptyString(message?.text?.content) : null;
|
||||
const promptMessage = dingtalkInboundMessage(message, {
|
||||
api: this.#api,
|
||||
clientId: this.#clientId,
|
||||
clientSecret: this.#clientSecret,
|
||||
});
|
||||
const text = promptMessage.content;
|
||||
const hasImages = hasInboundImages(promptMessage);
|
||||
const isPlainText = String(message?.msgtype).toLowerCase() === 'text';
|
||||
let cardStream = null;
|
||||
let cardStarted = false;
|
||||
try {
|
||||
if (!text) {
|
||||
await this.#send(sessionWebhook, '目前仅支持文字消息。');
|
||||
if (!text && !hasImages) {
|
||||
await this.#send(sessionWebhook, '目前支持文字和图片消息。');
|
||||
return;
|
||||
}
|
||||
|
||||
const command = text.toLowerCase();
|
||||
if (command === '/help') {
|
||||
if (isPlainText && !hasImages && command === '/help') {
|
||||
await this.#send(sessionWebhook, HELP_TEXT);
|
||||
return;
|
||||
}
|
||||
if (command === '/status') {
|
||||
if (isPlainText && !hasImages && command === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#send(sessionWebhook, '钉钉机器人与 DeepSeek Harness 连接正常。');
|
||||
return;
|
||||
}
|
||||
if (command === '/new') {
|
||||
if (isPlainText && !hasImages && command === '/new') {
|
||||
await this.#state.clearSession(key);
|
||||
await this.#send(sessionWebhook, '已开启新会话。请发送你的问题。');
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, key);
|
||||
const workspaceCommand = isPlainText && !hasImages
|
||||
? await runWorkspaceCommand(text, this.#harness, key)
|
||||
: null;
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#send(sessionWebhook, reply);
|
||||
}
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = isPlainText && !hasImages
|
||||
? await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
)
|
||||
: null;
|
||||
if (compactCommand) {
|
||||
await this.#send(sessionWebhook, compactCommand.message);
|
||||
return;
|
||||
}
|
||||
|
||||
const content = hasImages
|
||||
? await promptContentForMessage(promptMessage, { signal: this.#signal })
|
||||
: undefined;
|
||||
if (typeof this.#api.createAiCard === 'function'
|
||||
&& typeof this.#api.updateAiCard === 'function'
|
||||
&& typeof this.#api.finishAiCard === 'function') {
|
||||
|
|
@ -377,7 +509,7 @@ export class DingtalkHarnessBridge {
|
|||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
...(hasImages ? { content } : { text }),
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: {
|
||||
|
|
@ -400,13 +532,17 @@ export class DingtalkHarnessBridge {
|
|||
increment(this.#status, 'messagesReplied');
|
||||
this.#status.lastReplyAt = new Date().toISOString();
|
||||
this.#status.lastError = null;
|
||||
} catch {
|
||||
} catch (error) {
|
||||
if (this.#signal?.aborted) return;
|
||||
this.#status.lastError = '钉钉消息处理失败。';
|
||||
this.#logger.error?.('[dsh-dingtalk] failed to process an inbound message');
|
||||
this.#logger.error?.(
|
||||
'[dsh-dingtalk] failed to process an inbound message',
|
||||
safeErrorDiagnostic(error),
|
||||
);
|
||||
try {
|
||||
const streamed = cardStarted && await cardStream.finish(CARD_ERROR_TEXT);
|
||||
if (!streamed) await this.#send(sessionWebhook, CARD_ERROR_TEXT);
|
||||
const errorText = dingtalkImageErrorUserMessage(error) ?? CARD_ERROR_TEXT;
|
||||
const streamed = cardStarted && await cardStream.finish(errorText);
|
||||
if (!streamed) await this.#send(sessionWebhook, errorText);
|
||||
} catch {
|
||||
this.#logger.error?.('[dsh-dingtalk] failed to send the safe error reply');
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,9 +1,19 @@
|
|||
import { createEditableMessageStream, splitMessageText } from '../shared/editable-message-stream.mjs';
|
||||
import { fetchImageBuffer } from '../shared/image-prompt.mjs';
|
||||
import { DiscordApi } from './discord-api.mjs';
|
||||
import { createDiscordBridgeStatus, DiscordHarnessBridge } from './discord-bridge.mjs';
|
||||
|
||||
const DISCORD_GATEWAY_INTENTS = (1 << 0) | (1 << 9) | (1 << 12);
|
||||
const RECONNECT_DELAYS_MS = Object.freeze([1_000, 3_000, 5_000, 10_000, 30_000]);
|
||||
const IMAGE_MEDIA_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp', 'image/gif']);
|
||||
const IMAGE_FILE_TYPES = new Map([
|
||||
['.jpg', 'image/jpeg'],
|
||||
['.jpeg', 'image/jpeg'],
|
||||
['.png', 'image/png'],
|
||||
['.webp', 'image/webp'],
|
||||
['.gif', 'image/gif'],
|
||||
]);
|
||||
const DISCORD_IMAGE_HOSTS = Object.freeze(['cdn.discordapp.com']);
|
||||
|
||||
function socketUrl(value) {
|
||||
const url = new URL(value);
|
||||
|
|
@ -48,7 +58,37 @@ function stripBotMention(text, botId) {
|
|||
return text.replace(new RegExp(`<@!?${botId}>`, 'g'), '').trim();
|
||||
}
|
||||
|
||||
export function normalizeDiscordMessage(message, botId) {
|
||||
function attachmentMediaType(attachment) {
|
||||
const value = typeof attachment?.content_type === 'string'
|
||||
? attachment.content_type.split(';', 1)[0].trim().toLowerCase() : '';
|
||||
if (IMAGE_MEDIA_TYPES.has(value)) return value;
|
||||
const filename = typeof attachment?.filename === 'string' ? attachment.filename.toLowerCase() : '';
|
||||
for (const [extension, mediaType] of IMAGE_FILE_TYPES) {
|
||||
if (filename.endsWith(extension)) return mediaType;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function attachmentSize(value) {
|
||||
return Number.isSafeInteger(value) && value >= 0 ? value : undefined;
|
||||
}
|
||||
|
||||
function discordImageSource(attachment, fetchImpl) {
|
||||
const mediaType = attachmentMediaType(attachment);
|
||||
if (!mediaType || typeof attachment?.url !== 'string') return null;
|
||||
return {
|
||||
name: typeof attachment.filename === 'string' ? attachment.filename : undefined,
|
||||
mediaType,
|
||||
size: attachmentSize(attachment.size),
|
||||
load: (options) => fetchImageBuffer(attachment.url, {
|
||||
...options,
|
||||
fetchImpl,
|
||||
allowedHosts: DISCORD_IMAGE_HOSTS,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeDiscordMessage(message, botId, { fetchImpl = fetch } = {}) {
|
||||
if (!message?.id || !message?.channel_id || !message?.author?.id) return null;
|
||||
const direct = !message.guild_id;
|
||||
const addressed = direct
|
||||
|
|
@ -60,6 +100,9 @@ export function normalizeDiscordMessage(message, botId) {
|
|||
kind: direct ? 'direct' : 'group',
|
||||
conversationId: String(message.channel_id),
|
||||
content: stripBotMention(message.content ?? '', botId),
|
||||
images: Array.isArray(message.attachments)
|
||||
? message.attachments.map((attachment) => discordImageSource(attachment, fetchImpl)).filter(Boolean)
|
||||
: [],
|
||||
addressed,
|
||||
replyTarget: {
|
||||
channelId: String(message.channel_id),
|
||||
|
|
|
|||
|
|
@ -1,10 +1,16 @@
|
|||
import {
|
||||
conversationKey,
|
||||
extractInboundMessage,
|
||||
extractText,
|
||||
isAllowedSender,
|
||||
isBotSender,
|
||||
splitText,
|
||||
} from './message-utils.mjs';
|
||||
import {
|
||||
hasInboundImages,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from '../shared/image-prompt.mjs';
|
||||
import {
|
||||
harnessAnswerForQuestion,
|
||||
harnessQuestionText,
|
||||
|
|
@ -21,7 +27,7 @@ const RESOLVED_REPLY_TTL_MS = 30 * 60_000;
|
|||
const HELP_TEXT = [
|
||||
'北汇星河 AIOS 已连接 DeepSeek Harness。',
|
||||
'',
|
||||
'直接发送问题即可继续当前会话。',
|
||||
'直接发送文字或图片即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -273,7 +279,8 @@ export class FeishuHarnessBridge {
|
|||
await this.#finishReaction(messageId, processingReaction, 'ERROR');
|
||||
await this.#send(
|
||||
event.message.chat_id,
|
||||
'处理失败，请稍后重试。如果问题持续，请在 DeepSeek Harness 的飞书插件页面检查连接状态。',
|
||||
imagePromptUserMessage(error)
|
||||
?? '处理失败，请稍后重试。如果问题持续，请在 DeepSeek Harness 的飞书插件页面检查连接状态。',
|
||||
).catch(() => undefined);
|
||||
}
|
||||
|
||||
|
|
@ -297,40 +304,47 @@ export class FeishuHarnessBridge {
|
|||
this.#status.messagesReceived += 1;
|
||||
}
|
||||
|
||||
const text = extractText(event);
|
||||
if (!text) {
|
||||
await this.#send(event.message.chat_id, '目前仅支持文字消息。');
|
||||
const message = extractInboundMessage(event, this.#client);
|
||||
const text = message.content;
|
||||
const hasImages = hasInboundImages(message);
|
||||
const commandText = event.message.message_type === 'text' && !hasImages ? text : null;
|
||||
if (!text && !hasImages) {
|
||||
await this.#send(event.message.chat_id, '目前支持文字和图片消息。');
|
||||
return;
|
||||
}
|
||||
|
||||
if (text === '/help') {
|
||||
if (commandText === '/help') {
|
||||
await this.#send(event.message.chat_id, HELP_TEXT);
|
||||
return;
|
||||
}
|
||||
if (text === '/new') {
|
||||
if (commandText === '/new') {
|
||||
await this.#state.clearSession(key);
|
||||
await this.#send(event.message.chat_id, '已开启全新 Harness 会话。');
|
||||
return;
|
||||
}
|
||||
if (text === '/status') {
|
||||
if (commandText === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#send(event.message.chat_id, '飞书机器人与 DeepSeek Harness 连接正常。');
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, key);
|
||||
const workspaceCommand = commandText === null
|
||||
? null
|
||||
: await runWorkspaceCommand(text, this.#harness, key);
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#send(event.message.chat_id, reply);
|
||||
}
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = commandText === null
|
||||
? null
|
||||
: await runCompactCommand(
|
||||
commandText,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
if (compactCommand) {
|
||||
await this.#send(event.message.chat_id, compactCommand.message);
|
||||
return;
|
||||
|
|
@ -338,7 +352,7 @@ export class FeishuHarnessBridge {
|
|||
|
||||
this.#logger.info?.(`[dsh-feishu] processing ${event.message.chat_type} message ${messageId}`);
|
||||
try {
|
||||
await this.#answerWithStream(event, key, text);
|
||||
await this.#answerWithStream(event, key, message);
|
||||
this.#status.messagesReplied += 1;
|
||||
this.#status.lastReplyAt = new Date().toISOString();
|
||||
this.#status.lastError = null;
|
||||
|
|
@ -362,15 +376,20 @@ export class FeishuHarnessBridge {
|
|||
};
|
||||
}
|
||||
|
||||
async #answerWithStream(event, key, text) {
|
||||
async #answerWithStream(event, key, message) {
|
||||
const chatId = event.message.chat_id;
|
||||
const messageId = event.message.message_id;
|
||||
const text = message.content;
|
||||
const content = hasInboundImages(message)
|
||||
? await promptContentForMessage(message, { signal: this.#signal })
|
||||
: undefined;
|
||||
if (!this.#channel?.stream) {
|
||||
const { answer } = await askInWorkspaceSession({
|
||||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
content,
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: this.#interactionAskOptions(event, key),
|
||||
|
|
@ -398,6 +417,7 @@ export class FeishuHarnessBridge {
|
|||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
content,
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions,
|
||||
|
|
@ -425,6 +445,7 @@ export class FeishuHarnessBridge {
|
|||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
content,
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: this.#interactionAskOptions(event, key),
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import { ImagePromptError } from '../shared/image-prompt.mjs';
|
||||
|
||||
export function conversationKey(event) {
|
||||
const chatType = event?.message?.chat_type;
|
||||
if (chatType === 'p2p') {
|
||||
|
|
@ -10,19 +12,151 @@ export function conversationKey(event) {
|
|||
return `group:${chatId}`;
|
||||
}
|
||||
|
||||
export function extractText(event) {
|
||||
if (event?.message?.message_type !== 'text') return null;
|
||||
let parsed;
|
||||
function parsedMessageContent(event) {
|
||||
const value = event?.message?.content;
|
||||
if (value && typeof value === 'object') return value;
|
||||
if (typeof value !== 'string') return null;
|
||||
try {
|
||||
parsed = JSON.parse(event.message.content);
|
||||
const parsed = JSON.parse(value);
|
||||
return parsed && typeof parsed === 'object' ? parsed : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
let text = typeof parsed.text === 'string' ? parsed.text : '';
|
||||
for (const mention of event.message.mentions ?? []) {
|
||||
if (typeof mention.key === 'string' && mention.key) text = text.replaceAll(mention.key, '');
|
||||
}
|
||||
|
||||
function withoutMentions(text, event) {
|
||||
let result = typeof text === 'string' ? text : '';
|
||||
for (const mention of event?.message?.mentions ?? []) {
|
||||
if (typeof mention?.key === 'string' && mention.key) {
|
||||
result = result.replaceAll(mention.key, '');
|
||||
}
|
||||
}
|
||||
return text.trim();
|
||||
return result.trim();
|
||||
}
|
||||
|
||||
export function extractText(event) {
|
||||
if (event?.message?.message_type !== 'text') return null;
|
||||
const parsed = parsedMessageContent(event);
|
||||
return parsed ? withoutMentions(parsed.text, event) : null;
|
||||
}
|
||||
|
||||
function nonEmptyString(value) {
|
||||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function postContent(event, parsed = parsedMessageContent(event)) {
|
||||
if (event?.message?.message_type !== 'post') return null;
|
||||
if (!parsed) return null;
|
||||
|
||||
const lines = [];
|
||||
const title = nonEmptyString(withoutMentions(parsed.title, event));
|
||||
if (title) lines.push(title);
|
||||
const imageKeys = [];
|
||||
for (const paragraph of Array.isArray(parsed.content) ? parsed.content : []) {
|
||||
if (!Array.isArray(paragraph)) continue;
|
||||
let visibleText = '';
|
||||
for (const element of paragraph) {
|
||||
const tag = String(element?.tag ?? '').toLowerCase();
|
||||
if (tag === 'img') {
|
||||
const key = nonEmptyString(element?.image_key);
|
||||
if (key) imageKeys.push(key);
|
||||
} else if (tag === 'text' || tag === 'a' || tag === 'link') {
|
||||
if (typeof element?.text === 'string') visibleText += element.text;
|
||||
}
|
||||
}
|
||||
const line = nonEmptyString(withoutMentions(visibleText, event));
|
||||
if (line) lines.push(line);
|
||||
}
|
||||
|
||||
return {
|
||||
text: lines.join('\n'),
|
||||
imageKeys,
|
||||
};
|
||||
}
|
||||
|
||||
function headerValue(headers, name) {
|
||||
if (typeof headers?.get === 'function') return headers.get(name);
|
||||
return headers?.[name] ?? headers?.[name.toLowerCase()] ?? null;
|
||||
}
|
||||
|
||||
function declaredSize(headers) {
|
||||
const header = headerValue(headers, 'content-length');
|
||||
if (header === null || header === undefined || header === '') return null;
|
||||
const value = Number(header);
|
||||
return Number.isFinite(value) && value >= 0 ? value : null;
|
||||
}
|
||||
|
||||
async function readBoundedStream(stream, { signal, maxBytes }) {
|
||||
if (!stream || typeof stream[Symbol.asyncIterator] !== 'function') {
|
||||
throw new Error('Feishu image download returned no readable stream');
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
const abort = () => stream.destroy?.(
|
||||
signal.reason ?? new DOMException('Feishu image download aborted', 'AbortError'),
|
||||
);
|
||||
signal?.addEventListener('abort', abort, { once: true });
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
signal?.throwIfAborted();
|
||||
const data = Buffer.from(chunk);
|
||||
size += data.length;
|
||||
if (size > maxBytes) {
|
||||
stream.destroy?.();
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Feishu image exceeds ${maxBytes} bytes`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
chunks.push(data);
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
return Buffer.concat(chunks, size);
|
||||
} finally {
|
||||
signal?.removeEventListener('abort', abort);
|
||||
}
|
||||
}
|
||||
|
||||
function feishuImageSource(event, client, key) {
|
||||
return {
|
||||
async load({ signal, maxBytes }) {
|
||||
signal?.throwIfAborted();
|
||||
const resource = await client?.im?.v1?.messageResource?.get?.({
|
||||
path: {
|
||||
message_id: event.message.message_id,
|
||||
file_key: key,
|
||||
},
|
||||
params: { type: 'image' },
|
||||
});
|
||||
signal?.throwIfAborted();
|
||||
const size = declaredSize(resource?.headers);
|
||||
if (size !== null && size > maxBytes) {
|
||||
resource?.getReadableStream?.().destroy?.();
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Feishu image declares ${size} bytes; the limit is ${maxBytes}`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
return readBoundedStream(resource?.getReadableStream?.(), { signal, maxBytes });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function extractInboundMessage(event, client) {
|
||||
const messageType = event?.message?.message_type;
|
||||
const parsed = parsedMessageContent(event);
|
||||
const post = postContent(event, parsed);
|
||||
const standaloneImageKey = messageType === 'image'
|
||||
? nonEmptyString(parsed?.image_key)
|
||||
: null;
|
||||
const imageKeys = standaloneImageKey ? [standaloneImageKey] : post?.imageKeys ?? [];
|
||||
return {
|
||||
content: messageType === 'text' ? extractText(event) ?? '' : post?.text ?? '',
|
||||
images: imageKeys.map((key) => feishuImageSource(event, client, key)),
|
||||
};
|
||||
}
|
||||
|
||||
export function splitText(text, maxChars = 9000) {
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ export const FEISHU_SECRET_REF = 'DSH_FEISHU_APP_SECRET';
|
|||
export const REQUIRED_TENANT_SCOPES = Object.freeze([
|
||||
'im:message.p2p_msg:readonly',
|
||||
'im:message.group_at_msg:readonly',
|
||||
'im:message:readonly',
|
||||
'im:message:send_as_bot',
|
||||
'im:message.reactions:write_only',
|
||||
'im:message:recall',
|
||||
|
|
|
|||
|
|
@ -7,13 +7,30 @@ import {
|
|||
} from '../shared/harness-question.mjs';
|
||||
import { HarnessApprovalQueue } from '../shared/harness-approval.mjs';
|
||||
import { askInWorkspaceSession } from '../shared/workspace-session.mjs';
|
||||
import {
|
||||
fetchImageBuffer,
|
||||
hasInboundImages,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from '../shared/image-prompt.mjs';
|
||||
|
||||
const INTERACTION_RESOLVED_TEXT = '这个问题已在其他客户端处理，无需再次回答。';
|
||||
|
||||
export const QQ_IMAGE_HOSTS = Object.freeze([
|
||||
'.myqcloud.com',
|
||||
'.qpic.cn',
|
||||
'.qq.com',
|
||||
'.qq.com.cn',
|
||||
'.tencentcos.com',
|
||||
'.ugcimg.cn',
|
||||
]);
|
||||
|
||||
const QQ_IMAGE_FILENAME = /\.(?:gif|jpe?g|png|webp)$/i;
|
||||
|
||||
const HELP_TEXT = [
|
||||
'QQ 机器人已连接 DeepSeek Harness。',
|
||||
'',
|
||||
'直接发送文字即可继续当前会话。',
|
||||
'直接发送文字或图片即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -32,6 +49,51 @@ function safeText(message) {
|
|||
return typeof message?.content === 'string' ? message.content.trim() : '';
|
||||
}
|
||||
|
||||
function attachmentMediaType(attachment) {
|
||||
const value = nonEmptyString(attachment?.content_type ?? attachment?.contentType);
|
||||
if (!value) return null;
|
||||
return value.split(';', 1)[0].trim().toLowerCase();
|
||||
}
|
||||
|
||||
function isQqImageAttachment(attachment) {
|
||||
const mediaType = attachmentMediaType(attachment);
|
||||
return mediaType?.startsWith('image/') === true
|
||||
|| QQ_IMAGE_FILENAME.test(nonEmptyString(attachment?.filename) ?? '');
|
||||
}
|
||||
|
||||
function hasQqImageAttachments(message) {
|
||||
return Array.isArray(message?.attachments)
|
||||
&& message.attachments.some(isQqImageAttachment);
|
||||
}
|
||||
|
||||
/** Convert QQ's attachment metadata into lazily downloaded image references. */
|
||||
export function qqInboundMessage(message, { fetchImpl = fetch } = {}) {
|
||||
if (typeof fetchImpl !== 'function') throw new TypeError('fetchImpl must be a function');
|
||||
const images = [];
|
||||
for (const attachment of message?.attachments ?? []) {
|
||||
if (!isQqImageAttachment(attachment)) continue;
|
||||
const url = nonEmptyString(attachment?.url);
|
||||
const name = nonEmptyString(attachment?.filename) ?? undefined;
|
||||
const mediaType = attachmentMediaType(attachment);
|
||||
const declaredSize = Number(attachment?.size);
|
||||
images.push({
|
||||
...(name ? { name } : {}),
|
||||
...(mediaType?.startsWith('image/') ? { mediaType } : {}),
|
||||
...(Number.isFinite(declaredSize) && declaredSize >= 0 ? { size: declaredSize } : {}),
|
||||
load: ({ signal, maxBytes }) => {
|
||||
if (!url) throw new Error('QQ image attachment has no download URL');
|
||||
return fetchImageBuffer(url, {
|
||||
fetchImpl,
|
||||
signal,
|
||||
maxBytes,
|
||||
allowedHosts: QQ_IMAGE_HOSTS,
|
||||
});
|
||||
},
|
||||
});
|
||||
}
|
||||
return { content: safeText(message), images };
|
||||
}
|
||||
|
||||
function nonEmptyString(value) {
|
||||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
|
@ -40,6 +102,7 @@ function canClaimInteractionReply(message, pending) {
|
|||
return pending.questions[pending.index]
|
||||
&& nonEmptyString(message?.senderId) === pending.actor
|
||||
&& (message.kind !== 'group' || message.rawEventType === 'GROUP_AT_MESSAGE_CREATE')
|
||||
&& !hasQqImageAttachments(message)
|
||||
&& nonEmptyString(safeText(message));
|
||||
}
|
||||
|
||||
|
|
@ -64,6 +127,7 @@ export class QqHarnessBridge {
|
|||
#logger;
|
||||
#replyTimeoutMs;
|
||||
#signal;
|
||||
#fetchImpl;
|
||||
#queues = new Map();
|
||||
#pendingInteractions = new Map();
|
||||
#interactionKeys = new Map();
|
||||
|
|
@ -80,10 +144,12 @@ export class QqHarnessBridge {
|
|||
logger = console,
|
||||
replyTimeoutMs = 600_000,
|
||||
signal,
|
||||
fetchImpl = fetch,
|
||||
}) {
|
||||
if (!bot || typeof bot.sendText !== 'function') throw new TypeError('QQ bot client is required');
|
||||
if (!ownerUserOpenid) throw new TypeError('QQ scanner identity is required');
|
||||
if (!harness || !state) throw new TypeError('Harness client and state store are required');
|
||||
if (typeof fetchImpl !== 'function') throw new TypeError('fetchImpl must be a function');
|
||||
this.#bot = bot;
|
||||
this.#ownerUserOpenid = ownerUserOpenid;
|
||||
this.#harness = harness;
|
||||
|
|
@ -92,6 +158,7 @@ export class QqHarnessBridge {
|
|||
this.#logger = logger;
|
||||
this.#replyTimeoutMs = replyTimeoutMs;
|
||||
this.#signal = signal;
|
||||
this.#fetchImpl = fetchImpl;
|
||||
this.#approvals = new HarnessApprovalQueue({ label: 'qq', logger });
|
||||
}
|
||||
|
||||
|
|
@ -114,7 +181,7 @@ export class QqHarnessBridge {
|
|||
key,
|
||||
actor: sender,
|
||||
messageId,
|
||||
text: safeText(message),
|
||||
text: hasQqImageAttachments(message) ? '' : safeText(message),
|
||||
addressed: message.kind !== 'group' || message.rawEventType === 'GROUP_AT_MESSAGE_CREATE',
|
||||
hasPendingQuestion: Boolean(pending),
|
||||
questionCompletion: pending?.submitting || pending?.claimedReplyMessageId
|
||||
|
|
@ -210,32 +277,37 @@ export class QqHarnessBridge {
|
|||
if (message.kind === 'group' && message.rawEventType !== 'GROUP_AT_MESSAGE_CREATE') return;
|
||||
|
||||
const target = message.replyTarget;
|
||||
const text = safeText(message);
|
||||
const promptMessage = qqInboundMessage(message, { fetchImpl: this.#fetchImpl });
|
||||
const text = promptMessage.content;
|
||||
const hasImages = hasInboundImages(promptMessage);
|
||||
let stream = null;
|
||||
try {
|
||||
if (!text) {
|
||||
await this.#bot.sendText(target, '目前仅支持文字消息。');
|
||||
if (!text && !hasImages) {
|
||||
await this.#bot.sendText(target, '目前支持文字和图片消息。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const command = text.toLowerCase();
|
||||
if (command === '/help') {
|
||||
if (!hasImages && command === '/help') {
|
||||
await this.#bot.sendText(target, HELP_TEXT);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/status') {
|
||||
if (!hasImages && command === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#bot.sendText(target, 'QQ 机器人与 DeepSeek Harness 连接正常。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/new') {
|
||||
if (!hasImages && command === '/new') {
|
||||
await this.#state.clearSession(key);
|
||||
await this.#bot.sendText(target, '已开启新会话。请发送你的问题。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, key);
|
||||
const workspaceCommand = hasImages
|
||||
? null
|
||||
: await runWorkspaceCommand(text, this.#harness, key);
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#bot.sendText(target, reply);
|
||||
|
|
@ -243,20 +315,24 @@ export class QqHarnessBridge {
|
|||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = hasImages
|
||||
? null
|
||||
: await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
if (compactCommand) {
|
||||
await this.#bot.sendText(target, compactCommand.message);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
|
||||
let stream = null;
|
||||
const content = hasImages
|
||||
? await promptContentForMessage(promptMessage, { signal: this.#signal })
|
||||
: undefined;
|
||||
let streamFinished = false;
|
||||
if (message.kind === 'c2c' && target?.msgId && typeof this.#bot.openStream === 'function') {
|
||||
try {
|
||||
|
|
@ -271,7 +347,7 @@ export class QqHarnessBridge {
|
|||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
...(hasImages ? { content } : { text }),
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: {
|
||||
|
|
@ -316,11 +392,15 @@ export class QqHarnessBridge {
|
|||
this.#status.lastReplyAt = new Date().toISOString();
|
||||
this.#status.lastError = null;
|
||||
} catch (error) {
|
||||
stream?.cancel?.();
|
||||
if (this.#signal?.aborted) return;
|
||||
this.#status.lastError = error?.message ?? String(error);
|
||||
this.#logger.error?.('[dsh-im:qq] failed to process an inbound message:', error);
|
||||
try {
|
||||
await this.#bot.sendText(target, '消息处理失败，请稍后重试。');
|
||||
await this.#bot.sendText(
|
||||
target,
|
||||
imagePromptUserMessage(error) ?? '消息处理失败，请稍后重试。',
|
||||
);
|
||||
await this.#state.markSeen(messageId);
|
||||
} catch (sendError) {
|
||||
this.#logger.error?.('[dsh-im:qq] failed to send the safe error reply:', sendError);
|
||||
|
|
@ -345,7 +425,7 @@ export class QqHarnessBridge {
|
|||
|
||||
if (message.kind === 'group' && message.rawEventType !== 'GROUP_AT_MESSAGE_CREATE') return;
|
||||
const text = nonEmptyString(safeText(message));
|
||||
if (!text) {
|
||||
if (!text || hasQqImageAttachments(message)) {
|
||||
await this.#bot.sendText(message.replyTarget, '请用文字回答当前问题。');
|
||||
return;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -623,7 +623,7 @@ export class HarnessClient {
|
|||
return ownership ? { ownership, recovered: true } : null;
|
||||
}
|
||||
|
||||
async ask(sessionId, text, options = {}) {
|
||||
async ask(sessionId, prompt, options = {}) {
|
||||
if (typeof options === 'number') options = { timeoutMs: options };
|
||||
const timeoutMs = options.timeoutMs ?? 600_000;
|
||||
const signal = options.signal;
|
||||
|
|
@ -693,10 +693,16 @@ export class HarnessClient {
|
|||
]);
|
||||
}
|
||||
|
||||
const content = typeof prompt === 'string'
|
||||
? [{ type: 'text', text: prompt }]
|
||||
: prompt;
|
||||
if (!Array.isArray(content) || content.length === 0) {
|
||||
throw new TypeError('Harness prompt content is required');
|
||||
}
|
||||
await this.rpc('session.prompt', {
|
||||
sessionId,
|
||||
mode: 'queue',
|
||||
content: [{ type: 'text', text }],
|
||||
content,
|
||||
clientTimeZone: Intl.DateTimeFormat().resolvedOptions().timeZone,
|
||||
}, 30_000, { rpcId: promptRpcId, signal });
|
||||
|
||||
|
|
|
|||
268
src/channels/shared/image-prompt.mjs
Normal file
268
src/channels/shared/image-prompt.mjs
Normal file
|
|
@ -0,0 +1,268 @@
|
|||
const DEFAULT_MAX_IMAGE_BYTES = 5 * 1024 * 1024;
|
||||
const DEFAULT_MAX_IMAGES = 20;
|
||||
const DEFAULT_MAX_TOTAL_IMAGE_BYTES = 20 * 1024 * 1024;
|
||||
|
||||
export const DEFAULT_IMAGE_PROMPT = '请分析这张图片。';
|
||||
|
||||
export class ImagePromptError extends Error {
|
||||
constructor(code, message, userMessage, options = {}) {
|
||||
super(message, options);
|
||||
this.name = 'ImagePromptError';
|
||||
this.code = code;
|
||||
this.userMessage = userMessage;
|
||||
}
|
||||
}
|
||||
|
||||
function requestSignal(signal, timeoutMs) {
|
||||
const timeout = AbortSignal.timeout(timeoutMs);
|
||||
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
}
|
||||
|
||||
async function cancelResponseBody(response) {
|
||||
try {
|
||||
await response?.body?.cancel?.();
|
||||
} catch {
|
||||
// The original download error is more useful than a best-effort cleanup failure.
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchImageBuffer(url, {
|
||||
fetchImpl = fetch,
|
||||
headers,
|
||||
signal,
|
||||
maxBytes = DEFAULT_MAX_IMAGE_BYTES,
|
||||
timeoutMs = 15_000,
|
||||
allowedHosts,
|
||||
} = {}) {
|
||||
const target = new URL(url);
|
||||
if (target.protocol !== 'https:') throw new Error('Image download URL must use HTTPS');
|
||||
if (Array.isArray(allowedHosts) && !allowedHosts.some((rule) => (
|
||||
typeof rule === 'string'
|
||||
&& (target.hostname === rule
|
||||
|| (rule.startsWith('.')
|
||||
&& (target.hostname === rule.slice(1) || target.hostname.endsWith(rule))))
|
||||
))) {
|
||||
throw new Error('Image download URL is not hosted by the messaging platform');
|
||||
}
|
||||
const response = await fetchImpl(target, {
|
||||
method: 'GET',
|
||||
headers,
|
||||
signal: requestSignal(signal, timeoutMs),
|
||||
redirect: 'manual',
|
||||
});
|
||||
if (Number.isInteger(response?.status) && response.status >= 300 && response.status < 400) {
|
||||
await cancelResponseBody(response);
|
||||
throw new ImagePromptError(
|
||||
'image-redirect-blocked',
|
||||
`Image download redirect was blocked (HTTP ${response.status})`,
|
||||
'图片下载地址发生了重定向，暂时无法读取。',
|
||||
);
|
||||
}
|
||||
if (!response?.ok) {
|
||||
await cancelResponseBody(response);
|
||||
throw new ImagePromptError(
|
||||
'image-http-error',
|
||||
`Image download failed with HTTP ${response?.status ?? 'unknown'}`,
|
||||
`图片下载失败（HTTP ${response?.status ?? 'unknown'}），请重新发送后再试。`,
|
||||
);
|
||||
}
|
||||
const declaredLength = Number(response.headers?.get?.('content-length'));
|
||||
if (Number.isFinite(declaredLength) && declaredLength > maxBytes) {
|
||||
await cancelResponseBody(response);
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Image response declares ${declaredLength} bytes; the limit is ${maxBytes}`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
|
||||
if (response.body?.[Symbol.asyncIterator]) {
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
for await (const chunk of response.body) {
|
||||
const data = Buffer.from(chunk);
|
||||
size += data.length;
|
||||
if (size > maxBytes) {
|
||||
await response.body.cancel?.().catch?.(() => undefined);
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Image response exceeded ${maxBytes} bytes`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
chunks.push(data);
|
||||
}
|
||||
return Buffer.concat(chunks, size);
|
||||
}
|
||||
|
||||
const data = Buffer.from(await response.arrayBuffer());
|
||||
if (data.length > maxBytes) {
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Image response contains ${data.length} bytes; the limit is ${maxBytes}`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
function cleanText(value) {
|
||||
return typeof value === 'string' ? value.trim() : '';
|
||||
}
|
||||
|
||||
function imageSources(message) {
|
||||
return Array.isArray(message?.images) ? message.images.filter(Boolean) : [];
|
||||
}
|
||||
|
||||
function safeName(value) {
|
||||
if (typeof value !== 'string') return undefined;
|
||||
const name = value
|
||||
.replaceAll('\\', '/')
|
||||
.split('/')
|
||||
.at(-1)
|
||||
?.replace(/[\u0000-\u001f\u007f]/g, '')
|
||||
.trim()
|
||||
.slice(0, 255);
|
||||
return name || undefined;
|
||||
}
|
||||
|
||||
function detectedImageMediaType(data) {
|
||||
if (data.length >= 8
|
||||
&& data[0] === 0x89 && data[1] === 0x50 && data[2] === 0x4e && data[3] === 0x47
|
||||
&& data[4] === 0x0d && data[5] === 0x0a && data[6] === 0x1a && data[7] === 0x0a) {
|
||||
return 'image/png';
|
||||
}
|
||||
if (data.length >= 3 && data[0] === 0xff && data[1] === 0xd8 && data[2] === 0xff) {
|
||||
return 'image/jpeg';
|
||||
}
|
||||
if (data.length >= 6) {
|
||||
const signature = data.subarray(0, 6).toString('ascii');
|
||||
if (signature === 'GIF87a' || signature === 'GIF89a') return 'image/gif';
|
||||
}
|
||||
if (data.length >= 12
|
||||
&& data.subarray(0, 4).toString('ascii') === 'RIFF'
|
||||
&& data.subarray(8, 12).toString('ascii') === 'WEBP') {
|
||||
return 'image/webp';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function loadedImage(value) {
|
||||
if (Buffer.isBuffer(value) || value instanceof Uint8Array) {
|
||||
return { data: Buffer.from(value) };
|
||||
}
|
||||
const raw = value?.data ?? value?.buffer;
|
||||
if (Buffer.isBuffer(raw) || raw instanceof Uint8Array) {
|
||||
return {
|
||||
data: Buffer.from(raw),
|
||||
name: value?.name ?? value?.filename,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function hasInboundImages(message) {
|
||||
return imageSources(message).length > 0;
|
||||
}
|
||||
|
||||
export function hasInboundPrompt(message) {
|
||||
return Boolean(cleanText(message?.content)) || hasInboundImages(message);
|
||||
}
|
||||
|
||||
export async function promptContentForMessage(message, {
|
||||
signal,
|
||||
maxImageBytes = DEFAULT_MAX_IMAGE_BYTES,
|
||||
maxImages = DEFAULT_MAX_IMAGES,
|
||||
maxTotalImageBytes = DEFAULT_MAX_TOTAL_IMAGE_BYTES,
|
||||
} = {}) {
|
||||
const sources = imageSources(message);
|
||||
if (sources.length > maxImages) {
|
||||
throw new ImagePromptError(
|
||||
'too-many-images',
|
||||
`Image message contains ${sources.length} images; the limit is ${maxImages}`,
|
||||
`一次最多只能处理 ${maxImages} 张图片。`,
|
||||
);
|
||||
}
|
||||
|
||||
const text = cleanText(message?.content);
|
||||
const content = [];
|
||||
let totalImageBytes = 0;
|
||||
if (text) content.push({ type: 'text', text });
|
||||
else if (sources.length > 0) content.push({ type: 'text', text: DEFAULT_IMAGE_PROMPT });
|
||||
|
||||
for (const [index, source] of sources.entries()) {
|
||||
signal?.throwIfAborted();
|
||||
if (Number.isFinite(source?.size) && source.size > maxImageBytes) {
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Image ${index + 1} declares ${source.size} bytes; the limit is ${maxImageBytes}`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
if (Number.isFinite(source?.size) && totalImageBytes + source.size > maxTotalImageBytes) {
|
||||
throw new ImagePromptError(
|
||||
'images-too-large',
|
||||
`Images declare more than ${maxTotalImageBytes} bytes in total`,
|
||||
'一次发送的图片总大小过大，请减少图片数量或压缩后重试。',
|
||||
);
|
||||
}
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = source?.data === undefined
|
||||
? await source?.load?.({ signal, maxBytes: maxImageBytes })
|
||||
: source.data;
|
||||
} catch (error) {
|
||||
if (signal?.aborted || error?.name === 'AbortError' || error?.name === 'TimeoutError') throw error;
|
||||
if (error instanceof ImagePromptError) throw error;
|
||||
throw new ImagePromptError(
|
||||
'image-download-failed',
|
||||
`Unable to download image ${index + 1}: ${error?.message ?? String(error)}`,
|
||||
'图片下载失败，请重新发送后再试。',
|
||||
{ cause: error },
|
||||
);
|
||||
}
|
||||
const loaded = loadedImage(result);
|
||||
if (!loaded?.data.length) {
|
||||
throw new ImagePromptError(
|
||||
'invalid-image-data',
|
||||
`Image ${index + 1} returned no data`,
|
||||
'未能读取图片内容，请重新发送。',
|
||||
);
|
||||
}
|
||||
if (loaded.data.length > maxImageBytes) {
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Image ${index + 1} contains ${loaded.data.length} bytes; the limit is ${maxImageBytes}`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
if (totalImageBytes + loaded.data.length > maxTotalImageBytes) {
|
||||
throw new ImagePromptError(
|
||||
'images-too-large',
|
||||
`Images contain more than ${maxTotalImageBytes} bytes in total`,
|
||||
'一次发送的图片总大小过大，请减少图片数量或压缩后重试。',
|
||||
);
|
||||
}
|
||||
totalImageBytes += loaded.data.length;
|
||||
const mediaType = detectedImageMediaType(loaded.data);
|
||||
if (!mediaType) {
|
||||
throw new ImagePromptError(
|
||||
'unsupported-image-type',
|
||||
`Image ${index + 1} is not JPEG, PNG, GIF, or WebP`,
|
||||
'暂不支持该图片格式，请发送 JPEG、PNG、WebP 或 GIF 图片。',
|
||||
);
|
||||
}
|
||||
content.push({
|
||||
type: 'image',
|
||||
mediaType,
|
||||
data: loaded.data.toString('base64'),
|
||||
...(safeName(loaded.name ?? source?.name) ? { name: safeName(loaded.name ?? source?.name) } : {}),
|
||||
});
|
||||
}
|
||||
return content;
|
||||
}
|
||||
|
||||
export function imagePromptUserMessage(error) {
|
||||
return error instanceof ImagePromptError ? error.userMessage : null;
|
||||
}
|
||||
|
|
@ -2,6 +2,11 @@ import { runWorkspaceCommand } from './workspace-command.mjs';
|
|||
import { runCompactCommand } from './compact-command.mjs';
|
||||
import { askInWorkspaceSession } from './workspace-session.mjs';
|
||||
import { HarnessApprovalQueue } from './harness-approval.mjs';
|
||||
import {
|
||||
hasInboundImages,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from './image-prompt.mjs';
|
||||
import {
|
||||
harnessAnswerForQuestion,
|
||||
harnessQuestionText,
|
||||
|
|
@ -17,6 +22,7 @@ function cleanText(value) {
|
|||
function canClaimInteractionReply(message, pending, senderId) {
|
||||
return pending.actor === senderId
|
||||
&& (message.kind !== 'group' || message.addressed === true)
|
||||
&& !hasInboundImages(message)
|
||||
&& Boolean(cleanText(message.content));
|
||||
}
|
||||
|
||||
|
|
@ -98,7 +104,7 @@ export class TextHarnessBridge {
|
|||
key,
|
||||
actor: senderId,
|
||||
messageId,
|
||||
text: normalized.content,
|
||||
text: hasInboundImages(normalized) ? '' : normalized.content,
|
||||
addressed: normalized.kind !== 'group' || normalized.addressed === true,
|
||||
hasPendingQuestion: Boolean(pending),
|
||||
questionCompletion: pending?.submitting || pending?.claimedReplyMessageId
|
||||
|
|
@ -208,16 +214,17 @@ export class TextHarnessBridge {
|
|||
this.#status.lastRejectedAt = new Date().toISOString();
|
||||
return;
|
||||
}
|
||||
if (!text) {
|
||||
await this.#bot.sendText(target, '目前仅支持文字消息。');
|
||||
const hasImages = hasInboundImages(message);
|
||||
if (!text && !hasImages) {
|
||||
await this.#bot.sendText(target, '目前支持文字和图片消息。');
|
||||
return;
|
||||
}
|
||||
const command = text.toLowerCase();
|
||||
if (command === '/help') {
|
||||
if (!hasImages && command === '/help') {
|
||||
await this.#bot.sendText(target, [
|
||||
`${this.#descriptor.label}机器人已连接 DeepSeek Harness。`,
|
||||
'',
|
||||
'直接发送文字即可继续当前会话。',
|
||||
'直接发送文字或图片即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -229,30 +236,34 @@ export class TextHarnessBridge {
|
|||
].join('\n'));
|
||||
return;
|
||||
}
|
||||
if (command === '/status') {
|
||||
if (!hasImages && command === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#bot.sendText(target, `${this.#descriptor.label}机器人与 DeepSeek Harness 连接正常。`);
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, conversationKey);
|
||||
const workspaceCommand = !hasImages
|
||||
? await runWorkspaceCommand(text, this.#harness, conversationKey)
|
||||
: null;
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#bot.sendText(target, reply);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (command === '/new') {
|
||||
if (!hasImages && command === '/new') {
|
||||
await this.#state.clearSession(conversationKey);
|
||||
await this.#bot.sendText(target, '已开启新会话。请发送你的问题。');
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
conversationKey,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = !hasImages
|
||||
? await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
conversationKey,
|
||||
{ signal: this.#signal },
|
||||
)
|
||||
: null;
|
||||
if (compactCommand) {
|
||||
await this.#bot.sendText(target, compactCommand.message);
|
||||
return;
|
||||
|
|
@ -272,11 +283,15 @@ export class TextHarnessBridge {
|
|||
);
|
||||
}
|
||||
}
|
||||
const content = hasImages
|
||||
? await promptContentForMessage(message, { signal: this.#signal })
|
||||
: undefined;
|
||||
const { answer } = await askInWorkspaceSession({
|
||||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key: conversationKey,
|
||||
text,
|
||||
content,
|
||||
createOptions: this.#signal ? { signal: this.#signal } : undefined,
|
||||
existsOptions: this.#signal ? { signal: this.#signal } : undefined,
|
||||
askOptions: {
|
||||
|
|
@ -316,6 +331,18 @@ export class TextHarnessBridge {
|
|||
stream?.cancel?.();
|
||||
if (this.#signal?.aborted) return;
|
||||
this.#status.lastError = error?.message ?? String(error);
|
||||
const imageErrorMessage = imagePromptUserMessage(error);
|
||||
if (imageErrorMessage) {
|
||||
try {
|
||||
await this.#bot.sendText(target, imageErrorMessage);
|
||||
} catch (sendError) {
|
||||
this.#logger.error?.(
|
||||
`[dsh-im:${this.#descriptor.key}] failed to send the image error reply:`,
|
||||
sendError,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
this.#logger.error?.(`[dsh-im:${this.#descriptor.key}] failed to process a message:`, error);
|
||||
try {
|
||||
await this.#bot.sendText(target, '消息处理失败，请稍后重试。');
|
||||
|
|
@ -358,7 +385,7 @@ export class TextHarnessBridge {
|
|||
|
||||
const target = message.replyTarget;
|
||||
const text = cleanText(message.content);
|
||||
if (!text) {
|
||||
if (!text || hasInboundImages(message)) {
|
||||
try {
|
||||
await this.#bot.sendText(target, '请用文字回答当前问题。');
|
||||
} catch (error) {
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ export async function askInWorkspaceSession({
|
|||
state,
|
||||
key,
|
||||
text,
|
||||
content,
|
||||
createOptions,
|
||||
existsOptions,
|
||||
askOptions,
|
||||
|
|
@ -48,7 +49,7 @@ export async function askInWorkspaceSession({
|
|||
}
|
||||
return {
|
||||
sessionId,
|
||||
answer: await session.ask(text, askOptions),
|
||||
answer: await session.ask(content ?? text, askOptions),
|
||||
};
|
||||
} catch (error) {
|
||||
if (error?.code !== WORKSPACE_SESSION_STALE) throw error;
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ oauth_config:
|
|||
bot:
|
||||
- app_mentions:read
|
||||
- chat:write
|
||||
- files:read
|
||||
- im:history
|
||||
settings:
|
||||
event_subscriptions:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,10 @@
|
|||
import { fetchImageBuffer, ImagePromptError } from '../shared/image-prompt.mjs';
|
||||
|
||||
const DEFAULT_BASE_URL = 'https://slack.com/api/';
|
||||
const SLACK_FILE_HOST = 'files.slack.com';
|
||||
const LEGACY_SLACK_FILE_HOST = 'slack.com';
|
||||
const SLACK_FILE_PATH_PREFIX = '/files-pri/';
|
||||
const SLACK_FILE_HOSTS = Object.freeze([SLACK_FILE_HOST]);
|
||||
|
||||
function cleanString(value) {
|
||||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
|
|
@ -9,6 +15,61 @@ function requestSignal(signal, timeoutMs) {
|
|||
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
}
|
||||
|
||||
function isRedirectStatus(status) {
|
||||
return Number.isInteger(status) && status >= 300 && status < 400;
|
||||
}
|
||||
|
||||
async function cancelResponseBody(response) {
|
||||
try {
|
||||
await response?.body?.cancel?.();
|
||||
} catch {
|
||||
// Preserve the download failure when response cleanup also fails.
|
||||
}
|
||||
}
|
||||
|
||||
function secureSlackFileUrl(value) {
|
||||
const url = new URL(value);
|
||||
if (url.protocol !== 'https:') throw new Error('Image download URL must use HTTPS');
|
||||
if (url.username || url.password || (url.port && url.port !== '443')) {
|
||||
throw new Error('Image download URL is not hosted by the messaging platform');
|
||||
}
|
||||
if (url.hostname === LEGACY_SLACK_FILE_HOST && url.pathname.startsWith(SLACK_FILE_PATH_PREFIX)) {
|
||||
url.hostname = SLACK_FILE_HOST;
|
||||
}
|
||||
if (url.hostname !== SLACK_FILE_HOST || !url.pathname.startsWith(SLACK_FILE_PATH_PREFIX)) {
|
||||
throw new Error('Image download URL is not hosted by the messaging platform');
|
||||
}
|
||||
url.hash = '';
|
||||
return url;
|
||||
}
|
||||
|
||||
function redirectUrl(response, source) {
|
||||
const location = response?.headers?.get?.('location');
|
||||
if (!location) return null;
|
||||
try {
|
||||
return new URL(location, source);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function responseOAuthScopes(response) {
|
||||
const raw = response?.headers?.get?.('x-oauth-scopes');
|
||||
if (raw === null || raw === undefined) return null;
|
||||
return new Set(raw.split(',').map((scope) => scope.trim()).filter(Boolean));
|
||||
}
|
||||
|
||||
function isSlackWorkspaceRedirect(target) {
|
||||
return target?.protocol === 'https:'
|
||||
&& !target.username
|
||||
&& !target.password
|
||||
&& (!target.port || target.port === '443')
|
||||
&& target.hostname.endsWith('.slack.com')
|
||||
&& target.hostname !== SLACK_FILE_HOST
|
||||
&& target.pathname === '/'
|
||||
&& target.searchParams.has('redir');
|
||||
}
|
||||
|
||||
function delay(ms, signal) {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal?.aborted) {
|
||||
|
|
@ -79,6 +140,7 @@ export class SlackApi {
|
|||
#appToken;
|
||||
#fetch;
|
||||
#baseUrl;
|
||||
#botScopes = null;
|
||||
|
||||
constructor({ botToken, appToken, fetchImpl = fetch, baseUrl = DEFAULT_BASE_URL }) {
|
||||
if (botToken !== undefined && !validSlackBotToken(botToken)) {
|
||||
|
|
@ -177,6 +239,34 @@ export class SlackApi {
|
|||
});
|
||||
}
|
||||
|
||||
async downloadFile({ url, signal, maxBytes }) {
|
||||
if (!this.#botToken) throw new TypeError('Slack bot token is required for file download');
|
||||
const target = secureSlackFileUrl(url);
|
||||
const fetchSlackFile = async (requestUrl, options) => {
|
||||
const response = await this.#fetch(requestUrl, options);
|
||||
if (!isRedirectStatus(response?.status)) return response;
|
||||
|
||||
const next = redirectUrl(response, requestUrl);
|
||||
if ((this.#botScopes && !this.#botScopes.has('files:read'))
|
||||
|| isSlackWorkspaceRedirect(next)) {
|
||||
await cancelResponseBody(response);
|
||||
throw new ImagePromptError(
|
||||
'slack-file-access-required',
|
||||
'Slack redirected a private file request to the workspace because file access was not granted',
|
||||
'Slack 未授权机器人读取该文件。请为应用添加 files:read 后重新安装，再重新发送图片。',
|
||||
);
|
||||
}
|
||||
return response;
|
||||
};
|
||||
return fetchImageBuffer(target, {
|
||||
fetchImpl: fetchSlackFile,
|
||||
headers: { authorization: `Bearer ${this.#botToken}` },
|
||||
signal,
|
||||
maxBytes,
|
||||
allowedHosts: SLACK_FILE_HOSTS,
|
||||
});
|
||||
}
|
||||
|
||||
async #request(method, {
|
||||
tokenKind,
|
||||
body,
|
||||
|
|
@ -206,6 +296,11 @@ export class SlackApi {
|
|||
throw new Error(`Slack ${method} transport failed`);
|
||||
}
|
||||
|
||||
if (tokenKind === 'bot') {
|
||||
const scopes = responseOAuthScopes(response);
|
||||
if (scopes) this.#botScopes = scopes;
|
||||
}
|
||||
|
||||
let payload;
|
||||
try {
|
||||
payload = await response.json();
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import { createSlackBridgeStatus, SlackHarnessBridge } from './slack-bridge.mjs'
|
|||
const RECONNECT_DELAYS_MS = Object.freeze([1_000, 3_000, 5_000, 10_000, 30_000]);
|
||||
const SLACK_MESSAGE_LIMIT = 38_000;
|
||||
const SLACK_STREAM_CHUNK_LIMIT = 11_000;
|
||||
const IMAGE_MEDIA_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp', 'image/gif']);
|
||||
|
||||
function addSocketListener(socket, event, listener) {
|
||||
if (typeof socket.addEventListener === 'function') socket.addEventListener(event, listener);
|
||||
|
|
@ -42,13 +43,28 @@ function stripBotMention(value, botUserId) {
|
|||
.trim();
|
||||
}
|
||||
|
||||
export function normalizeSlackEvent(payload, botUserId) {
|
||||
function slackImageSource(file, loadFile) {
|
||||
const mediaType = typeof file?.mimetype === 'string' ? file.mimetype.toLowerCase() : '';
|
||||
const url = typeof file?.url_private_download === 'string' && file.url_private_download
|
||||
? file.url_private_download : file?.url_private;
|
||||
if (!IMAGE_MEDIA_TYPES.has(mediaType) || typeof url !== 'string' || !url) return null;
|
||||
return {
|
||||
name: typeof file.name === 'string' ? file.name : undefined,
|
||||
mediaType,
|
||||
size: Number.isSafeInteger(file.size) && file.size >= 0 ? file.size : undefined,
|
||||
load: (options) => loadFile(url, options),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeSlackEvent(payload, botUserId, { loadFile = async () => {
|
||||
throw new Error('Slack file downloader is unavailable');
|
||||
} } = {}) {
|
||||
const event = payload?.event;
|
||||
if (!event || !payload?.event_id || !event.channel || !event.user || !event.ts) return null;
|
||||
const direct = event.type === 'message' && event.channel_type === 'im';
|
||||
const mentioned = event.type === 'app_mention';
|
||||
if (!direct && !mentioned) return null;
|
||||
if (event.subtype || event.bot_id || event.app_id) return null;
|
||||
if ((event.subtype && event.subtype !== 'file_share') || event.bot_id || event.app_id) return null;
|
||||
const threadTs = String(event.thread_ts ?? event.ts);
|
||||
return {
|
||||
messageId: String(payload.event_id),
|
||||
|
|
@ -57,6 +73,9 @@ export function normalizeSlackEvent(payload, botUserId) {
|
|||
kind: direct ? 'direct' : 'group',
|
||||
conversationId: direct ? String(event.channel) : `${event.channel}:${threadTs}`,
|
||||
content: stripBotMention(event.text ?? '', botUserId),
|
||||
images: Array.isArray(event.files)
|
||||
? event.files.map((file) => slackImageSource(file, loadFile)).filter(Boolean)
|
||||
: [],
|
||||
addressed: direct || mentioned,
|
||||
replyTarget: {
|
||||
channelId: String(event.channel),
|
||||
|
|
@ -389,7 +408,9 @@ export class SlackRuntime {
|
|||
if (packet.type !== 'events_api' || packet.payload?.type !== 'event_callback') return;
|
||||
if (this.#appId && packet.payload.api_app_id
|
||||
&& packet.payload.api_app_id !== this.#appId) return;
|
||||
const message = normalizeSlackEvent(packet.payload, this.#config.platformId.split(':')[1]);
|
||||
const message = normalizeSlackEvent(packet.payload, this.#config.platformId.split(':')[1], {
|
||||
loadFile: (url, options) => this.#api.downloadFile({ url, ...options }),
|
||||
});
|
||||
const bridge = this.#bridge;
|
||||
if (message && bridge) {
|
||||
void bridge.accept(message).catch((error) => {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,7 @@
|
|||
import { fetchImageBuffer } from '../shared/image-prompt.mjs';
|
||||
|
||||
const DEFAULT_BASE_URL = 'https://api.telegram.org/';
|
||||
const TELEGRAM_FILE_HOSTS = Object.freeze(['api.telegram.org']);
|
||||
|
||||
function cleanString(value) {
|
||||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
|
|
@ -47,6 +50,40 @@ export class TelegramApi {
|
|||
});
|
||||
}
|
||||
|
||||
async getFile({ fileId, signal } = {}) {
|
||||
const id = cleanString(fileId);
|
||||
if (!id || !/^[A-Za-z0-9_-]{1,512}$/.test(id)) {
|
||||
throw new TypeError('Telegram file id is invalid');
|
||||
}
|
||||
return this.#call('getFile', { file_id: id }, { signal });
|
||||
}
|
||||
|
||||
async downloadFile({ fileId, signal, maxBytes } = {}) {
|
||||
const file = await this.getFile({ fileId, signal });
|
||||
const filePath = cleanString(file?.file_path);
|
||||
if (!filePath || filePath.startsWith('/') || filePath.includes('\\')
|
||||
|| filePath.includes('?') || filePath.includes('#')) {
|
||||
throw new Error('Telegram returned an invalid file path');
|
||||
}
|
||||
let decodedSegments;
|
||||
try {
|
||||
decodedSegments = filePath.split('/').map((segment) => decodeURIComponent(segment));
|
||||
} catch {
|
||||
throw new Error('Telegram returned an invalid file path');
|
||||
}
|
||||
if (decodedSegments.some((segment) => !segment || segment === '.' || segment === '..')) {
|
||||
throw new Error('Telegram returned an invalid file path');
|
||||
}
|
||||
const url = new URL(this.#baseUrl);
|
||||
url.pathname = `/file/bot${this.#token}/${filePath}`;
|
||||
return fetchImageBuffer(url, {
|
||||
fetchImpl: this.#fetch,
|
||||
signal,
|
||||
maxBytes,
|
||||
allowedHosts: TELEGRAM_FILE_HOSTS,
|
||||
});
|
||||
}
|
||||
|
||||
async sendMessage({ chatId, text, replyToMessageId, messageThreadId, signal }) {
|
||||
return this.#call('sendMessage', {
|
||||
chat_id: chatId,
|
||||
|
|
|
|||
|
|
@ -7,14 +7,17 @@ function escaped(value) {
|
|||
}
|
||||
|
||||
function mentionedUsername(message, username) {
|
||||
if (!username || typeof message?.text !== 'string' || !Array.isArray(message.entities)) return false;
|
||||
return message.entities.some((entity) => {
|
||||
if (entity?.type !== 'mention' || !Number.isInteger(entity.offset) || !Number.isInteger(entity.length)) {
|
||||
return false;
|
||||
}
|
||||
return message.text.slice(entity.offset, entity.offset + entity.length).toLowerCase()
|
||||
=== `@${username.toLowerCase()}`;
|
||||
});
|
||||
if (!username) return false;
|
||||
return [
|
||||
[message?.text, message?.entities],
|
||||
[message?.caption, message?.caption_entities],
|
||||
].some(([text, entities]) => typeof text === 'string' && Array.isArray(entities)
|
||||
&& entities.some((entity) => {
|
||||
if (entity?.type !== 'mention' || !Number.isInteger(entity.offset)
|
||||
|| !Number.isInteger(entity.length)) return false;
|
||||
return text.slice(entity.offset, entity.offset + entity.length).toLowerCase()
|
||||
=== `@${username.toLowerCase()}`;
|
||||
}));
|
||||
}
|
||||
|
||||
function withoutBotMention(text, username) {
|
||||
|
|
@ -22,7 +25,63 @@ function withoutBotMention(text, username) {
|
|||
return text.replace(new RegExp(`@${escaped(username)}\\b`, 'ig'), '').trim();
|
||||
}
|
||||
|
||||
export function normalizeTelegramUpdate(update, { botId, username }) {
|
||||
const IMAGE_MEDIA_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp', 'image/gif']);
|
||||
const IMAGE_FILE_TYPES = new Map([
|
||||
['.jpg', 'image/jpeg'],
|
||||
['.jpeg', 'image/jpeg'],
|
||||
['.png', 'image/png'],
|
||||
['.webp', 'image/webp'],
|
||||
['.gif', 'image/gif'],
|
||||
]);
|
||||
|
||||
function imageTypeForDocument(document) {
|
||||
const declaredType = document?.mime_type ?? document?.mimetype;
|
||||
const type = typeof declaredType === 'string' ? declaredType.toLowerCase() : '';
|
||||
if (IMAGE_MEDIA_TYPES.has(type)) return type;
|
||||
const filename = typeof document?.file_name === 'string' ? document.file_name.toLowerCase() : '';
|
||||
for (const [extension, mediaType] of IMAGE_FILE_TYPES) {
|
||||
if (filename.endsWith(extension)) return mediaType;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function fileSize(value) {
|
||||
return Number.isSafeInteger(value) && value >= 0 ? value : undefined;
|
||||
}
|
||||
|
||||
function photoScore(photo) {
|
||||
return fileSize(photo?.file_size) ?? ((Number(photo?.width) || 0) * (Number(photo?.height) || 0));
|
||||
}
|
||||
|
||||
function telegramImageSource(message, loadFile) {
|
||||
let file;
|
||||
let mediaType;
|
||||
let name;
|
||||
if (Array.isArray(message?.photo) && message.photo.length > 0) {
|
||||
file = message.photo.reduce((largest, candidate) => (
|
||||
photoScore(candidate) > photoScore(largest) ? candidate : largest
|
||||
));
|
||||
mediaType = 'image/jpeg';
|
||||
name = `${file.file_unique_id ?? file.file_id ?? 'telegram-photo'}.jpg`;
|
||||
} else if (message?.document) {
|
||||
const type = imageTypeForDocument(message.document);
|
||||
if (!type) return null;
|
||||
file = message.document;
|
||||
mediaType = type;
|
||||
name = typeof file.file_name === 'string' ? file.file_name : undefined;
|
||||
}
|
||||
if (!file || typeof file.file_id !== 'string') return null;
|
||||
return {
|
||||
name,
|
||||
mediaType,
|
||||
size: fileSize(file.file_size),
|
||||
load: (options) => loadFile(file.file_id, options),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeTelegramUpdate(update, { botId, username, loadFile = async () => {
|
||||
throw new Error('Telegram file downloader is unavailable');
|
||||
} }) {
|
||||
const message = update?.message;
|
||||
const chatId = message?.chat?.id;
|
||||
const senderId = message?.from?.id;
|
||||
|
|
@ -36,6 +95,7 @@ export function normalizeTelegramUpdate(update, { botId, username }) {
|
|||
|| mentionedUsername(message, username);
|
||||
const messageThreadId = Number.isSafeInteger(message.message_thread_id)
|
||||
? message.message_thread_id : undefined;
|
||||
const image = telegramImageSource(message, loadFile);
|
||||
return {
|
||||
messageId: String(update.update_id),
|
||||
senderId: String(senderId),
|
||||
|
|
@ -44,6 +104,7 @@ export function normalizeTelegramUpdate(update, { botId, username }) {
|
|||
conversationId: messageThreadId === undefined
|
||||
? String(chatId) : `${chatId}:${messageThreadId}`,
|
||||
content: withoutBotMention(message.text ?? message.caption ?? '', username),
|
||||
images: image ? [image] : [],
|
||||
addressed,
|
||||
replyTarget: {
|
||||
chatId,
|
||||
|
|
@ -250,6 +311,7 @@ export class TelegramRuntime {
|
|||
const message = normalizeTelegramUpdate(update, {
|
||||
botId: this.#config.platformId,
|
||||
username: this.#config.username,
|
||||
loadFile: (fileId, options) => this.#api.downloadFile({ fileId, ...options }),
|
||||
});
|
||||
if (message) {
|
||||
void this.#bridge.accept(message).catch((error) => {
|
||||
|
|
|
|||
|
|
@ -8,11 +8,17 @@ import { HarnessApprovalQueue } from '../shared/harness-approval.mjs';
|
|||
import { runCompactCommand } from '../shared/compact-command.mjs';
|
||||
import { runWorkspaceCommand } from '../shared/workspace-command.mjs';
|
||||
import { askInWorkspaceSession } from '../shared/workspace-session.mjs';
|
||||
import {
|
||||
hasInboundImages,
|
||||
ImagePromptError,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from '../shared/image-prompt.mjs';
|
||||
|
||||
const HELP_TEXT = [
|
||||
'企业微信机器人已连接 DeepSeek Harness。',
|
||||
'',
|
||||
'直接发送文字即可继续当前会话。',
|
||||
'直接发送文字或图片即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -23,6 +29,8 @@ const HELP_TEXT = [
|
|||
'/help 显示本帮助',
|
||||
].join('\n');
|
||||
const MAX_REPLY_BYTES = 18_000;
|
||||
const MAX_IMAGE_BYTES = 5 * 1024 * 1024;
|
||||
const MAX_PREFETCHED_IMAGES = 4;
|
||||
const INTERACTION_RESOLVED_TEXT = '这个问题已在其他客户端处理，无需再次回答。';
|
||||
|
||||
function nonEmptyString(value) {
|
||||
|
|
@ -59,6 +67,102 @@ function messageText(frame) {
|
|||
: text;
|
||||
}
|
||||
|
||||
function imageContents(frame) {
|
||||
const body = bodyOf(frame);
|
||||
if (body.msgtype === 'image') return [body.image];
|
||||
if (body.msgtype !== 'mixed' || !Array.isArray(body.mixed?.msg_item)) return [];
|
||||
return body.mixed.msg_item
|
||||
.filter((item) => item?.msgtype === 'image')
|
||||
.map((item) => item.image);
|
||||
}
|
||||
|
||||
function imageSource(client, image) {
|
||||
const url = nonEmptyString(image?.url);
|
||||
if (!url) return null;
|
||||
const aeskey = nonEmptyString(image?.aeskey) ?? undefined;
|
||||
return {
|
||||
async load({ signal, maxBytes }) {
|
||||
signal?.throwIfAborted();
|
||||
if (typeof client?.downloadFile !== 'function') {
|
||||
throw new Error('Enterprise WeChat image download is unavailable');
|
||||
}
|
||||
const result = await client.downloadFile(url, aeskey);
|
||||
signal?.throwIfAborted();
|
||||
const raw = result?.buffer;
|
||||
if (!Buffer.isBuffer(raw) && !(raw instanceof Uint8Array)) {
|
||||
throw new Error('Enterprise WeChat image download returned no data');
|
||||
}
|
||||
const data = Buffer.from(raw);
|
||||
if (Number.isFinite(maxBytes) && data.length > maxBytes) {
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Enterprise WeChat image exceeds ${maxBytes} bytes`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
return { data, name: result?.filename };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function wecomInboundMessage(frame, client) {
|
||||
return {
|
||||
content: messageText(frame),
|
||||
images: imageContents(frame).map((image) => imageSource(client, image)).filter(Boolean),
|
||||
};
|
||||
}
|
||||
|
||||
function prefetchInboundImages(message, signal) {
|
||||
if (!hasInboundImages(message)) return message;
|
||||
return {
|
||||
...message,
|
||||
images: message.images.map((source) => {
|
||||
const download = source.load({ signal, maxBytes: MAX_IMAGE_BYTES });
|
||||
// The conversation queue may not consume this promise immediately. Keep
|
||||
// an attached rejection handler while preserving the original outcome.
|
||||
download.catch(() => undefined);
|
||||
return {
|
||||
...source,
|
||||
async load({ signal: loadSignal, maxBytes = MAX_IMAGE_BYTES } = {}) {
|
||||
loadSignal?.throwIfAborted();
|
||||
const result = await download;
|
||||
loadSignal?.throwIfAborted();
|
||||
const raw = result?.data ?? result?.buffer ?? result;
|
||||
const size = Buffer.isBuffer(raw) || raw instanceof Uint8Array ? raw.length : 0;
|
||||
if (size > maxBytes) {
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`Enterprise WeChat image exceeds ${maxBytes} bytes`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
return result;
|
||||
},
|
||||
};
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function imageQueueFullMessage(message) {
|
||||
return {
|
||||
...message,
|
||||
images: message.images.map((source) => ({
|
||||
...source,
|
||||
async load() {
|
||||
throw new ImagePromptError(
|
||||
'image-queue-full',
|
||||
`Enterprise WeChat already has ${MAX_PREFETCHED_IMAGES} prefetched images`,
|
||||
'当前待处理图片较多，请稍后重新发送。',
|
||||
);
|
||||
},
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
function interactionReplyText(frame) {
|
||||
return bodyOf(frame).msgtype === 'text' ? messageText(frame) : '';
|
||||
}
|
||||
|
||||
function splitUtf8(text, maxBytes = MAX_REPLY_BYTES) {
|
||||
const source = String(text ?? '').trim();
|
||||
if (!source) return [];
|
||||
|
|
@ -89,7 +193,7 @@ function progressText(update) {
|
|||
function canClaimInteractionReply(frame, pending) {
|
||||
return pending.questions[pending.index]
|
||||
&& nonEmptyString(bodyOf(frame).from?.userid) === pending.actor
|
||||
&& nonEmptyString(messageText(frame));
|
||||
&& nonEmptyString(interactionReplyText(frame));
|
||||
}
|
||||
|
||||
export function createWecomBridgeStatus() {
|
||||
|
|
@ -119,6 +223,7 @@ export class WecomHarnessBridge {
|
|||
#acceptedMessageIds = new Set();
|
||||
#approvalTasks = new Set();
|
||||
#approvals;
|
||||
#prefetchedImageCount = 0;
|
||||
|
||||
constructor({
|
||||
client,
|
||||
|
|
@ -169,7 +274,7 @@ export class WecomHarnessBridge {
|
|||
key,
|
||||
actor: senderId,
|
||||
messageId,
|
||||
text: messageText(frame),
|
||||
text: interactionReplyText(frame),
|
||||
addressed: true,
|
||||
hasPendingQuestion: Boolean(pending),
|
||||
questionCompletion: pending?.submitting || pending?.claimedReplyMessageId
|
||||
|
|
@ -232,11 +337,28 @@ export class WecomHarnessBridge {
|
|||
releaseMessageId = true,
|
||||
alreadyRecorded = false,
|
||||
} = {}) {
|
||||
// WeCom image URLs expire after five minutes, while a conversation turn
|
||||
// may legally stay queued longer. Start the authenticated SDK download as
|
||||
// soon as the validated callback is accepted, then consume it in order.
|
||||
const inboundMessage = wecomInboundMessage(frame, this.#client);
|
||||
const imageCount = inboundMessage.images.length;
|
||||
let reservedImages = 0;
|
||||
let preparedMessage = inboundMessage;
|
||||
if (imageCount > 0) {
|
||||
if (this.#prefetchedImageCount + imageCount <= MAX_PREFETCHED_IMAGES) {
|
||||
reservedImages = imageCount;
|
||||
this.#prefetchedImageCount += reservedImages;
|
||||
preparedMessage = prefetchInboundImages(inboundMessage, this.#signal);
|
||||
} else {
|
||||
preparedMessage = imageQueueFullMessage(inboundMessage);
|
||||
}
|
||||
}
|
||||
const previous = this.#queues.get(key) ?? Promise.resolve();
|
||||
const current = previous
|
||||
.catch(() => undefined)
|
||||
.then(() => this.#process(frame, { alreadyRecorded }))
|
||||
.then(() => this.#process(frame, { alreadyRecorded, preparedMessage }))
|
||||
.finally(() => {
|
||||
this.#prefetchedImageCount -= reservedImages;
|
||||
if (releaseMessageId) this.#acceptedMessageIds.delete(messageId);
|
||||
if (this.#queues.get(key) === current) this.#queues.delete(key);
|
||||
});
|
||||
|
|
@ -275,7 +397,7 @@ export class WecomHarnessBridge {
|
|||
}
|
||||
}
|
||||
|
||||
async #process(frame, { alreadyRecorded = false } = {}) {
|
||||
async #process(frame, { alreadyRecorded = false, preparedMessage } = {}) {
|
||||
if (this.#signal?.aborted) return;
|
||||
const body = bodyOf(frame);
|
||||
const messageId = typeof body.msgid === 'string' ? body.msgid : '';
|
||||
|
|
@ -287,35 +409,39 @@ export class WecomHarnessBridge {
|
|||
this.#status.messagesReceived += 1;
|
||||
this.#status.lastMessageAt = new Date().toISOString();
|
||||
}
|
||||
const text = messageText(frame);
|
||||
const message = preparedMessage ?? wecomInboundMessage(frame, this.#client);
|
||||
const text = message.content;
|
||||
const hasImages = hasInboundImages(message);
|
||||
const key = conversationKey(frame);
|
||||
let streamId = null;
|
||||
let streamStarted = false;
|
||||
try {
|
||||
if (!text) {
|
||||
await this.#sendImmediate(frame, chatId, '目前支持文字、语音转写和图文混排中的文字消息。');
|
||||
if (!text && !hasImages) {
|
||||
await this.#sendImmediate(frame, chatId, '目前支持文字、图片和语音转写消息。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const command = text.toLowerCase();
|
||||
if (command === '/help') {
|
||||
if (!hasImages && command === '/help') {
|
||||
await this.#sendImmediate(frame, chatId, HELP_TEXT);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/status') {
|
||||
if (!hasImages && command === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#sendImmediate(frame, chatId, '企业微信机器人与 DeepSeek Harness 连接正常。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/new') {
|
||||
if (!hasImages && command === '/new') {
|
||||
await this.#state.clearSession(key);
|
||||
await this.#sendImmediate(frame, chatId, '已开启新会话。请发送你的问题。');
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, key);
|
||||
const workspaceCommand = hasImages
|
||||
? null
|
||||
: await runWorkspaceCommand(text, this.#harness, key);
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#sendImmediate(frame, chatId, reply);
|
||||
|
|
@ -323,13 +449,15 @@ export class WecomHarnessBridge {
|
|||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = hasImages
|
||||
? null
|
||||
: await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
if (compactCommand) {
|
||||
await this.#sendImmediate(frame, chatId, compactCommand.message);
|
||||
await this.#state.markSeen(messageId);
|
||||
|
|
@ -344,11 +472,15 @@ export class WecomHarnessBridge {
|
|||
this.#logger.warn?.('[dsh-im:wecom] unable to start a stream; using an active reply:', error);
|
||||
}
|
||||
|
||||
const content = hasImages
|
||||
? await promptContentForMessage(message, { signal: this.#signal })
|
||||
: undefined;
|
||||
const { answer } = await askInWorkspaceSession({
|
||||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
content,
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: {
|
||||
|
|
@ -392,11 +524,12 @@ export class WecomHarnessBridge {
|
|||
if (this.#signal?.aborted) return;
|
||||
this.#status.lastError = error?.message ?? String(error);
|
||||
this.#logger.error?.('[dsh-im:wecom] failed to process an inbound message');
|
||||
const errorText = imagePromptUserMessage(error) ?? '消息处理失败，请稍后重试。';
|
||||
try {
|
||||
if (streamStarted && streamId) {
|
||||
await this.#client.replyStream(frame, streamId, '消息处理失败，请稍后重试。', true);
|
||||
await this.#client.replyStream(frame, streamId, errorText, true);
|
||||
} else {
|
||||
await this.#sendImmediate(frame, chatId, '消息处理失败，请稍后重试。');
|
||||
await this.#sendImmediate(frame, chatId, errorText);
|
||||
}
|
||||
await this.#state.markSeen(messageId);
|
||||
} catch {
|
||||
|
|
@ -425,9 +558,9 @@ export class WecomHarnessBridge {
|
|||
this.#status.messagesReceived += 1;
|
||||
this.#status.lastMessageAt = new Date().toISOString();
|
||||
|
||||
const text = nonEmptyString(messageText(frame));
|
||||
const text = nonEmptyString(interactionReplyText(frame));
|
||||
if (!text) {
|
||||
await this.#sendImmediate(frame, chatId, '请用文字或语音回答当前问题。')
|
||||
await this.#sendImmediate(frame, chatId, '请用文字回答当前问题。')
|
||||
.catch(() => undefined);
|
||||
return;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,8 +1,13 @@
|
|||
import { randomBytes, randomUUID } from 'node:crypto';
|
||||
import { createDecipheriv, randomBytes, randomUUID } from 'node:crypto';
|
||||
|
||||
import { fetchImageBuffer } from '../shared/image-prompt.mjs';
|
||||
|
||||
export const WEIXIN_QR_BASE_URL = 'https://ilinkai.weixin.qq.com/';
|
||||
export const WEIXIN_PROTOCOL_VERSION = '2.4.6';
|
||||
export const DEFAULT_BOT_TYPE = '3';
|
||||
export const WEIXIN_CDN_BASE_URL = 'https://novac2c.cdn.weixin.qq.com/c2c';
|
||||
|
||||
const WEIXIN_CDN_HOST = 'novac2c.cdn.weixin.qq.com';
|
||||
|
||||
const ILINK_APP_ID = 'bot';
|
||||
const ILINK_CLIENT_VERSION = (2 << 16) | (4 << 8) | 6;
|
||||
|
|
@ -32,6 +37,93 @@ function nonEmptyString(value) {
|
|||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function strictBase64(value) {
|
||||
const text = nonEmptyString(value);
|
||||
if (!text || text.length % 4 !== 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(text)) return null;
|
||||
return Buffer.from(text, 'base64');
|
||||
}
|
||||
|
||||
/** Parse the two AES key encodings used by Weixin iLink image messages. */
|
||||
export function parseWeixinImageAesKey(imageItem) {
|
||||
const directHex = nonEmptyString(imageItem?.aeskey);
|
||||
if (directHex) {
|
||||
if (!/^[0-9a-fA-F]{32}$/.test(directHex)) {
|
||||
throw new WeixinApiError('invalid-image-key', '微信图片的加密密钥无效。');
|
||||
}
|
||||
return Buffer.from(directHex, 'hex');
|
||||
}
|
||||
|
||||
const encoded = strictBase64(imageItem?.media?.aes_key);
|
||||
if (encoded?.length === 16) return encoded;
|
||||
if (encoded?.length === 32 && /^[0-9a-fA-F]{32}$/.test(encoded.toString('ascii'))) {
|
||||
return Buffer.from(encoded.toString('ascii'), 'hex');
|
||||
}
|
||||
throw new WeixinApiError('invalid-image-key', '微信图片的加密密钥无效。');
|
||||
}
|
||||
|
||||
export function decryptWeixinImage(ciphertext, key) {
|
||||
const encrypted = Buffer.from(ciphertext);
|
||||
const aesKey = Buffer.from(key);
|
||||
if (aesKey.length !== 16 || encrypted.length === 0 || encrypted.length % 16 !== 0) {
|
||||
throw new WeixinApiError('invalid-image-ciphertext', '微信图片的加密数据无效。');
|
||||
}
|
||||
try {
|
||||
const decipher = createDecipheriv('aes-128-ecb', aesKey, null);
|
||||
return Buffer.concat([decipher.update(encrypted), decipher.final()]);
|
||||
} catch (error) {
|
||||
throw new WeixinApiError('image-decryption-failed', '微信图片解密失败。', { cause: error });
|
||||
}
|
||||
}
|
||||
|
||||
export function weixinImageDownloadUrl(media) {
|
||||
const query = nonEmptyString(media?.encrypt_query_param);
|
||||
if (query) {
|
||||
return `${WEIXIN_CDN_BASE_URL}/download?encrypted_query_param=${encodeURIComponent(query)}`;
|
||||
}
|
||||
|
||||
const fullUrl = nonEmptyString(media?.full_url);
|
||||
if (!fullUrl) throw new WeixinApiError('missing-image-url', '微信图片没有可用的下载地址。');
|
||||
let url;
|
||||
try {
|
||||
url = new URL(fullUrl);
|
||||
} catch {
|
||||
throw new WeixinApiError('invalid-image-url', '微信图片的下载地址无效。');
|
||||
}
|
||||
if (url.protocol !== 'https:' || url.hostname !== WEIXIN_CDN_HOST
|
||||
|| (url.port && url.port !== '443') || !url.pathname.startsWith('/c2c/')) {
|
||||
throw new WeixinApiError('untrusted-image-url', '微信图片的下载地址不受信任。');
|
||||
}
|
||||
url.username = '';
|
||||
url.password = '';
|
||||
url.hash = '';
|
||||
return url.toString();
|
||||
}
|
||||
|
||||
/** Convert iLink image items into lazily downloaded, decrypted image references. */
|
||||
export function extractWeixinImages(message, { fetchImpl = fetch } = {}) {
|
||||
if (typeof fetchImpl !== 'function') throw new TypeError('fetchImpl must be a function');
|
||||
const images = [];
|
||||
for (const item of message?.item_list ?? []) {
|
||||
const imageItem = item?.image_item;
|
||||
if (!imageItem || typeof imageItem !== 'object') continue;
|
||||
images.push({
|
||||
name: images.length === 0 ? 'image' : `image-${images.length + 1}`,
|
||||
load: async ({ signal, maxBytes }) => {
|
||||
const key = parseWeixinImageAesKey(imageItem);
|
||||
const url = weixinImageDownloadUrl(imageItem.media);
|
||||
const ciphertext = await fetchImageBuffer(url, {
|
||||
fetchImpl,
|
||||
signal,
|
||||
maxBytes: maxBytes + 16,
|
||||
allowedHosts: [WEIXIN_CDN_HOST],
|
||||
});
|
||||
return decryptWeixinImage(ciphertext, key);
|
||||
},
|
||||
});
|
||||
}
|
||||
return images;
|
||||
}
|
||||
|
||||
function isWeixinHost(hostname) {
|
||||
const normalized = hostname.toLowerCase().replace(/\.$/, '');
|
||||
return normalized === 'weixin.qq.com' || normalized.endsWith('.weixin.qq.com');
|
||||
|
|
@ -170,6 +262,10 @@ export function createWeixinApi({ fetchImpl = fetch } = {}) {
|
|||
if (typeof fetchImpl !== 'function') throw new TypeError('fetchImpl must be a function');
|
||||
|
||||
return Object.freeze({
|
||||
inboundImages(message) {
|
||||
return extractWeixinImages(message, { fetchImpl });
|
||||
},
|
||||
|
||||
async beginLogin({ localTokens = [], botType = DEFAULT_BOT_TYPE, signal } = {}) {
|
||||
const tokens = [...new Set(localTokens.map(nonEmptyString).filter(Boolean))].slice(-10);
|
||||
const response = await requestJson(fetchImpl, {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import {
|
||||
extractWeixinImages,
|
||||
extractWeixinText,
|
||||
splitWeixinText,
|
||||
weixinMessageId,
|
||||
|
|
@ -12,13 +13,18 @@ import { HarnessApprovalQueue } from '../shared/harness-approval.mjs';
|
|||
import { runCompactCommand } from '../shared/compact-command.mjs';
|
||||
import { runWorkspaceCommand } from '../shared/workspace-command.mjs';
|
||||
import { askInWorkspaceSession } from '../shared/workspace-session.mjs';
|
||||
import {
|
||||
hasInboundImages,
|
||||
imagePromptUserMessage,
|
||||
promptContentForMessage,
|
||||
} from '../shared/image-prompt.mjs';
|
||||
|
||||
const INTERACTION_RESOLVED_TEXT = '这个问题已在其他客户端处理，无需再次回答。';
|
||||
|
||||
const HELP_TEXT = [
|
||||
'微信已连接 DeepSeek Harness。',
|
||||
'',
|
||||
'直接发送文字或带文字识别结果的语音即可继续当前会话。',
|
||||
'直接发送文字、图片或带文字识别结果的语音即可继续当前会话。',
|
||||
'/new 开启一个全新会话',
|
||||
'/compact 压缩当前会话的较早上下文',
|
||||
'/workspace 工作区绝对路径 切换工作区',
|
||||
|
|
@ -37,9 +43,15 @@ function nonEmptyString(value) {
|
|||
return typeof value === 'string' && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function hasWeixinImageItems(message) {
|
||||
return Array.isArray(message?.item_list)
|
||||
&& message.item_list.some((item) => item?.image_item && typeof item.image_item === 'object');
|
||||
}
|
||||
|
||||
function canClaimInteractionReply(message, pending) {
|
||||
return pending.questions[pending.index]
|
||||
&& nonEmptyString(message?.from_user_id) === pending.actor
|
||||
&& !hasWeixinImageItems(message)
|
||||
&& nonEmptyString(extractWeixinText(message));
|
||||
}
|
||||
|
||||
|
|
@ -124,7 +136,7 @@ export class WeixinHarnessBridge {
|
|||
key,
|
||||
actor: sender,
|
||||
messageId,
|
||||
text: extractWeixinText(message),
|
||||
text: hasWeixinImageItems(message) ? '' : extractWeixinText(message),
|
||||
addressed: true,
|
||||
hasPendingQuestion: Boolean(pending),
|
||||
questionCompletion: pending?.submitting || pending?.claimedReplyMessageId
|
||||
|
|
@ -216,33 +228,40 @@ export class WeixinHarnessBridge {
|
|||
|
||||
const contextToken = typeof message.context_token === 'string' ? message.context_token : undefined;
|
||||
const runId = typeof message.run_id === 'string' ? message.run_id : undefined;
|
||||
const text = extractWeixinText(message);
|
||||
const text = extractWeixinText(message) ?? '';
|
||||
try {
|
||||
if (!text) {
|
||||
await this.#send(sender, '目前仅支持文字消息，以及微信已转成文字的语音消息。', contextToken, runId);
|
||||
const images = typeof this.#api.inboundImages === 'function'
|
||||
? this.#api.inboundImages(message)
|
||||
: extractWeixinImages(message);
|
||||
const promptMessage = { content: text, images };
|
||||
const hasImages = hasInboundImages(promptMessage);
|
||||
if (!text && !hasImages) {
|
||||
await this.#send(sender, '目前支持文字、图片，以及微信已转成文字的语音消息。', contextToken, runId);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
|
||||
const command = text.trim().toLowerCase();
|
||||
if (command === '/help') {
|
||||
if (!hasImages && command === '/help') {
|
||||
await this.#send(sender, HELP_TEXT, contextToken, runId);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/status') {
|
||||
if (!hasImages && command === '/status') {
|
||||
await this.#harness.ensureRunning({ signal: this.#signal });
|
||||
await this.#send(sender, '微信与 DeepSeek Harness 连接正常。', contextToken, runId);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
if (command === '/new') {
|
||||
if (!hasImages && command === '/new') {
|
||||
await this.#state.clearSession(key);
|
||||
await this.#send(sender, '已开启新会话。请发送你的问题。', contextToken, runId);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const workspaceCommand = await runWorkspaceCommand(text, this.#harness, key);
|
||||
const workspaceCommand = hasImages
|
||||
? null
|
||||
: await runWorkspaceCommand(text, this.#harness, key);
|
||||
if (workspaceCommand) {
|
||||
for (const reply of workspaceCommand.messages ?? [workspaceCommand.message]) {
|
||||
await this.#send(sender, reply, contextToken, runId);
|
||||
|
|
@ -250,26 +269,31 @@ export class WeixinHarnessBridge {
|
|||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
const compactCommand = await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
const compactCommand = hasImages
|
||||
? null
|
||||
: await runCompactCommand(
|
||||
text,
|
||||
this.#harness,
|
||||
this.#state,
|
||||
key,
|
||||
{ signal: this.#signal },
|
||||
);
|
||||
if (compactCommand) {
|
||||
await this.#send(sender, compactCommand.message, contextToken, runId);
|
||||
await this.#state.markSeen(messageId);
|
||||
return;
|
||||
}
|
||||
|
||||
const content = hasImages
|
||||
? await promptContentForMessage(promptMessage, { signal: this.#signal })
|
||||
: undefined;
|
||||
let answer;
|
||||
try {
|
||||
({ answer } = await askInWorkspaceSession({
|
||||
harness: this.#harness,
|
||||
state: this.#state,
|
||||
key,
|
||||
text,
|
||||
...(hasImages ? { content } : { text }),
|
||||
createOptions: { signal: this.#signal },
|
||||
existsOptions: { signal: this.#signal },
|
||||
askOptions: {
|
||||
|
|
@ -300,7 +324,12 @@ export class WeixinHarnessBridge {
|
|||
this.#status.lastError = error?.message ?? String(error);
|
||||
this.#logger.error?.('[dsh-weixin] failed to process an inbound message:', error);
|
||||
try {
|
||||
await this.#send(sender, '消息处理失败，请稍后重试。', contextToken, runId);
|
||||
await this.#send(
|
||||
sender,
|
||||
imagePromptUserMessage(error) ?? '消息处理失败，请稍后重试。',
|
||||
contextToken,
|
||||
runId,
|
||||
);
|
||||
await this.#state.markSeen(messageId);
|
||||
} catch (sendError) {
|
||||
this.#logger.error?.('[dsh-weixin] failed to send the safe error reply:', sendError);
|
||||
|
|
@ -326,7 +355,7 @@ export class WeixinHarnessBridge {
|
|||
const text = nonEmptyString(extractWeixinText(message));
|
||||
const contextToken = nonEmptyString(message?.context_token) ?? undefined;
|
||||
const runId = nonEmptyString(message?.run_id) ?? undefined;
|
||||
if (!text) {
|
||||
if (!text || hasWeixinImageItems(message)) {
|
||||
await this.#send(
|
||||
expected.actor,
|
||||
'请用文字回答当前问题。',
|
||||
|
|
|
|||
|
|
@ -1,12 +1,45 @@
|
|||
import {
|
||||
areJidsSameUser,
|
||||
downloadMediaMessage,
|
||||
normalizeMessageContent,
|
||||
} from '@whiskeysockets/baileys';
|
||||
|
||||
import { splitMessageText } from '../shared/editable-message-stream.mjs';
|
||||
import { ImagePromptError } from '../shared/image-prompt.mjs';
|
||||
import { createWhatsappBridgeStatus, WhatsappHarnessBridge } from './whatsapp-bridge.mjs';
|
||||
import { createWhatsappWebSession } from './whatsapp-web-session.mjs';
|
||||
|
||||
const IMAGE_MEDIA_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp', 'image/gif']);
|
||||
const DEFAULT_MAX_IMAGE_BYTES = 5 * 1024 * 1024;
|
||||
const IMAGE_DOWNLOAD_TIMEOUT_MS = 15_000;
|
||||
const MESSAGE_WRAPPER_KEYS = [
|
||||
'ephemeralMessage',
|
||||
'viewOnceMessage',
|
||||
'documentWithCaptionMessage',
|
||||
'viewOnceMessageV2',
|
||||
'viewOnceMessageV2Extension',
|
||||
'editedMessage',
|
||||
'associatedChildMessage',
|
||||
'groupStatusMessage',
|
||||
'groupStatusMessageV2',
|
||||
];
|
||||
const VIEW_ONCE_WRAPPER_KEYS = new Set([
|
||||
'viewOnceMessage',
|
||||
'viewOnceMessageV2',
|
||||
'viewOnceMessageV2Extension',
|
||||
]);
|
||||
|
||||
function hasViewOnceWrapper(content) {
|
||||
let current = content;
|
||||
for (let depth = 0; depth < 5 && current && typeof current === 'object'; depth += 1) {
|
||||
const wrapperKey = MESSAGE_WRAPPER_KEYS.find((key) => current[key]?.message);
|
||||
if (!wrapperKey) return false;
|
||||
if (VIEW_ONCE_WRAPPER_KEYS.has(wrapperKey)) return true;
|
||||
current = current[wrapperKey].message;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function messageContext(content) {
|
||||
return content?.extendedTextMessage?.contextInfo
|
||||
?? content?.imageMessage?.contextInfo
|
||||
|
|
@ -24,7 +57,114 @@ function messageText(content) {
|
|||
?? '';
|
||||
}
|
||||
|
||||
export function normalizeWhatsappMessage(message, accountJid) {
|
||||
function mediaSize(value) {
|
||||
if (Number.isSafeInteger(value) && value >= 0) return value;
|
||||
let converted;
|
||||
try {
|
||||
converted = Number(value?.toString?.());
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
return Number.isSafeInteger(converted) && converted >= 0 ? converted : undefined;
|
||||
}
|
||||
|
||||
async function downloadWhatsappImage(message, download, {
|
||||
signal,
|
||||
maxBytes = DEFAULT_MAX_IMAGE_BYTES,
|
||||
} = {}) {
|
||||
signal?.throwIfAborted();
|
||||
const timeout = AbortSignal.timeout(IMAGE_DOWNLOAD_TIMEOUT_MS);
|
||||
const downloadSignal = signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
const pendingStream = Promise.resolve().then(() => (
|
||||
download(message, 'stream', { options: { signal: downloadSignal } })
|
||||
));
|
||||
const stream = await new Promise((resolve, reject) => {
|
||||
let settled = false;
|
||||
const finish = (callback, value) => {
|
||||
if (settled) return false;
|
||||
settled = true;
|
||||
downloadSignal.removeEventListener('abort', onAbort);
|
||||
callback(value);
|
||||
return true;
|
||||
};
|
||||
const onAbort = () => finish(reject, downloadSignal.reason);
|
||||
downloadSignal.addEventListener('abort', onAbort, { once: true });
|
||||
pendingStream.then((value) => {
|
||||
if (!finish(resolve, value)) value?.destroy?.();
|
||||
}, (error) => finish(reject, error));
|
||||
if (downloadSignal.aborted) onAbort();
|
||||
}).catch((error) => {
|
||||
if (signal?.aborted) throw signal.reason ?? error;
|
||||
if (timeout.aborted) {
|
||||
throw new ImagePromptError(
|
||||
'image-download-failed',
|
||||
`WhatsApp image download timed out after ${IMAGE_DOWNLOAD_TIMEOUT_MS} ms`,
|
||||
'图片下载失败，请重新发送后再试。',
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
});
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
const abortStream = () => stream?.destroy?.(downloadSignal.reason);
|
||||
downloadSignal.addEventListener('abort', abortStream, { once: true });
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
downloadSignal.throwIfAborted();
|
||||
const data = Buffer.from(chunk);
|
||||
size += data.length;
|
||||
if (size > maxBytes) {
|
||||
stream.destroy?.();
|
||||
throw new ImagePromptError(
|
||||
'image-too-large',
|
||||
`WhatsApp image exceeded ${maxBytes} bytes`,
|
||||
'图片超过 5 MB，请压缩后重试。',
|
||||
);
|
||||
}
|
||||
chunks.push(data);
|
||||
}
|
||||
} catch (error) {
|
||||
if (signal?.aborted) throw signal.reason ?? error;
|
||||
if (timeout.aborted) {
|
||||
throw new ImagePromptError(
|
||||
'image-download-failed',
|
||||
`WhatsApp image stream timed out after ${IMAGE_DOWNLOAD_TIMEOUT_MS} ms`,
|
||||
'图片下载失败，请重新发送后再试。',
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
downloadSignal.removeEventListener('abort', abortStream);
|
||||
}
|
||||
return Buffer.concat(chunks, size);
|
||||
}
|
||||
|
||||
function whatsappImageSource(message, content, download, { viewOnce = false } = {}) {
|
||||
let media;
|
||||
let name;
|
||||
if (!viewOnce && content?.imageMessage && content.imageMessage.viewOnce !== true) {
|
||||
media = content.imageMessage;
|
||||
} else if (content?.documentMessage) {
|
||||
const type = typeof content.documentMessage.mimetype === 'string'
|
||||
? content.documentMessage.mimetype.toLowerCase() : '';
|
||||
if (!IMAGE_MEDIA_TYPES.has(type)) return null;
|
||||
media = content.documentMessage;
|
||||
name = typeof media.fileName === 'string' ? media.fileName : undefined;
|
||||
}
|
||||
if (!media) return null;
|
||||
const mediaType = typeof media.mimetype === 'string' ? media.mimetype.toLowerCase() : '';
|
||||
if (!IMAGE_MEDIA_TYPES.has(mediaType)) return null;
|
||||
return {
|
||||
name,
|
||||
mediaType,
|
||||
size: mediaSize(media.fileLength),
|
||||
load: (options) => downloadWhatsappImage(message, download, options),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeWhatsappMessage(message, accountJid, {
|
||||
download = downloadMediaMessage,
|
||||
} = {}) {
|
||||
const remoteJid = typeof message?.key?.remoteJid === 'string' ? message.key.remoteJid : '';
|
||||
const alternateRemoteJid = typeof message?.key?.remoteJidAlt === 'string'
|
||||
? message.key.remoteJidAlt : '';
|
||||
|
|
@ -38,12 +178,14 @@ export function normalizeWhatsappMessage(message, accountJid) {
|
|||
if (fromMe && !selfChat) return null;
|
||||
const senderJid = selfChat ? accountJid : group ? message.key.participant : remoteJid;
|
||||
if (typeof senderJid !== 'string' || !senderJid) return null;
|
||||
const viewOnce = hasViewOnceWrapper(message.message);
|
||||
const content = normalizeMessageContent(message.message);
|
||||
const context = messageContext(content);
|
||||
const mentioned = Array.isArray(context?.mentionedJid)
|
||||
&& context.mentionedJid.some((jid) => areJidsSameUser(jid, accountJid));
|
||||
const replyToSelf = typeof context?.participant === 'string'
|
||||
&& areJidsSameUser(context.participant, accountJid);
|
||||
const image = whatsappImageSource(message, content, download, { viewOnce });
|
||||
return {
|
||||
messageId: `${remoteJid}:${messageId}`,
|
||||
providerMessageId: messageId,
|
||||
|
|
@ -52,6 +194,7 @@ export function normalizeWhatsappMessage(message, accountJid) {
|
|||
kind: group ? 'group' : 'direct',
|
||||
conversationId: remoteJid,
|
||||
content: messageText(content),
|
||||
images: image ? [image] : [],
|
||||
addressed: !group || mentioned || replyToSelf,
|
||||
selfChat,
|
||||
replyTarget: { jid: remoteJid, quoted: message, selfChat },
|
||||
|
|
|
|||
|
|
@ -113,6 +113,145 @@ test('session replies use the fixed token endpoint, reject redirects, and cache
|
|||
});
|
||||
});
|
||||
|
||||
test('DingTalk image downloads exchange downloadCode with the callback robotCode and do not forward auth', async () => {
|
||||
const imageBytes = Buffer.from([
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x01, 0x02,
|
||||
]);
|
||||
const calls = [];
|
||||
const fetchImpl = async (url, options) => {
|
||||
const href = url.toString();
|
||||
calls.push({ url: href, options });
|
||||
if (href === `${DINGTALK_API_BASE_URL}v1.0/oauth2/accessToken`) {
|
||||
return jsonResponse({ accessToken: 'image-access-token', expireIn: 7_200 });
|
||||
}
|
||||
if (href === `${DINGTALK_API_BASE_URL}v1.0/robot/messageFiles/download`) {
|
||||
return jsonResponse({ downloadUrl: 'https://download.oss-cn-hangzhou.aliyuncs.com/opaque-image' });
|
||||
}
|
||||
return new Response(imageBytes, {
|
||||
headers: { 'content-length': String(imageBytes.length) },
|
||||
});
|
||||
};
|
||||
const api = createDingtalkApi({ fetchImpl });
|
||||
|
||||
const loaded = await api.downloadImage({
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
robotCode: 'robot-from-callback',
|
||||
downloadCode: 'opaque-download-code',
|
||||
maxBytes: 1_024,
|
||||
});
|
||||
|
||||
assert.equal(loaded.equals(imageBytes), true);
|
||||
assert.deepEqual(JSON.parse(calls[1].options.body), {
|
||||
downloadCode: 'opaque-download-code',
|
||||
robotCode: 'robot-from-callback',
|
||||
});
|
||||
assert.equal(
|
||||
calls[1].options.headers['x-acs-dingtalk-access-token'],
|
||||
'image-access-token',
|
||||
);
|
||||
assert.equal(calls[2].options.method, 'GET');
|
||||
assert.equal(calls[2].options.redirect, 'manual');
|
||||
assert.equal(calls[2].options.headers?.['x-acs-dingtalk-access-token'], undefined);
|
||||
});
|
||||
|
||||
test('DingTalk upgrades its HTTP temporary image URL to HTTPS without changing the signed request', async () => {
|
||||
const imageBytes = Buffer.from([0xff, 0xd8, 0xff, 0x01]);
|
||||
const calls = [];
|
||||
const fetchImpl = async (url, options) => {
|
||||
calls.push({ url: url.toString(), options });
|
||||
if (url.toString() === `${DINGTALK_API_BASE_URL}v1.0/oauth2/accessToken`) {
|
||||
return jsonResponse({ accessToken: 'image-access-token', expireIn: 7_200 });
|
||||
}
|
||||
if (url.toString() === `${DINGTALK_API_BASE_URL}v1.0/robot/messageFiles/download`) {
|
||||
return jsonResponse({
|
||||
downloadUrl: 'http://download.example.test:80/private/image?signature=opaque%2Bvalue',
|
||||
});
|
||||
}
|
||||
return new Response(imageBytes, { status: 200 });
|
||||
};
|
||||
const api = createDingtalkApi({ fetchImpl });
|
||||
|
||||
assert.deepEqual(await api.downloadImage({
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
robotCode: 'robot-from-callback',
|
||||
downloadCode: 'opaque-download-code',
|
||||
maxBytes: 1_024,
|
||||
}), imageBytes);
|
||||
|
||||
assert.equal(
|
||||
calls[2].url,
|
||||
'https://download.example.test/private/image?signature=opaque%2Bvalue',
|
||||
);
|
||||
assert.equal(calls[2].options.headers?.['content-type'], undefined);
|
||||
assert.equal(calls[2].options.headers?.['x-acs-dingtalk-access-token'], undefined);
|
||||
});
|
||||
|
||||
test('DingTalk image exchange errors preserve the provider code without exposing its message', async () => {
|
||||
const fetchImpl = async (url) => {
|
||||
if (url.toString() === `${DINGTALK_API_BASE_URL}v1.0/oauth2/accessToken`) {
|
||||
return jsonResponse({ accessToken: 'image-access-token', expireIn: 7_200 });
|
||||
}
|
||||
return jsonResponse({
|
||||
code: 'invalidParameter.robotCode.auth',
|
||||
message: 'response may contain platform details',
|
||||
}, { status: 400 });
|
||||
};
|
||||
const api = createDingtalkApi({ fetchImpl });
|
||||
|
||||
await assert.rejects(
|
||||
api.downloadImage({
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
robotCode: 'robot-from-callback',
|
||||
downloadCode: 'opaque-download-code',
|
||||
maxBytes: 1_024,
|
||||
}),
|
||||
(error) => {
|
||||
assert.equal(error.code, 'image-download-address-failed');
|
||||
assert.equal(error.status, 400);
|
||||
assert.equal(error.providerCode, 'invalidParameter.robotCode.auth');
|
||||
assert.equal(error.cause?.code, 'http-error');
|
||||
assert.doesNotMatch(error.message, /platform details/);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('DingTalk image content failures do not expose network or signed-URL details', async () => {
|
||||
const fetchImpl = async (url) => {
|
||||
if (url.toString() === `${DINGTALK_API_BASE_URL}v1.0/oauth2/accessToken`) {
|
||||
return jsonResponse({ accessToken: 'image-access-token', expireIn: 7_200 });
|
||||
}
|
||||
if (url.toString() === `${DINGTALK_API_BASE_URL}v1.0/robot/messageFiles/download`) {
|
||||
return jsonResponse({ downloadUrl: 'https://download.example.test/private?signature=hidden' });
|
||||
}
|
||||
const cause = new Error('socket details must stay private');
|
||||
cause.code = 'ECONNRESET';
|
||||
throw new TypeError('fetch failed', { cause });
|
||||
};
|
||||
const api = createDingtalkApi({ fetchImpl });
|
||||
|
||||
await assert.rejects(
|
||||
api.downloadImage({
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
robotCode: 'robot-from-callback',
|
||||
downloadCode: 'opaque-download-code',
|
||||
maxBytes: 1_024,
|
||||
}),
|
||||
(error) => {
|
||||
assert.equal(error.code, 'image-content-download-failed');
|
||||
assert.equal(error.providerCode, undefined);
|
||||
assert.equal(error.cause?.cause?.code, 'ECONNRESET');
|
||||
assert.doesNotMatch(error.message, /private|signature|socket details|hidden/);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('session webhook validation accepts only HTTPS DingTalk hosts on the default port', () => {
|
||||
assert.equal(
|
||||
normalizeDingtalkSessionWebhook('https://dingtalk.com/reply?ticket=one'),
|
||||
|
|
|
|||
|
|
@ -76,6 +76,193 @@ function stateFixture() {
|
|||
};
|
||||
}
|
||||
|
||||
const PNG_BYTES = Buffer.from([
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x01, 0x02, 0x03,
|
||||
]);
|
||||
|
||||
test('DingTalk resolves picture downloadCode lazily and sends image-only content to Harness', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:staff-approved', 'session-image');
|
||||
const downloads = [];
|
||||
const prompts = [];
|
||||
const sent = [];
|
||||
const bridge = new DingtalkHarnessBridge({
|
||||
api: {
|
||||
downloadImage: async (request) => {
|
||||
downloads.push(request);
|
||||
return PNG_BYTES;
|
||||
},
|
||||
sendText: async (request) => sent.push(request),
|
||||
},
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
prompts.push({ sessionId, content });
|
||||
return '钉钉图片已识别';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
});
|
||||
|
||||
await bridge.accept(message('dingtalk-picture', '', {
|
||||
msgtype: 'picture',
|
||||
text: undefined,
|
||||
content: JSON.stringify({ downloadCode: 'opaque-picture-code' }),
|
||||
robotCode: 'robot-from-callback',
|
||||
}));
|
||||
|
||||
assert.equal(downloads.length, 1);
|
||||
assert.equal(downloads[0].clientId, 'ding-client');
|
||||
assert.equal(downloads[0].clientSecret, 'host-secret');
|
||||
assert.equal(downloads[0].robotCode, 'robot-from-callback');
|
||||
assert.equal(downloads[0].downloadCode, 'opaque-picture-code');
|
||||
assert.equal(downloads[0].maxBytes, 5 * 1024 * 1024);
|
||||
assert.equal(prompts[0].sessionId, 'session-image');
|
||||
assert.deepEqual(prompts[0].content.map(({ type }) => type), ['text', 'image']);
|
||||
assert.equal(prompts[0].content[0].text, '请分析这张图片。');
|
||||
assert.equal(prompts[0].content[1].mediaType, 'image/png');
|
||||
assert.equal(Buffer.from(prompts[0].content[1].data, 'base64').equals(PNG_BYTES), true);
|
||||
assert.equal(sent.at(-1).text, '钉钉图片已识别');
|
||||
});
|
||||
|
||||
test('DingTalk richText preserves its caption and all picture download codes', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:staff-approved', 'session-rich-image');
|
||||
const codes = [];
|
||||
let prompt;
|
||||
const bridge = new DingtalkHarnessBridge({
|
||||
api: {
|
||||
downloadImage: async ({ downloadCode }) => {
|
||||
codes.push(downloadCode);
|
||||
return PNG_BYTES;
|
||||
},
|
||||
sendText: async () => {},
|
||||
},
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (_sessionId, content) => { prompt = content; return '完成'; },
|
||||
},
|
||||
state: fixture.state,
|
||||
});
|
||||
|
||||
await bridge.accept(message('dingtalk-rich-picture', '', {
|
||||
msgtype: 'richText',
|
||||
text: undefined,
|
||||
robotCode: 'robot-from-callback',
|
||||
content: {
|
||||
richText: [
|
||||
{ type: 'text', text: '请对比' },
|
||||
{ type: 'picture', pictureDownloadCode: 'picture-one' },
|
||||
{ type: 'text', text: { content: '这两张图' } },
|
||||
{ type: 'picture', downloadCode: 'picture-two' },
|
||||
],
|
||||
},
|
||||
}));
|
||||
|
||||
assert.deepEqual(codes, ['picture-one', 'picture-two']);
|
||||
assert.deepEqual(prompt.map(({ type }) => type), ['text', 'image', 'image']);
|
||||
assert.equal(prompt[0].text, '请对比\n这两张图');
|
||||
});
|
||||
|
||||
test('DingTalk checks the group mention before downloading a picture', async () => {
|
||||
let downloads = 0;
|
||||
let asks = 0;
|
||||
const bridge = new DingtalkHarnessBridge({
|
||||
api: {
|
||||
downloadImage: async () => { downloads += 1; return PNG_BYTES; },
|
||||
sendText: async () => {},
|
||||
},
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
harness: { ask: async () => { asks += 1; return 'unexpected'; } },
|
||||
state: stateFixture().state,
|
||||
});
|
||||
|
||||
await bridge.accept(message('dingtalk-unmentioned-picture', '', {
|
||||
msgtype: 'picture',
|
||||
text: undefined,
|
||||
content: { downloadCode: 'private-picture' },
|
||||
robotCode: 'robot-from-callback',
|
||||
conversationType: '2',
|
||||
conversationId: 'group-image',
|
||||
isInAtList: false,
|
||||
}));
|
||||
|
||||
assert.equal(downloads, 0);
|
||||
assert.equal(asks, 0);
|
||||
});
|
||||
|
||||
test('DingTalk returns a specific retry message when picture download fails', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:staff-approved', 'session-image');
|
||||
const sent = [];
|
||||
const bridge = new DingtalkHarnessBridge({
|
||||
api: {
|
||||
downloadImage: async () => { throw new Error('temporary URL expired'); },
|
||||
sendText: async (request) => sent.push(request),
|
||||
},
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async () => assert.fail('a failed image must not reach Harness'),
|
||||
},
|
||||
state: fixture.state,
|
||||
logger: { error() {} },
|
||||
});
|
||||
|
||||
await bridge.accept(message('dingtalk-picture-error', '', {
|
||||
msgtype: 'picture',
|
||||
text: undefined,
|
||||
content: { downloadCode: 'expired-picture' },
|
||||
robotCode: 'robot-from-callback',
|
||||
}));
|
||||
|
||||
assert.equal(sent.at(-1).text, '图片下载失败，请重新发送后再试。');
|
||||
});
|
||||
|
||||
test('DingTalk distinguishes download-address failures from temporary-file failures', async () => {
|
||||
for (const [code, expected] of [
|
||||
[
|
||||
'image-download-address-failed',
|
||||
'钉钉未能换取图片下载地址，请重新发送；若持续失败，请检查机器人的“企业内机器人发送消息权限”。',
|
||||
],
|
||||
['image-content-download-failed', '钉钉返回的图片临时地址无法读取，请重新发送。'],
|
||||
]) {
|
||||
const fixture = stateFixture();
|
||||
const sent = [];
|
||||
const bridge = new DingtalkHarnessBridge({
|
||||
api: {
|
||||
downloadImage: async () => {
|
||||
const error = new Error('safe stage marker');
|
||||
error.code = code;
|
||||
throw error;
|
||||
},
|
||||
sendText: async (request) => sent.push(request),
|
||||
},
|
||||
clientId: 'ding-client',
|
||||
clientSecret: 'host-secret',
|
||||
harness: { ask: async () => assert.fail('a failed image must not reach Harness') },
|
||||
state: fixture.state,
|
||||
logger: { error() {} },
|
||||
});
|
||||
|
||||
await bridge.accept(message(`dingtalk-${code}`, '', {
|
||||
msgtype: 'picture',
|
||||
text: undefined,
|
||||
content: { downloadCode: 'opaque-picture-code' },
|
||||
robotCode: 'robot-from-callback',
|
||||
}));
|
||||
|
||||
assert.equal(sent.at(-1).text, expected);
|
||||
}
|
||||
});
|
||||
|
||||
test('DingTalk executes /compact for the bound Session without prompting the model', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:staff-approved', 'session-compact');
|
||||
|
|
|
|||
|
|
@ -210,6 +210,63 @@ test('Discord normalizes DMs and only addressed server messages', () => {
|
|||
assert.equal(unmentionedReply.addressed, false);
|
||||
});
|
||||
|
||||
test('Discord exposes image attachments through bounded CDN loaders', async () => {
|
||||
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const calls = [];
|
||||
const message = normalizeDiscordMessage({
|
||||
id: '111111111111111120',
|
||||
channel_id: '222222222222222220',
|
||||
author: { id: '333333333333333330', bot: false },
|
||||
content: '',
|
||||
attachments: [{
|
||||
id: '555555555555555550',
|
||||
filename: 'screen.png',
|
||||
content_type: 'image/png',
|
||||
size: png.length,
|
||||
url: 'https://cdn.discordapp.com/attachments/222/555/screen.png?ex=test',
|
||||
}, {
|
||||
id: '555555555555555551',
|
||||
filename: 'notes.txt',
|
||||
content_type: 'text/plain',
|
||||
size: 4,
|
||||
url: 'https://cdn.discordapp.com/attachments/222/555/notes.txt',
|
||||
}, {
|
||||
id: '555555555555555552',
|
||||
filename: 'camera.jpg',
|
||||
size: png.length,
|
||||
url: 'https://cdn.discordapp.com/attachments/222/555/camera.jpg',
|
||||
}],
|
||||
}, '1234567890123456789', {
|
||||
fetchImpl: async (url, options) => {
|
||||
calls.push({ url, options });
|
||||
return new Response(png, { status: 200, headers: { 'content-length': String(png.length) } });
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(message.content, '');
|
||||
assert.equal(message.images.length, 2);
|
||||
assert.deepEqual({
|
||||
name: message.images[0].name,
|
||||
mediaType: message.images[0].mediaType,
|
||||
size: message.images[0].size,
|
||||
}, { name: 'screen.png', mediaType: 'image/png', size: png.length });
|
||||
assert.equal(message.images[1].mediaType, 'image/jpeg');
|
||||
assert.deepEqual(await message.images[0].load({ maxBytes: 100 }), png);
|
||||
assert.equal(calls[0].url.hostname, 'cdn.discordapp.com');
|
||||
assert.equal(calls[0].options.redirect, 'manual');
|
||||
|
||||
const unsafe = normalizeDiscordMessage({
|
||||
id: '111111111111111121',
|
||||
channel_id: '222222222222222220',
|
||||
author: { id: '333333333333333330', bot: false },
|
||||
attachments: [{
|
||||
filename: 'screen.png', content_type: 'image/png', size: 8,
|
||||
url: 'https://example.com/internal.png',
|
||||
}],
|
||||
}, '1234567890123456789', { fetchImpl: async () => assert.fail('must not fetch') });
|
||||
await assert.rejects(() => unsafe.images[0].load({ maxBytes: 100 }), /messaging platform/);
|
||||
});
|
||||
|
||||
class FakeSocket {
|
||||
#listeners = new Map();
|
||||
sent = [];
|
||||
|
|
|
|||
|
|
@ -1,6 +1,13 @@
|
|||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable } from 'node:stream';
|
||||
import { FeishuHarnessBridge } from '../../../src/channels/feishu/bridge.mjs';
|
||||
import { DEFAULT_IMAGE_PROMPT } from '../../../src/channels/shared/image-prompt.mjs';
|
||||
|
||||
const PNG_1X1 = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
function deferred() {
|
||||
let resolve;
|
||||
|
|
@ -206,6 +213,177 @@ test('bridge maps a Feishu conversation to a persistent Harness session and repl
|
|||
assert.equal(status.messagesRejected, 1);
|
||||
});
|
||||
|
||||
test('bridge downloads an inbound Feishu image once and submits structured Harness content', async () => {
|
||||
const fixture = stateFixture([['p2p:ou_user', 'session-image']]);
|
||||
const downloaded = [];
|
||||
const asked = [];
|
||||
const sent = [];
|
||||
const client = {
|
||||
im: { v1: {
|
||||
messageResource: { get: async (request) => {
|
||||
downloaded.push(request);
|
||||
return {
|
||||
headers: { 'content-length': String(PNG_1X1.length) },
|
||||
getReadableStream: () => Readable.from([PNG_1X1]),
|
||||
};
|
||||
} },
|
||||
message: { create: async (request) => {
|
||||
sent.push(JSON.parse(request.data.content).text);
|
||||
return { code: 0, data: { message_id: 'om_image_reply' } };
|
||||
} },
|
||||
} },
|
||||
};
|
||||
const bridge = new FeishuHarnessBridge({
|
||||
client,
|
||||
channel: {},
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
asked.push({ sessionId, content });
|
||||
return '看到了一张图片';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
status: bridgeStatus(),
|
||||
allowedSenderOpenIds: new Set(['ou_user']),
|
||||
});
|
||||
const imageEvent = event('om_image_input', '', {
|
||||
message_type: 'image',
|
||||
content: JSON.stringify({ image_key: 'img_input' }),
|
||||
});
|
||||
|
||||
await bridge.accept(imageEvent);
|
||||
await bridge.accept(imageEvent);
|
||||
await bridge.accept({
|
||||
...event('om_image_unauthorized', '', {
|
||||
message_type: 'image',
|
||||
content: JSON.stringify({ image_key: 'img_unauthorized' }),
|
||||
}),
|
||||
sender: { sender_type: 'user', sender_id: { open_id: 'ou_other' } },
|
||||
});
|
||||
await bridge.waitForIdle();
|
||||
|
||||
assert.deepEqual(downloaded, [{
|
||||
path: { message_id: 'om_image_input', file_key: 'img_input' },
|
||||
params: { type: 'image' },
|
||||
}]);
|
||||
assert.deepEqual(asked, [{
|
||||
sessionId: 'session-image',
|
||||
content: [
|
||||
{ type: 'text', text: DEFAULT_IMAGE_PROMPT },
|
||||
{ type: 'image', mediaType: 'image/png', data: PNG_1X1.toString('base64') },
|
||||
],
|
||||
}]);
|
||||
assert.deepEqual(sent, ['看到了一张图片']);
|
||||
});
|
||||
|
||||
test('bridge sends Feishu post text and all embedded images as one structured prompt', async () => {
|
||||
const fixture = stateFixture([['group:oc_post_group', 'session-post']]);
|
||||
const downloaded = [];
|
||||
const asked = [];
|
||||
const sent = [];
|
||||
const client = {
|
||||
im: { v1: {
|
||||
messageResource: { get: async (request) => {
|
||||
downloaded.push(request);
|
||||
return {
|
||||
headers: { 'content-length': String(PNG_1X1.length) },
|
||||
getReadableStream: () => Readable.from([PNG_1X1]),
|
||||
};
|
||||
} },
|
||||
message: { create: async (request) => {
|
||||
sent.push(JSON.parse(request.data.content).text);
|
||||
return { code: 0, data: { message_id: 'om_post_reply' } };
|
||||
} },
|
||||
} },
|
||||
};
|
||||
const bridge = new FeishuHarnessBridge({
|
||||
client,
|
||||
channel: {},
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
asked.push({ sessionId, content });
|
||||
return '两张图片都已收到';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
status: bridgeStatus(),
|
||||
allowedSenderOpenIds: new Set(['ou_user']),
|
||||
});
|
||||
const postEvent = event('om_post_input', '', {
|
||||
message_type: 'post',
|
||||
chat_type: 'group',
|
||||
chat_id: 'oc_post_group',
|
||||
mentions: [{ key: '@_bot_1' }],
|
||||
content: JSON.stringify({
|
||||
title: '比较截图',
|
||||
content: [
|
||||
[
|
||||
{ tag: 'at', user_id: '@_bot_1', user_name: '机器人' },
|
||||
{ tag: 'text', text: '@_bot_1 请比较 ' },
|
||||
{ tag: 'a', text: '这两张图', href: 'https://example.com' },
|
||||
],
|
||||
[{ tag: 'img', image_key: 'img_post_first' }],
|
||||
[{ tag: 'img', image_key: 'img_post_second' }],
|
||||
],
|
||||
}),
|
||||
});
|
||||
|
||||
await bridge.accept(postEvent);
|
||||
await bridge.waitForIdle();
|
||||
|
||||
assert.deepEqual(downloaded, [
|
||||
{
|
||||
path: { message_id: 'om_post_input', file_key: 'img_post_first' },
|
||||
params: { type: 'image' },
|
||||
},
|
||||
{
|
||||
path: { message_id: 'om_post_input', file_key: 'img_post_second' },
|
||||
params: { type: 'image' },
|
||||
},
|
||||
]);
|
||||
assert.deepEqual(asked, [{
|
||||
sessionId: 'session-post',
|
||||
content: [
|
||||
{ type: 'text', text: '比较截图\n请比较 这两张图' },
|
||||
{ type: 'image', mediaType: 'image/png', data: PNG_1X1.toString('base64') },
|
||||
{ type: 'image', mediaType: 'image/png', data: PNG_1X1.toString('base64') },
|
||||
],
|
||||
}]);
|
||||
assert.deepEqual(sent, ['两张图片都已收到']);
|
||||
});
|
||||
|
||||
test('text inside a Feishu post is a model prompt rather than a local command', async () => {
|
||||
const fixture = stateFixture([['p2p:ou_user', 'session-post-command']]);
|
||||
const asked = [];
|
||||
const sent = [];
|
||||
const bridge = new FeishuHarnessBridge({
|
||||
client: textClient(async ({ text }) => sent.push(text)),
|
||||
channel: {},
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
asked.push({ sessionId, content });
|
||||
return '按普通内容处理';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
status: bridgeStatus(),
|
||||
allowedSenderOpenIds: new Set(['ou_user']),
|
||||
});
|
||||
|
||||
await bridge.accept(event('om_post_command', '', {
|
||||
message_type: 'post',
|
||||
content: JSON.stringify({ content: [[{ tag: 'text', text: '/new' }]] }),
|
||||
}));
|
||||
await bridge.waitForIdle();
|
||||
|
||||
assert.equal(fixture.sessions.get('p2p:ou_user'), 'session-post-command');
|
||||
assert.deepEqual(asked, [{ sessionId: 'session-post-command', content: '/new' }]);
|
||||
assert.deepEqual(sent, ['按普通内容处理']);
|
||||
});
|
||||
|
||||
test('a threaded Feishu reply answers a pending Harness question before the original turn queue', async () => {
|
||||
const sent = [];
|
||||
const streamed = [];
|
||||
|
|
@ -676,7 +854,7 @@ test('a queued next prompt stays separate while a failed interaction response is
|
|||
assert.deepEqual(sent.slice(-2).map(({ text }) => text), ['第一轮完成', '第二轮完成']);
|
||||
});
|
||||
|
||||
test('a non-text pending reply does not block the valid answer behind it', async () => {
|
||||
test('a rich-post pending reply does not block the valid text answer behind it', async () => {
|
||||
const fixture = stateFixture([['p2p:ou_user', 'session-invalid-reply']]);
|
||||
const sent = [];
|
||||
const invalidNoticeStarted = deferred();
|
||||
|
|
@ -722,9 +900,14 @@ test('a non-text pending reply does not block the valid answer behind it', async
|
|||
|
||||
const first = bridge.accept(event('invalid-reply-start', '启动交互'));
|
||||
await eventually(() => sent.some(({ text }) => text.includes('请给出有效文字答案')));
|
||||
const invalid = bridge.accept(event('invalid-reply-image', '', {
|
||||
message_type: 'image',
|
||||
content: JSON.stringify({ image_key: 'img-test' }),
|
||||
const invalid = bridge.accept(event('invalid-reply-post', '', {
|
||||
message_type: 'post',
|
||||
content: JSON.stringify({
|
||||
content: [
|
||||
[{ tag: 'text', text: '这不是文字回答' }],
|
||||
[{ tag: 'img', image_key: 'img-test' }],
|
||||
],
|
||||
}),
|
||||
}));
|
||||
await invalidNoticeStarted.promise;
|
||||
const valid = bridge.accept(event('invalid-reply-valid', '真正的答案'));
|
||||
|
|
|
|||
|
|
@ -1,13 +1,20 @@
|
|||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable } from 'node:stream';
|
||||
import {
|
||||
conversationKey,
|
||||
extractInboundMessage,
|
||||
extractText,
|
||||
isAllowedSender,
|
||||
isBotSender,
|
||||
splitText,
|
||||
} from '../../../src/channels/feishu/message-utils.mjs';
|
||||
|
||||
const PNG_1X1 = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
test('extractText removes bot mentions', () => {
|
||||
const event = {
|
||||
message: {
|
||||
|
|
@ -19,6 +26,121 @@ test('extractText removes bot mentions', () => {
|
|||
assert.equal(extractText(event), '你好');
|
||||
});
|
||||
|
||||
test('extractInboundMessage lazily downloads a Feishu image resource as a bounded stream', async () => {
|
||||
const calls = [];
|
||||
const event = {
|
||||
message: {
|
||||
message_id: 'om_image',
|
||||
message_type: 'image',
|
||||
content: JSON.stringify({ image_key: 'img_test' }),
|
||||
},
|
||||
};
|
||||
const client = { im: { v1: { messageResource: { get: async (request) => {
|
||||
calls.push(request);
|
||||
return {
|
||||
headers: { 'content-length': String(PNG_1X1.length) },
|
||||
getReadableStream: () => Readable.from([
|
||||
PNG_1X1.subarray(0, 12),
|
||||
PNG_1X1.subarray(12),
|
||||
]),
|
||||
};
|
||||
} } } } };
|
||||
|
||||
const message = extractInboundMessage(event, client);
|
||||
assert.equal(message.content, '');
|
||||
assert.equal(message.images.length, 1);
|
||||
assert.deepEqual(await message.images[0].load({ maxBytes: 1024 }), PNG_1X1);
|
||||
assert.deepEqual(calls, [{
|
||||
path: { message_id: 'om_image', file_key: 'img_test' },
|
||||
params: { type: 'image' },
|
||||
}]);
|
||||
});
|
||||
|
||||
test('extractInboundMessage preserves visible Feishu post text and every embedded image', async () => {
|
||||
const calls = [];
|
||||
const event = {
|
||||
message: {
|
||||
message_id: 'om_post',
|
||||
message_type: 'post',
|
||||
mentions: [{ key: '@_bot_1' }],
|
||||
content: JSON.stringify({
|
||||
title: '截图比较',
|
||||
content: [
|
||||
[
|
||||
{ tag: 'at', user_id: '@_bot_1', user_name: '机器人' },
|
||||
{ tag: 'text', text: '@_bot_1 请查看 ' },
|
||||
{ tag: 'a', text: '第一处', href: 'https://example.com/one' },
|
||||
{ tag: 'link', text: ' 和第二处', href: 'https://example.com/two' },
|
||||
],
|
||||
[{ tag: 'img', image_key: 'img_first' }],
|
||||
[{ tag: 'text', text: '补充说明' }],
|
||||
[
|
||||
{ tag: 'img', image_key: 'img_second' },
|
||||
{ tag: 'img', image_key: ' ' },
|
||||
],
|
||||
],
|
||||
}),
|
||||
},
|
||||
};
|
||||
const client = { im: { v1: { messageResource: { get: async (request) => {
|
||||
calls.push(request);
|
||||
return {
|
||||
headers: { 'content-length': String(PNG_1X1.length) },
|
||||
getReadableStream: () => Readable.from([PNG_1X1]),
|
||||
};
|
||||
} } } } };
|
||||
|
||||
const message = extractInboundMessage(event, client);
|
||||
assert.equal(extractText(event), null, 'rich posts must not become interaction replies');
|
||||
assert.equal(message.content, '截图比较\n请查看 第一处 和第二处\n补充说明');
|
||||
assert.equal(message.images.length, 2);
|
||||
assert.deepEqual(await Promise.all(message.images.map((image) => image.load({ maxBytes: 1024 }))), [
|
||||
PNG_1X1,
|
||||
PNG_1X1,
|
||||
]);
|
||||
assert.deepEqual(calls, [
|
||||
{
|
||||
path: { message_id: 'om_post', file_key: 'img_first' },
|
||||
params: { type: 'image' },
|
||||
},
|
||||
{
|
||||
path: { message_id: 'om_post', file_key: 'img_second' },
|
||||
params: { type: 'image' },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test('Feishu image loading rejects declared or streamed data above the caller limit', async () => {
|
||||
for (const resource of [
|
||||
{
|
||||
headers: { 'content-length': '5' },
|
||||
getReadableStream: () => Readable.from([Buffer.alloc(5)]),
|
||||
},
|
||||
{
|
||||
headers: {},
|
||||
getReadableStream: () => Readable.from([Buffer.alloc(2), Buffer.alloc(3)]),
|
||||
},
|
||||
]) {
|
||||
const message = extractInboundMessage({
|
||||
message: {
|
||||
message_id: 'om_large',
|
||||
message_type: 'image',
|
||||
content: JSON.stringify({ image_key: 'img_large' }),
|
||||
},
|
||||
}, { im: { v1: { messageResource: { get: async () => resource } } } });
|
||||
await assert.rejects(message.images[0].load({ maxBytes: 4 }), /limit|exceeds/);
|
||||
}
|
||||
});
|
||||
|
||||
test('malformed Feishu image content does not create a downloadable image reference', () => {
|
||||
assert.deepEqual(extractInboundMessage({
|
||||
message: { message_type: 'image', content: '{not-json' },
|
||||
}, {}), { content: '', images: [] });
|
||||
assert.deepEqual(extractInboundMessage({
|
||||
message: { message_type: 'post', content: '{not-json' },
|
||||
}, {}), { content: '', images: [] });
|
||||
});
|
||||
|
||||
test('conversationKey isolates p2p users and groups', () => {
|
||||
assert.equal(conversationKey({
|
||||
sender: { sender_id: { open_id: 'ou_test' } },
|
||||
|
|
|
|||
|
|
@ -81,6 +81,7 @@ test('QR success stores the secret off-config and becomes immediately chat-ready
|
|||
assert.equal(fx.getSdkOptions().addons.preset, false);
|
||||
assert.deepEqual(fx.getSdkOptions().addons.events.items.tenant, ['im.message.receive_v1']);
|
||||
assert.ok(fx.getSdkOptions().addons.scopes.tenant.includes('im:message.p2p_msg:readonly'));
|
||||
assert.ok(fx.getSdkOptions().addons.scopes.tenant.includes('im:message:readonly'));
|
||||
assert.ok(fx.getSdkOptions().addons.scopes.tenant.includes('im:message:send_as_bot'));
|
||||
assert.ok(fx.getSdkOptions().addons.scopes.tenant.includes('cardkit:card:write'));
|
||||
fx.getSdkOptions().onQRCodeReady({ url: 'https://accounts.feishu.cn/qr', expireIn: 600 });
|
||||
|
|
|
|||
|
|
@ -51,6 +51,189 @@ function message(overrides = {}) {
|
|||
};
|
||||
}
|
||||
|
||||
const PNG_BYTES = Buffer.from([
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x00, 0x00, 0x00, 0x00,
|
||||
]);
|
||||
|
||||
test('QQ sends image-only attachments to Harness and accepts the SDK file MIME fallback', async () => {
|
||||
const fixture = stateFixture([['c2c:owner-openid', 'session-image']]);
|
||||
const prompts = [];
|
||||
const downloads = [];
|
||||
const sent = [];
|
||||
const bridge = new QqHarnessBridge({
|
||||
bot: { sendText: async (_target, text) => sent.push(text) },
|
||||
ownerUserOpenid: 'owner-openid',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
prompts.push({ sessionId, content });
|
||||
return '看到图片了';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
fetchImpl: async (url, init) => {
|
||||
downloads.push({ url: url.toString(), init });
|
||||
return new Response(PNG_BYTES, { headers: { 'content-type': 'application/octet-stream' } });
|
||||
},
|
||||
});
|
||||
|
||||
await bridge.accept(message({
|
||||
messageId: 'qq-image',
|
||||
content: '',
|
||||
attachments: [{
|
||||
content_type: 'file',
|
||||
filename: 'diagram.PNG',
|
||||
size: PNG_BYTES.length,
|
||||
url: 'https://multimedia.nt.qq.com.cn/download/opaque',
|
||||
}],
|
||||
}));
|
||||
|
||||
assert.equal(downloads.length, 1);
|
||||
assert.equal(downloads[0].init.method, 'GET');
|
||||
assert.equal(downloads[0].init.redirect, 'manual');
|
||||
assert.equal(prompts.length, 1);
|
||||
assert.equal(prompts[0].sessionId, 'session-image');
|
||||
assert.deepEqual(prompts[0].content.map(({ type }) => type), ['text', 'image']);
|
||||
assert.equal(prompts[0].content[0].text, '请分析这张图片。');
|
||||
assert.equal(prompts[0].content[1].mediaType, 'image/png');
|
||||
assert.equal(prompts[0].content[1].name, 'diagram.PNG');
|
||||
assert.equal(Buffer.from(prompts[0].content[1].data, 'base64').equals(PNG_BYTES), true);
|
||||
assert.deepEqual(sent, ['看到图片了']);
|
||||
assert.equal(fixture.seen.has('qq-image'), true);
|
||||
});
|
||||
|
||||
test('QQ checks sender and group mention before downloading image attachments', async () => {
|
||||
let downloads = 0;
|
||||
let asks = 0;
|
||||
const bridge = new QqHarnessBridge({
|
||||
bot: { sendText: async () => {} },
|
||||
ownerUserOpenid: 'owner-openid',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async () => { asks += 1; return 'unexpected'; },
|
||||
},
|
||||
state: stateFixture().state,
|
||||
fetchImpl: async () => { downloads += 1; return new Response(PNG_BYTES); },
|
||||
});
|
||||
const attachment = {
|
||||
content_type: 'image/png',
|
||||
filename: 'private.png',
|
||||
url: 'https://multimedia.nt.qq.com.cn/download/private',
|
||||
};
|
||||
|
||||
await bridge.accept(message({
|
||||
messageId: 'qq-image-other',
|
||||
senderId: 'other-openid',
|
||||
content: '',
|
||||
attachments: [attachment],
|
||||
}));
|
||||
await bridge.accept(message({
|
||||
kind: 'group',
|
||||
rawEventType: 'GROUP_MESSAGE_CREATE',
|
||||
groupOpenid: 'group-1',
|
||||
messageId: 'qq-image-unmentioned',
|
||||
content: '',
|
||||
attachments: [attachment],
|
||||
replyTarget: { scope: 'group', targetId: 'group-1', msgId: 'qq-image-unmentioned' },
|
||||
}));
|
||||
|
||||
assert.equal(downloads, 0);
|
||||
assert.equal(asks, 0);
|
||||
});
|
||||
|
||||
test('QQ rejects non-platform image URLs without fetching and returns a retryable image error', async () => {
|
||||
const fixture = stateFixture([['c2c:owner-openid', 'session-image']]);
|
||||
const sent = [];
|
||||
let downloads = 0;
|
||||
const bridge = new QqHarnessBridge({
|
||||
bot: { sendText: async (_target, text) => sent.push(text) },
|
||||
ownerUserOpenid: 'owner-openid',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async () => assert.fail('an untrusted image must not reach Harness'),
|
||||
},
|
||||
state: fixture.state,
|
||||
logger: { error() {} },
|
||||
fetchImpl: async () => { downloads += 1; return new Response(PNG_BYTES); },
|
||||
});
|
||||
|
||||
await bridge.accept(message({
|
||||
messageId: 'qq-image-untrusted',
|
||||
content: '这是什么',
|
||||
attachments: [{
|
||||
content_type: 'image/png',
|
||||
filename: 'photo.png',
|
||||
url: 'https://attacker.example/photo.png',
|
||||
}],
|
||||
}));
|
||||
|
||||
assert.equal(downloads, 0);
|
||||
assert.deepEqual(sent, ['图片下载失败，请重新发送后再试。']);
|
||||
assert.equal(fixture.seen.has('qq-image-untrusted'), true);
|
||||
});
|
||||
|
||||
test('QQ does not use an image caption as a pending Harness answer', async () => {
|
||||
const fixture = stateFixture([['c2c:owner-openid', 'session-question-image']]);
|
||||
const sent = [];
|
||||
const answered = deferred();
|
||||
let submitted;
|
||||
let downloads = 0;
|
||||
const bridge = new QqHarnessBridge({
|
||||
bot: { sendText: async (_target, text) => sent.push(text) },
|
||||
ownerUserOpenid: 'owner-openid',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
async ask(sessionId, _text, options) {
|
||||
await options.onInteraction({
|
||||
kind: 'question',
|
||||
interactionId: 'qq-question-image',
|
||||
rpcId: 'qq-question-image',
|
||||
sessionId,
|
||||
payload: {
|
||||
questions: [{
|
||||
id: 'environment',
|
||||
question: '请选择环境',
|
||||
options: [{ label: '生产环境' }],
|
||||
}],
|
||||
},
|
||||
async respond(result) {
|
||||
submitted = result;
|
||||
answered.resolve();
|
||||
return { accepted: true };
|
||||
},
|
||||
});
|
||||
await answered.promise;
|
||||
return '已完成';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
fetchImpl: async () => { downloads += 1; return new Response(PNG_BYTES); },
|
||||
});
|
||||
|
||||
const first = bridge.accept(message({ messageId: 'qq-question-start', content: '开始提问' }));
|
||||
await eventually(() => sent.some((text) => text.includes('请选择环境')));
|
||||
await bridge.accept(message({
|
||||
messageId: 'qq-question-image-answer',
|
||||
content: '生产环境',
|
||||
attachments: [{
|
||||
content_type: 'image/png',
|
||||
filename: 'answer.png',
|
||||
url: 'https://multimedia.nt.qq.com.cn/download/answer',
|
||||
}],
|
||||
}));
|
||||
assert.equal(downloads, 0);
|
||||
assert.equal(submitted, undefined);
|
||||
assert.equal(sent.at(-1), '请用文字回答当前问题。');
|
||||
|
||||
await bridge.accept(message({ messageId: 'qq-question-text-answer', content: '生产环境' }));
|
||||
await first;
|
||||
assert.deepEqual(submitted.value.answer.answers, [{
|
||||
id: 'environment',
|
||||
selected: ['生产环境'],
|
||||
}]);
|
||||
});
|
||||
|
||||
test('QQ executes /compact for the bound Session without prompting the model', async () => {
|
||||
const fixture = stateFixture([['c2c:owner-openid', 'session-compact']]);
|
||||
const sent = [];
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ import {
|
|||
SlackRuntime,
|
||||
normalizeSlackEvent,
|
||||
} from '../../../src/channels/slack/slack-runtime.mjs';
|
||||
import { SLACK_APP_MANIFEST_YAML } from '../../../src/channels/slack/manifest.mjs';
|
||||
import {
|
||||
SLACK_ENDPOINTS,
|
||||
createSlackRpcHandler,
|
||||
|
|
@ -133,6 +134,146 @@ test('Slack API uses native streaming methods and suppresses generated mass ment
|
|||
assert.equal(calls[3].body.text, '请通知 @channel 和 @U99999999');
|
||||
});
|
||||
|
||||
test('Slack downloads private files with the bot token from Slack hosts only', async () => {
|
||||
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const calls = [];
|
||||
const api = new SlackApi({
|
||||
botToken: BOT_TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
calls.push({ url, options });
|
||||
return new Response(png, { status: 200, headers: { 'content-length': String(png.length) } });
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await api.downloadFile({
|
||||
url: 'https://files.slack.com/files-pri/T123-F123/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), png);
|
||||
assert.equal(calls[0].options.headers.authorization, `Bearer ${BOT_TOKEN}`);
|
||||
assert.equal(calls[0].options.redirect, 'manual');
|
||||
assert.deepEqual(await api.downloadFile({
|
||||
url: 'https://slack.com/files-pri/T123-F124/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), png);
|
||||
assert.equal(calls[1].options.headers.authorization, `Bearer ${BOT_TOKEN}`);
|
||||
assert.equal(calls[1].url.hostname, 'files.slack.com');
|
||||
await assert.rejects(() => api.downloadFile({
|
||||
url: 'https://example.com/internal.png', maxBytes: 100,
|
||||
}), /messaging platform/);
|
||||
assert.equal(calls.length, 2);
|
||||
|
||||
const redirectCalls = [];
|
||||
const redirectingApi = new SlackApi({
|
||||
botToken: BOT_TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
redirectCalls.push({ url, options });
|
||||
return new Response(null, {
|
||||
status: 302,
|
||||
headers: { location: 'https://example.com/leak-token' },
|
||||
});
|
||||
},
|
||||
});
|
||||
await assert.rejects(() => redirectingApi.downloadFile({
|
||||
url: 'https://slack.com/files-pri/T123-F125/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), (error) => {
|
||||
assert.equal(error.code, 'image-redirect-blocked');
|
||||
return true;
|
||||
});
|
||||
assert.equal(redirectCalls.length, 1);
|
||||
assert.equal(redirectCalls[0].url.hostname, 'files.slack.com');
|
||||
assert.equal(redirectCalls[0].options.headers.authorization, `Bearer ${BOT_TOKEN}`);
|
||||
assert.match(SLACK_APP_MANIFEST_YAML, /\n\s+- files:read\n/);
|
||||
});
|
||||
|
||||
test('Slack refuses unsafe file redirects and explains stale files:read authorization', async () => {
|
||||
const workspaceCalls = [];
|
||||
const workspaceApi = new SlackApi({
|
||||
botToken: BOT_TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
workspaceCalls.push({ url, options });
|
||||
return new Response(null, {
|
||||
status: 302,
|
||||
headers: { location: 'https://workspace-name.slack.com/?redir=%2Ffiles-pri%2Fsecret' },
|
||||
});
|
||||
},
|
||||
});
|
||||
await assert.rejects(() => workspaceApi.downloadFile({
|
||||
url: 'https://files.slack.com/files-pri/T123-F126/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), (error) => {
|
||||
assert.equal(error.code, 'slack-file-access-required');
|
||||
assert.match(error.userMessage, /files:read/);
|
||||
return true;
|
||||
});
|
||||
assert.equal(workspaceCalls.length, 1);
|
||||
|
||||
const missingScopeCalls = [];
|
||||
const missingScopeApi = new SlackApi({
|
||||
botToken: BOT_TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
missingScopeCalls.push({ url, options });
|
||||
if (url.pathname.endsWith('/auth.test')) {
|
||||
return new Response(JSON.stringify({
|
||||
ok: true,
|
||||
team_id: 'T12345678',
|
||||
user_id: 'U12345678',
|
||||
bot_id: 'B12345678',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'x-oauth-scopes': 'app_mentions:read,chat:write,im:history',
|
||||
},
|
||||
});
|
||||
}
|
||||
return new Response(null, {
|
||||
status: 302,
|
||||
headers: {
|
||||
location: 'https://files-origin.slack.com/files-pri/T123-F126/download/image.png',
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
await missingScopeApi.authTest();
|
||||
await assert.rejects(() => missingScopeApi.downloadFile({
|
||||
url: 'https://files.slack.com/files-pri/T123-F126/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), (error) => {
|
||||
assert.equal(error.code, 'slack-file-access-required');
|
||||
assert.match(error.userMessage, /files:read/);
|
||||
return true;
|
||||
});
|
||||
assert.equal(missingScopeCalls.length, 2);
|
||||
assert.deepEqual(missingScopeCalls.map((call) => call.url.hostname), [
|
||||
'slack.com', 'files.slack.com',
|
||||
]);
|
||||
|
||||
for (const location of [
|
||||
'https://example.com/leak-token',
|
||||
'http://files-origin.slack.com/files-pri/T123-F126/download/image.png',
|
||||
'https://files-origin.slack.com/files-pri/T123-F126/download/image.png',
|
||||
'https://files-origin.slack.com/files-pri/T123-F999/download/image.png',
|
||||
'https://files-origin.slack.com/files-pri/T123-F126/download/image.png?changed=1',
|
||||
]) {
|
||||
const unsafeCalls = [];
|
||||
const unsafeApi = new SlackApi({
|
||||
botToken: BOT_TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
unsafeCalls.push({ url, options });
|
||||
return new Response(null, { status: 302, headers: { location } });
|
||||
},
|
||||
});
|
||||
await assert.rejects(() => unsafeApi.downloadFile({
|
||||
url: 'https://files.slack.com/files-pri/T123-F126/download/image.png',
|
||||
maxBytes: 100,
|
||||
}), (error) => {
|
||||
assert.equal(error.code, 'image-redirect-blocked');
|
||||
return true;
|
||||
});
|
||||
assert.equal(unsafeCalls.length, 1);
|
||||
}
|
||||
});
|
||||
|
||||
test('Slack controller stores two protected credential references and exposes neither token', async (t) => {
|
||||
const directory = await mkdtemp(join(tmpdir(), 'dsh-im-slack-'));
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
|
|
@ -260,6 +401,62 @@ test('Slack normalizes direct messages and addressed channel events', () => {
|
|||
assert.equal(botMessage, null);
|
||||
});
|
||||
|
||||
test('Slack accepts image file shares and keeps other files out of image prompts', async () => {
|
||||
const loads = [];
|
||||
const direct = normalizeSlackEvent({
|
||||
event_id: 'Ev010',
|
||||
team_id: 'T12345678',
|
||||
event: {
|
||||
type: 'message',
|
||||
subtype: 'file_share',
|
||||
channel_type: 'im',
|
||||
channel: 'D12345678',
|
||||
user: 'U87654321',
|
||||
ts: '1700000000.010',
|
||||
text: '识别一下',
|
||||
files: [{
|
||||
id: 'F12345678', name: 'screen.png', mimetype: 'image/png', size: 2_000,
|
||||
url_private_download: 'https://files.slack.com/files-pri/T123-F123/download/screen.png',
|
||||
}, {
|
||||
id: 'F12345679', name: 'notes.txt', mimetype: 'text/plain', size: 20,
|
||||
url_private: 'https://files.slack.com/files-pri/T123-F124/notes.txt',
|
||||
}],
|
||||
},
|
||||
}, 'U12345678', {
|
||||
loadFile: async (url, options) => {
|
||||
loads.push({ url, options });
|
||||
return Buffer.from('image');
|
||||
},
|
||||
});
|
||||
assert.equal(direct.images.length, 1);
|
||||
assert.equal(direct.images[0].name, 'screen.png');
|
||||
await direct.images[0].load({ maxBytes: 5_000 });
|
||||
assert.match(loads[0].url, /screen\.png$/);
|
||||
|
||||
const group = normalizeSlackEvent({
|
||||
event_id: 'Ev011',
|
||||
team_id: 'T12345678',
|
||||
event: {
|
||||
type: 'app_mention', channel: 'C12345678', user: 'U87654321', ts: '1700000000.011',
|
||||
text: '<@U12345678>',
|
||||
files: [{
|
||||
id: 'F12345680', name: 'photo.jpg', mimetype: 'image/jpeg', size: 1_000,
|
||||
url_private: 'https://files.slack.com/files-pri/T123-F125/photo.jpg',
|
||||
}],
|
||||
},
|
||||
}, 'U12345678');
|
||||
assert.equal(group.addressed, true);
|
||||
assert.equal(group.images.length, 1);
|
||||
|
||||
assert.equal(normalizeSlackEvent({
|
||||
event_id: 'Ev012',
|
||||
event: {
|
||||
type: 'message', subtype: 'message_changed', channel_type: 'im', channel: 'D12345678',
|
||||
user: 'U87654321', ts: '1700000000.012', text: '', files: [],
|
||||
},
|
||||
}, 'U12345678'), null);
|
||||
});
|
||||
|
||||
class FakeSocket {
|
||||
#listeners = new Map();
|
||||
sent = [];
|
||||
|
|
|
|||
|
|
@ -120,6 +120,37 @@ test('Telegram API validates a Bot Token without exposing it in requests or erro
|
|||
});
|
||||
});
|
||||
|
||||
test('Telegram API resolves and downloads files without exposing arbitrary paths', async () => {
|
||||
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const calls = [];
|
||||
const api = new TelegramApi({
|
||||
token: TOKEN,
|
||||
fetchImpl: async (url, options) => {
|
||||
calls.push({ url, options });
|
||||
if (url.pathname.endsWith('/getFile')) {
|
||||
return jsonResponse({ ok: true, result: { file_path: 'photos/file_1.png' } });
|
||||
}
|
||||
return new Response(png, { status: 200, headers: { 'content-length': String(png.length) } });
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await api.downloadFile({ fileId: 'AgAC_test-file', maxBytes: 100 }), png);
|
||||
assert.equal(calls.length, 2);
|
||||
assert.equal(calls[0].options.method, 'POST');
|
||||
assert.equal(calls[1].options.method, 'GET');
|
||||
assert.equal(calls[1].url.hostname, 'api.telegram.org');
|
||||
assert.match(calls[1].url.pathname, /\/file\/bot.+\/photos\/file_1\.png$/);
|
||||
|
||||
const unsafeApi = new TelegramApi({
|
||||
token: TOKEN,
|
||||
fetchImpl: async () => jsonResponse({ ok: true, result: { file_path: '../secret' } }),
|
||||
});
|
||||
await assert.rejects(() => unsafeApi.downloadFile({ fileId: 'AgAC_test-file' }), (error) => {
|
||||
assert.match(error.message, /invalid file path/);
|
||||
assert.doesNotMatch(error.message, new RegExp(TOKEN.replaceAll(':', '\\:')));
|
||||
return true;
|
||||
});
|
||||
});
|
||||
|
||||
test('Telegram config and controller store only a credential reference in bot data', async (t) => {
|
||||
const directory = await mkdtemp(join(tmpdir(), 'dsh-im-telegram-'));
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
|
|
@ -252,6 +283,73 @@ test('Telegram normalizes private messages and requires an explicit group addres
|
|||
assert.equal(topicTwo.replyTarget.messageThreadId, 200);
|
||||
});
|
||||
|
||||
test('Telegram normalizes photo captions and image documents into one downloadable image', async () => {
|
||||
const loads = [];
|
||||
const groupPhoto = normalizeTelegramUpdate({
|
||||
update_id: 20,
|
||||
message: {
|
||||
message_id: 10,
|
||||
chat: { id: -1001, type: 'supergroup' },
|
||||
from: { id: 43, is_bot: false },
|
||||
caption: '@HarnessBot 看看这张图',
|
||||
caption_entities: [{ type: 'mention', offset: 0, length: 11 }],
|
||||
photo: [
|
||||
{ file_id: 'small', file_unique_id: 'photo', width: 90, height: 90, file_size: 500 },
|
||||
{ file_id: 'large', file_unique_id: 'photo', width: 1280, height: 720, file_size: 2_000 },
|
||||
],
|
||||
},
|
||||
}, {
|
||||
botId: '123456789',
|
||||
username: 'HarnessBot',
|
||||
loadFile: async (fileId, options) => {
|
||||
loads.push({ fileId, options });
|
||||
return Buffer.from('image');
|
||||
},
|
||||
});
|
||||
assert.equal(groupPhoto.addressed, true);
|
||||
assert.equal(groupPhoto.content, '看看这张图');
|
||||
assert.equal(groupPhoto.images.length, 1);
|
||||
assert.equal(groupPhoto.images[0].size, 2_000);
|
||||
await groupPhoto.images[0].load({ maxBytes: 5_000 });
|
||||
assert.equal(loads[0].fileId, 'large');
|
||||
|
||||
const document = normalizeTelegramUpdate({
|
||||
update_id: 21,
|
||||
message: {
|
||||
message_id: 11,
|
||||
chat: { id: 88, type: 'private' },
|
||||
from: { id: 42, is_bot: false },
|
||||
document: {
|
||||
file_id: 'png-document', file_name: 'diagram.png', mime_type: 'image/png', file_size: 3_000,
|
||||
},
|
||||
},
|
||||
}, { botId: '123456789', username: 'HarnessBot' });
|
||||
assert.equal(document.images[0].name, 'diagram.png');
|
||||
assert.equal(document.images[0].mediaType, 'image/png');
|
||||
|
||||
const documentWithoutMime = normalizeTelegramUpdate({
|
||||
update_id: 23,
|
||||
message: {
|
||||
message_id: 13,
|
||||
chat: { id: 88, type: 'private' },
|
||||
from: { id: 42, is_bot: false },
|
||||
document: { file_id: 'webp-document', file_name: 'diagram.webp', file_size: 2_000 },
|
||||
},
|
||||
}, { botId: '123456789', username: 'HarnessBot' });
|
||||
assert.equal(documentWithoutMime.images[0].mediaType, 'image/webp');
|
||||
|
||||
const pdf = normalizeTelegramUpdate({
|
||||
update_id: 22,
|
||||
message: {
|
||||
message_id: 12,
|
||||
chat: { id: 88, type: 'private' },
|
||||
from: { id: 42, is_bot: false },
|
||||
document: { file_id: 'pdf-document', file_name: 'file.pdf', mime_type: 'application/pdf' },
|
||||
},
|
||||
}, { botId: '123456789', username: 'HarnessBot' });
|
||||
assert.deepEqual(pdf.images, []);
|
||||
});
|
||||
|
||||
test('Telegram bridge ignores unaddressed groups and streams direct replies', async () => {
|
||||
const sent = [];
|
||||
const updates = [];
|
||||
|
|
|
|||
|
|
@ -1,7 +1,16 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { WecomHarnessBridge } from '../../../src/channels/wecom/wecom-bridge.mjs';
|
||||
import {
|
||||
WecomHarnessBridge,
|
||||
wecomInboundMessage,
|
||||
} from '../../../src/channels/wecom/wecom-bridge.mjs';
|
||||
import { DEFAULT_IMAGE_PROMPT } from '../../../src/channels/shared/image-prompt.mjs';
|
||||
|
||||
const PNG_1X1 = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
function frame(overrides = {}) {
|
||||
return {
|
||||
|
|
@ -152,6 +161,204 @@ test('Enterprise WeChat messages stream Harness progress and finalize once', asy
|
|||
assert.equal(bridge.status.messagesReplied, 1);
|
||||
});
|
||||
|
||||
test('Enterprise WeChat downloads an image with its AES key and submits structured content once', async () => {
|
||||
const transport = testClient();
|
||||
const downloads = [];
|
||||
const asked = [];
|
||||
transport.client.downloadFile = async (url, aeskey) => {
|
||||
downloads.push({ url, aeskey });
|
||||
return { buffer: PNG_1X1, filename: 'photo.png' };
|
||||
};
|
||||
const bridge = new WecomHarnessBridge({
|
||||
client: transport.client,
|
||||
generateStreamId: () => 'stream-image',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
asked.push({ sessionId, content });
|
||||
return '图片识别完成';
|
||||
},
|
||||
},
|
||||
state: state(),
|
||||
});
|
||||
const imageFrame = frame({
|
||||
msgid: 'image-1',
|
||||
msgtype: 'image',
|
||||
text: undefined,
|
||||
image: { url: 'https://wecom.example/image', aeskey: 'aes-image' },
|
||||
});
|
||||
|
||||
await bridge.accept(imageFrame);
|
||||
await bridge.accept(imageFrame);
|
||||
|
||||
assert.deepEqual(downloads, [{
|
||||
url: 'https://wecom.example/image',
|
||||
aeskey: 'aes-image',
|
||||
}]);
|
||||
assert.deepEqual(asked, [{
|
||||
sessionId: 'session-existing',
|
||||
content: [
|
||||
{ type: 'text', text: DEFAULT_IMAGE_PROMPT },
|
||||
{
|
||||
type: 'image',
|
||||
mediaType: 'image/png',
|
||||
data: PNG_1X1.toString('base64'),
|
||||
name: 'photo.png',
|
||||
},
|
||||
],
|
||||
}]);
|
||||
assert.equal(transport.streamed.at(-1).content, '图片识别完成');
|
||||
});
|
||||
|
||||
test('Enterprise WeChat preserves mixed-message text and image order', async () => {
|
||||
const transport = testClient();
|
||||
const jpeg = Buffer.from([0xff, 0xd8, 0xff, 0xd9]);
|
||||
const gif = Buffer.from('GIF89a payload');
|
||||
transport.client.downloadFile = async (url) => ({
|
||||
buffer: url.endsWith('/one') ? jpeg : gif,
|
||||
});
|
||||
let prompt;
|
||||
const bridge = new WecomHarnessBridge({
|
||||
client: transport.client,
|
||||
generateStreamId: () => 'stream-mixed',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (_sessionId, content) => { prompt = content; return 'ok'; },
|
||||
},
|
||||
state: state(),
|
||||
});
|
||||
|
||||
await bridge.accept(frame({
|
||||
msgid: 'mixed-image',
|
||||
msgtype: 'mixed',
|
||||
text: undefined,
|
||||
mixed: { msg_item: [
|
||||
{ msgtype: 'text', text: { content: '比较这两张图' } },
|
||||
{ msgtype: 'image', image: { url: 'https://wecom.example/one', aeskey: 'one' } },
|
||||
{ msgtype: 'image', image: { url: 'https://wecom.example/two', aeskey: 'two' } },
|
||||
] },
|
||||
}));
|
||||
|
||||
assert.deepEqual(prompt, [
|
||||
{ type: 'text', text: '比较这两张图' },
|
||||
{ type: 'image', mediaType: 'image/jpeg', data: jpeg.toString('base64') },
|
||||
{ type: 'image', mediaType: 'image/gif', data: gif.toString('base64') },
|
||||
]);
|
||||
});
|
||||
|
||||
test('Enterprise WeChat starts image download before an earlier conversation turn finishes', async () => {
|
||||
const transport = testClient();
|
||||
const firstStarted = deferred();
|
||||
const releaseFirst = deferred();
|
||||
const downloads = [];
|
||||
const prompts = [];
|
||||
transport.client.downloadFile = async (url, aeskey) => {
|
||||
downloads.push({ url, aeskey });
|
||||
return { buffer: PNG_1X1, filename: 'queued.png' };
|
||||
};
|
||||
const bridge = new WecomHarnessBridge({
|
||||
client: transport.client,
|
||||
generateStreamId: (() => { let index = 0; return () => `queued-${++index}`; })(),
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
async ask(_sessionId, prompt) {
|
||||
prompts.push(prompt);
|
||||
if (prompt === '先处理这个慢任务') {
|
||||
firstStarted.resolve();
|
||||
await releaseFirst.promise;
|
||||
}
|
||||
return 'ok';
|
||||
},
|
||||
},
|
||||
state: state(),
|
||||
});
|
||||
|
||||
const first = bridge.accept(frame({
|
||||
msgid: 'queued-text',
|
||||
text: { content: '先处理这个慢任务' },
|
||||
}));
|
||||
await firstStarted.promise;
|
||||
const second = bridge.accept(frame({
|
||||
msgid: 'queued-image',
|
||||
msgtype: 'image',
|
||||
text: undefined,
|
||||
image: { url: 'https://wecom.example/expiring', aeskey: 'queued-key' },
|
||||
}));
|
||||
|
||||
await eventually(() => downloads.length === 1);
|
||||
assert.deepEqual(prompts, ['先处理这个慢任务']);
|
||||
releaseFirst.resolve();
|
||||
await Promise.all([first, second]);
|
||||
assert.deepEqual(downloads, [{
|
||||
url: 'https://wecom.example/expiring',
|
||||
aeskey: 'queued-key',
|
||||
}]);
|
||||
assert.equal(Array.isArray(prompts[1]), true);
|
||||
assert.equal(prompts[1][1].mediaType, 'image/png');
|
||||
});
|
||||
|
||||
test('Enterprise WeChat bounds prefetched image memory while a conversation is queued', async () => {
|
||||
const transport = testClient();
|
||||
const firstStarted = deferred();
|
||||
const releaseFirst = deferred();
|
||||
const downloads = [];
|
||||
const prompts = [];
|
||||
transport.client.downloadFile = async (url) => {
|
||||
downloads.push(url);
|
||||
return { buffer: PNG_1X1 };
|
||||
};
|
||||
const bridge = new WecomHarnessBridge({
|
||||
client: transport.client,
|
||||
generateStreamId: (() => { let index = 0; return () => `bounded-${++index}`; })(),
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
async ask(_sessionId, prompt) {
|
||||
prompts.push(prompt);
|
||||
if (prompt === '阻塞队列') {
|
||||
firstStarted.resolve();
|
||||
await releaseFirst.promise;
|
||||
}
|
||||
return 'ok';
|
||||
},
|
||||
},
|
||||
state: state(),
|
||||
logger: { error() {}, warn() {} },
|
||||
});
|
||||
|
||||
const pending = [bridge.accept(frame({
|
||||
msgid: 'bounded-start',
|
||||
text: { content: '阻塞队列' },
|
||||
}))];
|
||||
await firstStarted.promise;
|
||||
for (let index = 0; index < 5; index += 1) {
|
||||
pending.push(bridge.accept(frame({
|
||||
msgid: `bounded-image-${index}`,
|
||||
msgtype: 'image',
|
||||
text: undefined,
|
||||
image: { url: `https://wecom.example/bounded-${index}`, aeskey: `key-${index}` },
|
||||
})));
|
||||
}
|
||||
|
||||
await eventually(() => downloads.length === 4);
|
||||
releaseFirst.resolve();
|
||||
await Promise.all(pending);
|
||||
assert.equal(downloads.length, 4);
|
||||
assert.equal(prompts.length, 5);
|
||||
assert.equal(transport.streamed.some(({ content }) => (
|
||||
content === '当前待处理图片较多，请稍后重新发送。'
|
||||
)), true);
|
||||
});
|
||||
|
||||
test('Enterprise WeChat image references enforce the caller byte limit after SDK decryption', async () => {
|
||||
const message = wecomInboundMessage(frame({
|
||||
msgtype: 'image',
|
||||
image: { url: 'https://wecom.example/large', aeskey: 'large-key' },
|
||||
}), {
|
||||
downloadFile: async () => ({ buffer: Buffer.alloc(5) }),
|
||||
});
|
||||
await assert.rejects(message.images[0].load({ maxBytes: 4 }), /exceeds/);
|
||||
});
|
||||
|
||||
test('Enterprise WeChat visibility scope accepts direct and group conversations without local approval', async () => {
|
||||
let asks = 0;
|
||||
const client = {
|
||||
|
|
|
|||
|
|
@ -1,11 +1,15 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import { createCipheriv, randomBytes } from 'node:crypto';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
createWeixinApi,
|
||||
decryptWeixinImage,
|
||||
extractWeixinText,
|
||||
normalizeWeixinApiBaseUrl,
|
||||
parseWeixinImageAesKey,
|
||||
splitWeixinText,
|
||||
weixinImageDownloadUrl,
|
||||
WeixinApiError,
|
||||
} from '../../../src/channels/weixin/weixin-api.mjs';
|
||||
|
||||
|
|
@ -17,6 +21,76 @@ function jsonResponse(value, init) {
|
|||
});
|
||||
}
|
||||
|
||||
function encryptImage(plaintext, key) {
|
||||
const cipher = createCipheriv('aes-128-ecb', key, null);
|
||||
return Buffer.concat([cipher.update(plaintext), cipher.final()]);
|
||||
}
|
||||
|
||||
test('iLink image references download lazily from the canonical CDN and decrypt AES-128-ECB', async () => {
|
||||
const key = randomBytes(16);
|
||||
const plaintext = Buffer.from([
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x01, 0x02, 0x03,
|
||||
]);
|
||||
const ciphertext = encryptImage(plaintext, key);
|
||||
const calls = [];
|
||||
const api = createWeixinApi({
|
||||
fetchImpl: async (url, init) => {
|
||||
calls.push({ url: url.toString(), init });
|
||||
return new Response(ciphertext, {
|
||||
headers: { 'content-length': String(ciphertext.length) },
|
||||
});
|
||||
},
|
||||
});
|
||||
const images = api.inboundImages({
|
||||
item_list: [{
|
||||
type: 2,
|
||||
image_item: {
|
||||
aeskey: key.toString('hex'),
|
||||
media: { encrypt_query_param: 'one=two&three=four' },
|
||||
},
|
||||
}],
|
||||
});
|
||||
|
||||
assert.equal(images.length, 1);
|
||||
assert.equal(calls.length, 0);
|
||||
const loaded = await images[0].load({ maxBytes: 1_024 });
|
||||
|
||||
assert.equal(loaded.equals(plaintext), true);
|
||||
assert.equal(
|
||||
calls[0].url,
|
||||
'https://novac2c.cdn.weixin.qq.com/c2c/download?encrypted_query_param=one%3Dtwo%26three%3Dfour',
|
||||
);
|
||||
assert.equal(calls[0].init.method, 'GET');
|
||||
assert.equal(calls[0].init.redirect, 'manual');
|
||||
});
|
||||
|
||||
test('Weixin image keys support both documented CDN encodings and reject unsafe URLs', () => {
|
||||
const key = randomBytes(16);
|
||||
assert.equal(parseWeixinImageAesKey({ aeskey: key.toString('hex') }).equals(key), true);
|
||||
assert.equal(parseWeixinImageAesKey({
|
||||
media: { aes_key: key.toString('base64') },
|
||||
}).equals(key), true);
|
||||
assert.equal(parseWeixinImageAesKey({
|
||||
media: { aes_key: Buffer.from(key.toString('hex'), 'ascii').toString('base64') },
|
||||
}).equals(key), true);
|
||||
assert.throws(
|
||||
() => parseWeixinImageAesKey({ aeskey: 'not-a-key' }),
|
||||
(error) => error instanceof WeixinApiError && error.code === 'invalid-image-key',
|
||||
);
|
||||
assert.equal(
|
||||
weixinImageDownloadUrl({ full_url: 'https://novac2c.cdn.weixin.qq.com/c2c/download?id=one#fragment' }),
|
||||
'https://novac2c.cdn.weixin.qq.com/c2c/download?id=one',
|
||||
);
|
||||
assert.throws(
|
||||
() => weixinImageDownloadUrl({ full_url: 'https://attacker.example/c2c/download?id=one' }),
|
||||
(error) => error instanceof WeixinApiError && error.code === 'untrusted-image-url',
|
||||
);
|
||||
|
||||
const encrypted = encryptImage(Buffer.from('image payload'), key);
|
||||
assert.equal(decryptWeixinImage(encrypted, key).toString(), 'image payload');
|
||||
});
|
||||
|
||||
test('QR login uses the Tencent iLink headers and keeps local tokens out of the URL', async () => {
|
||||
const calls = [];
|
||||
const api = createWeixinApi({
|
||||
|
|
|
|||
|
|
@ -63,6 +63,101 @@ function stateFixture() {
|
|||
};
|
||||
}
|
||||
|
||||
const PNG_BYTES = Buffer.from([
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x01, 0x02, 0x03,
|
||||
]);
|
||||
|
||||
test('Weixin sends image-only messages to Harness as structured content', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:owner-user', 'session-image');
|
||||
const prompts = [];
|
||||
const sent = [];
|
||||
const bridge = new WeixinHarnessBridge({
|
||||
api: {
|
||||
inboundImages: () => [{ name: 'image', data: PNG_BYTES }],
|
||||
sendText: async (request) => sent.push(request),
|
||||
},
|
||||
baseUrl: 'https://ilinkai.weixin.qq.com/',
|
||||
token: 'host-token',
|
||||
ownerUserId: 'owner-user',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async (sessionId, content) => {
|
||||
prompts.push({ sessionId, content });
|
||||
return '微信图片已识别';
|
||||
},
|
||||
},
|
||||
state: fixture.state,
|
||||
});
|
||||
|
||||
await bridge.accept(message('weixin-image', '', {
|
||||
item_list: [{ type: 2, image_item: { media: {} } }],
|
||||
}));
|
||||
|
||||
assert.equal(prompts.length, 1);
|
||||
assert.equal(prompts[0].sessionId, 'session-image');
|
||||
assert.deepEqual(prompts[0].content.map(({ type }) => type), ['text', 'image']);
|
||||
assert.equal(prompts[0].content[0].text, '请分析这张图片。');
|
||||
assert.equal(prompts[0].content[1].mediaType, 'image/png');
|
||||
assert.equal(Buffer.from(prompts[0].content[1].data, 'base64').equals(PNG_BYTES), true);
|
||||
assert.equal(sent.at(-1).text, '微信图片已识别');
|
||||
assert.equal(sent.at(-1).contextToken, 'context-weixin-image');
|
||||
assert.equal(fixture.seen.has('weixin-image'), true);
|
||||
});
|
||||
|
||||
test('Weixin authorizes the sender before resolving encrypted image references', async () => {
|
||||
let imageExtractions = 0;
|
||||
let asks = 0;
|
||||
const bridge = new WeixinHarnessBridge({
|
||||
api: {
|
||||
inboundImages: () => { imageExtractions += 1; return [{ data: PNG_BYTES }]; },
|
||||
sendText: async () => assert.fail('an unauthorized sender must not receive a reply'),
|
||||
},
|
||||
baseUrl: 'https://ilinkai.weixin.qq.com/',
|
||||
token: 'host-token',
|
||||
ownerUserId: 'owner-user',
|
||||
harness: { ask: async () => { asks += 1; return 'unexpected'; } },
|
||||
state: stateFixture().state,
|
||||
});
|
||||
|
||||
await bridge.accept(message('weixin-image-other', '', {
|
||||
from_user_id: 'other-user',
|
||||
item_list: [{ type: 2, image_item: { media: {} } }],
|
||||
}));
|
||||
|
||||
assert.equal(imageExtractions, 0);
|
||||
assert.equal(asks, 0);
|
||||
});
|
||||
|
||||
test('Weixin returns a specific retry message when encrypted image loading fails', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:owner-user', 'session-image');
|
||||
const sent = [];
|
||||
const bridge = new WeixinHarnessBridge({
|
||||
api: {
|
||||
inboundImages: () => [{ load: async () => { throw new Error('CDN unavailable'); } }],
|
||||
sendText: async (request) => sent.push(request),
|
||||
},
|
||||
baseUrl: 'https://ilinkai.weixin.qq.com/',
|
||||
token: 'host-token',
|
||||
ownerUserId: 'owner-user',
|
||||
harness: {
|
||||
sessionExists: async () => true,
|
||||
ask: async () => assert.fail('a failed image must not reach Harness'),
|
||||
},
|
||||
state: fixture.state,
|
||||
logger: { error() {} },
|
||||
});
|
||||
|
||||
await bridge.accept(message('weixin-image-error', '', {
|
||||
item_list: [{ type: 2, image_item: { media: {} } }],
|
||||
}));
|
||||
|
||||
assert.equal(sent.at(-1).text, '图片下载失败，请重新发送后再试。');
|
||||
assert.equal(fixture.seen.has('weixin-image-error'), true);
|
||||
});
|
||||
|
||||
test('Weixin executes /compact for the bound Session without prompting the model', async () => {
|
||||
const fixture = stateFixture();
|
||||
fixture.sessions.set('p2p:owner-user', 'session-compact');
|
||||
|
|
@ -536,7 +631,10 @@ test('Weixin serializes an invalid pending reply before the following valid answ
|
|||
await eventually(() => sent.some(({ text }) => text.includes('请给出有效文字答案')));
|
||||
const invalid = bridge.accept(message('invalid-image', '', {
|
||||
message_type: 3,
|
||||
item_list: [{ type: 2 }],
|
||||
item_list: [
|
||||
{ type: 1, text_item: { text: '伪装成答案的图片说明' } },
|
||||
{ type: 2, image_item: { media: {} } },
|
||||
],
|
||||
}));
|
||||
await invalidNoticeStarted.promise;
|
||||
const valid = bridge.accept(message('invalid-valid', '真正的答案'));
|
||||
|
|
|
|||
|
|
@ -207,6 +207,127 @@ test('WhatsApp normalizes direct and explicitly mentioned group messages', () =>
|
|||
}, ACCOUNT_JID), null);
|
||||
});
|
||||
|
||||
test('WhatsApp exposes image media through a bounded Baileys download stream', async () => {
|
||||
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const calls = [];
|
||||
const controller = new AbortController();
|
||||
const group = normalizeWhatsappMessage({
|
||||
key: {
|
||||
remoteJid: '120363000000000000@g.us',
|
||||
participant: '16505550999@s.whatsapp.net',
|
||||
id: 'group-image-1',
|
||||
fromMe: false,
|
||||
},
|
||||
message: {
|
||||
imageMessage: {
|
||||
mimetype: 'image/png',
|
||||
caption: '看看这张图',
|
||||
fileLength: { toString: () => String(png.length) },
|
||||
url: 'https://mmg.whatsapp.net/image',
|
||||
contextInfo: { mentionedJid: [ACCOUNT_JID] },
|
||||
},
|
||||
},
|
||||
}, ACCOUNT_JID, {
|
||||
download: async (raw, type, options) => {
|
||||
calls.push({ raw, type, options });
|
||||
return {
|
||||
async *[Symbol.asyncIterator]() { yield png; },
|
||||
};
|
||||
},
|
||||
});
|
||||
assert.equal(group.addressed, true);
|
||||
assert.equal(group.content, '看看这张图');
|
||||
assert.equal(group.images.length, 1);
|
||||
assert.equal(group.images[0].size, png.length);
|
||||
assert.deepEqual(await group.images[0].load({ signal: controller.signal, maxBytes: 100 }), png);
|
||||
assert.equal(calls[0].type, 'stream');
|
||||
assert.equal(calls[0].options.options.signal instanceof AbortSignal, true);
|
||||
|
||||
const downloadStarted = Promise.withResolvers();
|
||||
const lateStream = Promise.withResolvers();
|
||||
let lateStreamDestroyed = false;
|
||||
const cancelled = normalizeWhatsappMessage({
|
||||
key: { remoteJid: '16505550999@s.whatsapp.net', id: 'cancelled-1', fromMe: false },
|
||||
message: {
|
||||
imageMessage: { mimetype: 'image/png', url: 'https://mmg.whatsapp.net/cancelled' },
|
||||
},
|
||||
}, ACCOUNT_JID, {
|
||||
download: async () => {
|
||||
downloadStarted.resolve();
|
||||
return lateStream.promise;
|
||||
},
|
||||
});
|
||||
const cancelledController = new AbortController();
|
||||
const cancelledLoad = cancelled.images[0].load({
|
||||
signal: cancelledController.signal,
|
||||
maxBytes: 100,
|
||||
});
|
||||
await downloadStarted.promise;
|
||||
cancelledController.abort(new DOMException('Stopped', 'AbortError'));
|
||||
await assert.rejects(cancelledLoad, { name: 'AbortError' });
|
||||
lateStream.resolve({ destroy() { lateStreamDestroyed = true; } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
assert.equal(lateStreamDestroyed, true);
|
||||
|
||||
const document = normalizeWhatsappMessage({
|
||||
key: { remoteJid: '16505550999@s.whatsapp.net', id: 'document-image-1', fromMe: false },
|
||||
message: {
|
||||
documentMessage: {
|
||||
mimetype: 'image/webp', fileName: 'diagram.webp', fileLength: 2_000,
|
||||
url: 'https://mmg.whatsapp.net/document',
|
||||
},
|
||||
},
|
||||
}, ACCOUNT_JID);
|
||||
assert.equal(document.images[0].name, 'diagram.webp');
|
||||
assert.equal(document.images[0].mediaType, 'image/webp');
|
||||
|
||||
for (const [index, wrapper] of [
|
||||
'viewOnceMessage',
|
||||
'viewOnceMessageV2',
|
||||
'viewOnceMessageV2Extension',
|
||||
].entries()) {
|
||||
const wrappedMessage = {
|
||||
[wrapper]: {
|
||||
message: {
|
||||
imageMessage: {
|
||||
mimetype: 'image/jpeg', url: `https://mmg.whatsapp.net/view-once-${index}`,
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const viewOnce = normalizeWhatsappMessage({
|
||||
key: {
|
||||
remoteJid: '16505550999@s.whatsapp.net',
|
||||
id: `view-once-${index}`,
|
||||
fromMe: false,
|
||||
},
|
||||
message: index === 1
|
||||
? { ephemeralMessage: { message: wrappedMessage } }
|
||||
: wrappedMessage,
|
||||
}, ACCOUNT_JID);
|
||||
assert.deepEqual(viewOnce.images, []);
|
||||
}
|
||||
|
||||
const oversized = normalizeWhatsappMessage({
|
||||
key: { remoteJid: '16505550999@s.whatsapp.net', id: 'oversized-1', fromMe: false },
|
||||
message: {
|
||||
imageMessage: { mimetype: 'image/jpeg', url: 'https://mmg.whatsapp.net/oversized' },
|
||||
},
|
||||
}, ACCOUNT_JID, {
|
||||
download: async () => ({
|
||||
async *[Symbol.asyncIterator]() {
|
||||
yield Buffer.alloc(4);
|
||||
yield Buffer.alloc(4);
|
||||
},
|
||||
destroy() {},
|
||||
}),
|
||||
});
|
||||
await assert.rejects(() => oversized.images[0].load({ maxBytes: 5 }), (error) => {
|
||||
assert.equal(error.code, 'image-too-large');
|
||||
return true;
|
||||
});
|
||||
});
|
||||
|
||||
test('WhatsApp runtime connects a linked device and replies through Harness', async () => {
|
||||
let callbacks;
|
||||
const calls = [];
|
||||
|
|
|
|||
183
test/image-bridge.test.mjs
Normal file
183
test/image-bridge.test.mjs
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { TextHarnessBridge } from '../src/channels/shared/text-harness-bridge.mjs';
|
||||
|
||||
const PNG_1X1 = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
function memoryState() {
|
||||
const sessions = new Map();
|
||||
const seen = new Set();
|
||||
return {
|
||||
sessionFor: (key) => sessions.get(key) ?? null,
|
||||
setSession: async (key, value) => sessions.set(key, value),
|
||||
clearSession: async (key) => sessions.delete(key),
|
||||
hasSeen: (id) => seen.has(id),
|
||||
markSeen: async (id) => seen.add(id),
|
||||
};
|
||||
}
|
||||
|
||||
function bridgeFixture() {
|
||||
const sent = [];
|
||||
const prompts = [];
|
||||
const bridge = new TextHarnessBridge({
|
||||
descriptor: { key: 'test', label: 'Test' },
|
||||
bot: { sendText: async (_target, text) => sent.push(text) },
|
||||
harness: {
|
||||
ensureRunning: async () => true,
|
||||
sessionExists: async () => true,
|
||||
createSession: async () => 'session-image',
|
||||
ask: async (_sessionId, prompt) => {
|
||||
prompts.push(prompt);
|
||||
return '识别成功';
|
||||
},
|
||||
},
|
||||
state: memoryState(),
|
||||
});
|
||||
return { bridge, sent, prompts };
|
||||
}
|
||||
|
||||
test('the shared bridge sends image and caption content to Harness', async () => {
|
||||
const { bridge, sent, prompts } = bridgeFixture();
|
||||
let loads = 0;
|
||||
await bridge.accept({
|
||||
messageId: 'image-1',
|
||||
senderId: 'user-1',
|
||||
kind: 'direct',
|
||||
conversationId: 'user-1',
|
||||
content: '图中是什么？',
|
||||
images: [{ async load() { loads += 1; return PNG_1X1; } }],
|
||||
replyTarget: {},
|
||||
});
|
||||
|
||||
assert.equal(loads, 1);
|
||||
assert.deepEqual(prompts, [[
|
||||
{ type: 'text', text: '图中是什么？' },
|
||||
{ type: 'image', mediaType: 'image/png', data: PNG_1X1.toString('base64') },
|
||||
]]);
|
||||
assert.deepEqual(sent, ['识别成功']);
|
||||
});
|
||||
|
||||
test('unaddressed group images are rejected before their bytes are downloaded', async () => {
|
||||
const { bridge, sent, prompts } = bridgeFixture();
|
||||
let loads = 0;
|
||||
await bridge.accept({
|
||||
messageId: 'image-2',
|
||||
senderId: 'user-1',
|
||||
kind: 'group',
|
||||
conversationId: 'group-1',
|
||||
content: '',
|
||||
addressed: false,
|
||||
images: [{ async load() { loads += 1; return PNG_1X1; } }],
|
||||
replyTarget: {},
|
||||
});
|
||||
|
||||
assert.equal(loads, 0);
|
||||
assert.deepEqual(prompts, []);
|
||||
assert.deepEqual(sent, []);
|
||||
});
|
||||
|
||||
test('image validation failures receive a specific safe reply', async () => {
|
||||
const { bridge, sent, prompts } = bridgeFixture();
|
||||
await bridge.accept({
|
||||
messageId: 'image-3',
|
||||
senderId: 'user-1',
|
||||
kind: 'direct',
|
||||
conversationId: 'user-1',
|
||||
content: '',
|
||||
images: [{ data: Buffer.from('not an image') }],
|
||||
replyTarget: {},
|
||||
});
|
||||
|
||||
assert.deepEqual(prompts, []);
|
||||
assert.deepEqual(sent, ['暂不支持该图片格式，请发送 JPEG、PNG、WebP 或 GIF 图片。']);
|
||||
});
|
||||
|
||||
test('an image caption cannot answer a pending Harness question', async () => {
|
||||
const sent = [];
|
||||
const questionSent = Promise.withResolvers();
|
||||
const answered = Promise.withResolvers();
|
||||
let interactionResult;
|
||||
let imageLoads = 0;
|
||||
const bridge = new TextHarnessBridge({
|
||||
descriptor: { key: 'test', label: 'Test' },
|
||||
bot: {
|
||||
async sendText(_target, text) {
|
||||
sent.push(text);
|
||||
if (text.includes('请选择环境')) questionSent.resolve();
|
||||
},
|
||||
},
|
||||
harness: {
|
||||
ensureRunning: async () => true,
|
||||
sessionExists: async () => true,
|
||||
createSession: async () => 'session-question',
|
||||
async ask(_sessionId, prompt, options) {
|
||||
assert.equal(prompt, '开始测试');
|
||||
await options.onInteraction({
|
||||
kind: 'question',
|
||||
interactionId: 'question-1',
|
||||
rpcId: 'question-1',
|
||||
sessionId: 'session-question',
|
||||
payload: {
|
||||
questions: [{
|
||||
id: 'environment',
|
||||
question: '请选择环境',
|
||||
options: [{ label: '生产环境' }],
|
||||
}],
|
||||
},
|
||||
async respond(result) {
|
||||
interactionResult = result;
|
||||
answered.resolve();
|
||||
return { accepted: true };
|
||||
},
|
||||
});
|
||||
await answered.promise;
|
||||
return '已完成';
|
||||
},
|
||||
},
|
||||
state: memoryState(),
|
||||
});
|
||||
|
||||
const original = bridge.accept({
|
||||
messageId: 'question-start',
|
||||
senderId: 'user-1',
|
||||
kind: 'direct',
|
||||
conversationId: 'user-1',
|
||||
content: '开始测试',
|
||||
replyTarget: {},
|
||||
});
|
||||
await questionSent.promise;
|
||||
|
||||
await bridge.accept({
|
||||
messageId: 'question-image',
|
||||
senderId: 'user-1',
|
||||
kind: 'direct',
|
||||
conversationId: 'user-1',
|
||||
content: '生产环境',
|
||||
images: [{ async load() { imageLoads += 1; return PNG_1X1; } }],
|
||||
replyTarget: {},
|
||||
});
|
||||
assert.equal(imageLoads, 0);
|
||||
assert.equal(interactionResult, undefined);
|
||||
assert.equal(sent.at(-1), '请用文字回答当前问题。');
|
||||
|
||||
await bridge.accept({
|
||||
messageId: 'question-answer',
|
||||
senderId: 'user-1',
|
||||
kind: 'direct',
|
||||
conversationId: 'user-1',
|
||||
content: '生产环境',
|
||||
replyTarget: {},
|
||||
});
|
||||
await original;
|
||||
assert.deepEqual(interactionResult, {
|
||||
ok: true,
|
||||
value: {
|
||||
sessionId: 'session-question',
|
||||
answer: { answers: [{ id: 'environment', selected: ['生产环境'] }] },
|
||||
},
|
||||
});
|
||||
});
|
||||
233
test/image-prompt.test.mjs
Normal file
233
test/image-prompt.test.mjs
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import {
|
||||
DEFAULT_IMAGE_PROMPT,
|
||||
ImagePromptError,
|
||||
fetchImageBuffer,
|
||||
hasInboundPrompt,
|
||||
promptContentForMessage,
|
||||
} from '../src/channels/shared/image-prompt.mjs';
|
||||
import { HarnessClient } from '../src/channels/shared/harness-client.mjs';
|
||||
|
||||
const PNG_1X1 = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
test('text-only messages retain the existing Harness content shape', async () => {
|
||||
assert.equal(hasInboundPrompt({ content: ' hello ' }), true);
|
||||
assert.deepEqual(await promptContentForMessage({ content: ' hello ' }), [
|
||||
{ type: 'text', text: 'hello' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('image-only messages lazily load bytes and receive a useful default instruction', async () => {
|
||||
let loadOptions;
|
||||
const content = await promptContentForMessage({
|
||||
content: '',
|
||||
images: [{
|
||||
name: '../photo.png',
|
||||
async load(options) {
|
||||
loadOptions = options;
|
||||
return PNG_1X1;
|
||||
},
|
||||
}],
|
||||
});
|
||||
|
||||
assert.equal(loadOptions.maxBytes, 5 * 1024 * 1024);
|
||||
assert.deepEqual(content, [
|
||||
{ type: 'text', text: DEFAULT_IMAGE_PROMPT },
|
||||
{
|
||||
type: 'image',
|
||||
mediaType: 'image/png',
|
||||
data: PNG_1X1.toString('base64'),
|
||||
name: 'photo.png',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test('captions and multiple image parts preserve their input order', async () => {
|
||||
const jpeg = Buffer.from([0xff, 0xd8, 0xff, 0xd9]);
|
||||
const gif = Buffer.from('GIF89a payload');
|
||||
assert.deepEqual(await promptContentForMessage({
|
||||
content: 'compare these',
|
||||
images: [{ data: jpeg }, { data: gif }],
|
||||
}), [
|
||||
{ type: 'text', text: 'compare these' },
|
||||
{ type: 'image', mediaType: 'image/jpeg', data: jpeg.toString('base64') },
|
||||
{ type: 'image', mediaType: 'image/gif', data: gif.toString('base64') },
|
||||
]);
|
||||
});
|
||||
|
||||
test('declared oversized images are rejected without downloading them', async () => {
|
||||
let loaded = false;
|
||||
await assert.rejects(
|
||||
promptContentForMessage({
|
||||
images: [{
|
||||
size: 5 * 1024 * 1024 + 1,
|
||||
async load() { loaded = true; return PNG_1X1; },
|
||||
}],
|
||||
}),
|
||||
(error) => error instanceof ImagePromptError && error.code === 'image-too-large',
|
||||
);
|
||||
assert.equal(loaded, false);
|
||||
});
|
||||
|
||||
test('multiple images share an aggregate byte limit', async () => {
|
||||
let secondLoaded = false;
|
||||
await assert.rejects(
|
||||
promptContentForMessage({
|
||||
images: [
|
||||
{ size: PNG_1X1.length, async load() { return PNG_1X1; } },
|
||||
{
|
||||
size: PNG_1X1.length,
|
||||
async load() { secondLoaded = true; return PNG_1X1; },
|
||||
},
|
||||
],
|
||||
}, { maxTotalImageBytes: PNG_1X1.length + 1 }),
|
||||
(error) => error instanceof ImagePromptError && error.code === 'images-too-large',
|
||||
);
|
||||
assert.equal(secondLoaded, false);
|
||||
});
|
||||
|
||||
test('unsupported image bytes receive a channel-safe error', async () => {
|
||||
await assert.rejects(
|
||||
promptContentForMessage({ images: [{ data: Buffer.from('not an image') }] }),
|
||||
(error) => error instanceof ImagePromptError
|
||||
&& error.code === 'unsupported-image-type'
|
||||
&& /JPEG/.test(error.userMessage),
|
||||
);
|
||||
});
|
||||
|
||||
test('bounded HTTPS downloads enforce response size before buffering', async () => {
|
||||
let cancelled = false;
|
||||
const fetchImpl = async (url, options) => {
|
||||
assert.equal(url.href, 'https://files.example/image.png');
|
||||
assert.equal(options.redirect, 'manual');
|
||||
return {
|
||||
ok: true,
|
||||
status: 200,
|
||||
headers: { get: (name) => (name === 'content-length' ? '9' : null) },
|
||||
body: { async cancel() { cancelled = true; } },
|
||||
async arrayBuffer() { throw new Error('must not buffer'); },
|
||||
};
|
||||
};
|
||||
await assert.rejects(
|
||||
fetchImageBuffer('https://files.example/image.png', { fetchImpl, maxBytes: 8 }),
|
||||
(error) => error instanceof ImagePromptError && error.code === 'image-too-large',
|
||||
);
|
||||
assert.equal(cancelled, true);
|
||||
});
|
||||
|
||||
test('image downloads reject insecure platform URLs', async () => {
|
||||
await assert.rejects(
|
||||
fetchImageBuffer('http://files.example/image.png'),
|
||||
/must use HTTPS/,
|
||||
);
|
||||
});
|
||||
|
||||
test('image downloads enforce messaging-platform host allowlists', async () => {
|
||||
await assert.rejects(
|
||||
fetchImageBuffer('https://cdn.attacker.example/image.png', {
|
||||
allowedHosts: ['cdn.discordapp.com', '.slack.com'],
|
||||
}),
|
||||
/not hosted by the messaging platform/,
|
||||
);
|
||||
const data = await fetchImageBuffer('https://files.slack.com/image.png', {
|
||||
allowedHosts: ['.slack.com'],
|
||||
fetchImpl: async () => ({
|
||||
ok: true,
|
||||
headers: { get: () => null },
|
||||
async arrayBuffer() { return PNG_1X1; },
|
||||
}),
|
||||
});
|
||||
assert.deepEqual(data, PNG_1X1);
|
||||
});
|
||||
|
||||
test('image downloads report redirects without following or exposing the target', async () => {
|
||||
let options;
|
||||
let cancelled = false;
|
||||
await assert.rejects(
|
||||
fetchImageBuffer('https://files.example.test/image', {
|
||||
fetchImpl: async (_url, requestOptions) => {
|
||||
options = requestOptions;
|
||||
return {
|
||||
ok: false,
|
||||
status: 302,
|
||||
headers: { get: () => null },
|
||||
body: { async cancel() { cancelled = true; } },
|
||||
};
|
||||
},
|
||||
}),
|
||||
(error) => {
|
||||
assert.equal(error.code, 'image-redirect-blocked');
|
||||
assert.doesNotMatch(error.message, /private|token|hidden/);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
assert.equal(options.redirect, 'manual');
|
||||
assert.equal(cancelled, true);
|
||||
});
|
||||
|
||||
test('image downloads report the safe HTTP status for non-success responses', async () => {
|
||||
let cancelled = false;
|
||||
await assert.rejects(
|
||||
fetchImageBuffer('https://files.example.test/image', {
|
||||
fetchImpl: async () => ({
|
||||
ok: false,
|
||||
status: 403,
|
||||
headers: { get: () => null },
|
||||
body: { async cancel() { cancelled = true; } },
|
||||
}),
|
||||
}),
|
||||
(error) => error.code === 'image-http-error'
|
||||
&& error.userMessage === '图片下载失败（HTTP 403），请重新发送后再试。',
|
||||
);
|
||||
assert.equal(cancelled, true);
|
||||
});
|
||||
|
||||
test('HarnessClient sends structured image content without rewriting it', async () => {
|
||||
const client = new HarnessClient({
|
||||
baseUrl: 'http://127.0.0.1:3080',
|
||||
workspace: '/tmp/image-prompt-test',
|
||||
});
|
||||
client.ensureRunning = async () => true;
|
||||
let promptPayload;
|
||||
let promptRpcId;
|
||||
let historyCalls = 0;
|
||||
client.rpc = async (method, payload, _timeoutMs, options) => {
|
||||
if (method === 'session.prompt') {
|
||||
promptPayload = payload;
|
||||
promptRpcId = options.rpcId;
|
||||
return {};
|
||||
}
|
||||
assert.equal(method, 'session.history');
|
||||
historyCalls += 1;
|
||||
if (historyCalls === 1) return { events: [] };
|
||||
return {
|
||||
events: [
|
||||
{ event: { seq: 1, type: 'turn/start', data: { turn: 1 } } },
|
||||
{ event: {
|
||||
seq: 2,
|
||||
type: 'user/message',
|
||||
data: { turn: 1, source: { rpcId: promptRpcId } },
|
||||
} },
|
||||
{ event: {
|
||||
seq: 3,
|
||||
type: 'assistant/message',
|
||||
data: { turn: 1, message: { content: [{ type: 'text', text: 'a cat' }] } },
|
||||
} },
|
||||
{ event: { seq: 4, type: 'turn/end', data: { turn: 1, reason: 'completed' } } },
|
||||
],
|
||||
};
|
||||
};
|
||||
const content = [
|
||||
{ type: 'text', text: 'what is this?' },
|
||||
{ type: 'image', mediaType: 'image/png', data: PNG_1X1.toString('base64') },
|
||||
];
|
||||
|
||||
assert.equal(await client.ask('session-image', content, { timeoutMs: 2_000 }), 'a cat');
|
||||
assert.deepEqual(promptPayload.content, content);
|
||||
assert.equal(promptPayload.sessionId, 'session-image');
|
||||
});
|
||||
|
|
@ -188,6 +188,33 @@ test('the next message continues the bound Session without creating a new one',
|
|||
assert.equal(createCalls, 0);
|
||||
});
|
||||
|
||||
test('workspace sessions forward structured multimodal prompt content unchanged', async (t) => {
|
||||
const { path, defaultWorkspace } = await fixture(t);
|
||||
const workspaces = await new BotWorkspaceStore(path, { defaultWorkspace }).load();
|
||||
await workspaces.ensure('bot_multimodal');
|
||||
const state = memoryState({ conversation: 'session-image' });
|
||||
const prompts = [];
|
||||
const content = [
|
||||
{ type: 'text', text: '这是什么？' },
|
||||
{ type: 'image', mediaType: 'image/png', data: 'AAAA' },
|
||||
];
|
||||
const scope = createBotWorkspaceScope({
|
||||
async sessionExists() { return true; },
|
||||
async ask(sessionId, prompt) {
|
||||
prompts.push({ sessionId, prompt });
|
||||
return 'image answer';
|
||||
},
|
||||
}, { botId: 'bot_multimodal', workspaces, state });
|
||||
|
||||
assert.deepEqual(await askInWorkspaceSession({
|
||||
harness: scope.harness,
|
||||
state: scope.state,
|
||||
key: 'conversation',
|
||||
content,
|
||||
}), { sessionId: 'session-image', answer: 'image answer' });
|
||||
assert.deepEqual(prompts, [{ sessionId: 'session-image', prompt: content }]);
|
||||
});
|
||||
|
||||
test('adoption errors and invalid adoption responses leave local state unchanged', async (t) => {
|
||||
const { path, defaultWorkspace, alternateWorkspace } = await fixture(t);
|
||||
const workspaces = await new BotWorkspaceStore(path, { defaultWorkspace }).load();
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue