feat: multimodal image clients, workspaces, Tushare skill, test fixes

- Replace monolithic image_message_client with HTTP/OCR/legacy modules; tighten OpenAI transport + tool schemas for multimodal downgrade to OCR specialist path.
- Add image/stock workspace prompts (META/SOUL/ROLE_SYSTEM); register experts; tweak specialist agent/direct loop/query_image_attachment.
- Add bundled runtime/skills/tushare-finance (references, api_client, SKILL metadata).
- Document OCR-related env vars; admin chat tweaks; README; weixin_install Ensure-OfficialPluginRuntimeDeps helper.
- Tests: multimodal downgrade + OCR coverage, strict tool pairing in attachment replay guard, workspace contract skips _internal/_system dirs, router/trace/prompt guards.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
oliver 2026-05-10 04:41:42 +08:00
parent 6307ade480
commit d1bcc4debe
266 changed files with 57024 additions and 839 deletions

View file

@ -308,47 +308,33 @@
- 作用:Gemini OpenAI 兼容下 tools 非流式开关
- 生效:`oclaw/platform/llm/chat_models.py`
## 图像能力
## OCR / 看多模态专线
- `AIA_IMAGE_MODEL`
- 默认:空(走 profile/模型默认)
- 作用:图像模型
- 生效:`oclaw/platform/llm/image_message_client.py`, `oclaw/runtime/agents/specialist_agent.py`
用于工具 **`query_image_attachment`**、纯文本模型降级 OCR、**图片专家**单次看图与底层 **`send_ocr_image_messages`**:向 **OpenAI-compatible** 网关发送带图 `messages`(通常为 `POST …/chat/completions`)。旧版 DashScope 形态载荷见 **`send_legacy_image_messages`**(一般不再用于图片专家)。与主聊天模型配置的 Key/Base URL **互不继承**。
- `AIA_IMAGE_BASE_URL`
- 默认:`https://api.openai.com/v1`
- 作用:图像服务 base URL
- 生效:`oclaw/platform/llm/image_message_client.py`
- `AIA_OCR_BASE_URL`
- 默认:无(必填,否则工具判为未配置)
- 作用:多模态网关根地址(例如 `https://api.example.com/v1`;实际请求为 `BASE_URL` + `CHAT_ENDPOINT`)
- 生效:`oclaw/platform/llm/image_ocr_client.py`
- `AIA_IMAGE_API_KEY`
- 默认:空
- 作用:图像 API Key
- 生效:`oclaw/platform/llm/image_message_client.py`
- `AIA_OCR_API_KEY`
- 默认:无(必填)
- 作用:上述网关的 Bearer API Key
- 生效:`oclaw/platform/llm/image_ocr_client.py`
- `AIA_IMAGE_CHAT_ENDPOINT`
- `AIA_OCR_MODEL`
- 默认:无(必填,**不提供默认模型 id**)
- 作用:网关上支持图+文的多模态模型名(由你的厂商决定)
- 生效:`oclaw/platform/llm/image_ocr_client.py`,`oclaw/runtime/agents/specialist_agent.py`
- `AIA_OCR_CHAT_ENDPOINT`
- 默认:`/chat/completions`
- 作用:图像接口 endpoint
- 生效:`oclaw/platform/llm/image_message_client.py`
- 作用:相对 `AIA_OCR_BASE_URL` 的路径
- 生效:`oclaw/platform/llm/image_ocr_client.py`
- `AIA_IMAGE_RETRIES`
- 默认:`3`
- 作用:图像请求重试次数
- 生效:`oclaw/platform/llm/image_message_client.py`
**动态参数:** 代码中也可在调用 `send_ocr_image_messages` / `send_legacy_image_messages` 时传入 `base_url` / `api_key` / `model`,优先级高于环境变量(见 `vision_llm_backend_status` 仅在未传参时使用 env 判断「是否已配置」)。
- `AIA_IMAGE_RETRY_BACKOFF_SEC`
- 默认:`0.8`
- 作用:图像请求重试退避秒数
- 生效:`oclaw/platform/llm/image_message_client.py`
- `AIA_IMAGE_STATUS_RETRIES`
- 默认:`4`
- 作用:图像状态轮询重试次数
- 生效:`oclaw/platform/llm/image_message_client.py`
- `AIA_IMAGE_STATUS_RETRY_BACKOFF_SEC`
- 默认:`5.0`
- 作用:图像状态轮询退避秒数
- 生效:`oclaw/platform/llm/image_message_client.py`
> **说明:** 历史上曾使用 `AIA_IMAGE_BASE_URL` / `AIA_IMAGE_API_KEY` / `AIA_IMAGE_MODEL` / `AIA_IMAGE_CHAT_ENDPOINT` 作为同一路由;当前实现 **不再读取** 上述变量作 OCR 通道配置,请统一改为 `AIA_OCR_*`。(`AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS` 等为 **另一用途**,与 OCR 网关无关,仍保留原名。)
## Memory / RAG

View file

@ -57,6 +57,23 @@
---
## 2026-05-10 / Unreleased
### Added
- `AIA_OCR_BASE_URL`, `AIA_OCR_API_KEY`, `AIA_OCR_MODEL`, `AIA_OCR_CHAT_ENDPOINT`
- 默认值:`CHAT_ENDPOINT` 为 `/chat/completions`,其余无默认(三项主体必填方视为已配置)
- 用途:`query_image_attachment` / `send_ocr_image_messages` 等多模态 HTTP 通道命名(与主 LLM 分离)
- 影响模块:`oclaw/platform/llm/image_ocr_client.py`、`image_legacy_client.py`,`oclaw/runtime/tools/public/query_image_attachment_tool.py`,`oclaw/runtime/agents/specialist_agent.py`(图片专家走 OCR 多模态 HTTP)
### Changed
- `send_ocr_image_messages` / `send_legacy_image_messages` 不再有隐式默认模型 id;未配 `AIA_OCR_MODEL`(且未传 `model`)将直接失败
### Removed(OCR 通道)
- `AIA_IMAGE_BASE_URL` / `AIA_IMAGE_API_KEY` / `AIA_IMAGE_MODEL` / `AIA_IMAGE_CHAT_ENDPOINT` **不再**作为看图/OCR 通道的环境变量读取(须改用 `AIA_OCR_*`)。与附件回放相关的 `AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS` 等 **不受影响**。
- 先前文档中的 `AIA_IMAGE_RETRIES` 等重试变量 **从未**由当前 OCR/legacy 客户端使用,示例 env 中已去除占位。
---
## 2026-04-26 / Unreleased
### Added