hansjone/oclaw
1
0
Fork 0
mirror of https://github.com/hansjone/oclaw.git synced 2026-10-09 00:40:45 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions

实现附件处理链路的统一引用化与可检索增强，避免大文件/多模态内容直接撑爆上下文并提升工具可用性。

Browse source 
本次补齐 text/image/video/archive 的标准化处理、会话级工具守卫、回放压缩、配置与文档对齐，并修复表格与流式输出相关体验问题。

Made-with: Cursor
This commit is contained in:
oliver 2026-04-26 15:25:50 +08:00
parent 9d2900db02
commit 37a2ef35f4
27 changed files with 3227 additions and 278 deletions
Download patch file Download diff file Expand all files Collapse all files

26
README.md
View file

@ -16,3 +16,29 @@ This repository is fully consolidated under `oclaw/`.
- Use `oclaw` consistently in paths, symbols, and docs.
- Avoid introducing legacy aliases or old naming variants.
## Attachment Replay Config
- Attachment-related limits are configured in `oclaw.json` under:
- `plugins.entries.memory-wiki.auto.attachments.tabular`
- Replay limits:
- `image_result_replay_cap_chars` (default `4000`, range `600..30000`)
- `video_result_replay_cap_chars` (default `4000`, range `600..30000`)
- Used to cap historical `query_image_attachment` / `query_video_attachment(task=transcript)` text replay in model context.
- Video transcript chunk defaults:
- `video_transcript_chunk_size` (default `1600`)
- `video_transcript_chunk_overlap` (default `200`)
- Unified archive budget defaults (zip/tar/tgz/gz):
- `archive_max_depth` (default `2`)
- `archive_max_file_count` (default `200`)
- `archive_max_entry_bytes` (default `10485760`)
- `archive_max_total_uncompressed_bytes` (default `52428800`)
- Archive parse errors now expose stable `error_code` values (for UI mapping and retries).
- Effective priority for replay-cap values:
- DB setting `AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS`
- DB setting `AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS`
- Environment variable `AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS`
- Environment variable `AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS`
- `oclaw.json` value
- Built-in default
See `docs/ENVIRONMENT_VARIABLES.md` for full runtime variable reference.

9
data/mcp_local.env.example
View file

@ -31,6 +31,15 @@
#
# Oclaw runtime 默认不回退 legacy manager/runner；若紧急排障需临时回退，可显式打开：
# AIA_OCLAW_ALLOW_LEGACY_FALLBACK=1
# 历史轮次里 query_image_attachment（OCR/描述）结果回放上限（字符数）：
# AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS=4000
# 历史轮次里 query_video_attachment(task=transcript) 结果回放上限（字符数）：
# AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS=4000
# 压缩包统一预算（zip/tar/tgz/gz）走 oclaw.json / Admin「附件」配置：
# archive_max_depth=2
# archive_max_file_count=200
# archive_max_entry_bytes=10485760
# archive_max_total_uncompressed_bytes=52428800
#
# Oclaw 仍生效的工具循环预算（可在 Admin -> Tool Policy 配置）：
# AIA_TURN_MAX_TOOL_WORKERS=8

28
docs/ENVIRONMENT_VARIABLES.md
View file

@ -181,6 +181,34 @@
- 作用：`tool_log` 中 args/result 截断上限
- 生效：`oclaw/platform/persistence/sqlite_store.py`
- `AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS`
- 默认：`4000`
- 作用：限制历史轮次中 `query_image_attachment`（OCR/描述）结果回放到模型上下文时的 `text` 长度上限
- 范围：`600..30000`
- 优先级：DB setting（同名） > 环境变量 > `oclaw.json`(`plugins.entries.memory-wiki.auto.attachments.tabular.image_result_replay_cap_chars`) > 默认值
- 生效：`oclaw/runtime/direct_loop.py`
- `AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS`
- 默认：`4000`
- 作用：限制历史轮次中 `query_video_attachment`（`task=transcript`）结果回放到模型上下文时的 `text` 长度上限
- 范围：`600..30000`
- 优先级：DB setting（同名） > 环境变量 > `oclaw.json`(`plugins.entries.memory-wiki.auto.attachments.tabular.video_result_replay_cap_chars`) > 默认值
- 生效：`oclaw/runtime/direct_loop.py`
- `video_transcript_chunk_size` / `video_transcript_chunk_overlap`（`oclaw.json` 配置项）
- 默认：`1600` / `200`
- 作用：`query_video_attachment(task=transcript)` 落库转写文本时的默认分块参数（可被工具入参覆盖）
- 范围：`size: 1..8000`，`overlap: 1..4000`（实际使用会约束 overlap < size）
- 路径：`plugins.entries.memory-wiki.auto.attachments.tabular`
- 生效：`oclaw/runtime/tools/experts/generalist/video_query.py`
- `archive_max_depth` / `archive_max_file_count` / `archive_max_entry_bytes` / `archive_max_total_uncompressed_bytes`（`oclaw.json` 配置项）
- 默认：`2` / `200` / `10485760` / `52428800`
- 作用：统一 `archive_processor`（zip/tar/tgz/gz）安全预算：限制嵌套深度、文件数量、单文件解压大小、总解压大小
- 路径：`plugins.entries.memory-wiki.auto.attachments.tabular`
- 生效：`oclaw/platform/files/archive_processor.py`, `oclaw/platform/files/file_attachments.py`
- 错误码（工具/上下文可见）：`archive_unsupported_format`, `archive_path_traversal`, `archive_max_depth_exceeded`, `archive_max_file_count_exceeded`, `archive_max_entry_bytes_exceeded`, `archive_max_total_uncompressed_bytes_exceeded`, `archive_link_entry_forbidden`, `archive_special_entry_forbidden`, `archive_parse_failed`
## MCP 与工具线侧
- `AIA_MCP_SPECIALISTS`

42
docs/ENVIRONMENT_VARIABLES_CHANGELOG.md
View file

@ -57,6 +57,48 @@
---
## 2026-04-26 / Unreleased
### Added
- `AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS`
- 默认值：`4000`
- 用途：限制历史轮次中 `query_image_attachment`（OCR/描述）结果回放到模型上下文时的字符上限
- 影响模块：`oclaw/runtime/direct_loop.py`, `oclaw/interfaces/admin/chat_api.py`, `oclaw/interfaces/admin/static/app.js`
- `AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS`
- 默认值：`4000`
- 用途：限制历史轮次中 `query_video_attachment`（`task=transcript`）结果回放到模型上下文时的字符上限
- 影响模块：`oclaw/runtime/direct_loop.py`, `oclaw/interfaces/admin/chat_api.py`, `oclaw/interfaces/admin/static/app.js`
### Changed
- Admin「附件」设置页新增 `image_result_replay_cap_chars` 可视化配置，并写入 `oclaw.json`：
- 路径：`plugins.entries.memory-wiki.auto.attachments.tabular.image_result_replay_cap_chars`
- 范围：`600..30000`
- 是否需要重启：否（新 turn 读取时生效）
- Admin「附件」设置页新增视频相关配置，并写入 `oclaw.json`：
- `video_result_replay_cap_chars`（范围 `600..30000`）
- `video_transcript_chunk_size`（范围 `1..8000`）
- `video_transcript_chunk_overlap`（范围 `1..4000`）
- 路径：`plugins.entries.memory-wiki.auto.attachments.tabular`
- 是否需要重启：否（新 turn 读取时生效）
- Admin「附件」设置页新增压缩包统一预算配置，并写入 `oclaw.json`：
- `archive_max_depth`（默认 `2`）
- `archive_max_file_count`（默认 `200`）
- `archive_max_entry_bytes`（默认 `10485760`）
- `archive_max_total_uncompressed_bytes`（默认 `52428800`）
- 统一错误码：`archive_unsupported_format`, `archive_path_traversal`, `archive_max_depth_exceeded`, `archive_max_file_count_exceeded`, `archive_max_entry_bytes_exceeded`, `archive_max_total_uncompressed_bytes_exceeded`, `archive_link_entry_forbidden`, `archive_special_entry_forbidden`, `archive_parse_failed`
- 路径：`plugins.entries.memory-wiki.auto.attachments.tabular`
- 影响模块：`oclaw/platform/files/archive_processor.py`, `oclaw/platform/files/file_attachments.py`
- 是否需要重启：否（新 turn 读取时生效）
### Migration Checklist
- [x] 已更新 `oclaw/docs/ENVIRONMENT_VARIABLES.md`
- [x] 已更新 `README.md`
- [x] 已更新示例 env 文件
- [x] 已验证 Admin 配置页（如适用）
- [x] 已执行编译/测试回归
---
## 2026-04-21 / Unreleased
### Added

317
interfaces/admin/chat_api.py
View file

@ -40,7 +40,7 @@ def _oclaw_config_path() -> Path:
p = Path(raw)
return p if p.is_absolute() else p.resolve()
# repo_root/src/admin/chat_api.py -> repo_root
return Path(__file__).resolve().parents[2] / "oclaw" / "oclaw.json"
return Path(__file__).resolve().parents[2] / "oclaw.json"
def _wiki_root_from_config() -> Path | None:
@ -103,6 +103,15 @@ _DISPATCH_REASON_LABELS_SETTING_KEY = "AIA_DISPATCH_REASON_LABELS_JSON"
_SPECIALIST_FLAGS_SETTING_KEY = "AIA_CHAT_SPECIALIST_FLAGS_JSON"
_CHAT_SPECIALIST_IDS: tuple[str, ...] = tuple(str(x) for x in SPECIALIST_IDS if str(x).strip())
DEFAULT_TABULAR_SQL_TIMEOUT_MS = 8_000
DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS = 5_000
DEFAULT_TEXT_INLINE_MAX_CHARS = 12_000
DEFAULT_TEXT_CHUNK_SIZE = 1_600
DEFAULT_TEXT_CHUNK_OVERLAP = 200
DEFAULT_TEXT_QUERY_TOP_K = 5
DEFAULT_IMAGE_RESULT_REPLAY_CAP_CHARS = 4_000
DEFAULT_VIDEO_RESULT_REPLAY_CAP_CHARS = 4_000
DEFAULT_VIDEO_TRANSCRIPT_CHUNK_SIZE = 1_600
DEFAULT_VIDEO_TRANSCRIPT_CHUNK_OVERLAP = 200
def _safe_rel_avatar_name(name: str) -> str:
@ -441,9 +450,31 @@ def _tabular_limits_from_oclaw_config() -> dict[str, int]:
"max_excel_sheets": _safe_int(tabular.get("max_excel_sheets"), DEFAULT_MAX_EXCEL_SHEETS, max_value=500),
"large_table_preview_rows": _safe_int(tabular.get("large_table_preview_rows"), 20, max_value=500),
"tool_mode_enabled": bool(tabular.get("tool_mode_enabled", True)),
"tool_mode_min_rows": _safe_int(tabular.get("tool_mode_min_rows"), 20_000),
"tool_mode_min_rows": _safe_int(tabular.get("tool_mode_min_rows"), DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS),
"tool_mode_max_bytes": _safe_int(tabular.get("tool_mode_max_bytes"), 30 * 1024 * 1024),
"sql_timeout_ms": _safe_timeout_ms(tabular.get("sql_timeout_ms"), DEFAULT_TABULAR_SQL_TIMEOUT_MS),
"text_inline_max_chars": _safe_int(tabular.get("text_inline_max_chars"), DEFAULT_TEXT_INLINE_MAX_CHARS, max_value=200_000),
"text_chunk_size": _safe_int(tabular.get("text_chunk_size"), DEFAULT_TEXT_CHUNK_SIZE, max_value=8_000),
"text_chunk_overlap": _safe_int(tabular.get("text_chunk_overlap"), DEFAULT_TEXT_CHUNK_OVERLAP, max_value=4_000),
"text_query_top_k": _safe_int(tabular.get("text_query_top_k"), DEFAULT_TEXT_QUERY_TOP_K, max_value=50),
"image_result_replay_cap_chars": _safe_int(
tabular.get("image_result_replay_cap_chars"), DEFAULT_IMAGE_RESULT_REPLAY_CAP_CHARS, min_value=600, max_value=30_000
),
"video_result_replay_cap_chars": _safe_int(
tabular.get("video_result_replay_cap_chars"), DEFAULT_VIDEO_RESULT_REPLAY_CAP_CHARS, min_value=600, max_value=30_000
),
"video_transcript_chunk_size": _safe_int(
tabular.get("video_transcript_chunk_size"), DEFAULT_VIDEO_TRANSCRIPT_CHUNK_SIZE, max_value=8_000
),
"video_transcript_chunk_overlap": _safe_int(
tabular.get("video_transcript_chunk_overlap"), DEFAULT_VIDEO_TRANSCRIPT_CHUNK_OVERLAP, max_value=4_000
),
"archive_max_depth": _safe_int(tabular.get("archive_max_depth"), 2, max_value=10),
"archive_max_file_count": _safe_int(tabular.get("archive_max_file_count"), 200, max_value=20_000),
"archive_max_entry_bytes": _safe_int(tabular.get("archive_max_entry_bytes"), 10 * 1024 * 1024, max_value=2_000_000_000),
"archive_max_total_uncompressed_bytes": _safe_int(
tabular.get("archive_max_total_uncompressed_bytes"), 50 * 1024 * 1024, max_value=5_000_000_000
),
}
@ -470,11 +501,69 @@ def _set_tabular_limits_into_oclaw_config(limits: dict[str, int]) -> dict[str, i
"max_excel_sheets": int(limits.get("max_excel_sheets") or DEFAULT_MAX_EXCEL_SHEETS),
"large_table_preview_rows": int(limits.get("large_table_preview_rows") or 20),
"tool_mode_enabled": bool(limits.get("tool_mode_enabled", True)),
"tool_mode_min_rows": int(limits.get("tool_mode_min_rows") or 20_000),
"tool_mode_min_rows": int(limits.get("tool_mode_min_rows") or DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS),
"tool_mode_max_bytes": int(limits.get("tool_mode_max_bytes") or (30 * 1024 * 1024)),
"sql_timeout_ms": int(
_safe_timeout_ms(limits.get("sql_timeout_ms"), DEFAULT_TABULAR_SQL_TIMEOUT_MS)
),
"text_inline_max_chars": int(
_safe_int(limits.get("text_inline_max_chars"), DEFAULT_TEXT_INLINE_MAX_CHARS, max_value=200_000)
),
"text_chunk_size": int(
_safe_int(limits.get("text_chunk_size"), DEFAULT_TEXT_CHUNK_SIZE, max_value=8_000)
),
"text_chunk_overlap": int(
_safe_int(limits.get("text_chunk_overlap"), DEFAULT_TEXT_CHUNK_OVERLAP, max_value=4_000)
),
"text_query_top_k": int(
_safe_int(limits.get("text_query_top_k"), DEFAULT_TEXT_QUERY_TOP_K, max_value=50)
),
"image_result_replay_cap_chars": int(
_safe_int(
limits.get("image_result_replay_cap_chars"),
DEFAULT_IMAGE_RESULT_REPLAY_CAP_CHARS,
min_value=600,
max_value=30_000,
)
),
"video_result_replay_cap_chars": int(
_safe_int(
limits.get("video_result_replay_cap_chars"),
DEFAULT_VIDEO_RESULT_REPLAY_CAP_CHARS,
min_value=600,
max_value=30_000,
)
),
"video_transcript_chunk_size": int(
_safe_int(
limits.get("video_transcript_chunk_size"),
DEFAULT_VIDEO_TRANSCRIPT_CHUNK_SIZE,
max_value=8_000,
)
),
"video_transcript_chunk_overlap": int(
_safe_int(
limits.get("video_transcript_chunk_overlap"),
DEFAULT_VIDEO_TRANSCRIPT_CHUNK_OVERLAP,
max_value=4_000,
)
),
"archive_max_depth": int(_safe_int(limits.get("archive_max_depth"), 2, max_value=10)),
"archive_max_file_count": int(
_safe_int(limits.get("archive_max_file_count"), 200, max_value=20_000)
),
"archive_max_entry_bytes": int(
_safe_int(
limits.get("archive_max_entry_bytes"), 10 * 1024 * 1024, max_value=2_000_000_000
)
),
"archive_max_total_uncompressed_bytes": int(
_safe_int(
limits.get("archive_max_total_uncompressed_bytes"),
50 * 1024 * 1024,
max_value=5_000_000_000,
)
),
}
}
}
@ -493,6 +582,72 @@ def _chat_session_mode_setting_key(*, tenant_id: str, user_id: str, session_id:
return f"chat.session.mode.{tenant_id}.{user_id}.{session_id}.{field}"
def _chat_user_mode_setting_key(*, tenant_id: str, user_id: str, field: str) -> str:
return f"chat.user.mode.{tenant_id}.{user_id}.{field}"
def _resolve_mode_settings(
*,
store: SqliteStore,
tenant_id: str,
user_id: str,
session_id: str,
) -> tuple[str, str, str]:
"""Resolve chat mode with global-user preference first, then session fallback."""
user_mode_key = _chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="interaction_mode")
user_specialist_key = _chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="specialist")
user_memory_mode_key = _chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="memory_mode")
session_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"
)
session_specialist_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"
)
session_memory_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"
)
mode_raw = str(store.get_setting(user_mode_key) or "").strip() or str(store.get_setting(session_mode_key) or "").strip()
specialist_raw = str(store.get_setting(user_specialist_key) or "").strip() or str(
store.get_setting(session_specialist_key) or ""
).strip()
memory_raw = str(store.get_setting(user_memory_mode_key) or "").strip() or str(store.get_setting(session_memory_mode_key) or "").strip()
interaction_mode = normalize_interaction_mode(mode_raw or "expert")
specialist = normalize_requested_specialist(specialist_raw or "generalist")
specialist = _apply_specialist_flags(store, specialist)
memory_mode = _normalize_memory_mode({"memory_mode": (memory_raw or "default")})
return interaction_mode, specialist, memory_mode
def _persist_mode_settings(
*,
store: SqliteStore,
tenant_id: str,
user_id: str,
session_id: str,
interaction_mode: str,
specialist: str,
memory_mode: str,
) -> None:
"""Persist as global user default and current-session compatibility snapshot."""
for key in (
_chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="interaction_mode"),
_chat_session_mode_setting_key(tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"),
):
store.set_setting(key, interaction_mode)
for key in (
_chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="specialist"),
_chat_session_mode_setting_key(tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"),
):
store.set_setting(key, specialist)
for key in (
_chat_user_mode_setting_key(tenant_id=tenant_id, user_id=user_id, field="memory_mode"),
_chat_session_mode_setting_key(tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"),
):
store.set_setting(key, memory_mode)
def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStore, str | None], dict[str, Any]]) -> None:
chat = APIRouter(prefix="/admin/api/chat", tags=["chat"])
@ -538,6 +693,21 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
user_id = str(ctx.get("user_id") or "")
title = str(payload.get("title") or "").strip() or ("新会话" if _api_lang(store) == "zh" else "New Chat")
s = store.create_session_for_user(title=title, tenant_id=tenant_id, user_id=user_id)
interaction_mode, specialist, memory_mode = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(s.id),
)
_persist_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(s.id),
interaction_mode=interaction_mode,
specialist=specialist,
memory_mode=memory_mode,
)
return {
"ok": True,
"session": {
@ -603,6 +773,21 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
title=("新会话" if lang == "zh" else "New Chat"), tenant_id=tenant_id, user_id=user_id
)
next_id = str(ns.id)
interaction_mode, specialist, memory_mode = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=next_id,
)
_persist_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=next_id,
interaction_mode=interaction_mode,
specialist=specialist,
memory_mode=memory_mode,
)
return {"ok": True, "next_session_id": next_id}
@chat.post("/sessions/{session_id}/fork")
@ -793,19 +978,12 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
sess = _resolve_chat_session(store, ctx, session_id)
if not sess:
raise HTTPException(status_code=404, detail="session_not_found")
mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"
interaction_mode, specialist, memory_mode = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
specialist_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"
)
memory_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"
)
interaction_mode = normalize_interaction_mode(store.get_setting(mode_key))
specialist = normalize_requested_specialist(store.get_setting(specialist_key))
specialist = _apply_specialist_flags(store, specialist)
memory_mode = _normalize_memory_mode({"memory_mode": store.get_setting(memory_mode_key)})
return {"ok": True, "interaction_mode": interaction_mode, "specialist": specialist, "memory_mode": memory_mode}
@chat.post("/sessions/{session_id}/mode")
@ -825,19 +1003,16 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
interaction_mode = normalize_interaction_mode(payload.get("interaction_mode"))
specialist = normalize_requested_specialist(payload.get("specialist"))
specialist = _apply_specialist_flags(store, specialist)
mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"
)
specialist_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"
)
memory_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"
)
memory_mode = _normalize_memory_mode(payload)
store.set_setting(mode_key, interaction_mode)
store.set_setting(specialist_key, specialist)
store.set_setting(memory_mode_key, memory_mode)
_persist_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
interaction_mode=interaction_mode,
specialist=specialist,
memory_mode=memory_mode,
)
return {"ok": True, "interaction_mode": interaction_mode, "specialist": specialist, "memory_mode": memory_mode}
@chat.get("/admin/user-stats")
@ -1066,9 +1241,21 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
"max_excel_sheets": DEFAULT_MAX_EXCEL_SHEETS,
"large_table_preview_rows": 20,
"tool_mode_enabled": True,
"tool_mode_min_rows": 20_000,
"tool_mode_min_rows": DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS,
"tool_mode_max_bytes": 30 * 1024 * 1024,
"sql_timeout_ms": DEFAULT_TABULAR_SQL_TIMEOUT_MS,
"text_inline_max_chars": DEFAULT_TEXT_INLINE_MAX_CHARS,
"text_chunk_size": DEFAULT_TEXT_CHUNK_SIZE,
"text_chunk_overlap": DEFAULT_TEXT_CHUNK_OVERLAP,
"text_query_top_k": DEFAULT_TEXT_QUERY_TOP_K,
"image_result_replay_cap_chars": DEFAULT_IMAGE_RESULT_REPLAY_CAP_CHARS,
"video_result_replay_cap_chars": DEFAULT_VIDEO_RESULT_REPLAY_CAP_CHARS,
"video_transcript_chunk_size": DEFAULT_VIDEO_TRANSCRIPT_CHUNK_SIZE,
"video_transcript_chunk_overlap": DEFAULT_VIDEO_TRANSCRIPT_CHUNK_OVERLAP,
"archive_max_depth": 2,
"archive_max_file_count": 200,
"archive_max_entry_bytes": 10 * 1024 * 1024,
"archive_max_total_uncompressed_bytes": 50 * 1024 * 1024,
}
saved = _set_tabular_limits_into_oclaw_config(limits)
return {"ok": True, "cleared": True, "limits": saved}
@ -1081,9 +1268,33 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
"max_excel_sheets": _safe_int(raw.get("max_excel_sheets"), DEFAULT_MAX_EXCEL_SHEETS, max_value=500),
"large_table_preview_rows": _safe_int(raw.get("large_table_preview_rows"), 20, max_value=500),
"tool_mode_enabled": bool(raw.get("tool_mode_enabled", True)),
"tool_mode_min_rows": _safe_int(raw.get("tool_mode_min_rows"), 20_000),
"tool_mode_min_rows": _safe_int(raw.get("tool_mode_min_rows"), DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS),
"tool_mode_max_bytes": _safe_int(raw.get("tool_mode_max_bytes"), 30 * 1024 * 1024),
"sql_timeout_ms": _safe_timeout_ms(raw.get("sql_timeout_ms"), DEFAULT_TABULAR_SQL_TIMEOUT_MS),
"text_inline_max_chars": _safe_int(raw.get("text_inline_max_chars"), DEFAULT_TEXT_INLINE_MAX_CHARS, max_value=200_000),
"text_chunk_size": _safe_int(raw.get("text_chunk_size"), DEFAULT_TEXT_CHUNK_SIZE, max_value=8_000),
"text_chunk_overlap": _safe_int(raw.get("text_chunk_overlap"), DEFAULT_TEXT_CHUNK_OVERLAP, max_value=4_000),
"text_query_top_k": _safe_int(raw.get("text_query_top_k"), DEFAULT_TEXT_QUERY_TOP_K, max_value=50),
"image_result_replay_cap_chars": _safe_int(
raw.get("image_result_replay_cap_chars"), DEFAULT_IMAGE_RESULT_REPLAY_CAP_CHARS, min_value=600, max_value=30_000
),
"video_result_replay_cap_chars": _safe_int(
raw.get("video_result_replay_cap_chars"), DEFAULT_VIDEO_RESULT_REPLAY_CAP_CHARS, min_value=600, max_value=30_000
),
"video_transcript_chunk_size": _safe_int(
raw.get("video_transcript_chunk_size"), DEFAULT_VIDEO_TRANSCRIPT_CHUNK_SIZE, max_value=8_000
),
"video_transcript_chunk_overlap": _safe_int(
raw.get("video_transcript_chunk_overlap"), DEFAULT_VIDEO_TRANSCRIPT_CHUNK_OVERLAP, max_value=4_000
),
"archive_max_depth": _safe_int(raw.get("archive_max_depth"), 2, max_value=10),
"archive_max_file_count": _safe_int(raw.get("archive_max_file_count"), 200, max_value=20_000),
"archive_max_entry_bytes": _safe_int(
raw.get("archive_max_entry_bytes"), 10 * 1024 * 1024, max_value=2_000_000_000
),
"archive_max_total_uncompressed_bytes": _safe_int(
raw.get("archive_max_total_uncompressed_bytes"), 50 * 1024 * 1024, max_value=5_000_000_000
),
}
saved = _set_tabular_limits_into_oclaw_config(next_limits)
return {"ok": True, "limits": saved}
@ -1253,20 +1464,26 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
interaction_mode, selected_specialist = _normalize_chat_mode(payload)
memory_mode = _normalize_memory_mode(payload)
if "interaction_mode" not in payload and "chat_mode" not in payload:
mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"
interaction_mode, _, _ = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
interaction_mode = normalize_interaction_mode(store.get_setting(mode_key))
if "specialist" not in payload:
specialist_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"
_, selected_specialist, _ = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
selected_specialist = normalize_requested_specialist(store.get_setting(specialist_key))
if "memory_mode" not in payload:
memory_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"
_, _, memory_mode = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
memory_mode = _normalize_memory_mode({"memory_mode": store.get_setting(memory_mode_key)})
selected_specialist = _apply_specialist_flags(store, selected_specialist)
if not text_raw and not attachments:
raise HTTPException(status_code=400, detail="text_or_attachments_required")
@ -1351,20 +1568,26 @@ def include_chat_routes(router: APIRouter, *, resolve_auth: Callable[[SqliteStor
interaction_mode, selected_specialist = _normalize_chat_mode(payload)
memory_mode = _normalize_memory_mode(payload)
if "interaction_mode" not in payload and "chat_mode" not in payload:
mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="interaction_mode"
interaction_mode, _, _ = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
interaction_mode = normalize_interaction_mode(store.get_setting(mode_key))
if "specialist" not in payload:
specialist_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="specialist"
_, selected_specialist, _ = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
selected_specialist = normalize_requested_specialist(store.get_setting(specialist_key))
if "memory_mode" not in payload:
memory_mode_key = _chat_session_mode_setting_key(
tenant_id=tenant_id, user_id=user_id, session_id=str(session_id), field="memory_mode"
_, _, memory_mode = _resolve_mode_settings(
store=store,
tenant_id=tenant_id,
user_id=user_id,
session_id=str(session_id),
)
memory_mode = _normalize_memory_mode({"memory_mode": store.get_setting(memory_mode_key)})
selected_specialist = _apply_specialist_flags(store, selected_specialist)
if not text_raw and not attachments:
raise HTTPException(status_code=400, detail="text_or_attachments_required")

80
interfaces/admin/static/app.js
View file

@ -45,9 +45,21 @@ const I18N = {
"attachments.toolModeMinRows": "触发工具模式最小行数",
"attachments.toolModeMaxBytes": "工具模式最大文件字节数",
"attachments.sqlTimeoutMs": "SQL 超时（ms）",
"attachments.imageReplayCapChars": "图片结果回放上限字符数",
"attachments.videoReplayCapChars": "视频转写结果回放上限字符数",
"attachments.videoTranscriptChunkSize": "视频转写分块大小",
"attachments.videoTranscriptChunkOverlap": "视频转写分块重叠",
"attachments.archiveMaxDepth": "压缩包最大嵌套深度",
"attachments.archiveMaxFileCount": "压缩包最大文件数",
"attachments.archiveMaxEntryBytes": "压缩包单文件最大解压字节",
"attachments.archiveMaxTotalBytes": "压缩包总解压字节上限",
"attachments.highPreviewWarn": "大表摘要预览行数超过 200，可能显著增加 token 消耗。确认继续？",
"attachments.invalidNumber": "请输入有效正整数",
"attachments.sqlTimeoutHint": "硬超时（wall-clock）。范围 100..120000，默认 8000。命中会返回 sql_timeout 结构化错误。",
"attachments.imageReplayCapHint": "历史轮次中，query_image_attachment 的 OCR/描述结果回放上限。范围 600..30000，默认 4000。",
"attachments.videoReplayCapHint": "历史轮次中，query_video_attachment 的 transcript 结果回放上限。范围 600..30000，默认 4000。",
"attachments.videoTranscriptChunkHint": "query_video_attachment(task=transcript) 默认使用该分块参数落库，便于后续 query_text_attachment 检索。",
"attachments.archivePolicyHint": "archive_processor 的统一预算：支持 zip/tar/tgz/gz，限制嵌套层级、文件数与解压体积。",
"attachments.loadError": "加载失败",
"attachments.saved": "已保存",
"attachments.save": "保存",
@ -452,9 +464,21 @@ const I18N = {
"attachments.toolModeMinRows": "Min rows to trigger tool mode",
"attachments.toolModeMaxBytes": "Max file bytes for tool mode",
"attachments.sqlTimeoutMs": "SQL timeout (ms)",
"attachments.imageReplayCapChars": "Image replay cap chars",
"attachments.videoReplayCapChars": "Video replay cap chars",
"attachments.videoTranscriptChunkSize": "Video transcript chunk size",
"attachments.videoTranscriptChunkOverlap": "Video transcript chunk overlap",
"attachments.archiveMaxDepth": "Archive max depth",
"attachments.archiveMaxFileCount": "Archive max file count",
"attachments.archiveMaxEntryBytes": "Archive max entry uncompressed bytes",
"attachments.archiveMaxTotalBytes": "Archive max total uncompressed bytes",
"attachments.highPreviewWarn": "Large-table preview rows is above 200, which may significantly increase token usage. Continue?",
"attachments.invalidNumber": "Please enter valid positive integers",
"attachments.sqlTimeoutHint": "Hard wall-clock timeout. Range 100..120000, default 8000. Timeout returns structured sql_timeout error.",
"attachments.imageReplayCapHint": "Replay cap for query_image_attachment OCR/description results in historical context. Range 600..30000, default 4000.",
"attachments.videoReplayCapHint": "Replay cap for query_video_attachment transcript results in historical context. Range 600..30000, default 4000.",
"attachments.videoTranscriptChunkHint": "Default chunk parameters used by query_video_attachment(task=transcript) when persisting transcript chunks for query_text_attachment retrieval.",
"attachments.archivePolicyHint": "Unified archive_processor budget for zip/tar/tgz/gz: limits depth, file count and uncompressed size.",
"attachments.loadError": "Load failed",
"attachments.saved": "Saved",
"attachments.save": "Save",
@ -6399,6 +6423,14 @@ async function renderAttachments() {
const toolMinRowsInput = el("input", { class: "input", type: "number", min: "1", step: "1" });
const toolMaxBytesInput = el("input", { class: "input", type: "number", min: "1", step: "1" });
const sqlTimeoutInput = el("input", { class: "input", type: "number", min: "100", max: "120000", step: "1" });
const imageReplayCapInput = el("input", { class: "input", type: "number", min: "600", max: "30000", step: "1" });
const videoReplayCapInput = el("input", { class: "input", type: "number", min: "600", max: "30000", step: "1" });
const videoTranscriptChunkSizeInput = el("input", { class: "input", type: "number", min: "1", max: "8000", step: "1" });
const videoTranscriptChunkOverlapInput = el("input", { class: "input", type: "number", min: "1", max: "4000", step: "1" });
const archiveMaxDepthInput = el("input", { class: "input", type: "number", min: "1", max: "10", step: "1" });
const archiveMaxFileCountInput = el("input", { class: "input", type: "number", min: "1", max: "20000", step: "1" });
const archiveMaxEntryBytesInput = el("input", { class: "input", type: "number", min: "1", step: "1" });
const archiveMaxTotalBytesInput = el("input", { class: "input", type: "number", min: "1", step: "1" });
const clampTimeout = (raw, fallback = 8000) => {
const n = parseInt(String(raw ?? "").trim(), 10);
@ -6421,6 +6453,14 @@ async function renderAttachments() {
const minRows = parsePositiveInt(toolMinRowsInput.value);
const maxBytes = parsePositiveInt(toolMaxBytesInput.value);
const sqlTimeoutMs = parsePositiveInt(sqlTimeoutInput.value);
const imageReplayCapChars = parsePositiveInt(imageReplayCapInput.value);
const videoReplayCapChars = parsePositiveInt(videoReplayCapInput.value);
const videoTranscriptChunkSize = parsePositiveInt(videoTranscriptChunkSizeInput.value);
const videoTranscriptChunkOverlap = parsePositiveInt(videoTranscriptChunkOverlapInput.value);
const archiveMaxDepth = parsePositiveInt(archiveMaxDepthInput.value);
const archiveMaxFileCount = parsePositiveInt(archiveMaxFileCountInput.value);
const archiveMaxEntryBytes = parsePositiveInt(archiveMaxEntryBytesInput.value);
const archiveMaxTotalBytes = parsePositiveInt(archiveMaxTotalBytesInput.value);
if (
rows === null ||
cols === null ||
@ -6429,7 +6469,15 @@ async function renderAttachments() {
previewRows === null ||
minRows === null ||
maxBytes === null ||
sqlTimeoutMs === null
sqlTimeoutMs === null ||
imageReplayCapChars === null ||
videoReplayCapChars === null ||
videoTranscriptChunkSize === null ||
videoTranscriptChunkOverlap === null ||
archiveMaxDepth === null ||
archiveMaxFileCount === null ||
archiveMaxEntryBytes === null ||
archiveMaxTotalBytes === null
) {
throw new Error(t("attachments.invalidNumber"));
}
@ -6446,6 +6494,14 @@ async function renderAttachments() {
tool_mode_min_rows: minRows,
tool_mode_max_bytes: maxBytes,
sql_timeout_ms: clampTimeout(sqlTimeoutMs, 8000),
image_result_replay_cap_chars: Math.max(600, Math.min(30000, imageReplayCapChars)),
video_result_replay_cap_chars: Math.max(600, Math.min(30000, videoReplayCapChars)),
video_transcript_chunk_size: Math.max(1, Math.min(8000, videoTranscriptChunkSize)),
video_transcript_chunk_overlap: Math.max(1, Math.min(4000, videoTranscriptChunkOverlap)),
archive_max_depth: Math.max(1, Math.min(10, archiveMaxDepth)),
archive_max_file_count: Math.max(1, Math.min(20000, archiveMaxFileCount)),
archive_max_entry_bytes: Math.max(1, archiveMaxEntryBytes),
archive_max_total_uncompressed_bytes: Math.max(1, archiveMaxTotalBytes),
};
};
@ -6457,9 +6513,17 @@ async function renderAttachments() {
maxSheetsInput.value = String(l.max_excel_sheets || 50);
largePreviewRowsInput.value = String(l.large_table_preview_rows || 20);
toolEnabledInput.checked = !!l.tool_mode_enabled;
toolMinRowsInput.value = String(l.tool_mode_min_rows || 20000);
toolMinRowsInput.value = String(l.tool_mode_min_rows || 5000);
toolMaxBytesInput.value = String(l.tool_mode_max_bytes || 31457280);
sqlTimeoutInput.value = String(clampTimeout(l.sql_timeout_ms, 8000));
imageReplayCapInput.value = String(Math.max(600, Math.min(30000, parsePositiveInt(l.image_result_replay_cap_chars) || 4000)));
videoReplayCapInput.value = String(Math.max(600, Math.min(30000, parsePositiveInt(l.video_result_replay_cap_chars) || 4000)));
videoTranscriptChunkSizeInput.value = String(Math.max(1, Math.min(8000, parsePositiveInt(l.video_transcript_chunk_size) || 1600)));
videoTranscriptChunkOverlapInput.value = String(Math.max(1, Math.min(4000, parsePositiveInt(l.video_transcript_chunk_overlap) || 200)));
archiveMaxDepthInput.value = String(Math.max(1, Math.min(10, parsePositiveInt(l.archive_max_depth) || 2)));
archiveMaxFileCountInput.value = String(Math.max(1, Math.min(20000, parsePositiveInt(l.archive_max_file_count) || 200)));
archiveMaxEntryBytesInput.value = String(Math.max(1, parsePositiveInt(l.archive_max_entry_bytes) || 10485760));
archiveMaxTotalBytesInput.value = String(Math.max(1, parsePositiveInt(l.archive_max_total_uncompressed_bytes) || 52428800));
};
const load = async () => {
@ -6538,6 +6602,18 @@ async function renderAttachments() {
inputRow(t("attachments.toolModeMaxBytes"), toolMaxBytesInput),
inputRow(t("attachments.sqlTimeoutMs"), sqlTimeoutInput),
el("div", { class: "muted", style: "margin-top:6px;line-height:1.45;", text: t("attachments.sqlTimeoutHint") }),
inputRow(t("attachments.imageReplayCapChars"), imageReplayCapInput),
el("div", { class: "muted", style: "margin-top:6px;line-height:1.45;", text: t("attachments.imageReplayCapHint") }),
inputRow(t("attachments.videoReplayCapChars"), videoReplayCapInput),
el("div", { class: "muted", style: "margin-top:6px;line-height:1.45;", text: t("attachments.videoReplayCapHint") }),
inputRow(t("attachments.videoTranscriptChunkSize"), videoTranscriptChunkSizeInput),
inputRow(t("attachments.videoTranscriptChunkOverlap"), videoTranscriptChunkOverlapInput),
el("div", { class: "muted", style: "margin-top:6px;line-height:1.45;", text: t("attachments.videoTranscriptChunkHint") }),
inputRow(t("attachments.archiveMaxDepth"), archiveMaxDepthInput),
inputRow(t("attachments.archiveMaxFileCount"), archiveMaxFileCountInput),
inputRow(t("attachments.archiveMaxEntryBytes"), archiveMaxEntryBytesInput),
inputRow(t("attachments.archiveMaxTotalBytes"), archiveMaxTotalBytesInput),
el("div", { class: "muted", style: "margin-top:6px;line-height:1.45;", text: t("attachments.archivePolicyHint") }),
el("div", { class: "row", style: "gap:8px;margin-top:12px;flex-wrap:wrap;" }, [loadBtn, resetBtn, saveBtn]),
el("div", { style: "height:8px" }),
status,

125
interfaces/admin/static/chat.js
View file

@ -288,8 +288,14 @@ const CHAT_URL_SCOPE_KEY = "ops_chat_url_scope";
const CHAT_SPECIALIST_PREF_KEY = "ops_chat_specialist_pref";
const CHAT_INTERACTION_MODE_KEY = "ops_chat_interaction_mode";
const CHAT_MEMORY_MODE_KEY = "ops_chat_memory_mode";
const CHAT_REASONING_TOGGLE_KEY = "ops_chat_reasoning_toggle";
const ADMIN_CHAT_SHOW_TOOL_OUTPUT_DEFAULT = false;
let adminChatShowToolOutput = ADMIN_CHAT_SHOW_TOOL_OUTPUT_DEFAULT;
const REASONING_BLOCK_MAX_CHARS = 12000;
const CHAT_ENABLE_WIKI_EVENT_POLLER = false;
const _rt = String(localStorage.getItem(CHAT_REASONING_TOGGLE_KEY) || "").trim().toLowerCase();
let adminChatShowToolOutput = _rt
? ["1", "true", "yes", "on"].includes(_rt)
: ADMIN_CHAT_SHOW_TOOL_OUTPUT_DEFAULT;
let authSession = null;
function _toolSummaryTitle(role) {
@ -303,10 +309,33 @@ function _normalizeEventType(v) {
}
function _collapsedBlockNode(title, text) {
const raw = String(text || "");
const clipped = raw.length > REASONING_BLOCK_MAX_CHARS ? raw.slice(0, REASONING_BLOCK_MAX_CHARS) : raw;
const suffix =
raw.length > REASONING_BLOCK_MAX_CHARS
? `\n\n[truncated ${raw.length - REASONING_BLOCK_MAX_CHARS} chars to keep UI responsive]`
: "";
const box = document.createElement("div");
box.className = "chat-msg__reasoning-block";
box.appendChild(el("div", { class: "chat-msg__reasoning-title", text: String(title || "") }));
box.appendChild(el("pre", { class: "chat-msg__reasoning-pre", text: String(text || "") || "—" }));
const pre = el("pre", { class: "chat-msg__reasoning-pre", text: (clipped + suffix) || "—" });
box.appendChild(pre);
if (raw.length > REASONING_BLOCK_MAX_CHARS) {
const btn = document.createElement("button");
btn.type = "button";
btn.className = "btn btn-sm";
btn.style.marginTop = "6px";
btn.textContent = currentLang === "zh" ? "展开全文（可能较慢）" : "Load full text (may be slow)";
btn.addEventListener("click", () => {
btn.disabled = true;
btn.textContent = currentLang === "zh" ? "加载中..." : "Loading...";
requestAnimationFrame(() => {
pre.textContent = raw || "—";
btn.remove();
});
});
box.appendChild(btn);
}
return box;
}
@ -1206,12 +1235,14 @@ function createStreamStitcher() {
};
}
function formatToolPanelText(name, payload) {
function formatToolPanelText(name, payload, options = {}) {
const streamMode = !!(options && options.streamMode);
const truncateToolPanel = (s) => {
const raw = String(s || "").trim();
const maxChars = 50;
if (streamMode) return raw;
const maxChars = 4000;
if (raw.length <= maxChars) return raw;
return `${raw.slice(0, maxChars)}…（详情请重新阅读）`;
return `${raw.slice(0, maxChars)}\n\n…（为保证界面性能，已截断 ${raw.length - maxChars} 字符）`;
};
const n = String(name || "").trim() || "tool";
const p = payload && typeof payload === "object" ? payload : {};
@ -1252,7 +1283,7 @@ function formatToolPanelText(name, payload) {
body = String(r || "");
}
}
body = normalizeStreamText(body).replace(/\\n/g, "").replace(/\n/g, "");
body = normalizeStreamText(body);
return `${n}\n${truncateToolPanel(body)}`.trim();
}
@ -1870,12 +1901,12 @@ async function renderChatUi() {
});
modeSelect.appendChild(el("option", { value: "comprehensive", text: t("chat.modeComprehensive") }));
modeSelect.appendChild(el("option", { value: "expert", text: t("chat.modeExpert") }));
const _im = String(localStorage.getItem(CHAT_INTERACTION_MODE_KEY) || "comprehensive").toLowerCase();
modeSelect.value = ["comprehensive", "expert"].includes(_im) ? _im : "comprehensive";
const _im = String(localStorage.getItem(CHAT_INTERACTION_MODE_KEY) || "expert").toLowerCase();
modeSelect.value = ["comprehensive", "expert"].includes(_im) ? _im : "expert";
const _mm = String(localStorage.getItem(CHAT_MEMORY_MODE_KEY) || "default").toLowerCase();
localStorage.setItem(CHAT_MEMORY_MODE_KEY, _mm === "store_only" ? "store_only" : "default");
modeSelect.addEventListener("change", () => {
localStorage.setItem(CHAT_INTERACTION_MODE_KEY, String(modeSelect.value || "comprehensive"));
localStorage.setItem(CHAT_INTERACTION_MODE_KEY, String(modeSelect.value || "expert"));
saveSessionModePreference();
});
const applySpecialistOptions = () => {
@ -1944,7 +1975,7 @@ async function renderChatUi() {
if (String(modeSelect.value || "").toLowerCase() !== "expert") {
specialistSelect.value = "generalist";
}
localStorage.setItem(CHAT_INTERACTION_MODE_KEY, String(modeSelect.value || "comprehensive"));
localStorage.setItem(CHAT_INTERACTION_MODE_KEY, String(modeSelect.value || "expert"));
localStorage.setItem(CHAT_SPECIALIST_PREF_KEY, String(specialistSelect.value || "generalist"));
const mml = String(localStorage.getItem(CHAT_MEMORY_MODE_KEY) || "default").toLowerCase();
localStorage.setItem(CHAT_MEMORY_MODE_KEY, mml === "store_only" ? "store_only" : "default");
@ -1955,9 +1986,9 @@ async function renderChatUi() {
if (!activeId) return;
try {
await apiPost(`/admin/api/chat/sessions/${encodeURIComponent(activeId)}/mode`, {
interaction_mode: String(modeSelect.value || "comprehensive"),
interaction_mode: String(modeSelect.value || "expert"),
specialist:
String(modeSelect.value || "comprehensive").toLowerCase() === "expert"
String(modeSelect.value || "expert").toLowerCase() === "expert"
? String(specialistSelect.value || "generalist")
: "generalist",
memory_mode: String(localStorage.getItem(CHAT_MEMORY_MODE_KEY) || "default"),
@ -1975,7 +2006,7 @@ async function renderChatUi() {
]);
const syncModeSpecialistUi = () => {
const m = String(modeSelect.value || "comprehensive").toLowerCase();
const m = String(modeSelect.value || "expert").toLowerCase();
const showSpecialist = m === "expert";
specialistLabelEl.style.display = showSpecialist ? "" : "none";
specialistSelect.style.display = showSpecialist ? "" : "none";
@ -2069,22 +2100,12 @@ async function renderChatUi() {
topTopic = String(k || "");
}
}
const actions = [
{
label: t("chat.wikiViewMerged"),
onClick: () => openWikiPreviewModal({ sessionId: String(activeId || ""), path: "inbox/merged-turns.md" }),
},
];
if (topTopic) {
actions.push({
label: t("chat.wikiViewTopic", { topic: topTopic }),
onClick: () => openWikiPreviewModal({ sessionId: String(activeId || ""), path: `topics/auto-${topTopic}.md` }),
});
}
const actions = [];
return { kind: "info", text: t("chat.wikiToastMerged", { merged: String(merged), skipped: String(skipped) }), actions };
};
const pollWikiEvents = async () => {
if (!CHAT_ENABLE_WIKI_EVENT_POLLER) return;
const sid = String(activeId || "");
if (!sid) return;
const after = String((wikiAfterFinishedAt && wikiAfterFinishedAt[sid]) || "");
@ -2129,6 +2150,7 @@ async function renderChatUi() {
};
const startWikiPoller = () => {
if (!CHAT_ENABLE_WIKI_EVENT_POLLER) return;
if (wikiPollTimerId != null) return;
wikiPollTimerId = setInterval(pollWikiEvents, 2200);
// kick once
@ -2171,6 +2193,7 @@ async function renderChatUi() {
toolToggleBtn.addEventListener("click", () => {
showToolOutput = !showToolOutput;
adminChatShowToolOutput = showToolOutput;
localStorage.setItem(CHAT_REASONING_TOGGLE_KEY, showToolOutput ? "1" : "0");
syncToolToggleBtn();
const isStreaming = composerShell.classList.contains("chat-composer-shell--busy");
if (isStreaming) {
@ -2464,6 +2487,7 @@ async function renderChatUi() {
statusBar.textContent = t("chat.loading");
const resp = await apiPost("/admin/api/chat/sessions", {});
adoptCreatedSession(resp);
await loadSessionModePreference();
paintSessions();
messagesEl.innerHTML = "";
messagesEl.appendChild(el("div", { class: "muted", text: t("chat.empty") }));
@ -2614,7 +2638,7 @@ async function renderChatUi() {
attachments: Array.isArray(attachments) ? attachments : [],
idempotencyKey: String(idempotencyKey || `idem_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`),
thinking: "default",
interaction_mode: String(interactionMode || "comprehensive"),
interaction_mode: String(interactionMode || "expert"),
specialist: String(specialist || "generalist"),
memory_mode: String(memoryMode || "default"),
},
@ -2710,7 +2734,7 @@ async function renderChatUi() {
let currentStreamAbortController = null;
let currentWsTransport = null;
let currentAbortMeta = { sessionId: "", runId: "" };
const WS_CHAT_SEND_TIMEOUT_MS = 60000;
const WS_CHAT_SEND_TIMEOUT_MS = 180000;
const isAbortError = (err) => {
const name = String(err && err.name ? err.name : "");
const msg = String(err && err.message ? err.message : err || "");
@ -2856,10 +2880,12 @@ async function renderChatUi() {
const chunks = [];
for (const seg of chatStreamSegments) {
if (!seg || seg.type !== "text") continue;
const txt = String(seg.text || "").trim();
const txt = String(seg.text || "");
if (txt) chunks.push(txt);
}
return decodeEscapedNewlines(chunks.join("\n"));
// Keep 1:1 character mapping with original text segments.
// Never inject separators here; otherwise typewriter offsets drift.
return decodeEscapedNewlines(chunks.join(""));
};
const _renderStreamCompositeNow = () => {
const bubble = ensureStreamBubble();
@ -2902,7 +2928,7 @@ async function renderChatUi() {
const showLen = Math.max(0, Math.min(full.length, streamDisplayShown.length - textIdx));
const shown = full.slice(0, showLen);
textIdx += full.length;
if (!shown.trim()) continue;
if (!shown) continue;
blocks.push(`<div class="chat-msg__plain">${escapeHtml(decodeEscapedNewlines(shown))}</div>`);
} else if (seg.type === "tool") {
if (!showToolOutput) continue;
@ -2968,8 +2994,9 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
renderStreamComposite();
return;
}
// Slightly faster typing while keeping typewriter feel for正文.
const step = 2;
// Adaptive typing speed: when backlog grows, catch up quickly
// to avoid fragmented/laggy streaming and end-of-turn text bursts.
const step = Math.max(4, Math.min(180, Math.ceil(remain / 10)));
streamDisplayShown = streamDisplayTarget.slice(0, streamDisplayShown.length + step);
renderStreamComposite();
if (typingTimerId != null) {
@ -2979,7 +3006,7 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
typingTimerId = setTimeout(() => {
typingTimerId = null;
_scheduleTypingTick();
}, 22);
}, 14);
});
};
const setStreamText = (text) => {
@ -2996,11 +3023,12 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
const appendToolSegment = (payload) => {
const p = payload && typeof payload === "object" ? payload : {};
const name = String(p.name || "tool");
const liveTag = currentLang === "zh" ? "[实时全量]" : "[LIVE FULL]";
const key = `${name}:${++toolSeq}`;
const rawPayload = p.payload != null ? p.payload : p;
const body = formatToolPanelText(name, rawPayload);
const body = formatToolPanelText(name, rawPayload, { streamMode: true });
const sqlAudit = extractSqlAuditPayload(rawPayload);
chatStreamSegments.push({ type: "tool", key, title: name, body, sqlAudit });
chatStreamSegments.push({ type: "tool", key, title: `${name} ${liveTag}`, body, sqlAudit });
};
const appendFinalAssistant = async (message, fallbackText) => {
const normalized = _normalizeAssistantMessage(message, { requireRole: false, requireContentArray: false });
@ -3053,10 +3081,10 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
sessionId: activeId,
text: userText,
attachments: attachmentPayload,
interactionMode: String(modeSelect.value || "comprehensive"),
interactionMode: String(modeSelect.value || "expert"),
idempotencyKey: String(turnId || ""),
specialist:
String(modeSelect.value || "comprehensive").toLowerCase() === "expert"
String(modeSelect.value || "expert").toLowerCase() === "expert"
? String(specialistSelect.value || "generalist")
: "generalist",
memoryMode: String(localStorage.getItem(CHAT_MEMORY_MODE_KEY) || "default"),
@ -3144,6 +3172,11 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
chatStream = "";
chatRunId = null;
chatStreamSegments = [];
// Final assistant body may arrive before reasoning/tool rows are fully persisted.
// Reload once shortly after final to avoid "reasoning disappears until session switch".
setTimeout(() => {
loadMessagesForActive().catch(() => {});
}, 350);
if (!ok) statusBar.textContent = "";
return;
}
@ -3209,6 +3242,24 @@ ${autoLimit ? `<div style="margin-top:8px;"><span class="muted">auto-added claus
}, 350);
return doneMeta;
}
// WS timeout may happen while backend still computes and persists final reply.
// Try recovering from persisted history before surfacing a hard error.
if (emsg.includes("ws_send_timeout")) {
try {
for (let i = 0; i < 12; i++) {
if (await _recoverLatestAssistantFromHistory()) {
statusBar.textContent =
currentLang === "zh"
? "请求超时，已从历史补齐本轮结果。"
: "Request timed out; recovered this turn from history.";
return doneMeta || { __transport: "ws_timeout_history_recovered" };
}
await _sleep(600 + i * 400);
}
} catch (_) {
// fall through to generic ws-like failure recovery
}
}
// WS may time out/close while backend persists final message slightly later.
// Poll history briefly before surfacing hard failure.
try {

12
oclaw.json
View file

@ -6,7 +6,7 @@
},
"entries": {
"memory-wiki": {
"wiki_root": "oclaw/docs/memory-system/wiki",
"wiki_root": "data/wiki",
"max_search_results": 20,
"max_get_lines": 800,
"auto": {
@ -20,7 +20,15 @@
"large_table_preview_rows": 20,
"tool_mode_enabled": true,
"tool_mode_min_rows": 20000,
"tool_mode_max_bytes": 31457280
"tool_mode_max_bytes": 31457280,
"image_result_replay_cap_chars": 4000,
"video_result_replay_cap_chars": 4000,
"video_transcript_chunk_size": 1600,
"video_transcript_chunk_overlap": 200,
"archive_max_depth": 2,
"archive_max_file_count": 200,
"archive_max_entry_bytes": 10485760,
"archive_max_total_uncompressed_bytes": 52428800
}
},
"inject": {

30
platform/__init__.py
View file

@ -0,0 +1,30 @@
from __future__ import annotations
import importlib.util
import sysconfig
from pathlib import Path
def _load_stdlib_platform() -> object | None:
stdlib_dir = Path(sysconfig.get_paths().get("stdlib") or "")
platform_py = (stdlib_dir / "platform.py").resolve()
if not platform_py.exists():
return None
spec = importlib.util.spec_from_file_location("_stdlib_platform", str(platform_py))
if spec is None or spec.loader is None:
return None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
_stdlib = _load_stdlib_platform()
if _stdlib is not None:
for _name in dir(_stdlib):
if _name in {"__name__", "__package__", "__loader__", "__spec__", "__file__", "__cached__"}:
continue
globals()[_name] = getattr(_stdlib, _name)
__all__ = list(getattr(_stdlib, "__all__", []))
else:
__all__ = []

202
platform/files/archive_processor.py Normal file
View file

@ -0,0 +1,202 @@
from __future__ import annotations
import gzip
import io
import tarfile
import zipfile
from pathlib import PurePosixPath
from typing import Any, Callable
MAX_ARCHIVE_DEPTH = 2
MAX_ARCHIVE_FILE_COUNT = 200
MAX_ARCHIVE_ENTRY_BYTES = 10 * 1024 * 1024
MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES = 50 * 1024 * 1024
MAX_ARCHIVE_MEMBER_NAME_LENGTH = 240
def is_safe_archive_member_name(name: str) -> bool:
n = str(name or "").replace("\\", "/").strip()
if not n:
return False
if len(n) > MAX_ARCHIVE_MEMBER_NAME_LENGTH:
return False
if any(ord(ch) < 32 for ch in n):
return False
p = PurePosixPath(n)
if p.is_absolute():
return False
if any(part in ("..", "") for part in p.parts):
return False
return True
def detect_archive_kind(name: str) -> str | None:
n = str(name or "").strip().lower()
if n.endswith(".tar.gz") or n.endswith(".tgz"):
return "tgz"
if n.endswith(".zip"):
return "zip"
if n.endswith(".tar"):
return "tar"
if n.endswith(".gz"):
return "gz"
if n.endswith(".rar"):
return "rar"
if n.endswith(".7z"):
return "7z"
return None
def sniff_archive_kind(data: bytes) -> str | None:
b = bytes(data or b"")
if len(b) >= 4 and b[:4] == b"PK\x03\x04":
return "zip"
if len(b) >= 2 and b[:2] == b"\x1f\x8b":
return "gz"
if len(b) >= 6 and b[:6] == b"Rar!\x1a\x07":
return "rar"
if len(b) >= 6 and b[:6] == b"7z\xbc\xaf\x27\x1c":
return "7z"
if len(b) >= 262 and b[257:262] == b"ustar":
return "tar"
return None
def process_archive(
*,
archive_name: str,
archive_data: bytes,
process_member: Callable[[str, bytes, int], list[dict[str, Any]]],
depth: int = 0,
limits: dict[str, int] | None = None,
) -> list[dict[str, Any]]:
lim = limits or {}
max_depth = int(lim.get("max_depth") or MAX_ARCHIVE_DEPTH)
max_file_count = int(lim.get("max_file_count") or MAX_ARCHIVE_FILE_COUNT)
max_entry_bytes = int(lim.get("max_entry_bytes") or MAX_ARCHIVE_ENTRY_BYTES)
max_total_uncompressed_bytes = int(
lim.get("max_total_uncompressed_bytes") or MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES
)
kind_from_sig = sniff_archive_kind(archive_data)
kind_from_name = detect_archive_kind(archive_name)
# For .tgz/.tar.gz, signature looks like gzip; keep richer name hint.
if kind_from_name == "tgz" and kind_from_sig == "gz":
kind = "tgz"
else:
kind = kind_from_sig or kind_from_name
if kind is None:
return []
if depth > max_depth:
return [{"type": "text", "name": "zip-error", "content": f"ZIP nesting too deep (>{max_depth})"}]
out: list[dict[str, Any]] = []
def _err(msg: str, *, error_code: str) -> list[dict[str, Any]]:
base = {
"type": "text",
"name": ("zip-error" if kind == "zip" else "archive-error"),
"error_code": str(error_code or "archive_error"),
"content": (
f"[ArchiveError]\n- error_code: {error_code}\n- format: {kind}\n- detail: {msg}"
),
}
return [base]
try:
if kind in {"rar", "7z"}:
raise ValueError(
f"unsupported archive format: {kind}; currently supported formats are zip/tar/tgz/gz"
)
if kind == "zip":
with zipfile.ZipFile(io.BytesIO(archive_data)) as zf:
infos = [x for x in zf.infolist() if not x.is_dir()]
if len(infos) > max_file_count:
raise ValueError(f"ZIP has too many files ({len(infos)} > {max_file_count})")
total_uncompressed = 0
for info in infos:
if not is_safe_archive_member_name(info.filename):
raise ValueError(f"ZIP contains unsafe path: {info.filename}")
fsize = int(info.file_size or 0)
total_uncompressed += fsize
if fsize > max_entry_bytes:
raise ValueError(f"ZIP entry too large: {info.filename} ({fsize} > {max_entry_bytes})")
if total_uncompressed > max_total_uncompressed_bytes:
raise ValueError(
f"ZIP total uncompressed size too large ({total_uncompressed} > {max_total_uncompressed_bytes})"
)
with zf.open(info) as f:
out.extend(process_member(info.filename, f.read(), depth + 1))
return out
if kind in {"tar", "tgz"}:
mode = "r:gz" if kind == "tgz" else "r:"
with tarfile.open(fileobj=io.BytesIO(archive_data), mode=mode) as tf:
all_members = list(tf.getmembers())
for m in all_members:
if m.issym() or m.islnk():
raise ValueError(f"TAR contains link entry: {m.name}")
if m.ischr() or m.isblk() or m.isfifo():
raise ValueError(f"TAR contains device/fifo entry: {m.name}")
members = [m for m in all_members if m.isfile()]
if len(members) > max_file_count:
raise ValueError(f"TAR has too many files ({len(members)} > {max_file_count})")
total_uncompressed = 0
for m in members:
if not is_safe_archive_member_name(m.name):
raise ValueError(f"TAR contains unsafe path: {m.name}")
fsize = int(m.size or 0)
total_uncompressed += fsize
if fsize > max_entry_bytes:
raise ValueError(f"TAR entry too large: {m.name} ({fsize} > {max_entry_bytes})")
if total_uncompressed > max_total_uncompressed_bytes:
raise ValueError(
f"TAR total uncompressed size too large ({total_uncompressed} > {max_total_uncompressed_bytes})"
)
ext = tf.extractfile(m)
if ext is None:
continue
out.extend(process_member(m.name, ext.read(), depth + 1))
return out
if kind == "gz":
raw = gzip.decompress(archive_data)
if len(raw) > max_entry_bytes:
raise ValueError(f"GZ entry too large ({len(raw)} > {max_entry_bytes})")
name = str(archive_name or "file.gz")
inner_name = name[:-3] if name.lower().endswith(".gz") else f"{name}.out"
out.extend(process_member(inner_name, raw, depth + 1))
return out
except Exception as e:
s = str(e).lower()
code = "archive_parse_failed"
if "unsupported archive format" in s:
code = "archive_unsupported_format"
elif "unsafe path" in s:
code = "archive_path_traversal"
elif "nesting too deep" in s:
code = "archive_max_depth_exceeded"
elif "too many files" in s:
code = "archive_max_file_count_exceeded"
elif "entry too large" in s:
code = "archive_max_entry_bytes_exceeded"
elif "total uncompressed size too large" in s:
code = "archive_max_total_uncompressed_bytes_exceeded"
elif "link entry" in s:
code = "archive_link_entry_forbidden"
elif "device/fifo entry" in s:
code = "archive_special_entry_forbidden"
return _err(str(e), error_code=code)
return out
__all__ = [
"MAX_ARCHIVE_DEPTH",
"MAX_ARCHIVE_ENTRY_BYTES",
"MAX_ARCHIVE_FILE_COUNT",
"MAX_ARCHIVE_MEMBER_NAME_LENGTH",
"MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES",
"detect_archive_kind",
"sniff_archive_kind",
"is_safe_archive_member_name",
"process_archive",
]

431
platform/files/file_attachments.py
View file

@ -6,10 +6,10 @@ import io
import json
import os
import re
import subprocess
import zipfile
from functools import lru_cache
from pathlib import Path
from pathlib import PurePosixPath
from typing import Any
import pandas as pd
@ -18,8 +18,24 @@ from PIL import Image
from PyPDF2 import PdfReader
from oclaw.platform.config.paths import PROJECT_ROOT
from oclaw.platform.files.archive_processor import (
MAX_ARCHIVE_DEPTH,
MAX_ARCHIVE_ENTRY_BYTES,
MAX_ARCHIVE_FILE_COUNT,
MAX_ARCHIVE_MEMBER_NAME_LENGTH,
MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES,
detect_archive_kind,
is_safe_archive_member_name,
process_archive,
)
from oclaw.platform.files.attachment_assets import AttachmentAssetStore
from oclaw.platform.files.tabular_attachment_store import save_dataframe, save_workbook
from oclaw.platform.files.text_attachment_store import (
DEFAULT_TEXT_CHUNK_OVERLAP,
DEFAULT_TEXT_CHUNK_SIZE,
DEFAULT_TEXT_INLINE_MAX_CHARS,
save_text_document,
)
def _sniff_image_mime(data: bytes) -> str | None:
@ -36,25 +52,101 @@ def _sniff_image_mime(data: bytes) -> str | None:
return None
def _ffprobe_exists() -> bool:
try:
p = subprocess.run(["ffprobe", "-version"], capture_output=True, text=True, timeout=3)
return p.returncode == 0
except Exception:
return False
def _extract_video_meta_from_ffprobe(obj: dict[str, Any]) -> dict[str, Any]:
fmt = obj.get("format") if isinstance(obj.get("format"), dict) else {}
streams = obj.get("streams") if isinstance(obj.get("streams"), list) else []
duration_sec = None
if isinstance(fmt, dict) and fmt.get("duration") is not None:
try:
duration_sec = float(fmt.get("duration"))
except Exception:
duration_sec = None
width = None
height = None
fps = None
for s in streams:
if not isinstance(s, dict):
continue
if str(s.get("codec_type") or "") != "video":
continue
if s.get("width") is not None and s.get("height") is not None:
try:
width = int(s.get("width") or 0) or None
height = int(s.get("height") or 0) or None
except Exception:
width = width
height = height
fr = str(s.get("avg_frame_rate") or s.get("r_frame_rate") or "").strip()
if fr and fr != "0/0" and "/" in fr:
try:
a, b = fr.split("/", 1)
fa = float(a)
fb = float(b)
if fb:
fps = fa / fb
except Exception:
fps = fps
break
out: dict[str, Any] = {}
if duration_sec is not None:
out["duration_sec"] = duration_sec
if width is not None:
out["width"] = width
if height is not None:
out["height"] = height
if fps is not None:
out["fps"] = fps
return out
def _ffprobe_video_meta(path: Path) -> dict[str, Any]:
if not _ffprobe_exists():
return {}
try:
p = subprocess.run(
["ffprobe", "-v", "error", "-print_format", "json", "-show_format", "-show_streams", str(path)],
capture_output=True,
text=True,
timeout=8,
)
if p.returncode != 0:
return {}
obj = json.loads(p.stdout or "{}")
if not isinstance(obj, dict):
return {}
return _extract_video_meta_from_ffprobe(obj)
except Exception:
return {}
MAX_TEXT_CHARS = 12_000
EXCEL_PREVIEW_ROWS = 80
LARGE_TABLE_PREVIEW_ROWS = 20
MAX_ZIP_DEPTH = 2
MAX_ZIP_FILE_COUNT = 200
MAX_ZIP_ENTRY_BYTES = 10 * 1024 * 1024
MAX_ZIP_TOTAL_UNCOMPRESSED_BYTES = 50 * 1024 * 1024
MAX_ZIP_DEPTH = MAX_ARCHIVE_DEPTH
MAX_ZIP_FILE_COUNT = MAX_ARCHIVE_FILE_COUNT
MAX_ZIP_ENTRY_BYTES = MAX_ARCHIVE_ENTRY_BYTES
MAX_ZIP_TOTAL_UNCOMPRESSED_BYTES = MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES
DEFAULT_TABULAR_ROWS_READ = 5000
DEFAULT_TABULAR_COLUMNS = 200
DEFAULT_TABULAR_CELL_CHARS = 500
DEFAULT_TABULAR_TOOL_MODE_ENABLED = True
DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS = 20_000
DEFAULT_TABULAR_TOOL_MODE_MIN_ROWS = 5_000
DEFAULT_TABULAR_TOOL_MODE_MAX_BYTES = 30 * 1024 * 1024
DEFAULT_LARGE_TABLE_PREVIEW_ROWS = LARGE_TABLE_PREVIEW_ROWS
MAX_ZIP_MEMBER_NAME_LENGTH = 240
MAX_ZIP_MEMBER_NAME_LENGTH = MAX_ARCHIVE_MEMBER_NAME_LENGTH
MAX_EXCEL_ZIP_FILE_COUNT = 500
MAX_EXCEL_ZIP_ENTRY_BYTES = 20 * 1024 * 1024
MAX_EXCEL_ZIP_TOTAL_UNCOMPRESSED_BYTES = 120 * 1024 * 1024
DEFAULT_MAX_EXCEL_SHEETS = 50
DEFAULT_TEXT_QUERY_TOP_K = 5
def _truncate_text(text: str, limit: int = MAX_TEXT_CHARS) -> str:
@ -64,6 +156,24 @@ def _truncate_text(text: str, limit: int = MAX_TEXT_CHARS) -> str:
return t[:limit] + f"\n\n...[truncated to {limit} chars]"
def _text_ref_summary(*, name: str, chars: int, chunks: int, source_kind: str, preview: str, top_k: int) -> str:
return _truncate_text(
(
f"# Text Document Summary\n"
f"- name: {name}\n"
f"- source_kind: {source_kind}\n"
f"- chars: {chars}\n"
f"- chunks: {chunks}\n"
f"- mode: text_ref\n"
f"- query_tool: query_text_attachment\n"
f"- note: use `text_id` from accompanying `text_ref` attachment; recommended top_k <= {top_k}\n\n"
f"## Preview (first {min(len(preview), 1500)} chars)\n"
f"{preview[:1500]}"
),
limit=MAX_TEXT_CHARS,
)
def _decode_text_bytes(data: bytes) -> str:
try:
return data.decode("utf-8")
@ -120,6 +230,26 @@ def _attachments_limits() -> dict[str, Any]:
"tool_mode_max_bytes": _safe_int(
tabular_cfg.get("tool_mode_max_bytes"), DEFAULT_TABULAR_TOOL_MODE_MAX_BYTES, max_value=500 * 1024 * 1024
),
"text_inline_max_chars": _safe_int(
tabular_cfg.get("text_inline_max_chars"), DEFAULT_TEXT_INLINE_MAX_CHARS, max_value=200_000
),
"text_chunk_size": _safe_int(tabular_cfg.get("text_chunk_size"), DEFAULT_TEXT_CHUNK_SIZE, max_value=8_000),
"text_chunk_overlap": _safe_int(
tabular_cfg.get("text_chunk_overlap"), DEFAULT_TEXT_CHUNK_OVERLAP, max_value=4_000
),
"text_query_top_k": _safe_int(tabular_cfg.get("text_query_top_k"), DEFAULT_TEXT_QUERY_TOP_K, max_value=50),
"archive_max_depth": _safe_int(tabular_cfg.get("archive_max_depth"), MAX_ARCHIVE_DEPTH, max_value=10),
"archive_max_file_count": _safe_int(
tabular_cfg.get("archive_max_file_count"), MAX_ARCHIVE_FILE_COUNT, max_value=20_000
),
"archive_max_entry_bytes": _safe_int(
tabular_cfg.get("archive_max_entry_bytes"), MAX_ARCHIVE_ENTRY_BYTES, max_value=2_000_000_000
),
"archive_max_total_uncompressed_bytes": _safe_int(
tabular_cfg.get("archive_max_total_uncompressed_bytes"),
MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES,
max_value=5_000_000_000,
),
}
@ -160,6 +290,11 @@ def _dataframe_summary_text(
dtypes = [f"{str(k)}:{str(v)}" for k, v in df.dtypes.items()]
show_rows = max(1, int(preview_rows or EXCEL_PREVIEW_ROWS))
sample = df.head(show_rows).to_string(index=False)
rows_scope_line = (
f"## Preview (first {min(rows, show_rows)} rows)"
if rows > show_rows
else f"## Full table included ({rows} rows)"
)
body = (
f"# Table Summary\n"
f"- rows: {rows}\n"
@ -169,7 +304,7 @@ def _dataframe_summary_text(
f"- clipped_cells: {'yes' if clipped_cells else 'no'}\n"
f"- columns: {', '.join(header)}\n"
f"- dtypes: {', '.join(dtypes)}\n\n"
f"## Preview (first {min(rows, show_rows)} rows)\n"
f"{rows_scope_line}\n"
f"{sample}"
)
return _truncate_text(body)
@ -191,19 +326,7 @@ def _pdf_summary_text(reader: PdfReader) -> str:
def _is_safe_zip_member_name(name: str) -> bool:
n = str(name or "").replace("\\", "/").strip()
if not n:
return False
if len(n) > MAX_ZIP_MEMBER_NAME_LENGTH:
return False
if any(ord(ch) < 32 for ch in n):
return False
p = PurePosixPath(n)
if p.is_absolute():
return False
if any(part in ("..", "") for part in p.parts):
return False
return True
return is_safe_archive_member_name(name)
def _validate_excel_zip_payload(data: bytes) -> str | None:
@ -231,48 +354,21 @@ def _validate_excel_zip_payload(data: bytes) -> str | None:
def process_zip(zip_data: bytes, *, _depth: int = 0) -> list[dict[str, Any]]:
if _depth > MAX_ZIP_DEPTH:
return [
{
"type": "text",
"name": "zip-error",
"content": f"ZIP nesting too deep (>{MAX_ZIP_DEPTH})",
}
]
attachments: list[dict[str, Any]] = []
try:
with zipfile.ZipFile(io.BytesIO(zip_data)) as zf:
infos = [x for x in zf.infolist() if not x.is_dir()]
if len(infos) > MAX_ZIP_FILE_COUNT:
raise ValueError(f"ZIP has too many files ({len(infos)} > {MAX_ZIP_FILE_COUNT})")
total_uncompressed = 0
for info in infos:
if not _is_safe_zip_member_name(info.filename):
raise ValueError(f"ZIP contains unsafe path: {info.filename}")
total_uncompressed += int(info.file_size or 0)
if int(info.file_size or 0) > MAX_ZIP_ENTRY_BYTES:
raise ValueError(
f"ZIP entry too large: {info.filename} ({int(info.file_size or 0)} > {MAX_ZIP_ENTRY_BYTES})"
)
if total_uncompressed > MAX_ZIP_TOTAL_UNCOMPRESSED_BYTES:
raise ValueError(
"ZIP total uncompressed size too large "
f"({total_uncompressed} > {MAX_ZIP_TOTAL_UNCOMPRESSED_BYTES})"
)
with zf.open(info) as f:
data = f.read()
att = process_file_data(info.filename, data, _zip_depth=_depth + 1)
if att:
attachments.extend(att)
except Exception as e:
attachments.append(
{
"type": "text",
"name": "zip-error",
"content": f"Error parsing zip: {e}",
}
)
return attachments
limits = _attachments_limits()
return process_archive(
archive_name="bundle.zip",
archive_data=zip_data,
process_member=lambda n, b, d: process_file_data(n, b, _zip_depth=d),
depth=_depth,
limits={
"max_depth": int(limits.get("archive_max_depth") or MAX_ARCHIVE_DEPTH),
"max_file_count": int(limits.get("archive_max_file_count") or MAX_ARCHIVE_FILE_COUNT),
"max_entry_bytes": int(limits.get("archive_max_entry_bytes") or MAX_ARCHIVE_ENTRY_BYTES),
"max_total_uncompressed_bytes": int(
limits.get("archive_max_total_uncompressed_bytes") or MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES
),
},
)
def process_file_data(name: str, data: bytes, *, _zip_depth: int = 0) -> list[dict[str, Any]]:
@ -336,6 +432,67 @@ def process_file_data(name: str, data: bytes, *, _zip_depth: int = 0) -> list[di
)
return attachments
if ext in ("mp4", "mov", "mkv", "webm", "avi", "m4v"):
mime = "video/mp4"
if ext == "webm":
mime = "video/webm"
elif ext == "mov":
mime = "video/quicktime"
elif ext == "mkv":
mime = "video/x-matroska"
elif ext == "avi":
mime = "video/x-msvideo"
meta = AttachmentAssetStore().save_bytes(data, filename=name, mime=mime)
probe_meta: dict[str, Any] = {}
try:
p = AttachmentAssetStore().get_local_path(str(meta.attachment_id))
if p is not None:
probe_meta = _ffprobe_video_meta(p)
except Exception:
probe_meta = {}
attachments.append(
{
"type": "video_ref",
"name": meta.name,
"mime": meta.mime,
"attachment_id": meta.attachment_id,
"bytes": meta.bytes,
**probe_meta,
}
)
attachments.append(
{
"type": "text",
"name": name,
"content": _truncate_text(
"".join(
[
"# Video Attachment\n",
f"- name: {meta.name}\n",
f"- mime: {meta.mime}\n",
f"- bytes: {meta.bytes}\n",
(
f"- duration_sec: {probe_meta.get('duration_sec')}\n"
if probe_meta.get("duration_sec") is not None
else ""
),
(
f"- size: {probe_meta.get('width')}x{probe_meta.get('height')}\n"
if probe_meta.get("width") and probe_meta.get("height")
else ""
),
(f"- fps: {probe_meta.get('fps')}\n" if probe_meta.get("fps") is not None else ""),
"- mode: video_ref\n",
"- query_tool: query_video_attachment\n",
"- note: use `attachment_id` from accompanying `video_ref` attachment.\n",
]
),
limit=MAX_TEXT_CHARS,
),
}
)
return attachments
if ext in ("xlsx", "xls", "csv"):
try:
if ext == "xlsx":
@ -453,14 +610,97 @@ def process_file_data(name: str, data: bytes, *, _zip_depth: int = 0) -> list[di
try:
doc = Document(io.BytesIO(data))
text = "\n".join([p.text for p in doc.paragraphs])
attachments.append({"type": "text", "name": name, "content": _truncate_text(text)})
limits = _attachments_limits()
inline_cap = int(limits.get("text_inline_max_chars") or DEFAULT_TEXT_INLINE_MAX_CHARS)
if len(str(text or "")) <= inline_cap:
attachments.append({"type": "text", "name": name, "content": _truncate_text(text, limit=inline_cap)})
else:
meta = AttachmentAssetStore().save_bytes(
data,
filename=name,
mime="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
)
text_meta = save_text_document(
attachment_id=str(meta.attachment_id),
name=name,
text=str(text or ""),
source_kind="docx",
chunk_size=int(limits.get("text_chunk_size") or DEFAULT_TEXT_CHUNK_SIZE),
chunk_overlap=int(limits.get("text_chunk_overlap") or DEFAULT_TEXT_CHUNK_OVERLAP),
)
top_k = int(limits.get("text_query_top_k") or DEFAULT_TEXT_QUERY_TOP_K)
attachments.append(
{
"type": "text",
"name": name,
"content": _text_ref_summary(
name=name,
chars=int(text_meta.get("chars") or 0),
chunks=int(text_meta.get("chunks") or 0),
source_kind="docx",
preview=str(text or ""),
top_k=top_k,
),
}
)
attachments.append(
{
"type": "text_ref",
"name": str(name),
"attachment_id": str(meta.attachment_id),
"text_id": str(text_meta.get("text_id") or ""),
"chars": int(text_meta.get("chars") or 0),
"chunks": int(text_meta.get("chunks") or 0),
"source_kind": "docx",
}
)
except Exception as e:
attachments.append({"type": "text", "name": name, "content": _truncate_text(f"Error parsing docx: {e}")})
elif ext == "pdf":
try:
reader = PdfReader(io.BytesIO(data))
attachments.append({"type": "text", "name": name, "content": _pdf_summary_text(reader)})
text = _pdf_summary_text(reader)
limits = _attachments_limits()
inline_cap = int(limits.get("text_inline_max_chars") or DEFAULT_TEXT_INLINE_MAX_CHARS)
if len(str(text or "")) <= inline_cap:
attachments.append({"type": "text", "name": name, "content": _truncate_text(text, limit=inline_cap)})
else:
meta = AttachmentAssetStore().save_bytes(data, filename=name, mime="application/pdf")
text_meta = save_text_document(
attachment_id=str(meta.attachment_id),
name=name,
text=str(text or ""),
source_kind="pdf",
chunk_size=int(limits.get("text_chunk_size") or DEFAULT_TEXT_CHUNK_SIZE),
chunk_overlap=int(limits.get("text_chunk_overlap") or DEFAULT_TEXT_CHUNK_OVERLAP),
)
top_k = int(limits.get("text_query_top_k") or DEFAULT_TEXT_QUERY_TOP_K)
attachments.append(
{
"type": "text",
"name": name,
"content": _text_ref_summary(
name=name,
chars=int(text_meta.get("chars") or 0),
chunks=int(text_meta.get("chunks") or 0),
source_kind="pdf",
preview=str(text or ""),
top_k=top_k,
),
}
)
attachments.append(
{
"type": "text_ref",
"name": str(name),
"attachment_id": str(meta.attachment_id),
"text_id": str(text_meta.get("text_id") or ""),
"chars": int(text_meta.get("chars") or 0),
"chunks": int(text_meta.get("chunks") or 0),
"source_kind": "pdf",
}
)
except Exception as e:
attachments.append({"type": "text", "name": name, "content": _truncate_text(f"Error parsing pdf: {e}")})
@ -468,10 +708,65 @@ def process_file_data(name: str, data: bytes, *, _zip_depth: int = 0) -> list[di
text = _decode_text_bytes(data)
if ext == "html":
text = _clean_html_to_text(text)
attachments.append({"type": "text", "name": name, "content": _truncate_text(text)})
limits = _attachments_limits()
inline_cap = int(limits.get("text_inline_max_chars") or DEFAULT_TEXT_INLINE_MAX_CHARS)
if len(str(text or "")) <= inline_cap:
attachments.append({"type": "text", "name": name, "content": _truncate_text(text, limit=inline_cap)})
else:
meta = AttachmentAssetStore().save_bytes(data, filename=name, mime="text/plain")
text_meta = save_text_document(
attachment_id=str(meta.attachment_id),
name=name,
text=str(text or ""),
source_kind=ext or "text",
chunk_size=int(limits.get("text_chunk_size") or DEFAULT_TEXT_CHUNK_SIZE),
chunk_overlap=int(limits.get("text_chunk_overlap") or DEFAULT_TEXT_CHUNK_OVERLAP),
)
top_k = int(limits.get("text_query_top_k") or DEFAULT_TEXT_QUERY_TOP_K)
attachments.append(
{
"type": "text",
"name": name,
"content": _text_ref_summary(
name=name,
chars=int(text_meta.get("chars") or 0),
chunks=int(text_meta.get("chunks") or 0),
source_kind=str(text_meta.get("source_kind") or ext or "text"),
preview=str(text or ""),
top_k=top_k,
),
}
)
attachments.append(
{
"type": "text_ref",
"name": str(name),
"attachment_id": str(meta.attachment_id),
"text_id": str(text_meta.get("text_id") or ""),
"chars": int(text_meta.get("chars") or 0),
"chunks": int(text_meta.get("chunks") or 0),
"source_kind": str(text_meta.get("source_kind") or ext or "text"),
}
)
elif ext == "zip":
attachments.extend(process_zip(data, _depth=_zip_depth))
elif detect_archive_kind(name):
limits = _attachments_limits()
attachments.extend(
process_archive(
archive_name=name,
archive_data=data,
process_member=lambda n, b, d: process_file_data(n, b, _zip_depth=d),
depth=_zip_depth,
limits={
"max_depth": int(limits.get("archive_max_depth") or MAX_ARCHIVE_DEPTH),
"max_file_count": int(limits.get("archive_max_file_count") or MAX_ARCHIVE_FILE_COUNT),
"max_entry_bytes": int(limits.get("archive_max_entry_bytes") or MAX_ARCHIVE_ENTRY_BYTES),
"max_total_uncompressed_bytes": int(
limits.get("archive_max_total_uncompressed_bytes") or MAX_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES
),
},
)
)
return attachments

83
platform/files/tabular_attachment_store.py
View file

@ -35,6 +35,11 @@ def _meta_path(table_id: str) -> Path:
return _tabular_root() / f"{table_id}.meta.json"
def _is_valid_table_id(table_id: str) -> bool:
tid = str(table_id or "").strip().lower()
return bool(re.fullmatch(r"[0-9a-f]{64}", tid))
def _normalize_columns(raw_cols: list[Any]) -> tuple[list[str], list[dict[str, Any]]]:
out: list[str] = []
mapping: list[dict[str, Any]] = []
@ -311,7 +316,10 @@ def save_workbook(*, attachment_id: str, name: str, sheets: dict[str, pd.DataFra
def get_table_meta(table_id: str) -> dict[str, Any] | None:
p = _meta_path(str(table_id or "").strip())
tid = str(table_id or "").strip()
if not _is_valid_table_id(tid):
return None
p = _meta_path(tid)
if not p.exists():
return None
try:
@ -352,6 +360,29 @@ def _columns_for_sheet(meta: dict[str, Any], table_name: str) -> list[str]:
return []
def _sheet_hint_payload(meta: dict[str, Any], *, selected_sheet: str | None, table_name: str) -> dict[str, Any] | None:
sheets = meta.get("sheets") if isinstance(meta.get("sheets"), list) else []
valid_rows = [x for x in sheets if isinstance(x, dict)]
if len(valid_rows) <= 1:
return None
if str(selected_sheet or "").strip():
return None
names = [str((x or {}).get("sheet_name") or "") for x in valid_rows]
names = [x for x in names if x]
if len(names) <= 1:
return None
default_sheet_name = ""
for row in valid_rows:
if str(row.get("table_name") or "") == str(table_name):
default_sheet_name = str(row.get("sheet_name") or "")
break
return {
"available_sheets": names[:20],
"default_sheet": default_sheet_name,
"note": "Workbook contains multiple sheets; pass `sheet` to target a specific sheet.",
}
def query_table(
*,
table_id: str,
@ -361,6 +392,12 @@ def query_table(
where_contains: dict[str, str] | None = None,
sheet: str | None = None,
) -> dict[str, Any]:
if not _is_valid_table_id(table_id):
return {
"ok": False,
"error": "table_id_invalid_format",
"hint": "Use table_id from tabular_ref attachment (64-char hex id).",
}
meta = get_table_meta(table_id)
if not isinstance(meta, dict):
return {"ok": False, "error": "table_not_found"}
@ -392,6 +429,7 @@ def query_table(
out_rows, engine = _run_select_sql(db_path=db, sql=sql, params=params)
with sqlite3.connect(str(db)) as conn:
total = int(conn.execute(f'SELECT COUNT(1) AS n FROM "{table_name}"').fetchone()[0] or 0)
hint = _sheet_hint_payload(meta, selected_sheet=sheet, table_name=table_name)
return {
"ok": True,
"table_id": str(table_id),
@ -404,6 +442,7 @@ def query_table(
"limit": lim,
"offset": off,
"engine": engine,
**({"sheet_hint": hint} if isinstance(hint, dict) else {}),
}
@ -417,6 +456,12 @@ def aggregate_table(
top_n: int = 20,
sheet: str | None = None,
) -> dict[str, Any]:
if not _is_valid_table_id(table_id):
return {
"ok": False,
"error": "table_id_invalid_format",
"hint": "Use table_id from tabular_ref attachment (64-char hex id).",
}
meta = get_table_meta(table_id)
if not isinstance(meta, dict):
return {"ok": False, "error": "table_not_found"}
@ -465,7 +510,7 @@ def aggregate_table(
"value": float(r.get("metric_value") or 0.0),
}
)
return {
out = {
"ok": True,
"table_id": str(table_id),
"sheet": str(sheet or ""),
@ -477,10 +522,14 @@ def aggregate_table(
"top_n": lim,
"engine": engine,
}
hint = _sheet_hint_payload(meta, selected_sheet=sheet, table_name=table_name)
if isinstance(hint, dict):
out["sheet_hint"] = hint
return out
sql = f'SELECT {metric_sql} AS metric_value FROM "{table_name}"{where_sql} '
raw_rows, engine = _run_select_sql(db_path=db, sql=sql, params=params)
row = raw_rows[0] if raw_rows else {}
return {
out = {
"ok": True,
"table_id": str(table_id),
"sheet": str(sheet or ""),
@ -491,6 +540,10 @@ def aggregate_table(
"value": float((row.get("metric_value") if isinstance(row, dict) else 0.0) or 0.0),
"engine": engine,
}
hint = _sheet_hint_payload(meta, selected_sheet=sheet, table_name=table_name)
if isinstance(hint, dict):
out["sheet_hint"] = hint
return out
def run_table_sql(
@ -500,6 +553,12 @@ def run_table_sql(
limit: int = 200,
sheet: str | None = None,
) -> dict[str, Any]:
if not _is_valid_table_id(table_id):
return {
"ok": False,
"error": "table_id_invalid_format",
"hint": "Use table_id from tabular_ref attachment (64-char hex id).",
}
meta = get_table_meta(table_id)
if not isinstance(meta, dict):
return {"ok": False, "error": "table_not_found"}
@ -547,7 +606,7 @@ def run_table_sql(
},
}
out_rows = rows[:lim]
return {
out = {
"ok": True,
"table_id": str(table_id),
"name": str(meta.get("name") or ""),
@ -568,6 +627,10 @@ def run_table_sql(
"limit": lim,
"engine": engine,
}
hint = _sheet_hint_payload(meta, selected_sheet=sheet, table_name=table_name)
if isinstance(hint, dict):
out["sheet_hint"] = hint
return out
def analyze_table_full_scan(
@ -577,6 +640,12 @@ def analyze_table_full_scan(
sheet: str | None = None,
top_values_limit: int = 3,
) -> dict[str, Any]:
if not _is_valid_table_id(table_id):
return {
"ok": False,
"error": "table_id_invalid_format",
"hint": "Use table_id from tabular_ref attachment (64-char hex id).",
}
meta = get_table_meta(table_id)
if not isinstance(meta, dict):
return {"ok": False, "error": "table_not_found"}
@ -628,7 +697,7 @@ def analyze_table_full_scan(
item["top_values"] = [{"value": str(r[0]), "count": int(r[1] or 0)} for r in tops]
col_stats.append(item)
elapsed_ms = int((time.perf_counter() - started) * 1000)
return {
out = {
"ok": True,
"table_id": str(table_id),
"name": str(meta.get("name") or ""),
@ -645,6 +714,10 @@ def analyze_table_full_scan(
},
"engine": "builtin_sqlite_fullscan",
}
hint = _sheet_hint_payload(meta, selected_sheet=sheet, table_name=table_name)
if isinstance(hint, dict):
out["sheet_hint"] = hint
return out
__all__ = [

197
platform/files/text_attachment_store.py Normal file
View file

@ -0,0 +1,197 @@
from __future__ import annotations
import hashlib
import json
import re
import sqlite3
from pathlib import Path
from typing import Any
from oclaw.platform.config.paths import attachments_dir
DEFAULT_TEXT_INLINE_MAX_CHARS = 12_000
DEFAULT_TEXT_CHUNK_SIZE = 1_600
DEFAULT_TEXT_CHUNK_OVERLAP = 200
def _text_root() -> Path:
p = (attachments_dir() / "textual").resolve()
p.mkdir(parents=True, exist_ok=True)
return p
def _text_id(attachment_id: str, name: str) -> str:
raw = f"{attachment_id}:{name}".encode("utf-8", errors="ignore")
return hashlib.sha256(raw).hexdigest()
def _db_path(text_id: str) -> Path:
return _text_root() / f"{text_id}.sqlite"
def _meta_path(text_id: str) -> Path:
return _text_root() / f"{text_id}.meta.json"
def _is_valid_text_id(text_id: str) -> bool:
tid = str(text_id or "").strip().lower()
return bool(re.fullmatch(r"[0-9a-f]{64}", tid))
def _chunk_text(*, text: str, chunk_size: int, overlap: int) -> list[dict[str, Any]]:
body = str(text or "")
if not body:
return []
size = max(200, min(int(chunk_size or DEFAULT_TEXT_CHUNK_SIZE), 8_000))
ov = max(0, min(int(overlap or DEFAULT_TEXT_CHUNK_OVERLAP), size - 1))
chunks: list[dict[str, Any]] = []
i = 0
idx = 0
n = len(body)
while i < n:
end = min(n, i + size)
chunk = body[i:end]
chunks.append({"chunk_index": idx, "start_char": i, "end_char": end, "content": chunk})
if end >= n:
break
i = end - ov
idx += 1
return chunks
def save_text_document(
*,
attachment_id: str,
name: str,
text: str,
source_kind: str,
chunk_size: int = DEFAULT_TEXT_CHUNK_SIZE,
chunk_overlap: int = DEFAULT_TEXT_CHUNK_OVERLAP,
) -> dict[str, Any]:
tid = _text_id(attachment_id=attachment_id, name=name)
db = _db_path(tid)
meta = _meta_path(tid)
chunks = _chunk_text(text=str(text or ""), chunk_size=chunk_size, overlap=chunk_overlap)
with sqlite3.connect(str(db)) as conn:
conn.execute("DROP TABLE IF EXISTS text_chunks")
conn.execute(
(
"CREATE TABLE text_chunks ("
"chunk_index INTEGER NOT NULL, "
"start_char INTEGER NOT NULL, "
"end_char INTEGER NOT NULL, "
"content TEXT NOT NULL)"
)
)
for c in chunks:
conn.execute(
"INSERT INTO text_chunks(chunk_index,start_char,end_char,content) VALUES(?,?,?,?)",
[
int(c.get("chunk_index") or 0),
int(c.get("start_char") or 0),
int(c.get("end_char") or 0),
str(c.get("content") or ""),
],
)
conn.commit()
payload = {
"text_id": tid,
"attachment_id": str(attachment_id or ""),
"name": str(name or ""),
"source_kind": str(source_kind or "text"),
"chars": int(len(str(text or ""))),
"chunks": int(len(chunks)),
"chunk_size": int(chunk_size),
"chunk_overlap": int(chunk_overlap),
"db_path": str(db),
}
meta.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
return payload
def get_text_meta(text_id: str) -> dict[str, Any] | None:
tid = str(text_id or "").strip()
if not _is_valid_text_id(tid):
return None
p = _meta_path(tid)
if not p.exists():
return None
try:
obj = json.loads(p.read_text(encoding="utf-8"))
return obj if isinstance(obj, dict) else None
except Exception:
return None
def query_text_document(
*,
text_id: str,
query: str | None = None,
top_k: int = 5,
offset: int = 0,
) -> dict[str, Any]:
tid = str(text_id or "").strip()
if not _is_valid_text_id(tid):
return {"ok": False, "error": "text_id_invalid_format"}
meta = get_text_meta(tid)
if not isinstance(meta, dict):
return {"ok": False, "error": "text_not_found"}
db = Path(str(meta.get("db_path") or ""))
if not db.exists():
return {"ok": False, "error": "text_store_missing"}
lim = max(1, min(int(top_k or 5), 50))
off = max(0, int(offset or 0))
q = str(query or "").strip()
rows: list[dict[str, Any]] = []
with sqlite3.connect(str(db)) as conn:
conn.row_factory = sqlite3.Row
total = int(conn.execute("SELECT COUNT(1) AS n FROM text_chunks").fetchone()[0] or 0)
if q:
q_like = f"%{q.lower()}%"
sql = (
"SELECT chunk_index,start_char,end_char,content, "
"(LENGTH(LOWER(content)) - LENGTH(REPLACE(LOWER(content), LOWER(?), ''))) / MAX(1, LENGTH(?)) AS score "
"FROM text_chunks WHERE LOWER(content) LIKE ? "
"ORDER BY score DESC, chunk_index ASC LIMIT ? OFFSET ?"
)
out = conn.execute(sql, [q, q, q_like, lim, off]).fetchall()
else:
out = conn.execute(
"SELECT chunk_index,start_char,end_char,content, 0 AS score FROM text_chunks ORDER BY chunk_index ASC LIMIT ? OFFSET ?",
[lim, off],
).fetchall()
for r in out:
rows.append(
{
"chunk_index": int(r["chunk_index"] or 0),
"start_char": int(r["start_char"] or 0),
"end_char": int(r["end_char"] or 0),
"content": str(r["content"] or ""),
"score": float(r["score"] or 0.0),
}
)
return {
"ok": True,
"text_id": tid,
"name": str(meta.get("name") or ""),
"source_kind": str(meta.get("source_kind") or "text"),
"chars": int(meta.get("chars") or 0),
"chunks_total": int(meta.get("chunks") or 0),
"rows_total": int(total),
"query": q,
"rows": rows,
"top_k": lim,
"offset": off,
}
__all__ = [
"DEFAULT_TEXT_INLINE_MAX_CHARS",
"DEFAULT_TEXT_CHUNK_SIZE",
"DEFAULT_TEXT_CHUNK_OVERLAP",
"save_text_document",
"get_text_meta",
"query_text_document",
]

41
runtime/chat/agent_messages.py
View file

@ -177,6 +177,9 @@ def _summarize_unpaired_tool_content(raw: str, *, cap: int) -> str:
def _truncate_tool_context(text: str, *, lang: str) -> str:
# NOTE: this is a history-replay safety clamp only.
# Never apply this to current-turn tool rows; otherwise the model may
# incorrectly infer runtime truncation from UI-facing copy.
s = str(text or "").strip()
if not s:
return s
@ -315,6 +318,39 @@ def build_llm_messages(
),
}
)
elif att_type == "text_ref":
name = str(att.get("name") or "document")
text_id = str(att.get("text_id") or "")
chars = int(att.get("chars") or 0)
chunks = int(att.get("chunks") or 0)
source_kind = str(att.get("source_kind") or "text")
aid = str(att.get("attachment_id") or "")
content_list.append(
{
"type": "text",
"text": (
f"[LongTextAttachment]\n"
f"- name: {name}\n"
f"- text_id: {text_id}\n"
f"- attachment_id: {aid}\n"
f"- source_kind: {source_kind}\n"
f"- chars: {chars}\n"
f"- chunks: {chunks}\n"
f"- tools: query_text_attachment\n"
"- note: for detailed evidence, call `query_text_attachment` with text_id."
),
}
)
elif att_type == "video_ref":
name = str(att.get("name") or "video")
mime = str(att.get("mime") or "video/*")
aid = str(att.get("attachment_id") or "")
sz = att.get("bytes")
meta_line = f"[VideoAttachment]\n- name: {name}\n- mime: {mime}\n- attachment_id: {aid}"
if sz:
meta_line += f"\n- bytes: {sz}"
meta_line += "\n- tools: query_video_attachment"
content_list.append({"type": "text", "text": meta_line})
elif att_type == "relay_pointer":
p_uri = str(att.get("pointer_uri") or "").strip()
if not p_uri:
@ -471,7 +507,7 @@ def build_llm_messages(
tool_content_out = raw_tc_content[: max(1, cap - 80)] + "\n...<truncated>"
except Exception:
tool_content_out = raw_tc_content[: max(1, cap - 80)] + "\n...<truncated>"
if tool_context_truncate_enabled:
if tool_context_truncate_enabled and str(tool_call_id) in historical_tool_ids:
# Preserve explicit guard markers from upstream context guards.
if "_tool_result_guarded" not in str(tool_content_out or ""):
tool_content_out = _truncate_tool_context(tool_content_out, lang=lang)
@ -494,8 +530,7 @@ def build_llm_messages(
r = getattr(m, "content", "") or ""
cap2 = tool_llm_message_max_chars()
pretty2 = _summarize_unpaired_tool_content(r, cap=cap2)
if tool_context_truncate_enabled:
pretty2 = _truncate_tool_context(pretty2, lang=lang)
# Unpaired tool rows are already summarized; avoid extra 50-char clipping.
out.append(
{
"role": "assistant",

422
runtime/chat/tool_runtime.py
View file

@ -11,6 +11,7 @@ import logging
import time
import os
import threading
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed
from concurrent.futures import TimeoutError as FuturesTimeoutError
from dataclasses import dataclass
@ -20,7 +21,10 @@ from oclaw.platform.persistence.sqlite_store import SqliteStore
from oclaw.runtime.tools.base import ToolRegistry
from oclaw.platform.llm.chat_models import LLMToolCall
from oclaw.runtime.tools.tool_validation import validate_tool_arguments
from oclaw.runtime.tools.experts.workspace.workspace_base import workspace_path_access_scope
from oclaw.runtime.tools.experts.workspace.workspace_base import (
workspace_path_access_scope,
workspace_write_namespace_scope,
)
logger = logging.getLogger(__name__)
_tool_exec_log = logging.getLogger("oclaw.tool_exec")
@ -33,6 +37,170 @@ _SQL_REPLAY_COMPACT_TOOL_NAMES = {
"run_tabular_sql",
"analyze_tabular_attachment_full_scan",
}
_TABULAR_QUERY_TOOL_NAMES = {
"query_tabular_attachment",
"run_tabular_sql",
"analyze_tabular_attachment_full_scan",
}
_TEXT_QUERY_TOOL_NAMES = {
"query_text_attachment",
}
_IMAGE_QUERY_TOOL_NAMES = {
"query_image_attachment",
}
_VIDEO_QUERY_TOOL_NAMES = {
"query_video_attachment",
}
def _message_has_tabular_ref(raw_attachments: Any) -> bool:
if raw_attachments is None:
return False
obj = raw_attachments
if isinstance(raw_attachments, str):
s = str(raw_attachments or "").strip()
if not s:
return False
try:
obj = json.loads(s)
except Exception:
return False
if isinstance(obj, dict):
items = [obj]
elif isinstance(obj, list):
items = obj
else:
return False
for it in items:
if not isinstance(it, dict):
continue
if str(it.get("type") or "").strip().lower() == "tabular_ref":
return True
return False
def _message_has_text_ref(raw_attachments: Any) -> bool:
if raw_attachments is None:
return False
obj = raw_attachments
if isinstance(raw_attachments, str):
s = str(raw_attachments or "").strip()
if not s:
return False
try:
obj = json.loads(s)
except Exception:
return False
if isinstance(obj, dict):
items = [obj]
elif isinstance(obj, list):
items = obj
else:
return False
for it in items:
if not isinstance(it, dict):
continue
if str(it.get("type") or "").strip().lower() == "text_ref":
return True
return False
def _message_has_image_ref(raw_attachments: Any) -> bool:
if raw_attachments is None:
return False
obj = raw_attachments
if isinstance(raw_attachments, str):
s = str(raw_attachments or "").strip()
if not s:
return False
try:
obj = json.loads(s)
except Exception:
return False
if isinstance(obj, dict):
items = [obj]
elif isinstance(obj, list):
items = obj
else:
return False
for it in items:
if not isinstance(it, dict):
continue
t = str(it.get("type") or "").strip().lower()
if t in {"image_ref", "image", "input_image"}:
return True
return False
def _message_has_video_ref(raw_attachments: Any) -> bool:
if raw_attachments is None:
return False
obj = raw_attachments
if isinstance(raw_attachments, str):
s = str(raw_attachments or "").strip()
if not s:
return False
try:
obj = json.loads(s)
except Exception:
return False
if isinstance(obj, dict):
items = [obj]
elif isinstance(obj, list):
items = obj
else:
return False
for it in items:
if not isinstance(it, dict):
continue
t = str(it.get("type") or "").strip().lower()
if t == "video_ref":
return True
return False
def _session_has_tabular_ref(store: Any, session_id: str, *, limit: int = 300) -> bool:
try:
rows = store.get_messages(session_id=session_id, limit=max(1, int(limit)))
except Exception:
return False
for m in rows or []:
if _message_has_tabular_ref(getattr(m, "attachments", None)):
return True
return False
def _session_has_text_ref(store: Any, session_id: str, *, limit: int = 300) -> bool:
try:
rows = store.get_messages(session_id=session_id, limit=max(1, int(limit)))
except Exception:
return False
for m in rows or []:
if _message_has_text_ref(getattr(m, "attachments", None)):
return True
return False
def _session_has_image_ref(store: Any, session_id: str, *, limit: int = 300) -> bool:
try:
rows = store.get_messages(session_id=session_id, limit=max(1, int(limit)))
except Exception:
return False
for m in rows or []:
if _message_has_image_ref(getattr(m, "attachments", None)):
return True
return False
def _session_has_video_ref(store: Any, session_id: str, *, limit: int = 300) -> bool:
try:
rows = store.get_messages(session_id=session_id, limit=max(1, int(limit)))
except Exception:
return False
for m in rows or []:
if _message_has_video_ref(getattr(m, "attachments", None)):
return True
return False
def normalize_tool_result(result: Any) -> dict[str, Any]:
@ -206,6 +374,7 @@ class ToolExecutionContext:
#: If ``get_ui_session_owner`` fails, load allowlist for this (tenant, user) from the HTTP/gateway request (``metadata``).
path_policy_tenant_id: str | None = None
path_policy_user_id: str | None = None
workspace_dir: str | None = None
turn_uuid: str | None = None
@ -235,13 +404,21 @@ class ToolExecutor:
timeout_s = 30.0
def _call() -> Any:
ws_ns = ""
raw_ws = str(ctx.workspace_dir or "").strip()
if raw_ws:
try:
wp = Path(raw_ws)
ws_ns = str(wp.name or wp.stem or "").strip()
except Exception:
ws_ns = ""
with workspace_path_access_scope(
ctx.store,
ctx.session_id,
owner_fallback_session_id=ctx.workspace_owner_session_id,
allowlist_tenant_id=ctx.path_policy_tenant_id,
allowlist_user_id=ctx.path_policy_user_id,
):
), workspace_write_namespace_scope(ws_ns):
return tool.handler(tc.arguments)
if isinstance(timeout_s, (int, float)) and float(timeout_s) > 0:
@ -400,65 +577,98 @@ class ToolExecutor:
)
return counts, observed_rows
def _compact_tool_result_for_history(
*,
tool_name: str,
result: dict[str, Any],
call_index: int,
threshold: int,
observed_rows_this_call: int,
observed_rows_cumulative_in_turn: int,
) -> dict[str, Any]:
out: dict[str, Any] = {
"ok": bool(result.get("ok")),
"_history_compacted": True,
"_history_compact_reason": "repeated_tool_calls_in_turn",
"tool_name": str(tool_name or ""),
"call_index_in_turn_for_tool": int(call_index),
"compact_threshold": int(threshold),
"_tool_observed_rows_this_call": int(observed_rows_this_call),
"_tool_observed_rows_cumulative_in_turn": int(observed_rows_cumulative_in_turn),
"result_keys": sorted(list(result.keys()))[:30],
"result_bytes": int(_json_blob_size(result)),
"hint": (
"Repeated tool calls in this turn were compacted in chat history to avoid context bloat. "
"Full payload remains in tool logs."
),
"audit_note": (
"History is compacted by system optimization. If more detail is needed, continue querying "
"with the same SQL/tool parameters from this turn."
),
}
for key in ("error_code", "error", "rows_returned", "limit", "table_id", "engine"):
if key in result:
out[key] = result.get(key)
for key in ("input_sql", "executed_sql"):
v = str(result.get(key) or "").strip()
if v:
out[key] = v[:1200]
guard = result.get("sql_guard")
if isinstance(guard, dict):
out["sql_guard"] = {
"readonly_enforced": bool(guard.get("readonly_enforced")),
"auto_limit_applied": bool(guard.get("auto_limit_applied")),
"result_row_cap": int(guard.get("result_row_cap") or 0),
}
return out
_check_stop()
if not tool_uses:
return [], {}
history_summary_threshold = int(tool_history_summary_after_calls())
turn_tool_name_counts, turn_tool_observed_rows = _load_turn_tool_stats()
local_turn_tool_name_counts: dict[str, int] = {}
local_turn_tool_observed_rows: dict[str, int] = {}
local_turn_written_tool_msgs: dict[str, list[dict[str, Any]]] = {}
has_tabular_ref_in_session = _session_has_tabular_ref(ctx.store, ctx.session_id)
has_text_ref_in_session = _session_has_text_ref(ctx.store, ctx.session_id)
has_image_ref_in_session = _session_has_image_ref(ctx.store, ctx.session_id)
has_video_ref_in_session = _session_has_video_ref(ctx.store, ctx.session_id)
results_by_id: dict[str, tuple[dict[str, Any], int]] = {}
runnable_tool_uses: list[LLMToolCall] = []
sig_seen: dict[str, int] = {}
budget = max(1, min(int(signature_budget or 2), 8))
for tc in tool_uses:
if tc.name in _TABULAR_QUERY_TOOL_NAMES and not has_tabular_ref_in_session:
results_by_id[tc.id] = (
{
"ok": False,
"error_code": "tabular_ref_missing",
"error": "tabular_ref_missing",
"hint": "No tabular_ref attachment found in this session. Query tools require table_id from tabular_ref.",
},
0,
)
_trace(
"tabular_query_guard",
{
"tool_name": tc.name,
"blocked": True,
"reason": "tabular_ref_missing",
},
)
continue
if tc.name in _TEXT_QUERY_TOOL_NAMES and not has_text_ref_in_session:
results_by_id[tc.id] = (
{
"ok": False,
"error_code": "text_ref_missing",
"error": "text_ref_missing",
"hint": "No text_ref attachment found in this session. Query tools require text_id from text_ref.",
},
0,
)
_trace(
"text_query_guard",
{
"tool_name": tc.name,
"blocked": True,
"reason": "text_ref_missing",
},
)
continue
if tc.name in _IMAGE_QUERY_TOOL_NAMES and not has_image_ref_in_session:
results_by_id[tc.id] = (
{
"ok": False,
"error_code": "image_ref_missing",
"error": "image_ref_missing",
"hint": "No image_ref attachment found in this session. Query tools require attachment_id from image_ref.",
},
0,
)
_trace(
"image_query_guard",
{
"tool_name": tc.name,
"blocked": True,
"reason": "image_ref_missing",
},
)
continue
if tc.name in _VIDEO_QUERY_TOOL_NAMES and not has_video_ref_in_session:
results_by_id[tc.id] = (
{
"ok": False,
"error_code": "video_ref_missing",
"error": "video_ref_missing",
"hint": "No video_ref attachment found in this session. Query tools require attachment_id from video_ref.",
},
0,
)
_trace(
"video_query_guard",
{
"tool_name": tc.name,
"blocked": True,
"reason": "video_ref_missing",
},
)
continue
sig = f"{tc.name}:{self._json_dumps_safe(dict(tc.arguments or {}))}"
count = int(sig_seen.get(sig, 0))
if count >= budget:
@ -539,27 +749,14 @@ class ToolExecutor:
duration_ms=duration_ms,
)
tool_log_write_ms = int((time.perf_counter() - t_db1) * 1000)
# Full payload stays in tool_log; chat history must stay under provider per-message limits.
# Keep full payload during the active turn. History compaction is deferred
# until the turn finishes, so current-round model context remains lossless.
t_trunc = time.perf_counter()
observed_rows_this_call = int(_estimate_observed_rows(result))
result_for_llm = truncate_tool_result_for_llm_messages(result)
should_compact_history = tc.name in _SQL_REPLAY_COMPACT_TOOL_NAMES
if history_summary_threshold > 0 and should_compact_history:
prior = int(turn_tool_name_counts.get(tc.name, 0))
result_for_llm = dict(result or {})
if tc.name in _SQL_REPLAY_COMPACT_TOOL_NAMES:
current = int(local_turn_tool_name_counts.get(tc.name, 0))
call_index = prior + current + 1
prior_rows = int(turn_tool_observed_rows.get(tc.name, 0))
current_rows = int(local_turn_tool_observed_rows.get(tc.name, 0))
observed_rows_cumulative_in_turn = prior_rows + current_rows + observed_rows_this_call
if call_index >= history_summary_threshold:
result_for_llm = _compact_tool_result_for_history(
tool_name=tc.name,
result=result,
call_index=call_index,
threshold=history_summary_threshold,
observed_rows_this_call=observed_rows_this_call,
observed_rows_cumulative_in_turn=observed_rows_cumulative_in_turn,
)
local_turn_tool_name_counts[tc.name] = current + 1
local_turn_tool_observed_rows[tc.name] = current_rows + observed_rows_this_call
trunc_ms = int((time.perf_counter() - t_trunc) * 1000)
@ -576,47 +773,6 @@ class ToolExecutor:
)
tool_msg_write_ms = int((time.perf_counter() - t_db2) * 1000)
tool_messages.append({"role": "tool", "tool_call_id": tc.id, "content": tool_content, "name": tc.name})
tool_messages_idx = len(tool_messages) - 1
call_index_for_tool = int(turn_tool_name_counts.get(tc.name, 0)) + int(local_turn_tool_name_counts.get(tc.name, 0))
local_turn_written_tool_msgs.setdefault(tc.name, []).append(
{
"message_id": int(getattr(msg_row, "id", 0) or 0),
"tool_messages_idx": int(tool_messages_idx),
"result": dict(result or {}),
"observed_rows": int(observed_rows_this_call),
"call_index": int(call_index_for_tool),
"compacted": bool(isinstance(result_for_llm, dict) and result_for_llm.get("_history_compacted")),
}
)
# When threshold is reached for one SQL tool in the turn, retro-compact earlier same-tool tool messages too.
if history_summary_threshold > 0 and should_compact_history and call_index_for_tool >= history_summary_threshold:
running_rows = int(turn_tool_observed_rows.get(tc.name, 0))
entries = list(local_turn_written_tool_msgs.get(tc.name) or [])
for ent in entries:
running_rows += int(ent.get("observed_rows") or 0)
compacted_payload = _compact_tool_result_for_history(
tool_name=tc.name,
result=dict(ent.get("result") or {}),
call_index=int(ent.get("call_index") or 0),
threshold=history_summary_threshold,
observed_rows_this_call=int(ent.get("observed_rows") or 0),
observed_rows_cumulative_in_turn=int(running_rows),
)
compacted_content = self._json_dumps_safe(compacted_payload)
if not bool(ent.get("compacted")):
try:
ctx.store.update_message_content(
session_id=ctx.session_id,
message_id=int(ent.get("message_id") or 0),
content=compacted_content,
event_payload={"tool_name": tc.name, "observed_rows": int(ent.get("observed_rows") or 0)},
)
except Exception:
pass
ent["compacted"] = True
ti = int(ent.get("tool_messages_idx") or -1)
if 0 <= ti < len(tool_messages):
tool_messages[ti]["content"] = compacted_content
_trace(
"tool_result",
{
@ -650,6 +806,57 @@ class ToolExecutor:
on_tool_ui("tool_use_result", payload)
return tool_messages, results_by_id
def compact_turn_tool_messages_for_storage(
*,
store: Any,
session_id: str,
turn_uuid: str | None,
) -> dict[str, int]:
"""Compact persisted tool messages after turn completion.
This intentionally runs *after* the active turn so current-round model
context is not affected by truncation/compaction.
"""
tid = str(turn_uuid or "").strip()
if not tid:
return {"scanned": 0, "updated": 0}
try:
rows = store.get_messages(session_id=session_id, limit=800)
except Exception:
return {"scanned": 0, "updated": 0}
scanned = 0
updated = 0
for m in rows or []:
if str(getattr(m, "role", "") or "") != "tool":
continue
if str(getattr(m, "turn_uuid", "") or "") != tid:
continue
scanned += 1
raw = str(getattr(m, "content", "") or "")
if not raw.strip():
continue
try:
obj = json.loads(raw)
except Exception:
continue
if not isinstance(obj, dict):
continue
compacted = truncate_tool_result_for_llm_messages(obj)
if compacted == obj:
continue
try:
store.update_message_content(
session_id=session_id,
message_id=int(getattr(m, "id", 0) or 0),
content=json.dumps(compacted, ensure_ascii=False, default=str),
event_payload=getattr(m, "event_payload", None),
)
updated += 1
except Exception:
continue
return {"scanned": int(scanned), "updated": int(updated)}
__all__ = [
"ToolExecutionConfig",
"ToolExecutionContext",
@ -658,4 +865,5 @@ __all__ = [
"partition_tool_use_batches",
"tool_llm_message_max_chars",
"truncate_tool_result_for_llm_messages",
"compact_turn_tool_messages_for_storage",
]

316
runtime/direct_loop.py
View file

@ -8,6 +8,7 @@ import uuid
import copy
import threading
from dataclasses import dataclass
from pathlib import Path
from types import SimpleNamespace
from typing import Any, Callable, Optional
@ -24,6 +25,8 @@ from oclaw.runtime.tools.base import ToolRegistry
from oclaw.runtime.hooks_runtime import trigger_hook_event
_OCLAW_TOOL_RESULT_HARD_CAP_CHARS = 24_000
_OCLAW_ATTACHMENT_TEXT_REPLAY_CAP_CHARS = 4_000
_OCLAW_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS = 4_000
_DIRECT_LOOP_OC_STAGE: dict[str, str] = {
"tool_wire_filter": "wire_filter",
@ -39,6 +42,82 @@ _TOOL_WIRE_LAST_WARM_ROLES: tuple[str, ...] = ()
_TOOL_WIRE_LAST_WARM_COUNT: int = 0
def _safe_int(raw: Any, default: int, *, min_value: int = 1, max_value: int = 2_000_000) -> int:
try:
value = int(raw)
except Exception:
return default
if value < min_value:
return default
return min(value, max_value)
def _oclaw_config_path() -> Path:
raw = str(os.getenv("AIA_OCLAW_CONFIG_PATH") or "").strip()
if raw:
p = Path(raw)
return p if p.is_absolute() else p.resolve()
return Path(__file__).resolve().parents[1] / "oclaw.json"
def _image_tool_result_replay_cap_chars(store: Any) -> int:
default = _OCLAW_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS
raw_setting = ""
try:
raw_setting = str(store.get_setting("AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS") or "").strip()
except Exception:
raw_setting = ""
if raw_setting:
return _safe_int(raw_setting, default, min_value=600, max_value=30_000)
raw_env = str(os.getenv("AIA_IMAGE_TOOL_RESULT_REPLAY_CAP_CHARS") or "").strip()
if raw_env:
return _safe_int(raw_env, default, min_value=600, max_value=30_000)
try:
cfg_path = _oclaw_config_path()
if cfg_path.exists() and cfg_path.is_file():
obj = json.loads(cfg_path.read_text(encoding="utf-8"))
tab = (
(((obj.get("plugins") or {}).get("entries") or {}).get("memory-wiki") or {})
.get("auto", {})
.get("attachments", {})
.get("tabular", {})
)
if isinstance(tab, dict):
return _safe_int(tab.get("image_result_replay_cap_chars"), default, min_value=600, max_value=30_000)
except Exception:
pass
return default
def _video_tool_result_replay_cap_chars(store: Any) -> int:
default = 4_000
raw_setting = ""
try:
raw_setting = str(store.get_setting("AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS") or "").strip()
except Exception:
raw_setting = ""
if raw_setting:
return _safe_int(raw_setting, default, min_value=600, max_value=30_000)
raw_env = str(os.getenv("AIA_VIDEO_TOOL_RESULT_REPLAY_CAP_CHARS") or "").strip()
if raw_env:
return _safe_int(raw_env, default, min_value=600, max_value=30_000)
try:
cfg_path = _oclaw_config_path()
if cfg_path.exists() and cfg_path.is_file():
obj = json.loads(cfg_path.read_text(encoding="utf-8"))
tab = (
(((obj.get("plugins") or {}).get("entries") or {}).get("memory-wiki") or {})
.get("auto", {})
.get("attachments", {})
.get("tabular", {})
)
if isinstance(tab, dict):
return _safe_int(tab.get("video_result_replay_cap_chars"), default, min_value=600, max_value=30_000)
except Exception:
pass
return default
def _tool_wire_freeze_enabled(store: Any) -> bool:
raw = ""
try:
@ -218,6 +297,7 @@ def _guard_tool_results_for_llm_context(
run_id: str | None = None,
attempt_no: int | None = None,
lang: str = "",
active_turn_uuid: str | None = None,
) -> list[Any]:
"""Hard-guard overlarge `role=tool` message contents before sending to model.
@ -225,28 +305,87 @@ def _guard_tool_results_for_llm_context(
in-flight LLM context to prevent provider context overflow spirals.
"""
cap = max(4096, min(int(hard_cap_chars or _OCLAW_TOOL_RESULT_HARD_CAP_CHARS), 500_000))
image_cap = _image_tool_result_replay_cap_chars(store)
video_cap = _video_tool_result_replay_cap_chars(store)
out: list[Any] = []
for m in store_messages or []:
role = str(getattr(m, "role", "") or "")
if role != "tool":
out.append(m)
continue
raw = str(getattr(m, "content", "") or "")
if len(raw) <= cap:
if str(getattr(m, "turn_uuid", "") or "") == str(active_turn_uuid or "") and str(active_turn_uuid or "").strip():
out.append(m)
continue
# Best-effort parse tool JSON for a minimal summary.
raw = str(getattr(m, "content", "") or "")
# Best-effort parse tool JSON for image-query specific guard and overflow metadata.
ok = None
error_code = ""
error = ""
obj: dict[str, Any] | None = None
try:
obj = json.loads(raw)
if isinstance(obj, dict):
parsed = json.loads(raw)
if isinstance(parsed, dict):
obj = parsed
ok = obj.get("ok")
error_code = str(obj.get("error_code") or "").strip()
error = str(obj.get("error") or "").strip()
except Exception:
obj = None
if isinstance(obj, dict):
task = str(obj.get("task") or "").strip().lower()
text = str(obj.get("text") or "")
has_attachment_id = bool(str(obj.get("attachment_id") or "").strip())
# Guard image describe/OCR result replay aggressively to avoid long visual transcripts
# occupying context across future rounds.
if task in {"describe", "ocr"} and has_attachment_id and len(text) > image_cap:
preview = text[:image_cap] + "\n...<image_tool_result_truncated_for_context_replay>"
guarded_obj = dict(obj)
guarded_obj["text"] = preview
guarded_obj["_image_tool_result_guarded"] = True
guarded_obj["image_result_original_chars"] = len(text)
guarded_obj["image_result_replay_cap_chars"] = image_cap
guarded_obj["image_result_hint"] = (
"Image analysis result was truncated for context replay. "
"Refine query_image_attachment(question=...) for narrower evidence. / "
"图片分析结果在上下文回放中已截断，请缩小 query_image_attachment 的问题范围。"
)
guarded = _json_dumps_safe(guarded_obj)
out.append(
SimpleNamespace(
id=getattr(m, "id", 0),
session_id=getattr(m, "session_id", session_id),
role="tool",
content=guarded,
tool_calls=getattr(m, "tool_calls", None),
timestamp=getattr(m, "timestamp", ""),
attachments=getattr(m, "attachments", None),
)
)
continue
# Guard video transcript replay similarly (usually long).
if str(obj.get("task") or "").strip().lower() == "transcript" and has_attachment_id and len(text) > video_cap:
preview = text[:video_cap] + "\n...<video_tool_result_truncated_for_context_replay>"
guarded_obj = dict(obj)
guarded_obj["text"] = preview
guarded_obj["_video_tool_result_guarded"] = True
guarded_obj["video_result_original_chars"] = len(text)
guarded_obj["video_result_replay_cap_chars"] = video_cap
guarded = _json_dumps_safe(guarded_obj)
out.append(
SimpleNamespace(
id=getattr(m, "id", 0),
session_id=getattr(m, "session_id", session_id),
role="tool",
content=guarded,
tool_calls=getattr(m, "tool_calls", None),
timestamp=getattr(m, "timestamp", ""),
attachments=getattr(m, "attachments", None),
)
)
continue
if len(raw) <= cap:
out.append(m)
continue
preview = raw[: max(1, min(4000, cap - 400))] + "\n...<tool_result_guard_truncated>"
guarded_obj = {
"ok": bool(ok) if ok is not None else None,
@ -294,6 +433,118 @@ def _guard_tool_results_for_llm_context(
return out
def _guard_text_attachments_for_llm_context(
*,
store_messages: list[Any],
cap_chars: int,
active_turn_uuid: str | None = None,
) -> list[Any]:
"""Guard overlarge user text attachments for model context replay.
This does NOT rewrite DB history. It only guards the in-flight LLM context to
prevent large attachments from overwhelming context windows.
"""
cap = max(800, min(int(cap_chars or _OCLAW_ATTACHMENT_TEXT_REPLAY_CAP_CHARS), 80_000))
out: list[Any] = []
for m in store_messages or []:
role = str(getattr(m, "role", "") or "")
if role != "user":
out.append(m)
continue
# Never guard the active user turn.
if str(getattr(m, "turn_uuid", "") or "") == str(active_turn_uuid or "") and str(active_turn_uuid or "").strip():
out.append(m)
continue
raw_att = getattr(m, "attachments", None)
if not raw_att:
out.append(m)
continue
try:
att_obj = json.loads(raw_att) if isinstance(raw_att, str) else raw_att
except Exception:
out.append(m)
continue
if isinstance(att_obj, dict):
atts = [att_obj]
elif isinstance(att_obj, list):
atts = att_obj
else:
out.append(m)
continue
has_text_ref = any(
isinstance(a, dict) and str(a.get("type") or "").strip().lower() == "text_ref" for a in atts
)
changed = False
next_atts: list[dict[str, Any]] = []
for a in atts:
if not isinstance(a, dict):
continue
if str(a.get("type") or "").strip().lower() != "text":
next_atts.append(a)
continue
content = str(a.get("content") or "")
# If this user message already has a text_ref, keep inline text very small in replay context.
# The model can retrieve evidence via query_text_attachment(text_id=...).
if has_text_ref and content:
changed = True
name = str(a.get("name") or "attachment")
next_atts.append(
{
**a,
"content": (
"# Attachment (collapsed; text_ref available)\n"
f"- name: {name}\n"
"- note: use `query_text_attachment` with `text_id` from `text_ref` for details.\n"
"...<attachment_collapsed_for_context_replay>"
),
"_attachment_context_guarded": True,
"_attachment_context_collapsed": True,
}
)
continue
if len(content) <= cap:
next_atts.append(a)
continue
changed = True
name = str(a.get("name") or "attachment")
hint_lines = [
"# Attachment (summarized for context replay)",
f"- name: {name}",
f"- original_chars: {len(content)}",
f"- replay_cap_chars: {cap}",
]
if has_text_ref:
hint_lines.append("- note: use `query_text_attachment` with `text_id` from `text_ref` for details.")
else:
hint_lines.append("- note: attachment was large; re-upload or provide a smaller excerpt if needed.")
preview = content[: min(1200, cap)]
next_atts.append(
{
**a,
"content": "\n".join(hint_lines) + "\n\n## Preview\n" + preview + "\n\n...<attachment_truncated_for_context_replay>",
"_attachment_context_guarded": True,
}
)
if not changed:
out.append(m)
continue
out.append(
SimpleNamespace(
id=getattr(m, "id", 0),
session_id=getattr(m, "session_id", ""),
role="user",
content=getattr(m, "content", ""),
tool_calls=getattr(m, "tool_calls", None),
timestamp=getattr(m, "timestamp", ""),
attachments=next_atts,
turn_uuid=getattr(m, "turn_uuid", ""),
event_type=getattr(m, "event_type", ""),
event_payload=getattr(m, "event_payload", None),
)
)
return out
def _check_stop(should_stop: Optional[Callable[[], bool]]) -> None:
if should_stop and should_stop():
raise RuntimeError("generation interrupted by user")
@ -318,6 +569,7 @@ def _build_model_context(
skill_binding_role: str | None = None,
user_text: str = "",
prompt_build_context: dict[str, Any] | None = None,
active_turn_uuid: str | None = None,
) -> list[dict[str, Any]]:
rows = store.get_messages(session_id=session_id, limit=int(max_messages))
rows = _guard_tool_results_for_llm_context(
@ -330,6 +582,12 @@ def _build_model_context(
run_id=run_id,
attempt_no=attempt_no,
lang=lang,
active_turn_uuid=active_turn_uuid,
)
rows = _guard_text_attachments_for_llm_context(
store_messages=rows,
cap_chars=_OCLAW_ATTACHMENT_TEXT_REPLAY_CAP_CHARS,
active_turn_uuid=active_turn_uuid,
)
final_system = build_oclaw_executor_system_prompt(
store=store,
@ -672,10 +930,11 @@ def run_oclaw_direct_loop(
skill_binding_role: str | None = None,
wire_policy_role: str | None = None,
prompt_build_context: dict[str, Any] | None = None,
turn_uuid: str | None = None,
) -> TurnRunOutcome:
"""A minimal oclaw-style loop: model -> tool_uses -> execute -> tool_results -> continue."""
_check_stop(should_stop)
turn_uuid = str(uuid.uuid4())
turn_uuid = str(turn_uuid or "").strip() or str(uuid.uuid4())
if persist_user_message:
store.add_message(
session_id=session_id,
@ -689,10 +948,12 @@ def run_oclaw_direct_loop(
skill_exec = SkillExecutor(config=ToolExecutionConfig(max_workers=max(1, min(int(max_tool_workers or 8), 32))))
tool_traces: list[dict[str, Any]] = []
final_text = ""
hit_tool_round_limit = False
base_url = str(getattr(model, "base_url", "") or "")
for round_idx in range(max(1, int(max_tool_rounds or 1))):
max_rounds = max(1, int(max_tool_rounds or 1))
for round_idx in range(max_rounds):
_check_stop(should_stop)
if on_progress:
on_progress(f"oclaw: think ({round_idx + 1})…")
@ -715,6 +976,7 @@ def run_oclaw_direct_loop(
skill_binding_role=skill_binding_role,
user_text=str(user_text or ""),
prompt_build_context=prompt_build_context,
active_turn_uuid=turn_uuid,
)
llm_tools = _prepare_llm_tools(
store=store,
@ -744,6 +1006,10 @@ def run_oclaw_direct_loop(
final_text = step.assistant_text
if not step.llm_tool_calls:
break
if round_idx == (max_rounds - 1):
# Reached tool-round cap with pending tool calls. Execute this batch, then
# force one no-tool synthesis pass to guarantee a visible assistant body.
hit_tool_round_limit = True
elapsed_ms, results_by_id = _execute_tool_step(
skill_exec=skill_exec,
@ -783,6 +1049,42 @@ def run_oclaw_direct_loop(
if on_progress:
on_progress(f"oclaw: tools done ({elapsed_ms}ms)")
if hit_tool_round_limit:
_check_stop(should_stop)
if on_progress:
on_progress("oclaw: finalize…")
msgs = _build_model_context(
store=store,
session_id=session_id,
max_messages=max_messages,
system_prompt=system_prompt,
model=model,
lang=lang,
memory_context=memory_context,
trace_id=trace_id,
parent_span_id=parent_span_id,
tools=tools,
base_url=base_url,
run_id=run_id,
attempt_no=attempt_no,
workspace_dir=workspace_dir,
skill_binding_role=skill_binding_role,
user_text=str(user_text or ""),
prompt_build_context=prompt_build_context,
active_turn_uuid=turn_uuid,
)
# Final pass forbids extra tool calls; model must synthesize answer.
resp = model.chat(msgs, [], on_token=on_token)
step = _persist_assistant_step(
store=store,
session_id=session_id,
turn_uuid=turn_uuid,
assistant_text=str(getattr(resp, "content", "") or ""),
reasoning_text=str(getattr(resp, "reasoning_content", "") or ""),
llm_tool_calls=[],
)
final_text = step.assistant_text
return TurnRunOutcome(
final_text=str(final_text or ""),
tool_traces=tuple(tool_traces),

99
runtime/gateway.py
View file

@ -34,6 +34,7 @@ from oclaw.runtime.memory_stage import after_turn_memory
from oclaw.runtime.router import decide_route
from oclaw.runtime.worker import ensure_worker_started
from oclaw.runtime.orchestration.trace import new_span_id, new_trace_id
from oclaw.runtime.chat.tool_runtime import compact_turn_tool_messages_for_storage
_OC_STAGE_BY_EVENT: dict[str, str] = {
"gateway_received": "ingress",
@ -91,6 +92,7 @@ class OclawGateway:
start = t.find("{")
if start < 0:
return None
executed_turn_uuid = ""
try:
obj, _end = json.JSONDecoder().raw_decode(t[start:])
except Exception:
@ -414,6 +416,36 @@ class OclawGateway:
return True
return False
@staticmethod
def _has_text_ref_attachments(msg: StandardMessage) -> bool:
atts = msg.attachments if isinstance(msg.attachments, list) else []
for a in atts:
if isinstance(a, dict) and str(a.get("type") or "").strip().lower() == "text_ref":
return True
return False
@staticmethod
def _has_image_ref_attachments(msg: StandardMessage) -> bool:
atts = msg.attachments if isinstance(msg.attachments, list) else []
for a in atts:
if not isinstance(a, dict):
continue
t = str(a.get("type") or "").strip().lower()
if t in {"image_ref", "image", "input_image"}:
return True
return False
@staticmethod
def _has_video_ref_attachments(msg: StandardMessage) -> bool:
atts = msg.attachments if isinstance(msg.attachments, list) else []
for a in atts:
if not isinstance(a, dict):
continue
t = str(a.get("type") or "").strip().lower()
if t == "video_ref":
return True
return False
@staticmethod
def _tabular_query_system_hint(lang: str) -> str:
limits = OclawGateway._tabular_limits_from_config()
@ -431,6 +463,38 @@ class OclawGateway:
"如果需要更多行或更细节，请通过数据库工具（`query_tabular_attachment` / `run_tabular_sql`）结合 table_id 查询。"
)
@staticmethod
def _text_query_system_hint(lang: str) -> str:
if str(lang or "").startswith("en"):
return (
"For long text attachments: context may contain only summary/preview. "
"For detailed evidence, use `query_text_attachment` with `text_id` from `text_ref` attachment."
)
return (
"对于长文本附件：上下文可能只包含摘要/预览。"
"如需细节证据，请使用 `text_ref` 提供的 text_id 调用 `query_text_attachment`。"
)
@staticmethod
def _image_query_system_hint(lang: str) -> str:
if str(lang or "").startswith("en"):
return (
"For image attachments: use `query_image_attachment` with attachment_id "
"for OCR/description when visual evidence is required."
)
return (
"对于图片附件：如需 OCR 或图像细节，请使用 attachment_id 调用 `query_image_attachment`。"
)
@staticmethod
def _video_query_system_hint(lang: str) -> str:
if str(lang or "").startswith("en"):
return (
"For video attachments: use `query_video_attachment` with attachment_id from `video_ref` "
"to get metadata or transcript (if enabled)."
)
return "对于视频附件：请使用 `video_ref` 提供的 attachment_id 调用 `query_video_attachment` 获取元信息/转写。"
@staticmethod
def _tabular_limits_from_config() -> dict[str, int]:
cfg_path_raw = str(os.getenv("AIA_OCLAW_CONFIG_PATH") or "").strip()
@ -767,6 +831,25 @@ class OclawGateway:
},
started_at=t0,
)
if str(manager_instruction_text or "").strip() and not bool(manager_memory_mode):
try:
assignment_title = "Task assignment" if str(lang or "").startswith("en") else "任务分配"
assignment_text = (
f"{assignment_title}\n"
f"specialist={str(manager_specialist or '')}\n"
f"instruction:\n{str(manager_instruction_text or '').strip()}"
)
self.store.add_message(
session_id=msg.session_id,
tenant_id=msg.tenant_id,
user_id=msg.user_id,
role="assistant",
content=assignment_text,
tool_calls=None,
event_type="reasoning",
)
except Exception:
pass
# Build specialist/dynamic executor and dispatch only manager instruction to it.
specialist_input_msg = StandardMessage(
session_id=msg.session_id,
@ -984,6 +1067,12 @@ class OclawGateway:
sys_prompt = str(getattr(selected_executor, "system_prompt", "") or "")
if self._has_tabular_ref_attachments(msg):
sys_prompt = f"{sys_prompt}\n\n{self._tabular_query_system_hint(lang)}".strip()
if self._has_text_ref_attachments(msg):
sys_prompt = f"{sys_prompt}\n\n{self._text_query_system_hint(lang)}".strip()
if self._has_image_ref_attachments(msg):
sys_prompt = f"{sys_prompt}\n\n{self._image_query_system_hint(lang)}".strip()
if self._has_video_ref_attachments(msg):
sys_prompt = f"{sys_prompt}\n\n{self._video_query_system_hint(lang)}".strip()
def _get_int_setting(key: str, default: int, lo: int, hi: int) -> int:
try:
@ -1042,6 +1131,7 @@ class OclawGateway:
wire_policy_role="manager" if interaction_mode == "comprehensive" else str(requested_specialist),
),
)
executed_turn_uuid = str(getattr(core_out.outcome, "turn_uuid", "") or "")
specialist_reply = str(core_out.outcome.final_text or "")
if manager_memory_mode:
# manager_memory: write memory silently, but keep dialog output independent.
@ -1095,6 +1185,15 @@ class OclawGateway:
)
except Exception:
pass
if str(executed_turn_uuid or "").strip():
try:
compact_turn_tool_messages_for_storage(
store=self.store,
session_id=msg.session_id,
turn_uuid=executed_turn_uuid,
)
except Exception:
pass
_trace_local(
event_type="response_sent",
payload={"ok": bool(str(reply or "").strip()), "elapsed_ms": elapsed_ms, "mode": "sync_direct"},

75
runtime/tools/experts/generalist/image_query.py Normal file
View file

@ -0,0 +1,75 @@
from __future__ import annotations
from typing import Any
from oclaw.platform.files.attachment_assets import attachment_id_to_data_url
from oclaw.platform.llm.image_message_client import send_image_messages
from oclaw.runtime.tools.base import ToolSpec
def query_image_attachment_tool() -> ToolSpec:
def handler(args: dict[str, Any]) -> dict[str, Any]:
attachment_id = str(args.get("attachment_id") or "").strip()
if not attachment_id:
return {"ok": False, "error": "attachment_id_required"}
task = str(args.get("task") or "describe").strip().lower()
question = str(args.get("question") or "").strip()
if task not in {"describe", "ocr"}:
return {"ok": False, "error": "invalid_task"}
data_url = attachment_id_to_data_url(attachment_id=attachment_id)
if not data_url:
return {"ok": False, "error": "attachment_not_found"}
prompt = (
(
"请详细描述这张图片的主要内容、对象、场景和可见文字。"
"回答请使用要点列表，避免臆测。"
)
if task == "describe"
else (
"请只提取图片中可见文字并按阅读顺序输出。"
"如果有表格，保持行列结构；不确定的内容标注为[unclear]。"
)
)
if question:
prompt = f"{prompt}\n\n用户问题：{question}"
out = send_image_messages(images=[data_url], prompt=prompt)
if not bool(out.get("ok")):
return {
"ok": False,
"error": str(out.get("error") or "image_query_failed"),
"task": task,
"attachment_id": attachment_id,
}
text = str(out.get("text") or "")
if len(text) > 12_000:
text = text[:12_000] + "\n\n...[truncated image analysis output]"
return {
"ok": True,
"task": task,
"attachment_id": attachment_id,
"text": text,
"input_kind": list(out.get("input_kind") or []),
"backend_shape": str(out.get("backend_shape") or ""),
}
return ToolSpec(
name="query_image_attachment",
description="Analyze an uploaded image by attachment_id (describe or OCR).",
parameters={
"type": "object",
"properties": {
"attachment_id": {"type": "string"},
"task": {"type": "string", "enum": ["describe", "ocr"]},
"question": {"type": "string"},
},
"required": ["attachment_id"],
"additionalProperties": False,
},
handler=handler,
read_only=True,
tags=frozenset({"image", "read"}),
)
__all__ = ["query_image_attachment_tool"]

41
runtime/tools/experts/generalist/text_query.py Normal file
View file

@ -0,0 +1,41 @@
from __future__ import annotations
from typing import Any
from oclaw.platform.files.text_attachment_store import query_text_document
from oclaw.runtime.tools.base import ToolSpec
def query_text_attachment_tool() -> ToolSpec:
def handler(args: dict[str, Any]) -> dict[str, Any]:
text_id = str(args.get("text_id") or "").strip()
if not text_id:
return {"ok": False, "error": "text_id_required"}
return query_text_document(
text_id=text_id,
query=str(args.get("query") or "").strip() or None,
top_k=int(args.get("top_k") or 5),
offset=int(args.get("offset") or 0),
)
return ToolSpec(
name="query_text_attachment",
description="Query long text attachment chunks by text_id with optional keyword search.",
parameters={
"type": "object",
"properties": {
"text_id": {"type": "string"},
"query": {"type": "string"},
"top_k": {"type": "integer", "minimum": 1, "maximum": 50},
"offset": {"type": "integer", "minimum": 0},
},
"required": ["text_id"],
"additionalProperties": False,
},
handler=handler,
read_only=True,
)
__all__ = ["query_text_attachment_tool"]

278
runtime/tools/experts/generalist/video_query.py Normal file
View file

@ -0,0 +1,278 @@
from __future__ import annotations
import os
import subprocess
import tempfile
import io
import json
from pathlib import Path
from typing import Any
from oclaw.platform.files.attachment_assets import AttachmentAssetStore
from oclaw.platform.files.text_attachment_store import (
DEFAULT_TEXT_CHUNK_OVERLAP,
DEFAULT_TEXT_CHUNK_SIZE,
save_text_document,
)
from oclaw.runtime.extensions.openai.api import OPENAI_DEFAULT_AUDIO_TRANSCRIPTION_MODEL
from oclaw.runtime.tools.base import ToolSpec
def _ffmpeg_exists() -> bool:
try:
p = subprocess.run(["ffmpeg", "-version"], capture_output=True, text=True, timeout=3)
return p.returncode == 0
except Exception:
return False
def _ffprobe_json(path: Path) -> dict[str, Any] | None:
try:
p = subprocess.run(
[
"ffprobe",
"-v",
"error",
"-print_format",
"json",
"-show_format",
"-show_streams",
str(path),
],
capture_output=True,
text=True,
timeout=8,
)
if p.returncode != 0:
return None
obj = json.loads(p.stdout or "{}")
return obj if isinstance(obj, dict) else None
except Exception:
return None
def _safe_int(raw: Any, default: int, *, min_value: int = 1, max_value: int = 2_000_000) -> int:
try:
value = int(raw)
except Exception:
return default
if value < min_value:
return default
return min(value, max_value)
def _oclaw_config_path() -> Path:
raw = str(os.getenv("AIA_OCLAW_CONFIG_PATH") or "").strip()
if raw:
p = Path(raw)
return p if p.is_absolute() else p.resolve()
return Path(__file__).resolve().parents[4] / "oclaw.json"
def _video_transcript_chunk_defaults() -> tuple[int, int]:
size = DEFAULT_TEXT_CHUNK_SIZE
overlap = DEFAULT_TEXT_CHUNK_OVERLAP
try:
cfg_path = _oclaw_config_path()
if cfg_path.exists() and cfg_path.is_file():
obj = json.loads(cfg_path.read_text(encoding="utf-8"))
tab = (
(((obj.get("plugins") or {}).get("entries") or {}).get("memory-wiki") or {})
.get("auto", {})
.get("attachments", {})
.get("tabular", {})
)
if isinstance(tab, dict):
size = _safe_int(tab.get("video_transcript_chunk_size"), size, min_value=200, max_value=8_000)
overlap = _safe_int(tab.get("video_transcript_chunk_overlap"), overlap, min_value=0, max_value=4_000)
except Exception:
pass
overlap = max(0, min(overlap, max(0, size - 1)))
return size, overlap
def _normalized_video_meta(ffprobe_obj: dict[str, Any] | None) -> dict[str, Any]:
out: dict[str, Any] = {}
if not isinstance(ffprobe_obj, dict):
return out
fmt = ffprobe_obj.get("format") if isinstance(ffprobe_obj.get("format"), dict) else {}
streams = ffprobe_obj.get("streams") if isinstance(ffprobe_obj.get("streams"), list) else []
if isinstance(fmt, dict) and fmt.get("duration") is not None:
try:
out["duration_sec"] = float(fmt.get("duration"))
except Exception:
pass
for s in streams:
if not isinstance(s, dict):
continue
if str(s.get("codec_type") or "") != "video":
continue
try:
if s.get("width") is not None:
out["width"] = int(s.get("width"))
if s.get("height") is not None:
out["height"] = int(s.get("height"))
except Exception:
pass
fr = str(s.get("avg_frame_rate") or s.get("r_frame_rate") or "").strip()
if fr and fr != "0/0" and "/" in fr:
try:
a, b = fr.split("/", 1)
fa = float(a)
fb = float(b)
if fb:
out["fps"] = fa / fb
except Exception:
pass
break
return out
def query_video_attachment_tool() -> ToolSpec:
def handler(args: dict[str, Any]) -> dict[str, Any]:
attachment_id = str(args.get("attachment_id") or "").strip()
task = str(args.get("task") or "meta").strip().lower()
lang = str(args.get("lang") or "").strip().lower()
if not attachment_id:
return {"ok": False, "error": "attachment_id_required"}
if task not in {"meta", "transcript"}:
return {"ok": False, "error": "invalid_task"}
store = AttachmentAssetStore()
p = store.get_local_path(attachment_id)
meta = store.get_meta(attachment_id)
if p is None:
return {"ok": False, "error": "attachment_not_found"}
# Basic metadata (no heavy deps). Prefer ffprobe if present.
if task == "meta":
fp = _ffprobe_json(p)
norm = _normalized_video_meta(fp)
return {
"ok": True,
"task": "meta",
"attachment_id": attachment_id,
"name": (meta.name if meta else p.name),
"mime": (meta.mime if meta else "video/*"),
"bytes": int(meta.bytes if meta else (p.stat().st_size if p.exists() else 0)),
"duration_sec": norm.get("duration_sec"),
"width": norm.get("width"),
"height": norm.get("height"),
"fps": norm.get("fps"),
"ffprobe": fp if fp else None,
"note": "Use task=transcript to extract audio transcript (requires ffmpeg + OpenAI key).",
}
# transcript: extract audio then transcribe via OpenAI, store as text chunks.
if task == "transcript":
if not _ffmpeg_exists():
return {
"ok": False,
"error": "ffmpeg_missing",
"hint": "Install ffmpeg (ffmpeg/ffprobe on PATH) to enable transcript extraction.",
}
api_key = str(os.getenv("OPENAI_API_KEY") or "").strip()
if not api_key:
return {"ok": False, "error": "OPENAI_API_KEY_missing"}
try:
with tempfile.TemporaryDirectory() as td:
wav = Path(td) / "audio.wav"
subprocess.run(
["ffmpeg", "-y", "-i", str(p), "-vn", "-ac", "1", "-ar", "16000", str(wav)],
capture_output=True,
text=True,
timeout=60,
)
if not wav.exists() or wav.stat().st_size <= 0:
return {"ok": False, "error": "audio_extract_failed"}
try:
from openai import OpenAI
except Exception as e:
return {"ok": False, "error": f"openai_package_missing: {type(e).__name__}: {e}"}
base_url = str(os.getenv("OPENAI_BASE_URL") or "").strip()
client_kwargs: dict[str, Any] = {"api_key": api_key}
if base_url:
client_kwargs["base_url"] = base_url
client = OpenAI(**client_kwargs)
model = str(args.get("model") or os.getenv("OPENAI_AUDIO_TRANSCRIPTION_MODEL") or OPENAI_DEFAULT_AUDIO_TRANSCRIPTION_MODEL).strip()
prompt = str(args.get("prompt") or "").strip()
# OpenAI SDK expects a file-like object with a name.
wav_bytes = wav.read_bytes()
f = io.BytesIO(wav_bytes)
f.name = "audio.wav" # type: ignore[attr-defined]
# Best-effort: different gateways may accept different param names; keep it minimal.
try:
resp = client.audio.transcriptions.create( # type: ignore[attr-defined]
model=model,
file=f,
**({"prompt": prompt} if prompt else {}),
)
text = str(getattr(resp, "text", "") or "")
except Exception as e:
return {"ok": False, "error": f"transcription_failed: {type(e).__name__}: {e}"}
if not text.strip():
return {"ok": False, "error": "empty_transcript"}
# Persist transcript as a long text document so the model can query evidence by text_id.
name = str(meta.name if meta else p.name)
text_name = f"{name}.transcript.txt"
cfg_chunk_size, cfg_chunk_overlap = _video_transcript_chunk_defaults()
chunk_size = int(args.get("chunk_size") or cfg_chunk_size)
chunk_overlap = int(args.get("chunk_overlap") or cfg_chunk_overlap)
text_meta = save_text_document(
attachment_id=str(attachment_id),
name=text_name,
text=text,
source_kind="video_transcript",
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
)
preview = text[:1200]
note = (
"Use query_text_attachment(text_id=...) to retrieve exact evidence with offsets."
if lang.startswith("en")
else "后续请用 query_text_attachment(text_id=...) 按需检索证据（支持 offset/top_k/关键词）。"
)
return {
"ok": True,
"task": "transcript",
"attachment_id": attachment_id,
"name": text_name,
"text_id": str(text_meta.get("text_id") or ""),
"chars": int(text_meta.get("chars") or 0),
"chunks": int(text_meta.get("chunks") or 0),
"preview": preview,
"note": note,
}
except Exception as e:
return {"ok": False, "error": f"transcript_failed: {type(e).__name__}: {e}"}
return ToolSpec(
name="query_video_attachment",
description="Query a video attachment by attachment_id (meta or transcript).",
parameters={
"type": "object",
"properties": {
"attachment_id": {"type": "string"},
"task": {"type": "string", "enum": ["meta", "transcript"]},
"lang": {"type": "string", "description": "Optional hint: zh/en."},
"model": {"type": "string", "description": "Optional transcription model override."},
"prompt": {"type": "string", "description": "Optional transcription prompt/context."},
"chunk_size": {"type": "integer", "description": "Transcript chunk size (chars)."},
"chunk_overlap": {"type": "integer", "description": "Transcript chunk overlap (chars)."},
},
"required": ["attachment_id"],
"additionalProperties": False,
},
handler=handler,
read_only=True,
tags=frozenset({"video", "read"}),
)
__all__ = ["query_video_attachment_tool"]

6
runtime/tools/experts/network_ops/image_query.py Normal file
View file

@ -0,0 +1,6 @@
from __future__ import annotations
from oclaw.runtime.tools.experts.generalist.image_query import query_image_attachment_tool
__all__ = ["query_image_attachment_tool"]

6
runtime/tools/experts/network_ops/text_query.py Normal file
View file

@ -0,0 +1,6 @@
from __future__ import annotations
from oclaw.runtime.tools.experts.generalist.text_query import query_text_attachment_tool
__all__ = ["query_text_attachment_tool"]

6
runtime/tools/experts/network_ops/video_query.py Normal file
View file

@ -0,0 +1,6 @@
from __future__ import annotations
from oclaw.runtime.tools.experts.generalist.video_query import query_video_attachment_tool
__all__ = ["query_video_attachment_tool"]

166
tests/test_attachment_text_replay_guard.py Normal file
View file

@ -0,0 +1,166 @@
from __future__ import annotations
import json
from pathlib import Path
from oclaw.platform.persistence.sqlite_store import SqliteStore
from oclaw.runtime.direct_loop import _build_model_context
from oclaw.platform.llm.chat_models import RuleBasedChatModel
def test_large_text_attachment_is_guarded_in_history_context(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "t.sqlite"))
sess = store.create_session("t")
big = "X" * 10000
attachments = [{"type": "text", "name": "big.txt", "content": big}]
store.add_message(session_id=sess.id, role="user", content="hi", attachments=attachments)
msgs = _build_model_context(
store=store,
session_id=sess.id,
max_messages=50,
system_prompt="sys",
model=RuleBasedChatModel(),
lang="en",
memory_context=None,
trace_id=None,
parent_span_id=None,
tools=None,
base_url="",
user_text="",
prompt_build_context=None,
active_turn_uuid="different-turn",
)
# Ensure guard marker appears in injected context.
joined = json.dumps(msgs, ensure_ascii=False)
assert "Attachment (summarized for context replay)" in joined
assert "attachment_truncated_for_context_replay" in joined
def test_text_attachment_is_collapsed_when_text_ref_present(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "t2.sqlite"))
sess = store.create_session("t")
big = "Y" * 6000
attachments = [
{"type": "text", "name": "doc.txt", "content": big},
{"type": "text_ref", "name": "doc.txt", "text_id": "a" * 64, "chars": 6000, "chunks": 4, "source_kind": "txt"},
]
store.add_message(session_id=sess.id, role="user", content="hi", attachments=attachments)
msgs = _build_model_context(
store=store,
session_id=sess.id,
max_messages=50,
system_prompt="sys",
model=RuleBasedChatModel(),
lang="en",
memory_context=None,
trace_id=None,
parent_span_id=None,
tools=None,
base_url="",
user_text="",
prompt_build_context=None,
active_turn_uuid="different-turn",
)
joined = json.dumps(msgs, ensure_ascii=False)
assert "Attachment (collapsed; text_ref available)" in joined
assert "attachment_collapsed_for_context_replay" in joined
def test_large_image_tool_result_is_guarded_in_history_context(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "t3.sqlite"))
sess = store.create_session("t")
long_text = "OCR-LINE\n" * 1200
payload = {
"ok": True,
"task": "ocr",
"attachment_id": "b" * 64,
"text": long_text,
"backend_shape": "multi",
}
store.add_message(session_id=sess.id, role="tool", content=json.dumps(payload, ensure_ascii=False))
msgs = _build_model_context(
store=store,
session_id=sess.id,
max_messages=50,
system_prompt="sys",
model=RuleBasedChatModel(),
lang="en",
memory_context=None,
trace_id=None,
parent_span_id=None,
tools=None,
base_url="",
user_text="",
prompt_build_context=None,
active_turn_uuid="different-turn",
)
joined = json.dumps(msgs, ensure_ascii=False)
assert "image_tool_result_truncated_for_context_replay" in joined
assert "_image_tool_result_guarded" in joined
def test_small_image_tool_result_not_guarded(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "t4.sqlite"))
sess = store.create_session("t")
payload = {
"ok": True,
"task": "describe",
"attachment_id": "c" * 64,
"text": "A concise description of an icon.",
}
store.add_message(session_id=sess.id, role="tool", content=json.dumps(payload, ensure_ascii=False))
msgs = _build_model_context(
store=store,
session_id=sess.id,
max_messages=50,
system_prompt="sys",
model=RuleBasedChatModel(),
lang="en",
memory_context=None,
trace_id=None,
parent_span_id=None,
tools=None,
base_url="",
user_text="",
prompt_build_context=None,
active_turn_uuid="different-turn",
)
joined = json.dumps(msgs, ensure_ascii=False)
assert "image_tool_result_truncated_for_context_replay" not in joined
def test_large_video_transcript_tool_result_is_guarded_in_history_context(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "t5.sqlite"))
sess = store.create_session("t")
long_text = "LINE\n" * 2000
payload = {
"ok": True,
"task": "transcript",
"attachment_id": "d" * 64,
"text": long_text,
}
store.add_message(session_id=sess.id, role="tool", content=json.dumps(payload, ensure_ascii=False))
msgs = _build_model_context(
store=store,
session_id=sess.id,
max_messages=50,
system_prompt="sys",
model=RuleBasedChatModel(),
lang="en",
memory_context=None,
trace_id=None,
parent_span_id=None,
tools=None,
base_url="",
user_text="",
prompt_build_context=None,
active_turn_uuid="different-turn",
)
joined = json.dumps(msgs, ensure_ascii=False)
assert "video_tool_result_truncated_for_context_replay" in joined

179
tests/test_file_attachments.py
View file

@ -2,6 +2,8 @@ from __future__ import annotations
import io
import json
import gzip
import tarfile
import zipfile
from pathlib import Path
@ -15,6 +17,7 @@ from oclaw.platform.files.tabular_attachment_store import (
run_table_sql,
save_workbook,
)
from oclaw.platform.files.text_attachment_store import query_text_document
def _zip_bytes(files: dict[str, bytes]) -> bytes:
@ -25,6 +28,17 @@ def _zip_bytes(files: dict[str, bytes]) -> bytes:
return buf.getvalue()
def _tar_bytes(files: dict[str, bytes], *, gz: bool = False) -> bytes:
buf = io.BytesIO()
mode = "w:gz" if gz else "w"
with tarfile.open(fileobj=buf, mode=mode) as tf:
for name, data in files.items():
ti = tarfile.TarInfo(name=name)
ti.size = len(data)
tf.addfile(ti, io.BytesIO(data))
return buf.getvalue()
def test_csv_is_summarized_not_full_dump() -> None:
payload = (
"c1,c2,c3\n"
@ -37,6 +51,32 @@ def test_csv_is_summarized_not_full_dump() -> None:
assert "# Table Summary" in content
assert "rows: 2" in content
assert "cols: 3" in content
assert "## Full table included (2 rows)" in content
def test_long_txt_emits_text_ref_and_can_query() -> None:
payload = ("A" * 15000 + "\nneedle\n" + "B" * 2000).encode("utf-8")
out = process_file_data("notes.txt", payload)
text_refs = [x for x in out if isinstance(x, dict) and str(x.get("type") or "") == "text_ref"]
assert text_refs
text_id = str(text_refs[0].get("text_id") or "")
assert text_id
got = query_text_document(text_id=text_id, query="needle", top_k=3, offset=0)
assert bool(got.get("ok"))
rows = got.get("rows") or []
assert rows
assert any("needle" in str(x.get("content") or "") for x in rows)
def test_video_upload_emits_video_ref() -> None:
# Minimal MP4 header-like bytes; parser should not crash and should still store as video_ref.
payload = b"\x00\x00\x00\x18ftypmp42\x00\x00\x00\x00mp42isom"
out = process_file_data("clip.mp4", payload)
vrefs = [x for x in out if isinstance(x, dict) and str(x.get("type") or "") == "video_ref"]
assert vrefs
vr = vrefs[0]
assert str(vr.get("attachment_id") or "")
assert str(vr.get("mime") or "").startswith("video/")
def test_zip_file_count_limit_returns_error_attachment() -> None:
@ -48,6 +88,104 @@ def test_zip_file_count_limit_returns_error_attachment() -> None:
assert "too many files" in str(first.get("content") or "").lower()
def test_tar_path_traversal_is_blocked() -> None:
payload = _tar_bytes({"../evil.txt": b"x"})
out = process_file_data("unsafe.tar", payload)
assert out
first = out[0]
assert str(first.get("name") or "") == "archive-error"
assert "unsafe path" in str(first.get("content") or "").lower()
def test_tgz_is_parsed_and_members_processed() -> None:
payload = _tar_bytes({"ok.txt": b"hello"}, gz=True)
out = process_file_data("bundle.tgz", payload)
assert out
assert any(str(x.get("type") or "") == "text" and "hello" in str(x.get("content") or "") for x in out)
def test_tar_link_entry_is_explicitly_rejected() -> None:
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tf:
ti = tarfile.TarInfo(name="target.txt")
data = b"ok"
ti.size = len(data)
tf.addfile(ti, io.BytesIO(data))
lnk = tarfile.TarInfo(name="sym")
lnk.type = tarfile.SYMTYPE
lnk.linkname = "target.txt"
tf.addfile(lnk)
out = process_file_data("has-link.tar", buf.getvalue())
assert out
first = out[0]
assert str(first.get("name") or "") == "archive-error"
assert str(first.get("error_code") or "") == "archive_link_entry_forbidden"
def test_gz_single_file_is_decompressed_and_processed() -> None:
payload = gzip.compress(b"hello-gz")
out = process_file_data("note.txt.gz", payload)
assert out
assert any(str(x.get("type") or "") == "text" and "hello-gz" in str(x.get("content") or "") for x in out)
def test_rar_returns_explicit_unsupported_error() -> None:
out = process_file_data("bundle.rar", b"not-a-real-rar")
assert out
first = out[0]
assert str(first.get("name") or "") == "archive-error"
assert "unsupported archive format" in str(first.get("content") or "").lower()
def test_7z_returns_explicit_unsupported_error() -> None:
out = process_file_data("bundle.7z", b"not-a-real-7z")
assert out
first = out[0]
assert str(first.get("name") or "") == "archive-error"
assert "unsupported archive format" in str(first.get("content") or "").lower()
assert str(first.get("error_code") or "") == "archive_unsupported_format"
def test_archive_signature_detection_overrides_misleading_extension() -> None:
payload = _zip_bytes({"ok.txt": b"sig-ok"})
out = process_file_data("misleading.tar", payload)
assert out
assert any(str(x.get("type") or "") == "text" and "sig-ok" in str(x.get("content") or "") for x in out)
def test_archive_limits_can_be_overridden_by_config(tmp_path: Path, monkeypatch) -> None:
cfg = {
"plugins": {
"entries": {
"memory-wiki": {
"auto": {
"attachments": {
"tabular": {
"archive_max_depth": 1,
"archive_max_file_count": 2,
"archive_max_entry_bytes": 20,
"archive_max_total_uncompressed_bytes": 30,
}
}
}
}
}
}
}
cfg_path = tmp_path / "oclaw.json"
cfg_path.write_text(json.dumps(cfg), encoding="utf-8")
monkeypatch.setenv("AIA_OCLAW_CONFIG_PATH", str(cfg_path))
fa._attachments_limits.cache_clear()
try:
payload = _zip_bytes({"a.txt": b"x" * 50})
out = process_file_data("limited.zip", payload)
finally:
fa._attachments_limits.cache_clear()
assert out
first = out[0]
assert "entry too large" in str(first.get("content") or "").lower()
def test_nested_zip_depth_limit_is_enforced() -> None:
z4 = _zip_bytes({"deep.txt": b"hello"})
z3 = _zip_bytes({"z4.zip": z4})
@ -197,6 +335,14 @@ def test_large_csv_emits_tabular_ref_and_can_query() -> None:
assert str(got.get("engine") or "") in {"builtin_sqlite", "mcp_sqlite"}
def test_medium_csv_enters_tool_mode_with_default_threshold() -> None:
rows = ["c1,c2"] + [f"{i},row-{i}" for i in range(0, 6000)]
payload = ("\n".join(rows)).encode("utf-8")
out = process_file_data("medium-default.csv", payload)
tab_refs = [x for x in out if isinstance(x, dict) and str(x.get("type") or "") == "tabular_ref"]
assert tab_refs
def test_large_csv_can_aggregate_grouped_sum() -> None:
rows = ["dept,amount"] + [f"{'A' if i % 2 == 0 else 'B'},{i}" for i in range(0, 25010)]
payload = ("\n".join(rows)).encode("utf-8")
@ -324,6 +470,20 @@ def test_query_can_target_specific_excel_sheet() -> None:
assert rows and str(rows[0].get("k") or "") == "b"
def test_multi_sheet_query_without_sheet_returns_sheet_hint() -> None:
s1 = pd.DataFrame([["a", "1"]], columns=["k", "v"])
s2 = pd.DataFrame([["b", "2"]], columns=["k", "v"])
meta = save_workbook(attachment_id="aid3", name="multi2.xlsx", sheets={"Main": s1, "Backup": s2})
tid = str(meta.get("table_id") or "")
got = query_table(table_id=tid, columns=["k", "v"], limit=5, offset=0)
assert bool(got.get("ok"))
hint = got.get("sheet_hint") or {}
assert isinstance(hint, dict)
names = hint.get("available_sheets") or []
assert "Main" in names and "Backup" in names
assert str(hint.get("default_sheet") or "") == "Main"
def test_full_scan_analyzes_all_rows_and_returns_audit() -> None:
rows = ["dept,score"] + [f"{'A' if i % 2 == 0 else 'B'},{i % 5}" for i in range(0, 25025)]
payload = ("\n".join(rows)).encode("utf-8")
@ -345,6 +505,25 @@ def test_full_scan_analyzes_all_rows_and_returns_audit() -> None:
assert isinstance(tops, list) and len(tops) <= 2
def test_tabular_tools_reject_non_table_id_filename() -> None:
bad_id = "日志.xlsx"
q = query_table(table_id=bad_id, columns=["a"], limit=5, offset=0)
assert not bool(q.get("ok"))
assert str(q.get("error") or "") == "table_id_invalid_format"
ag = aggregate_table(table_id=bad_id, metric="count")
assert not bool(ag.get("ok"))
assert str(ag.get("error") or "") == "table_id_invalid_format"
sql = run_table_sql(table_id=bad_id, sql="SELECT 1", limit=5)
assert not bool(sql.get("ok"))
assert str(sql.get("error") or "") == "table_id_invalid_format"
fs = analyze_table_full_scan(table_id=bad_id, columns=["a"], top_values_limit=1)
assert not bool(fs.get("ok"))
assert str(fs.get("error") or "") == "table_id_invalid_format"
def test_xlsx_zip_safety_blocks_unsafe_paths() -> None:
payload = _zip_bytes({"../xl/workbook.xml": b"x"})
out = process_file_data("bad.xlsx", payload)

269
tests/test_tool_loop_guard.py
View file

@ -123,3 +123,272 @@ def test_repeated_non_sql_tools_are_not_compacted(tmp_path: Path) -> None:
assert payloads
assert not any(bool(p.get("_history_compacted")) for p in payloads)
def test_tabular_tools_blocked_without_tabular_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g4.sqlite"))
sess = store.create_session("t")
# A normal user turn without tabular_ref attachments.
store.add_message(session_id=sess.id, role="user", content="analyze this file", attachments=[{"type": "text", "name": "x"}])
def _handler(_args):
return {"ok": True, "rows": []}
reg = ToolRegistry(
[
ToolSpec(
name="query_tabular_attachment",
description="query table",
parameters={"type": "object", "properties": {"table_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_tabular_attachment", arguments={"table_id": "日志.xlsx"})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
blocked, _ = results["c1"]
assert not bool(blocked.get("ok"))
assert str(blocked.get("error_code") or "") == "tabular_ref_missing"
def test_tabular_tools_allowed_with_tabular_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g5.sqlite"))
sess = store.create_session("t")
store.add_message(
session_id=sess.id,
role="user",
content="uploaded table",
attachments=[{"type": "tabular_ref", "table_id": "a" * 64, "name": "ok.xlsx"}],
)
calls = {"n": 0}
def _handler(_args):
calls["n"] += 1
return {"ok": True, "rows": [{"x": 1}]}
reg = ToolRegistry(
[
ToolSpec(
name="query_tabular_attachment",
description="query table",
parameters={"type": "object", "properties": {"table_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_tabular_attachment", arguments={"table_id": "a" * 64})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
ok_res, _ = results["c1"]
assert bool(ok_res.get("ok"))
assert calls["n"] == 1
def test_text_tools_blocked_without_text_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g6.sqlite"))
sess = store.create_session("t")
store.add_message(session_id=sess.id, role="user", content="summarize", attachments=[{"type": "text", "name": "a.txt"}])
def _handler(_args):
return {"ok": True, "rows": []}
reg = ToolRegistry(
[
ToolSpec(
name="query_text_attachment",
description="query text",
parameters={"type": "object", "properties": {"text_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_text_attachment", arguments={"text_id": "a" * 64})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
blocked, _ = results["c1"]
assert not bool(blocked.get("ok"))
assert str(blocked.get("error_code") or "") == "text_ref_missing"
def test_text_tools_allowed_with_text_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g7.sqlite"))
sess = store.create_session("t")
store.add_message(
session_id=sess.id,
role="user",
content="uploaded long text",
attachments=[{"type": "text_ref", "text_id": "b" * 64, "name": "long.txt"}],
)
calls = {"n": 0}
def _handler(_args):
calls["n"] += 1
return {"ok": True, "rows": [{"x": 1}]}
reg = ToolRegistry(
[
ToolSpec(
name="query_text_attachment",
description="query text",
parameters={"type": "object", "properties": {"text_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_text_attachment", arguments={"text_id": "b" * 64})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
ok_res, _ = results["c1"]
assert bool(ok_res.get("ok"))
assert calls["n"] == 1
def test_image_tools_blocked_without_image_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g8.sqlite"))
sess = store.create_session("t")
store.add_message(session_id=sess.id, role="user", content="analyze image", attachments=[{"type": "text", "name": "x"}])
def _handler(_args):
return {"ok": True}
reg = ToolRegistry(
[
ToolSpec(
name="query_image_attachment",
description="query image",
parameters={"type": "object", "properties": {"attachment_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_image_attachment", arguments={"attachment_id": "abc"})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
blocked, _ = results["c1"]
assert not bool(blocked.get("ok"))
assert str(blocked.get("error_code") or "") == "image_ref_missing"
def test_image_tools_allowed_with_image_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g9.sqlite"))
sess = store.create_session("t")
store.add_message(
session_id=sess.id,
role="user",
content="uploaded image",
attachments=[{"type": "image_ref", "attachment_id": "abc", "mime": "image/png"}],
)
calls = {"n": 0}
def _handler(_args):
calls["n"] += 1
return {"ok": True}
reg = ToolRegistry(
[
ToolSpec(
name="query_image_attachment",
description="query image",
parameters={"type": "object", "properties": {"attachment_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_image_attachment", arguments={"attachment_id": "abc"})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
ok_res, _ = results["c1"]
assert bool(ok_res.get("ok"))
assert calls["n"] == 1
def test_video_tools_blocked_without_video_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g10.sqlite"))
sess = store.create_session("t")
store.add_message(session_id=sess.id, role="user", content="analyze video", attachments=[{"type": "text", "name": "x"}])
def _handler(_args):
return {"ok": True}
reg = ToolRegistry(
[
ToolSpec(
name="query_video_attachment",
description="query video",
parameters={"type": "object", "properties": {"attachment_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_video_attachment", arguments={"attachment_id": "abc"})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
blocked, _ = results["c1"]
assert not bool(blocked.get("ok"))
assert str(blocked.get("error_code") or "") == "video_ref_missing"
def test_video_tools_allowed_with_video_ref(tmp_path: Path) -> None:
store = SqliteStore(str(tmp_path / "g11.sqlite"))
sess = store.create_session("t")
store.add_message(
session_id=sess.id,
role="user",
content="uploaded video",
attachments=[{"type": "video_ref", "attachment_id": "abc", "mime": "video/mp4"}],
)
calls = {"n": 0}
def _handler(_args):
calls["n"] += 1
return {"ok": True}
reg = ToolRegistry(
[
ToolSpec(
name="query_video_attachment",
description="query video",
parameters={"type": "object", "properties": {"attachment_id": {"type": "string"}}},
handler=_handler,
read_only=True,
)
]
)
tool_uses = [LLMToolCall(id="c1", name="query_video_attachment", arguments={"attachment_id": "abc"})]
_, results = ToolExecutor().execute_tool_uses(
ctx=ToolExecutionContext(store=store, tools=reg, session_id=sess.id),
assistant_msg_id=1,
tool_uses=tool_uses,
)
ok_res, _ = results["c1"]
assert bool(ok_res.get("ok"))
assert calls["n"] == 1

19
tests/test_video_query_tool.py Normal file
View file

@ -0,0 +1,19 @@
from __future__ import annotations
import oclaw.runtime.tools.experts.generalist.video_query as vq
def test_normalized_video_meta_extracts_top_level_fields() -> None:
obj = {
"format": {"duration": "12.50"},
"streams": [
{"codec_type": "audio", "sample_rate": "48000"},
{"codec_type": "video", "width": 1920, "height": 1080, "avg_frame_rate": "30000/1001"},
],
}
out = vq._normalized_video_meta(obj)
assert float(out.get("duration_sec") or 0.0) > 12.0
assert int(out.get("width") or 0) == 1920
assert int(out.get("height") or 0) == 1080
assert float(out.get("fps") or 0.0) > 29.0