diff --git a/README.md b/README.md index 8a4d5e9..c6cd32b 100644 --- a/README.md +++ b/README.md @@ -169,20 +169,24 @@ Fool-proof Windows delivery via `NetX-Setup-*.exe` (bundled or external Postgres **完整说明（安装、配置、更新、排错）见：[docs/MCP.md](docs/MCP.md)** **拓扑画布独立 MCP** 见：[docs/MCP_TOPOLOGY.md](docs/MCP_TOPOLOGY.md) +**割接 / 业务监控独立 MCP** 见：[docs/MCP_BIZMONITOR.md](docs/MCP_BIZMONITOR.md)。三组一起使用时无需 netxops，完整配置为 [mcp.all.json](mcp.all.json)。 + 速查： ```powershell pip install -e ./packages/netx-mcp # 拓扑（可选，单独安装） pip install -e ./packages/netx-topology-mcp -# 配置见 mcp.json，运行： +pip install -e ./packages/netx-bizmonitor-mcp +# 三组配置见 mcp.all.json，分别运行： python -m netx_mcp python -m netx_topology_mcp +python -m netx_bizmonitor_mcp ``` - 客户端配置：[`mcp.json`](mcp.json)（含 `netx` + 可选 `netx-topology`） - oclaw payload：[`mcp_install_payload.json`](mcp_install_payload.json) / [`mcp_topology_install_payload.json`](mcp_topology_install_payload.json) -- 子包：[`packages/netx-mcp`](packages/netx-mcp/README.md)、[`packages/netx-topology-mcp`](packages/netx-topology-mcp/README.md) +- 子包：[`packages/netx-mcp`](packages/netx-mcp/README.md)、[`packages/netx-topology-mcp`](packages/netx-topology-mcp/README.md)、[`packages/netx-bizmonitor-mcp`](packages/netx-bizmonitor-mcp/README.md) ## Useful API endpoints diff --git a/docs/MCP.md b/docs/MCP.md index d07e90d..156b96a 100644 --- a/docs/MCP.md +++ b/docs/MCP.md @@ -116,7 +116,7 @@ pip install "git+https://github.com/hansjone/netx.git#subdirectory=packages/netx 1. 完成上文 **§1**（用 **oclaw 同机同一个 `python`** 安装 `netx-mcp`）。 2. Admin → MCP → 粘贴 `mcp.json` 全文 → 点击 **Install from JSON**（安装状态在下方一行小字）。 -3. **Health** → **Sync Tools**（应看到 **14** 个工具）。 +3. **Health** → **Sync Tools**（完整 scope 下应看到 **15** 个工具）。 4. 在 **MCP 专家绑定** 中为 ops 专家勾选 `server_id=netx`。 更细的 oclaw 说明（双轨内置工具、锚点注入等）见 oclaw 仓库： @@ -141,9 +141,11 @@ pip install "git+https://github.com/hansjone/netx.git#subdirectory=packages/netx | NMS 原始/SQL | `queryNmsAlarmsRaw`, `aggregateNmsAlarmsRaw`, `listNmsAlarmFields`, `sqlQueryNms` | | common（CLI + 路径） | `listManagedNe`, `getManagedNe`, `execManagedNe`, `getNeExecJob`, `listCliTargets`, `findTopologyPaths` | -参数优先 `nms_ne_id` / `nms_ne_ids`（保留 `ume_*` 别名）。拓扑 Fabric / 画布工具在 **[`netx-topology-mcp`](./MCP_TOPOLOGY.md)**。oclaw 中名称带前缀：`mcp__netx__`；DSH：`netx__`。 +参数使用 `nms_ne_id` / `nms_ne_ids`，旧 `ume_*` 模型参数已删除；`getManagedNe` 只接受 `ne_id`。拓扑 Fabric / 画布工具在 **[`netx-topology-mcp`](./MCP_TOPOLOGY.md)**。oclaw 中名称带前缀：`mcp__netx__`；DSH：`netx__`。 -**业务监控 / 割接分析**：不走 netx-mcp。DSH **netxops** 宿主工具组 `bizMonitor`（`netx__listBizMonitors` / `netx__getBizMonitorContext` 等）直连 netx REST（`/v1/biz-migration/*`、`/v1/biz-state/*`，需 token scope `biz-monitor:read`）。 +**业务监控 / 割接分析**：独立 [`netx-bizmonitor-mcp`](./MCP_BIZMONITOR.md) 提供 8 个工具，需 `biz-monitor:read`；netxops 的 bizMonitor 组使用相同 stem。三组统一配置见根 [`mcp.all.json`](../mcp.all.json)，不需要安装 netxops。 + +**CLI 安全边界**：独立 MCP 强制单行只读 `show` / `display`，可用只读输出过滤；配置、shell 操作符、重定向和多行脚本在 HTTP 前拒绝。后端开放 exec_policy 或 capability 的脚本提示也不能解除这个 MCP 门禁，Linux/RouterOS 原生 CLI 当前未暴露。异步 job_id 仅表示开始，按 poll_after_ms 有界轮询；terminal 后检查 result.ok，批量还须检查 summary.failed 和每条 results[].ok，不能只凭作业 succeeded 宣称全部成功。 --- @@ -191,10 +193,10 @@ pip show netx-mcp | 旧方式 | 新方式 | |--------|--------| -| `python -m netx_api.mcp` | `python -m netx_mcp`（推荐） | +| `python -m netx_api.mcp` | `python -m netx_mcp`（唯一 Python 模块入口） | | `NETX_MCP_MODE=db` 直连库 | 已废弃；使用 HTTP 模式 | -根目录 `python -m netx_api.mcp` 仍会委托到 `netx_mcp`，新环境请只装 `netx-mcp` 包。 +旧入口 `netx_api.mcp`、`netx_api/mcp_server.py` 和 `NETX_MCP_MODE=db` 已删除。请安装独立 `netx-mcp` 包并迁移配置；不再保留兼容委托。旧 DB 模式独有的导入告警查询/聚合/批次/诊断、同步触发、历史告警查询仍有 HTTP API，但不属于当前 ops 15 工具。 --- @@ -233,7 +235,7 @@ $p = Start-Process python -ArgumentList "-m","netx_mcp" -RedirectStandardInput p | 工具调用连不上 API | 检查 `NETX_API_URL`、防火墙、远端 API 是否启动 | | Windows 乱码 / JSON 解析失败 | 配置里保留 `PYTHONIOENCODING=utf-8`、`PYTHONUTF8=1`（见 `mcp.json`） | | oclaw 工具数为 0 | Admin **Health → Sync Tools**；确认 `server_id=netx` 已绑定 ops 专家 | -| 仍想用仓库脚本路径 | 未 pip 安装时可临时 `"args": ["D:/.../netx_api/mcp_server.py"]`（开发用） | +| 旧脚本路径或模块不存在 | 安装 `packages/netx-mcp`，配置改为 `"args": ["-m", "netx_mcp"]` | --- diff --git a/docs/MCP_BIZMONITOR.md b/docs/MCP_BIZMONITOR.md new file mode 100644 index 0000000..7db37ae --- /dev/null +++ b/docs/MCP_BIZMONITOR.md @@ -0,0 +1,41 @@ +# 割接与业务监控 MCP + +`netx-bizmonitor-mcp` 独立提供 bizMonitor 的 8 个只读工具。不安装 netxops 也能读取已有项目、批次、看板、红票、差异和采集原文。它只依赖 Python 3.11+ 与 httpx，通过 HTTP 使用正在运行的 netx。 + +## 三组一起配置 + +在运行 MCP 客户端的 Python 环境分别安装： + +```powershell +python -m pip install -e ./packages/netx-mcp +python -m pip install -e ./packages/netx-topology-mcp +python -m pip install -e ./packages/netx-bizmonitor-mcp +``` + +将根 [mcp.all.json](../mcp.all.json) 的三个 server 放入客户端 MCP 配置。也可以不安装包，分别将该包的 `src` 绝对路径设为对应 server 的 `PYTHONPATH`。三个模块分别为 `netx_mcp`、`netx_topology_mcp`、`netx_bizmonitor_mcp`；console scripts 属于各独立包，安装 API 本体不会提供一个缺少依赖的 MCP 假入口。 + +统一设置 `NETX_API_URL`。凭据优先用 `NETX_API_TOKEN`，否则读取 `NETX_MCP_TOKEN_FILE`，再尝试工作目录 `data/auth/mcp_token`。客户端工作目录不确定时，给 `NETX_MCP_TOKEN_FILE` 配置绝对路径。不要把真实令牌提交到仓库。 + +业务组需 `biz-monitor:read`；ops 需 `alarms:read`、`ne:read`、`ne:exec`，SQL 另需 `sql:query`；topology 读取需 `ne:read`，建图/布局需 `ne:write`。默认 bootstrap token 不含 SQL 和拓扑写权限，因此工具数量会按 scope 缩减；有完整权限时分别列出 **15、16、8** 个工具。 + +## 入口与证据链 + +MCP 使用裸名；netxops 使用 `netx__` 加相同名称。 + +1. `listBizMonitors`：无 ID 的入口，取得 project_id/task_id。 +2. `getBizMonitorContext`：读取模板、归一化、端口映射、两侧任务和基线。 +3. `listBizMonitorBatches(project_id,include_runs=true)`：取得割接批次与 run_id；也可 `kind=runs,batch_id=...`。 +4. `getBizMonitorBoard(batch_id)`：检查完成状态、缺失数据和统计。 +5. `listBizMonitorReds(project_id)` / `getBizMonitorDiffs(run_id,color=red)`：逐页取得红项和证据。`q` 自动映射后端 `kw`。 +6. `getBizCollectBatch`：采集批次的命令目录；需要时由 `listBizMonitorBatches(task_id)` 找采集批次。 +7. `getBizCollectCommandRaw(batch_id,command_id)`：两侧原文取证。这里使用采集 batch_id，不能传割接 batch_id。 + +`old_key/new_key` 保留设备原始 A/B；`match_*` 和归一化映射用于内部配对。原文按需获取，可能很大；凭据会脱敏，不把内部匹配键冒充设备原始端口。 + +本组不会创建任务/模板、启动采集、执行 evaluate、关闭红票。没有批次时不能得出“割接通过”。完整行为和参数见 [包 README](../packages/netx-bizmonitor-mcp/README.md) 与 [skill](../skills/biz-monitor/netx-biz-monitor/SKILL.md)。 + +## 可复现验收 + +`scripts/mcp_acceptance.py` 启动当前 worktree 的真实 stdio MCP，记录去敏 JSONL；仅允许本机 8890 或隔离样例 8891。工具源目录可用 `--source-root` 显式指定，不会静默导入主仓库的 editable 包。 + +没有现场割接数据时，可用 `scripts/seed_mcp_acceptance.py` 在 `agent-wt/netx-mcp-lead/artifacts/` 的新 acceptance SQLite 文件生成明确标注的离线样例，真实运行业务评价，再通过本地 API/MCP 验证读取链路。样例不代表现场故障；不要将样例库替换应用库。脚本拒绝覆盖、外部路径、网络连接和监听端口。 diff --git a/docs/MCP_TOPOLOGY.md b/docs/MCP_TOPOLOGY.md index b149c10..db3ec37 100644 --- a/docs/MCP_TOPOLOGY.md +++ b/docs/MCP_TOPOLOGY.md @@ -60,11 +60,11 @@ pip install "git+https://github.com/hansjone/netx.git#subdirectory=packages/netx 与告警 MCP 并存时，把两个 server 都放进 `mcpServers` 即可；未勾选/未安装的不会加载工具。 -oclaw：Install from JSON → Health → Sync Tools（应看到 **14** 个工具，含 `layoutTopologyView` / `sinkTopologyDualUnits` / `queryTopologyFabricNodes`）→ 专家绑定勾选 `server_id=netx-topology`。 +oclaw：Install from JSON → Health → Sync Tools（完整 scope 下应看到 **16** 个工具，含 `layoutTopologyView` / `sinkTopologyDualUnits` / `queryTopologyFabricNodes`）→ 专家绑定勾选 `server_id=netx-topology`。 Cursor：改包后若 Sync 工具数不对，请 **禁用/启用** `netx-topology` 或重载窗口（须重启 stdio 进程）。可选在 env 加 `PYTHONPATH=.../packages/netx-topology-mcp/src` 强制最新源码。 -配套 Agent Skill（dual_unit 排水 → 扫角/polish → 手拖）：[`.cursor/skills/netx-topology/SKILL.md`](../.cursor/skills/netx-topology/SKILL.md)。Cursor / oclaw 读 skill 后再调 MCP。 +配套 Agent Skill（dual_unit 排水 → 扫角/polish → 手拖）：[`skills/topology/netx-topology/SKILL.md`](../skills/topology/netx-topology/SKILL.md)。Cursor / oclaw 读 skill 后再调 MCP。 --- @@ -97,7 +97,7 @@ Cursor：改包后若 Sync 工具数不对，请 **禁用/启用** `netx-topolog | `removeTopologyViewNodes` | 筛选或 id 从画布移除（不删 Fabric），摘要 | | `sinkTopologyDualUnits` | 根图 dual_units 分批沉入子区域；默认 `layout_batch`→`layout_dual_unit`→块扫挂载；日常禁 `until_empty` | | `copyTopologyViewNodes` | 一键克隆画布成员+坐标到另一画布（`clear_target` 可选）；源画布不动，测沙箱用 | -| `layoutTopologyView` `move_nodes` | 指定 `fabric_node_ids` 从 `source_view_id`→`view_id`（默认同移出源）；对调两 view 回迁；别名 `sink_nodes`；`park=true` 扫角停靠 | +| `layoutTopologyView` `move_nodes` | 指定 `fabric_node_ids` 从 `source_view_id`→`view_id`（默认同移出源）；对调两 view 回迁；`park=true` 扫角停靠 | | `updateTopologyViewPositions` | **优先** `layout=grid\|offset\|stack` + 筛选，API 自己挪点；`positions[]` 仅少量微调 | | `layoutTopologyView` | **布图/局部修**：公开 `action=layout\|layout_dual_unit\|orbit_sweep\|clear_edge_hits\|pull_far_chains\|compact_bbox\|polish_crossings\|fix_overlaps\|untangle\|straighten_channels\|level_bands\|move_nodes`；recipe 仅 `compact\|corridor\|rings\|unstick`。眼 sink 主路径：`until_limit(crossing→total)` → `clear_edge_hits` → `pull_far_chains` → `compact_bbox` → 手拖；禁对眼跑 polish/fix_overlaps/untangle/round。stall 且 moved≈0 即算法到头。巨图 apply 可能返回 `job_id` → 轮询 `job_status` / `job_cancel`。Job=**子进程+`data/runtime/layout_jobs` 落盘**。`mode=preview\|apply` | | `projectTopologyNeighbors` | 投影**已有** LLDP 邻居到画布；区域画布务必传 `region_folder_id`，读 `out_of_region_skipped` | @@ -120,8 +120,11 @@ Cursor：改包后若 Sync 工具数不对，请 **禁用/启用** `netx-topolog | 包 | server_id | 职责 | |----|-----------|------| -| `netx-mcp` | `netx` | 告警、UME、托管网元 CLI（**13** 工具） | -| `netx-topology-mcp` | `netx-topology` | 拓扑画布 / Fabric 只读 + 分类打标 + 安全画图（**15** 工具） | +| `netx-mcp` | `netx` | NMS 告警、网元和只读 CLI（**15** 工具） | +| `netx-topology-mcp` | `netx-topology` | topology 组 15 工具及额外 sinkTopologyDualUnits（共 **16**） | +| `netx-bizmonitor-mcp` | `netx-bizmonitor` | 割接 / 业务状态只读分析（**8** 工具） | + +显式后台保存：`layoutTopologyView(view_id=...,action=layout,recipe=compact,mode=apply,params={background:true})`。用 `layoutTopologyView(action=job_status,params={job_id:...})` 轮询，成功后 getTopologyView 回读。任务是 MCP 本地子进程与落盘 JSON，没有 HTTP layout job API；跨目录或机器不能轮询同一 job。失败后可能已有部分写入，先回读，不盲重试。布局保存成功与评分诊断分别判断；评分校准另行安排，本轮不改算法/阈值。 `queryTopologyEdges` 已从 `netx-mcp` **迁出**到本包，避免重复。 diff --git a/mcp.all.json b/mcp.all.json new file mode 100644 index 0000000..a28560d --- /dev/null +++ b/mcp.all.json @@ -0,0 +1,34 @@ +{ + "mcpServers": { + "netx": { + "command": "python", + "args": ["-m", "netx_mcp"], + "env": { + "NETX_API_URL": "http://127.0.0.1:8890", + "NETX_LANG": "zh", + "PYTHONIOENCODING": "utf-8", + "PYTHONUTF8": "1" + } + }, + "netx-topology": { + "command": "python", + "args": ["-m", "netx_topology_mcp"], + "env": { + "NETX_API_URL": "http://127.0.0.1:8890", + "NETX_LANG": "zh", + "PYTHONIOENCODING": "utf-8", + "PYTHONUTF8": "1" + } + }, + "netx-bizmonitor": { + "command": "python", + "args": ["-m", "netx_bizmonitor_mcp"], + "env": { + "NETX_API_URL": "http://127.0.0.1:8890", + "NETX_LANG": "zh", + "PYTHONIOENCODING": "utf-8", + "PYTHONUTF8": "1" + } + } + } +} diff --git a/netx_api/managed_ne_router.py b/netx_api/managed_ne_router.py index 6c656d2..a12ff6d 100644 --- a/netx_api/managed_ne_router.py +++ b/netx_api/managed_ne_router.py @@ -7,6 +7,7 @@ from fastapi.responses import Response from sqlalchemy.orm import Session from .auth_deps import AuthContext, require_user +from .auth_scopes import has_scope from .auth_service import write_audit from .db import get_db from .device_types import SUPPORTED_VENDORS @@ -275,6 +276,10 @@ def api_create_exec_job( uid, uname = _actor(ctx) payload = body.model_dump(exclude_none=True) + if not has_scope(ctx.scopes, "ne:exec"): + raise HTTPException(status_code=403, detail={"error": "insufficient_scope", "required": ["ne:exec"]}) + if not uid: + raise HTTPException(status_code=401, detail={"error": "job_owner_required"}) is_batch = bool( payload.get("targets") or payload.get("ne_ids") @@ -330,7 +335,7 @@ def api_create_exec_job( ) return out - ack = start_ne_exec_job(kind=kind, arguments=payload, runner=_runner) + ack = start_ne_exec_job(kind=kind, arguments=payload, owner_user_id=uid, runner=_runner) write_audit( db, action="ne.exec_job_start", @@ -349,9 +354,16 @@ def api_get_exec_job( job_id: str, ctx: Annotated[AuthContext, Depends(require_user)], ): - """Poll a background exec job started via POST /exec-jobs.""" - _ = ctx - return get_ne_exec_job(job_id) + """Poll own jobs; admins (role or admin:users scope) may read any job.""" + if not has_scope(ctx.scopes, "ne:exec"): + raise HTTPException(status_code=403, detail={"error": "insufficient_scope", "required": ["ne:exec"]}) + uid, _ = _actor(ctx) + is_admin = ctx.user.role == "admin" or has_scope(ctx.scopes, "admin:users") + out = get_ne_exec_job(job_id, user_id=uid, is_admin=is_admin) + if out.get("error_code") in {"job_forbidden", "invalid_job_id", "job_id_required"}: + status = 403 if out["error_code"] == "job_forbidden" else 400 + raise HTTPException(status_code=status, detail=out) + return out @router.post("/connect-test") diff --git a/netx_api/mcp/__init__.py b/netx_api/mcp/__init__.py deleted file mode 100644 index 35f01fb..0000000 --- a/netx_api/mcp/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -"""Backward-compatible re-exports; prefer ``pip install netx-mcp`` and ``import netx_mcp``.""" - -from netx_mcp.http_tools import HTTP_MCP_TOOLS, call_http_tool - -__all__ = ["HTTP_MCP_TOOLS", "call_http_tool"] diff --git a/netx_api/mcp/__main__.py b/netx_api/mcp/__main__.py deleted file mode 100644 index f3a2a9b..0000000 --- a/netx_api/mcp/__main__.py +++ /dev/null @@ -1,4 +0,0 @@ -from netx_mcp.server import main - -if __name__ == "__main__": - main() diff --git a/netx_api/mcp/db_server.py b/netx_api/mcp/db_server.py deleted file mode 100644 index 7e1b8f4..0000000 --- a/netx_api/mcp/db_server.py +++ /dev/null @@ -1,573 +0,0 @@ -from __future__ import annotations - -import json -import sys -from typing import Any - -from .db import Base, SessionLocal, engine -from .importer import aggregate_alarms, query_alarms -from .models import AlarmBatch, UmeAlarmCurrent, UmeAlarmHistory, UmeInventoryNE -from .ume_client import UMEClient -from .ume_sync_service import sync_alarms_current, sync_alarms_history_full, sync_inventory_full, sync_topology_full -from .ume_token_store import ( - clear_shared_token, - load_shared_token, - release_refresh_lock, - save_shared_token, - try_acquire_refresh_lock, - wait_for_token_update, -) - -_UME_CLIENT_SINGLETON = UMEClient( - token_loader=lambda: load_shared_token(), - token_saver=lambda token, exp: save_shared_token(token, exp), - token_clearer=lambda: clear_shared_token(), - lock_acquirer=lambda: try_acquire_refresh_lock(), - lock_releaser=lambda: release_refresh_lock(), - token_waiter=lambda min_exp: wait_for_token_update(min_expires_at_epoch_s=float(min_exp)), -) - - -def _ok(rid: Any, result: dict[str, Any]) -> None: - sys.stdout.write(json.dumps({"jsonrpc": "2.0", "id": rid, "result": result}, ensure_ascii=False) + "\n") - sys.stdout.flush() - - -def _err(rid: Any, code: int, message: str) -> None: - sys.stdout.write( - json.dumps({"jsonrpc": "2.0", "id": rid, "error": {"code": code, "message": message}}, ensure_ascii=False) - + "\n" - ) - sys.stdout.flush() - - -def _tool_list() -> list[dict[str, Any]]: - return [ - { - "name": "queryAlarms", - "description": "Query normalized alarms with filters and pagination.", - "inputSchema": { - "type": "object", - "properties": { - "batch_id": {"type": "string"}, - "alarm_code": {"type": "string"}, - "severity": {"type": "string"}, - "ne_name": {"type": "string"}, - "page": {"type": "integer", "minimum": 1}, - "page_size": {"type": "integer", "minimum": 1, "maximum": 200}, - }, - "additionalProperties": False, - }, - }, - { - "name": "aggregateAlarms", - "description": "Aggregate alarms by severity_norm, alarm_code or ne_name.", - "inputSchema": { - "type": "object", - "properties": { - "group_by": { - "type": "string", - "enum": ["severity_norm", "alarm_code", "ne_name"], - }, - "batch_id": {"type": "string"}, - }, - "required": ["group_by"], - "additionalProperties": False, - }, - }, - { - "name": "getImportBatch", - "description": "Get imported batch summary by batch_id.", - "inputSchema": { - "type": "object", - "properties": {"batch_id": {"type": "string"}}, - "required": ["batch_id"], - "additionalProperties": False, - }, - }, - { - "name": "runDiagnostics", - "description": "Generate quick diagnostics summary by batch.", - "inputSchema": { - "type": "object", - "properties": {"batch_id": {"type": "string"}}, - "required": ["batch_id"], - "additionalProperties": False, - }, - }, - { - "name": "umeSync", - "description": "Trigger UME sync for inventory/current/history domains.", - "inputSchema": { - "type": "object", - "properties": { - "domains": { - "type": "array", - "items": {"type": "string", "enum": ["inventory", "alarms_current", "alarms_history", "topology"]}, - }, - "trigger_mode": {"type": "string", "enum": ["manual", "schedule"]}, - }, - "additionalProperties": False, - }, - }, - { - "name": "umeListNE", - "description": "List UME network elements from inventory table.", - "inputSchema": { - "type": "object", - "properties": { - "keyword": {"type": "string"}, - "page": {"type": "integer", "minimum": 1}, - "page_size": {"type": "integer", "minimum": 1, "maximum": 500}, - }, - "additionalProperties": False, - }, - }, - { - "name": "umeGetNE", - "description": "Get UME network element by ne_id.", - "inputSchema": { - "type": "object", - "properties": {"ne_id": {"type": "string"}}, - "required": ["ne_id"], - "additionalProperties": False, - }, - }, - { - "name": "umeListCurrentAlarms", - "description": "List current UME alarms from current table.", - "inputSchema": { - "type": "object", - "properties": { - "severity": {"type": "string"}, - "is_cleared": {"type": "string"}, - "ne_id": {"type": "string"}, - "keyword": {"type": "string"}, - "page": {"type": "integer", "minimum": 1}, - "page_size": {"type": "integer", "minimum": 1, "maximum": 500}, - }, - "additionalProperties": False, - }, - }, - { - "name": "umeListHistoryAlarms", - "description": "List historical UME alarms from history table.", - "inputSchema": { - "type": "object", - "properties": { - "severity": {"type": "string"}, - "ne_id": {"type": "string"}, - "keyword": {"type": "string"}, - "page": {"type": "integer", "minimum": 1}, - "page_size": {"type": "integer", "minimum": 1, "maximum": 500}, - }, - "additionalProperties": False, - }, - }, - ] - - -def _call_tool(name: str, args: dict[str, Any]) -> dict[str, Any]: - with SessionLocal() as db: - if name == "queryAlarms": - total, rows = query_alarms( - db, - batch_id=str(args.get("batch_id") or "").strip() or None, - alarm_code=str(args.get("alarm_code") or "").strip() or None, - severity=str(args.get("severity") or "").strip() or None, - ne_name=str(args.get("ne_name") or "").strip() or None, - page=max(1, int(args.get("page") or 1)), - page_size=min(200, max(1, int(args.get("page_size") or 50))), - ) - payload = { - "total": total, - "items": [ - { - "id": r.id, - "batch_id": r.batch_id, - "alarm_time": r.alarm_time.isoformat(), - "severity_norm": r.severity_norm, - "ne_name": r.ne_name, - "alarm_code": r.alarm_code, - "alarm_name": r.alarm_name, - } - for r in rows - ], - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "aggregateAlarms": - group_by = str(args.get("group_by") or "").strip() - rows = aggregate_alarms( - db, - group_by=group_by, - batch_id=str(args.get("batch_id") or "").strip() or None, - ) - payload = {"group_by": group_by, "buckets": [{"key": k, "count": v} for k, v in rows]} - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "getImportBatch": - batch_id = str(args.get("batch_id") or "").strip() - batch = db.get(AlarmBatch, batch_id) - if not batch: - return {"content": [{"type": "text", "text": json.dumps({"error": "batch_not_found"})}], "isError": True} - payload = { - "batch_id": batch.batch_id, - "total_rows": batch.total_rows, - "success_rows": batch.success_rows, - "failed_rows": batch.failed_rows, - "status": batch.status, - "created_at": batch.created_at.isoformat(), - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "runDiagnostics": - batch_id = str(args.get("batch_id") or "").strip() - sev_rows = aggregate_alarms(db, group_by="severity_norm", batch_id=batch_id) - code_rows = aggregate_alarms(db, group_by="alarm_code", batch_id=batch_id)[:5] - ne_rows = aggregate_alarms(db, group_by="ne_name", batch_id=batch_id)[:5] - sev_map = {k: v for k, v in sev_rows} - findings: list[str] = [] - actions: list[str] = [] - risk_level = "low" - if int(sev_map.get("critical", 0)) > 0: - findings.append("critical 告警存在，建议优先确认核心网元影响。") - actions.append("优先处理 critical 告警，确认影响面并升级。") - risk_level = "high" - if int(sev_map.get("warning", 0)) > int(sev_map.get("major", 0)): - findings.append("warning 占比较高，疑似阈值型告警风暴。") - actions.append("检查高频 warning 告警码是否集中在单一阈值策略。") - if risk_level != "high": - risk_level = "medium" - if not findings: - findings.append("分布相对均衡，建议按 top 告警码进一步排查。") - actions.append("按 top 告警码和网元继续下钻分析。") - payload = { - "batch_id": batch_id, - "risk_level": risk_level, - "severity_summary": [{"key": k, "count": v} for k, v in sev_rows], - "top_alarm_codes": [{"key": k, "count": v} for k, v in code_rows], - "top_ne": [{"key": k, "count": v} for k, v in ne_rows], - "findings": findings, - "actions": actions, - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "umeSync": - domains_raw = args.get("domains") - domains = [] - if isinstance(domains_raw, list): - domains = [str(x).strip() for x in domains_raw if str(x).strip()] - if not domains: - domains = ["inventory", "alarms_current", "alarms_history"] - trigger_mode = str(args.get("trigger_mode") or "manual").strip().lower() - if trigger_mode not in {"manual", "schedule"}: - trigger_mode = "manual" - client = _UME_CLIENT_SINGLETON - results: list[dict[str, Any]] = [] - if "inventory" in domains: - j = sync_inventory_full(db, client, trigger_mode=trigger_mode) - results.append( - { - "domain": "inventory", - "status": j.status, - "pulled_count": int(j.pulled_count or 0), - "inserted_count": int(j.inserted_count or 0), - "updated_count": int(j.updated_count or 0), - "error_message": str(j.error_message or ""), - } - ) - if "alarms_current" in domains: - j, b = sync_alarms_current(db, client, trigger_mode=trigger_mode) - results.append( - { - "domain": "alarms_current", - "status": j.status, - "batch_id": str(b.batch_id), - "pulled_count": int(j.pulled_count or 0), - "inserted_count": int(j.inserted_count or 0), - "updated_count": int(j.updated_count or 0), - "error_message": str(j.error_message or ""), - } - ) - if "alarms_history" in domains: - j, b = sync_alarms_history_full(db, client, trigger_mode=trigger_mode) - results.append( - { - "domain": "alarms_history", - "status": j.status, - "batch_id": str(b.batch_id), - "pulled_count": int(j.pulled_count or 0), - "inserted_count": int(j.inserted_count or 0), - "updated_count": int(j.updated_count or 0), - "error_message": str(j.error_message or ""), - } - ) - if "topology" in domains: - j = sync_topology_full(db, client, trigger_mode=trigger_mode) - results.append( - { - "domain": "topology", - "status": j.status, - "pulled_count": int(j.pulled_count or 0), - "inserted_count": int(j.inserted_count or 0), - "updated_count": int(j.updated_count or 0), - "error_message": str(j.error_message or ""), - } - ) - return {"content": [{"type": "text", "text": json.dumps({"ok": True, "jobs": results}, ensure_ascii=False)}]} - if name == "umeListNE": - stmt = db.query(UmeInventoryNE) - keyword = str(args.get("keyword") or "").strip() - if keyword: - stmt = stmt.filter( - UmeInventoryNE.ne_id.contains(keyword) - | UmeInventoryNE.ne_name.contains(keyword) - | UmeInventoryNE.user_label.contains(keyword) - | UmeInventoryNE.ip_address.contains(keyword) - ) - page = max(1, int(args.get("page") or 1)) - page_size = min(500, max(1, int(args.get("page_size") or 50))) - total = int(stmt.count()) - rows = stmt.order_by(UmeInventoryNE.ne_id.asc()).offset((page - 1) * page_size).limit(page_size).all() - payload = { - "total": total, - "page": page, - "page_size": page_size, - "items": [ - { - "ne_id": str(x.ne_id or ""), - "ne_name": str(x.ne_name or ""), - "user_label": str(x.user_label or ""), - "ip_address": str(x.ip_address or ""), - "ne_type": str(x.ne_type or ""), - } - for x in rows - ], - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "umeGetNE": - ne_id = str(args.get("ne_id") or "").strip() - row = db.get(UmeInventoryNE, ne_id) - if not row: - return {"content": [{"type": "text", "text": json.dumps({"error": "ume_ne_not_found"})}], "isError": True} - payload = { - "ne_id": str(row.ne_id or ""), - "ne_name": str(row.ne_name or ""), - "user_label": str(row.user_label or ""), - "ip_address": str(row.ip_address or ""), - "ne_type": str(row.ne_type or ""), - "vendor": str(row.vendor or ""), - "raw_json": str(row.raw_json or "{}"), - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "umeListCurrentAlarms": - stmt = db.query(UmeAlarmCurrent, UmeInventoryNE).outerjoin( - UmeInventoryNE, UmeAlarmCurrent.ne_id == UmeInventoryNE.ne_id - ) - severity = str(args.get("severity") or "").strip() - is_cleared = str(args.get("is_cleared") or "").strip() - ne_id = str(args.get("ne_id") or "").strip() - keyword = str(args.get("keyword") or "").strip() - if severity: - stmt = stmt.filter(UmeAlarmCurrent.perceived_severity == severity) - if is_cleared: - stmt = stmt.filter(UmeAlarmCurrent.is_cleared == is_cleared) - if ne_id: - stmt = stmt.filter(UmeAlarmCurrent.ne_id == ne_id) - if keyword: - stmt = stmt.filter( - UmeAlarmCurrent.alarm_key.contains(keyword) - | UmeAlarmCurrent.object_name.contains(keyword) - | UmeAlarmCurrent.host_name.contains(keyword) - | UmeInventoryNE.ne_name.contains(keyword) - | UmeInventoryNE.user_label.contains(keyword) - | UmeInventoryNE.ip_address.contains(keyword) - | UmeInventoryNE.host_name.contains(keyword) - ) - page = max(1, int(args.get("page") or 1)) - page_size = min(500, max(1, int(args.get("page_size") or 50))) - total = int(stmt.count()) - rows = ( - stmt.order_by( - UmeAlarmCurrent.time_created.desc(), - UmeAlarmCurrent.last_seen_at.desc(), - UmeAlarmCurrent.alarm_key.desc(), - ) - .offset((page - 1) * page_size) - .limit(page_size) - .all() - ) - payload = { - "total": total, - "page": page, - "page_size": page_size, - "items": [ - { - "alarm_key": str(alarm.alarm_key or ""), - "ne_id": str(alarm.ne_id or ""), - "host_name": str(alarm.host_name or (ne.host_name if ne else "") or ""), - "ne_name": str((ne.ne_name if ne else "") or ""), - "user_label": str((ne.user_label if ne else "") or ""), - "object_name": str(alarm.object_name or ""), - "event_type": str(alarm.event_type or ""), - "native_probable_cause": str(alarm.native_probable_cause or ""), - "perceived_severity": str(alarm.perceived_severity or ""), - "is_cleared": str(alarm.is_cleared or ""), - "time_created": str(alarm.time_created or ""), - } - for alarm, ne in rows - ], - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - if name == "umeListHistoryAlarms": - stmt = db.query(UmeAlarmHistory, UmeInventoryNE).outerjoin( - UmeInventoryNE, UmeAlarmHistory.ne_id == UmeInventoryNE.ne_id - ) - severity = str(args.get("severity") or "").strip() - ne_id = str(args.get("ne_id") or "").strip() - keyword = str(args.get("keyword") or "").strip() - if severity: - stmt = stmt.filter(UmeAlarmHistory.perceived_severity == severity) - if ne_id: - stmt = stmt.filter(UmeAlarmHistory.ne_id == ne_id) - if keyword: - stmt = stmt.filter( - UmeAlarmHistory.alarm_key.contains(keyword) - | UmeAlarmHistory.object_name.contains(keyword) - | UmeAlarmHistory.host_name.contains(keyword) - | UmeInventoryNE.ne_name.contains(keyword) - | UmeInventoryNE.user_label.contains(keyword) - | UmeInventoryNE.ip_address.contains(keyword) - | UmeInventoryNE.host_name.contains(keyword) - ) - page = max(1, int(args.get("page") or 1)) - page_size = min(500, max(1, int(args.get("page_size") or 50))) - total = int(stmt.count()) - rows = stmt.order_by(UmeAlarmHistory.last_seen_at.desc()).offset((page - 1) * page_size).limit(page_size).all() - payload = { - "total": total, - "page": page, - "page_size": page_size, - "items": [ - { - "alarm_key": str(alarm.alarm_key or ""), - "ne_id": str(alarm.ne_id or ""), - "host_name": str(alarm.host_name or (ne.host_name if ne else "") or ""), - "ne_name": str((ne.ne_name if ne else "") or ""), - "user_label": str((ne.user_label if ne else "") or ""), - "object_name": str(alarm.object_name or ""), - "event_type": str(alarm.event_type or ""), - "native_probable_cause": str(alarm.native_probable_cause or ""), - "perceived_severity": str(alarm.perceived_severity or ""), - "is_cleared": str(alarm.is_cleared or ""), - "time_created": str(alarm.time_created or ""), - } - for alarm, ne in rows - ], - } - return {"content": [{"type": "text", "text": json.dumps(payload, ensure_ascii=False)}]} - raise ValueError(f"unknown tool: {name}") - - -def _ensure_db_schema() -> None: - try: - Base.metadata.create_all(bind=engine) - with engine.begin() as conn: - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS device_level VARCHAR(64) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS host_name VARCHAR(256) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS location VARCHAR(512) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS ipv6_address VARCHAR(128) DEFAULT ''") - conn.exec_driver_sql( - "ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS hardware_version VARCHAR(128) DEFAULT ''" - ) - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS loopback VARCHAR(128) DEFAULT ''") - conn.exec_driver_sql( - "ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS consistent_state VARCHAR(64) DEFAULT ''" - ) - conn.exec_driver_sql( - "ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS interface_version VARCHAR(128) DEFAULT ''" - ) - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS mac VARCHAR(128) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS admin_status VARCHAR(64) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS address_type VARCHAR(64) DEFAULT ''") - conn.exec_driver_sql( - "ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS connection_status VARCHAR(64) DEFAULT ''" - ) - conn.exec_driver_sql( - "ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS maintain_status VARCHAR(64) DEFAULT ''" - ) - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS net_mask VARCHAR(128) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS create_time VARCHAR(64) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_inventory_ne ADD COLUMN IF NOT EXISTS creator VARCHAR(128) DEFAULT ''") - conn.exec_driver_sql("ALTER TABLE ume_alarms_current DROP COLUMN IF EXISTS ne_name") - conn.exec_driver_sql("ALTER TABLE ume_alarms_current DROP COLUMN IF EXISTS user_label") - conn.exec_driver_sql("ALTER TABLE ume_alarms_history DROP COLUMN IF EXISTS ne_name") - conn.exec_driver_sql("ALTER TABLE ume_alarms_history DROP COLUMN IF EXISTS user_label") - conn.exec_driver_sql("COMMENT ON TABLE ume_inventory_ne IS '网元对象详细信息'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.ne_id IS '网元uuid'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.ne_name IS '资源名称'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.ne_type IS '网元类型'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.user_label IS '用户标签'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.address_type IS '管理地址类型(1:IPv4,2:IPv6)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.ip_address IS '网元IPv4地址'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.net_mask IS '管理IPv4掩码(点分十进制)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.ipv6_address IS 'IPv6地址'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.admin_status IS '管理状态(0-离线,1-在线)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.connection_status IS '连接状态(0-断链,1-正常)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.consistent_state IS '数据一致性状态(1一致,2不一致,3冲突)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.maintain_status IS '工程状态(0普通,1调测,2新建)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.vendor IS '网元提供商'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.interface_version IS '网元接口版本号'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.hardware_version IS '硬件版本'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.mac IS '设备机架MAC地址'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.loopback IS '业务环回IP(IPv4)'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.device_level IS '网元层次'") - conn.exec_driver_sql("COMMENT ON COLUMN ume_inventory_ne.host_name IS '主机名称'") - except Exception: - pass - - -def run_stdio_loop() -> None: - """Legacy MCP: direct PostgreSQL access (deprecated; use HTTP mode).""" - _ensure_db_schema() - for line in sys.stdin: - raw = line.strip() - if not raw: - continue - try: - req = json.loads(raw) - except Exception: - continue - rid = req.get("id") - method = str(req.get("method") or "") - params = req.get("params") if isinstance(req.get("params"), dict) else {} - - try: - if method == "initialize": - _ok( - rid, - { - "protocolVersion": "2024-11-05", - "capabilities": {"tools": {}}, - "serverInfo": {"name": "netx-mcp", "version": "0.1.0"}, - }, - ) - continue - if method == "notifications/initialized": - continue - if method == "tools/list": - _ok(rid, {"tools": _tool_list()}) - continue - if method == "tools/call": - name = str(params.get("name") or "") - args = params.get("arguments") if isinstance(params.get("arguments"), dict) else {} - _ok(rid, _call_tool(name, args)) - continue - _err(rid, -32601, f"method not found: {method}") - except Exception as exc: - _err(rid, -32000, str(exc)) - - -def main() -> None: - run_stdio_loop() - - -if __name__ == "__main__": - main() diff --git a/netx_api/mcp/http_client.py b/netx_api/mcp/http_client.py deleted file mode 100644 index 0288d66..0000000 --- a/netx_api/mcp/http_client.py +++ /dev/null @@ -1,3 +0,0 @@ -"""Re-export from ``netx-mcp`` package (install: ``pip install -e packages/netx-mcp``).""" - -from netx_mcp.http_client import * # noqa: F403 diff --git a/netx_api/mcp/http_tools.py b/netx_api/mcp/http_tools.py deleted file mode 100644 index d77966c..0000000 --- a/netx_api/mcp/http_tools.py +++ /dev/null @@ -1,3 +0,0 @@ -"""Re-export from ``netx-mcp`` package (install: ``pip install -e packages/netx-mcp``).""" - -from netx_mcp.http_tools import * # noqa: F403 diff --git a/netx_api/mcp_server.py b/netx_api/mcp_server.py deleted file mode 100644 index 4f31601..0000000 --- a/netx_api/mcp_server.py +++ /dev/null @@ -1,32 +0,0 @@ -"""Backward-compatible entry for ``python netx_api/mcp_server.py`` / ``netx-mcp`` script on netx-ops install. - -HTTP MCP: delegates to ``netx_mcp`` (``pip install -e packages/netx-mcp``). -Legacy DB mode (``NETX_MCP_MODE=db``): requires full ``netx-ops`` install only. -""" - -from __future__ import annotations - -import os -import sys -from pathlib import Path - -# Script-path launch without pip install netx-mcp: add packages/netx-mcp/src -_repo_root = Path(__file__).resolve().parent.parent -_mcp_src = _repo_root / "packages" / "netx-mcp" / "src" -if _mcp_src.is_dir() and str(_mcp_src) not in sys.path: - sys.path.insert(0, str(_mcp_src)) - - -def main() -> None: - if str(os.getenv("NETX_MCP_MODE") or "http").strip().lower() == "db": - from netx_api.mcp.db_server import run_stdio_loop - - run_stdio_loop() - return - from netx_mcp.server import main as mcp_main - - mcp_main() - - -if __name__ == "__main__": - main() diff --git a/netx_api/ne_collect_runner.py b/netx_api/ne_collect_runner.py index bca7e06..0e637c7 100644 --- a/netx_api/ne_collect_runner.py +++ b/netx_api/ne_collect_runner.py @@ -87,21 +87,28 @@ def _collect_on_device( if conn_holder is not None: conn_holder["conn"] = conn conn_holder["session_log"] = log_buf - # Belt-and-suspenders: hop/bastion nested CLIs and missed session_prep. - try: - disable_target_paging( - conn, - vendor=str(creds.get("vendor") or ""), - device_type=str(creds.get("device_type") or ""), - ) - except Exception: - _log.debug("collection paging disable failed", exc_info=True) - prompt = str(conn.find_prompt() or "") - # find_prompt / paging-off often leave an extra prompt in the channel; - # drain before the first show or Netmiko returns empty (IOSv). - try: - from .ne_netmiko import drain_read_channel + from .ne_netmiko import drain_read_channel, is_cisco_ios_device_type + device_type = str(creds.get("device_type") or "") + # Direct Cisco session_prep already set width/length. A second timing-based + # ``terminal length 0`` plus find_prompt() re-introduces leftover prompts + # and line-buffer races on IOSv. Keep the extra paging pass for hop/other. + need_paging_pass = bool(creds.get("hop_enabled")) or not is_cisco_ios_device_type( + device_type + ) + if need_paging_pass: + try: + disable_target_paging( + conn, + vendor=str(creds.get("vendor") or ""), + device_type=device_type, + ) + except Exception: + _log.debug("collection paging disable failed", exc_info=True) + # Transcript header only — never find_prompt() here (sends Enter). + base = str(getattr(conn, "base_prompt", "") or "").strip() + prompt = f"{base}#" if base else "" + try: drain_read_channel(conn) except Exception: _log.debug("collection channel drain failed", exc_info=True) diff --git a/netx_api/ne_exec_jobs.py b/netx_api/ne_exec_jobs.py index 315d37f..dffacae 100644 --- a/netx_api/ne_exec_jobs.py +++ b/netx_api/ne_exec_jobs.py @@ -8,6 +8,7 @@ from __future__ import annotations import json import os +import re import threading import time import uuid @@ -24,6 +25,8 @@ _INFLIGHT = 0 _DEFAULT_TIMEOUT_S = 900 _DEFAULT_MAX_CONCURRENT = 3 _DEFAULT_ASYNC_MIN_NES = 4 +_JOB_ID_RE = re.compile(r"[0-9a-f]{32}") +_MAX_JOB_TARGETS = 20 # Same combined limit as ne_exec._normalize_batch_targets. def _env_int(name: str, default: int, *, min_v: int, max_v: int) -> int: @@ -72,7 +75,14 @@ def _jobs_dir() -> Path: def _job_path(job_id: str) -> Path: - return _jobs_dir() / f"{job_id}.json" + if not isinstance(job_id, str) or not _JOB_ID_RE.fullmatch(job_id): + raise ValueError("invalid_job_id") + directory = _jobs_dir().resolve() + path = (directory / f"{job_id}.json").resolve() + # Also reject a valid-looking filename symlinked outside the job directory. + if path.parent != directory: + raise ValueError("invalid_job_id") + return path def _atomic_write(path: Path, payload: dict[str, Any]) -> None: @@ -90,13 +100,11 @@ def _read_job(job_id: str) -> dict[str, Any] | None: data = json.loads(path.read_text(encoding="utf-8")) except Exception: return None - return data if isinstance(data, dict) else None + return data if isinstance(data, dict) and data.get("job_id") == job_id else None def _write_job(job: dict[str, Any]) -> None: - jid = str(job.get("job_id") or "").strip() - if not jid: - return + jid = job.get("job_id") _atomic_write(_job_path(jid), job) @@ -116,17 +124,16 @@ def default_job_timeout_s() -> int: def count_exec_ne_targets(args: dict[str, Any] | None) -> int: a = args if isinstance(args, dict) else {} n = 0 - for key in ("ne_ids", "ume_ne_ids", "nms_ne_ids"): + for key in ("ne_ids", "ume_ne_ids"): val = a.get(key) if isinstance(val, list): - n = max(n, len([x for x in val if str(x or "").strip()])) + n += len([x for x in val if str(x or "").strip()]) targets = a.get("targets") if isinstance(targets, list): - n = max(n, len([t for t in targets if isinstance(t, dict)])) + n += len([t for t in targets if isinstance(t, dict)]) if n == 0 and ( str(a.get("ne_id") or "").strip() or str(a.get("ume_ne_id") or "").strip() - or str(a.get("nms_ne_id") or "").strip() ): return 1 return int(n) @@ -164,11 +171,18 @@ def start_ne_exec_job( *, kind: str, arguments: dict[str, Any], + owner_user_id: str, runner: Callable[[], dict[str, Any]], timeout_s: int | None = None, ) -> dict[str, Any]: """Start background runner; returns immediate ack with job_id.""" global _INFLIGHT + owner = str(owner_user_id or "").strip() + if not owner: + return {"ok": False, "error_code": "job_owner_required", "error": "job_owner_required"} + ne_count = count_exec_ne_targets(arguments) + if ne_count > _MAX_JOB_TARGETS: + return {"ok": False, "error_code": "too_many_targets", "error": "too_many_targets", "max_targets": _MAX_JOB_TARGETS} max_c = max_concurrent_jobs() with _LOCK: if _INFLIGHT >= max_c: @@ -189,10 +203,11 @@ def start_ne_exec_job( now = int(time.time() * 1000) job = { "job_id": job_id, + "owner_user_id": owner, "status": _STATUS_RUNNING, "kind": str(kind or "exec"), "arguments": dict(arguments or {}), - "ne_count": count_exec_ne_targets(arguments), + "ne_count": ne_count, "created_at_ms": now, "updated_at_ms": now, "timeout_s": timeout, @@ -260,13 +275,26 @@ def start_ne_exec_job( } -def get_ne_exec_job(job_id: str) -> dict[str, Any]: - jid = str(job_id or "").strip() +def get_ne_exec_job(job_id: str, *, user_id: str, is_admin: bool = False) -> dict[str, Any]: + """Read only the caller's jobs; explicit admins may inspect any persisted job. + + Ownerless legacy records are never granted to ordinary callers. + """ + jid = job_id if not jid: return {"ok": False, "error_code": "job_id_required", "error": "job_id_required"} - job = _read_job(jid) + try: + job = _read_job(jid) + except ValueError: + return {"ok": False, "error_code": "invalid_job_id", "error": "invalid_job_id"} + requester = str(user_id or "").strip() + if not requester: + return {"ok": False, "error_code": "job_forbidden", "error": "job_forbidden"} if not job: return {"ok": False, "error_code": "job_not_found", "error": "job_not_found", "job_id": jid} + owner = str(job.get("owner_user_id") or "").strip() + if not is_admin and (not owner or owner != requester): + return {"ok": False, "error_code": "job_forbidden", "error": "job_forbidden"} status = str(job.get("status") or "") out: dict[str, Any] = { "ok": True, diff --git a/netx_api/ne_netmiko.py b/netx_api/ne_netmiko.py index 5353af1..9814d3d 100644 --- a/netx_api/ne_netmiko.py +++ b/netx_api/ne_netmiko.py @@ -2,6 +2,7 @@ from __future__ import annotations +import re import time from typing import Any @@ -138,6 +139,35 @@ def drain_read_channel( def _send_command_expect_prompt(conn: Any, cmd: str, *, read_timeout: int) -> str: device_type = getattr(conn, "device_type", "") + if isinstance(device_type, str) and is_cisco_ios_device_type(device_type): + # IOSv has the same late-prompt race as RouterOS: auto_find_prompt sends + # a newline after our drain and its reply can terminate the next show. + # Use the session's cached hostname, but require a full prompt line so + # "R2 uptime ..." inside show version cannot end the command early. + # + # cmd_verify must stay False: Netmiko hardcodes a 10s echo wait, and + # IOSv often echoes ``show version | include ...`` slower than that + # (live: ~13s) even when the command itself succeeds. Correctness comes + # from drain_read_channel + full-line expect_string, not echo verify. + base_prompt = getattr(conn, "base_prompt", "") + if not isinstance(base_prompt, str) or not base_prompt.strip(): + raise ValueError("cisco_base_prompt_missing") + prompt_pattern = rf"(?m:^{re.escape(base_prompt.strip())}[>#][ \t]*$)" + out = str( + conn.send_command( + command_string=cmd, + read_timeout=read_timeout, + auto_find_prompt=False, + expect_string=prompt_pattern, + cmd_verify=False, + ) + or "" + ) + # Leftover-prompt races still yield a silent empty body with ok=true at + # higher layers. Fail loud here — never re-send the command. + if not out.strip(): + raise ValueError("cisco_empty_show_output") + return out if isinstance(device_type, str) and device_type.lower().startswith("mikrotik"): # find_prompt sends an extra Enter *after* our channel drain. RouterOS # can repaint that old prompt after send_command starts, ending the read @@ -179,8 +209,10 @@ def send_show_command(conn: Any, command: str, *, read_timeout: int = 120) -> st Prefer ``send_command`` over ``send_command_timing`` for long config dumps: idle during ``Building configuration...`` truncates timing reads. - ``cmd_verify=False``: Netmiko's default echo check often raises - ``Pattern not detected: 'show\\ lldp\\ ...'`` on IOSv / hop / slow echo paths. + Cisco uses ``cmd_verify=False`` with a full-line prompt expect after drain; + Netmiko's echo verify is hardcoded to 10s and false-fails slow IOSv echoes. + Other drivers also use ``cmd_verify=False``: the default echo check often + raises ``Pattern not detected`` on hop / slow echo paths. """ cmd = str(command or "").strip() if not cmd: @@ -188,5 +220,9 @@ def send_show_command(conn: Any, command: str, *, read_timeout: int = 120) -> st # IOSv / hop: find_prompt / paging-off often leave an extra prompt in the # channel; drain so send_command does not match that leftover immediately. - drain_read_channel(conn) + device_type = getattr(conn, "device_type", "") + if isinstance(device_type, str) and is_cisco_ios_device_type(device_type): + drain_read_channel(conn, idle_reads=6, pause_sec=0.08) + else: + drain_read_channel(conn) return _send_command_expect_prompt(conn, cmd, read_timeout=read_timeout) diff --git a/netx_api/ne_session_connect.py b/netx_api/ne_session_connect.py index 049a850..601e669 100644 --- a/netx_api/ne_session_connect.py +++ b/netx_api/ne_session_connect.py @@ -304,24 +304,23 @@ class _ProgressSessionLog: def _cisco_ios_collection_driver_class(base_cls: type) -> type: - """Cisco IOSv-friendly session prep: avoid cmd_verify on terminal width/length.""" + """Cisco IOSv collection prep: establish hostname prompt only (like interactive).""" class _CiscoIosCollectionSession(base_cls): # type: ignore[misc,valid-type] def session_preparation(self) -> None: - # Default Netmiko waits for exact echo of "terminal width 511" (ReadTimeout on IOSv). - self._test_channel_read(pattern=r"[>#]") - try: - self.set_terminal_width(command="terminal width 511", pattern=r"[>#]") - except Exception: - pass - try: - self.disable_paging(command="terminal length 0", cmd_verify=False, pattern=r"[>#]") - except Exception: - try: - self.send_command_timing("terminal length 0", read_timeout=15) - except Exception: - pass - self.set_base_prompt() + # Stock Netmiko width/paging on IOSv+CR races leftover prompts + # (``terexit`` / ``terminal lengexit``) or blocks forever waiting for + # setup echo once VTY pressure builds. The interactive WebCRT path + # skips these commands and still returns complete show output; MCP + # show/display bodies are short enough that --More-- is not required. + # Keep hostname discovery so send_show_command can use a full-line + # prompt expect without auto_find_prompt. + from .ne_netmiko import drain_read_channel + + initial_pattern = r"(?m:^[^\r\n#>]+[>#][ \t]*$)" + self._test_channel_read(pattern=initial_pattern) + self.set_base_prompt(pattern=initial_pattern) + drain_read_channel(self, idle_reads=8, pause_sec=0.1) _CiscoIosCollectionSession.__name__ = ( f"CiscoIosCollection{getattr(base_cls, '__name__', 'Netmiko')}" @@ -508,6 +507,13 @@ def _base_connect_kwargs( "auth_timeout": timeout, "banner_timeout": timeout, } + # IOSv SSH accepts CR as Enter. LF alone can leave command input pending + # until a later write, producing missing echo, empty output or prompt timeouts. + # Keep Telnet's own line ending and other vendors' defaults unchanged. + from .ne_netmiko import is_cisco_ios_device_type + + if is_cisco_ios_device_type(device_type) and "telnet" not in device_type.lower(): + dev["default_enter"] = "\r" if session_timeout is not None: dev["session_timeout"] = session_timeout if keepalive is not None and int(keepalive) > 0: diff --git a/netx_api/topology_views_graph.py b/netx_api/topology_views_graph.py index 6409861..8fad1b0 100644 --- a/netx_api/topology_views_graph.py +++ b/netx_api/topology_views_graph.py @@ -749,6 +749,8 @@ def patch_view_positions( updated += 1 else: from .topology_region_canvas import add_missing_region_placement, is_region_canvas_node_id + from .ume_topology_world import is_ume_canvas_view, is_world_flat_view + from .ume_topology_world_graph import get_level_view_graph positions = list(body.positions or []) if len(positions) > VIEW_GRAPH_NODE_HARD_CAP: @@ -769,6 +771,24 @@ def patch_view_positions( .filter(TopoFabricNode.id.in_(check_ids)) .all() } + # UME canvases synthesize their members instead of adding them via /nodes. + # Persist placements only for actual members of this canvas; a position + # patch must never add unrelated fabric nodes to an ordinary view. + ume_member_ids: set[str] = set() + if is_ume_canvas_view(view) and any( + str(p.fabric_node_id or "").strip() not in existing for p in positions + ): + if is_world_flat_view(view): + ume_member_ids = { + str(row[0]) + for row in db.query(TopoFabricNode.id).filter( + TopoFabricNode.id.in_(check_ids), + TopoFabricNode.world_x.isnot(None), + TopoFabricNode.world_y.isnot(None), + ).all() + } + else: + ume_member_ids = {node.fabric_node_id for node in get_level_view_graph(db, view).nodes} for p in positions: fid = str(p.fabric_node_id or "").strip() if not fid: @@ -777,9 +797,17 @@ def patch_view_positions( raise HTTPException(status_code=400, detail=f"fabric_node_not_found:{fid}") row = existing.get(fid) if row is None: + if fid in ume_member_ids: + row = TopoViewNode( + id=uuid4().hex, view_id=view.id, fabric_node_id=fid, + x=float(p.x), y=float(p.y), label="", locked=False, + created_at=now, updated_at=now, + ) + db.add(row) # Child-region icons may be synthesized without a placement row; persist those. # Fabric NEs must not bypass membership caps / freeze (MCP adds via /nodes). - row = add_missing_region_placement(db, view, fid, x=p.x, y=p.y) + else: + row = add_missing_region_placement(db, view, fid, x=p.x, y=p.y) if row is None: skipped_missing += 1 continue diff --git a/netx_api/ume_topology_world_graph.py b/netx_api/ume_topology_world_graph.py index 135b51d..4d379df 100644 --- a/netx_api/ume_topology_world_graph.py +++ b/netx_api/ume_topology_world_graph.py @@ -46,9 +46,6 @@ WORLD_FLAT_DETAIL_CAP = VIEW_GRAPH_NODE_HARD_CAP # Viewport span (display/world units) at/under which we prefer a full region layout # instead of stride-thinning — stops “zoomed in still a peppered blob”. WORLD_FLAT_CLOSE_SPAN = 28000.0 -# Legacy crush span (pre-LOD). Flat graph is now 1:1 with fabric world_*; -# this constant only detects stale TopoViewNode overrides from that era. -WORLD_FLAT_DISPLAY_SPAN = 6000.0 def apply_persisted_view_positions( @@ -553,8 +550,6 @@ def get_flat_view_graph( full_max_x = float(extent_row[1]) full_min_y = float(extent_row[2]) full_max_y = float(extent_row[3]) - span_x = max(1.0, full_max_x - full_min_x) - span_y = max(1.0, full_max_y - full_min_y) # 1:1 with fabric world_* (only origin-shift to ~0). Infinite zoom keeps # each packed region block identical to its UME local relative layout. scale = 1.0 @@ -768,25 +763,9 @@ def get_flat_view_graph( ) for n in nodes ] - # Drag overrides in display space (world - origin). Skip legacy rows saved - # under the old ~6000 crush — they would yank nodes to the wrong place. - override_rows = ( - db.query(TopoViewNode).filter(TopoViewNode.view_id == view.id).limit(8).all() - ) - display_span = max(span_x, span_y) - overrides_ok = False - if override_rows and display_span > 0: - oxs = [abs(float(r.x or 0)) for r in override_rows] - oys = [abs(float(r.y or 0)) for r in override_rows] - peak = max(oxs + oys) if (oxs or oys) else 0.0 - legacy_crush = display_span > WORLD_FLAT_DISPLAY_SPAN * 2 and peak <= WORLD_FLAT_DISPLAY_SPAN * 1.5 - overrides_ok = (not legacy_crush) and all( - 0.0 <= float(r.x or 0) <= display_span * 1.2 - and 0.0 <= float(r.y or 0) <= display_span * 1.2 - for r in override_rows - ) - if overrides_ok: - nodes_out = apply_persisted_view_positions(db, view.id, nodes_out) + # Saved placements are authoritative display coordinates. Do not infer an + # obsolete coordinate system from their magnitude or reject valid negatives. + nodes_out = apply_persisted_view_positions(db, view.id, nodes_out) edges_out: list[ViewEdgeOut] = [] if include_edges and fids: diff --git a/packages/netx-bizmonitor-mcp/README.md b/packages/netx-bizmonitor-mcp/README.md new file mode 100644 index 0000000..b4ce924 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/README.md @@ -0,0 +1,62 @@ +# netx-bizmonitor-mcp + +独立 Python stdio MCP，只读访问 netx 的割接对比与业务状态 HTTP API。运行依赖仅 `httpx`，不依赖 `netx_api`、数据库驱动或 netxops。采用独立包与现有 topology 包、一组一 skill 的组织方式一致，客户端可按组挂载最小权限，避免扩大 ops 工具域及引入重依赖。 + +## 配置与启动 + +安装本包后使用 `python -m netx_bizmonitor_mcp`，或命令 `netx-bizmonitor-mcp`。无需安装时，将本包 `src` 的绝对路径设置为 `PYTHONPATH`。`mcp.json` 是客户端配置示例，需改为实际 Python/src 路径与令牌。 + +使用凭据文件时，应删除样例的 NETX_API_TOKEN 占位值或设为空；同时为客户端设定正确工作目录，或用 NETX_MCP_TOKEN_FILE 指定绝对路径。 + +- `NETX_API_URL`：默认 `http://127.0.0.1:8890`，支持 HTTP(S) 和路径前缀；禁止内嵌用户名/密码、查询参数、片段。 +- `NETX_API_TOKEN`：最高优先级 Bearer API 令牌；为空时依次读 `NETX_MCP_TOKEN_FILE` 指定的 UTF-8 文件、当前工作目录的 `data/auth/mcp_token`。文件为空、缺失、不可读或编码错误时继续下一候选。后端要求 **`biz-monitor:read`**，`ne:read` 或 `ops:write` 不代替此 scope。无可用令牌时返回 `netx_token_missing`，不发送请求。 +- `NETX_LANG`：默认 `zh`；`en` 时附加 `lang=en`。 +- Windows 建议 `PYTHONIOENCODING=utf-8`、`PYTHONUTF8=1`；程序也会将 stdio 配为 UTF-8。 + +令牌读取遵循现有独立 MCP 的凭据文件约定，仅在内存使用，文件内容与路径不回显；响应脱敏覆盖文件令牌，包括请求中途文件轮换前实际使用的令牌。所有业务请求仅 GET；不会触发采集、评估、CLI 执行或设备写入。通过 `GET /v1/auth/me` 探测权限，每次工具调用重新读取，避免缓存权限撤销；探测失败时仍列出工具，由目标 GET 的后端权限检查最终裁决。 + +## 工具与 HTTP 路由 + +工具名与 netxops bizMonitor 组相同，去掉 `netx__` 前缀，仅提供以下 8 个工具： + +1. `listBizMonitors` → `GET /v1/biz-migration/projects` 与/或 `GET /v1/biz-state/tasks`。 + `kind=all|projects|tasks`，`purpose` 只转发任务查询；`status/q` 本地过滤，`limit` 默认 100、上限 500，各列表分别限额。项目压缩为名称、状态、任务 ID、设备摘要、模板名称、高频参数与更新时间；任务保留后端目录字段。counts 区分匹配总数与返回数量。 +2. `listBizMonitorBatches` → `GET /v1/biz-migration/projects/{project_id}/batches`、`GET /v1/biz-state/tasks/{task_id}/batches`、`GET /v1/biz-migration/batches/{batch_id}/runs`。 + `kind=auto|all|cutover|collect|runs`；auto 根据项目/任务/批次 ID 选择。`all` 需要 project_id 和 task_id；`runs` 需要割接 batch_id。批次 limit 默认 50、上限 200；runs_limit 默认 10、上限 50。include_runs 最多展开前 5 个已筛选割接批次；单个展开失败保留批次并附 runs_error，直接 runs 查询失败则返回工具错误。只保留批次摘要与运行 anomaly/progress，不回传大 summary；counts 为本次返回数量。 +3. `getBizMonitorContext` → `GET /v1/biz-migration/monitor-context`。 + project_id/task_id 至少一个；传同名查询参数。保留后端定义包的模板、映射、任务和命令。 +4. `getBizMonitorBoard` → `GET /v1/biz-migration/batches/{batch_id}/board`。 + 必填割接 batch_id，可选 run_id；保留看板统计与证据。 +5. `listBizMonitorReds` → `GET /v1/biz-migration/projects/{project_id}/red-tickets`。 + 必填 project_id，可选 status、limit（1..500）。未传 limit 时沿用后端默认 200；保留原始 A/B 键及设备、命令、时间证据。 +6. `getBizMonitorDiffs` → `GET /v1/biz-migration/runs/{run_id}/diffs`。 + 必填 run_id，可选 metric_id/sheet_id/verdict/color/q/limit/offset。**MCP `q` 映射 HTTP `kw`**，修正 TS 将 q 原样传递导致搜索失效的问题。limit 限 1..500，未传时后端默认 100；offset 下限 0。原始 A/B 和 match_* 键保持不变。 +7. `getBizCollectBatch` → `GET /v1/biz-state/batches/{batch_id}`。 + 必填采集 batch_id。后端返回工作簿摘要、命令及 sheet 目录，不含原始 CLI 全文和全量指标行。本包保持后端响应，避免误称其会返回全部指标数据。 +8. `getBizCollectCommandRaw` → `GET /v1/biz-state/batches/{batch_id}/commands/{command_id}`。 + 两个 ID 必填；返回完整 raw_text 和采集证据，按 netxops 不做字符/行数截断。大原文只在明确调用此工具时获取。 + +路径 ID 均作为单个段编码。数字参数按 netxops 截为整数并限幅；未知字段/类型错误返回可修正的工具错误，不接受过时单数 kind 或旧工具别名。 + +## 推荐调用链 + +- 无 ID 时先 `listBizMonitors({"q":"站点名"})`，取得项目 ID。 +- `getBizMonitorContext({"project_id":"..."})` 理解模板与 old/new 任务。 +- `listBizMonitorBatches({"project_id":"...","include_runs":true})` 找割接批次及运行；也可 `kind=runs,batch_id=...` 单独查运行。 +- `getBizMonitorBoard({"batch_id":"...","run_id":"..."})` 看结论，再以 `listBizMonitorReds({"project_id":"..."})` 与 `getBizMonitorDiffs({"run_id":"...","color":"red","limit":100,"offset":0})` 查看证据和分页差异。 +- 需要原文时，用上下文中的任务 ID 调 `listBizMonitorBatches({"task_id":"..."})` 找采集批次，`getBizCollectBatch` 找 command_id，再 `getBizCollectCommandRaw` 获取指定命令全文。割接批次 ID 与采集批次 ID 不可混用。 + +所有返回使用 `{ok,data}` 文本 JSON。参数或上游业务/HTTP 错误使用 MCP `result.isError=true`；JSON-RPC 的解析、请求结构、方法和 tools/call 结构错误分别为 -32700/-32600/-32601/-32602。通知永不回包且不触发业务请求；stdout 只输出协议消息。 + +凭据字段递归删除，配置令牌、Bearer 与常见 `password=...`/`secret:...` 文本脱敏。HTTP 错误体、异常文本、URL 不回显，避免凭据泄露；业务 evidence、raw A/B、show 命令、时间字段保持。原始设备输出可能含任意格式的敏感业务信息，部署方仍应控制授权采集内容。 + +## 验证 + +在 worktree 根目录运行（不安装依赖，不启动项目服务，不连接真实设备）： + +```powershell +$env:PYTHONPATH = (Resolve-Path packages/netx-bizmonitor-mcp/src).Path +& D:/project/chatgpt/netx/.venv/Scripts/python.exe -m pytest packages/netx-bizmonitor-mcp/tests -q +``` + +测试使用 HTTP mock；stdio 子进程清空令牌环境变量并在无令牌文件的临时目录运行，因此不发送网络请求。覆盖 8 工具、目录/批次裁剪与展开、q→kw、scope、业务与协议错误、通知、凭据来源优先级、文件错误回退、令牌轮换与脱敏及长原文不截断。 diff --git a/packages/netx-bizmonitor-mcp/mcp.json b/packages/netx-bizmonitor-mcp/mcp.json new file mode 100644 index 0000000..d8f96f6 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/mcp.json @@ -0,0 +1,16 @@ +{ + "mcpServers": { + "netx-bizmonitor": { + "command": "python", + "args": ["-m", "netx_bizmonitor_mcp"], + "env": { + "NETX_API_URL": "http://127.0.0.1:8890", + "NETX_API_TOKEN": "nxt_replace_with_biz_monitor_read_token", + "NETX_LANG": "zh", + "PYTHONIOENCODING": "utf-8", + "PYTHONUTF8": "1", + "PYTHONPATH": "D:/project/chatgpt/netx/packages/netx-bizmonitor-mcp/src" + } + } + } +} diff --git a/packages/netx-bizmonitor-mcp/pyproject.toml b/packages/netx-bizmonitor-mcp/pyproject.toml new file mode 100644 index 0000000..c67c759 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/pyproject.toml @@ -0,0 +1,18 @@ +[build-system] +requires = ["setuptools>=68", "wheel"] +build-backend = "setuptools.build_meta" + +[project] +name = "netx-bizmonitor-mcp" +version = "0.1.0" +description = "Read-only stdio MCP for netx business monitoring and cutover evidence" +readme = "README.md" +requires-python = ">=3.11" +license = { text = "MIT" } +dependencies = ["httpx>=0.27.0"] + +[project.scripts] +netx-bizmonitor-mcp = "netx_bizmonitor_mcp.server:main" + +[tool.setuptools.packages.find] +where = ["src"] diff --git a/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__init__.py b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__init__.py new file mode 100644 index 0000000..9bf849d --- /dev/null +++ b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__init__.py @@ -0,0 +1,3 @@ +"""Standalone, read-only netx business-monitor MCP.""" + +__version__ = "0.1.0" diff --git a/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__main__.py b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__main__.py new file mode 100644 index 0000000..bc40eef --- /dev/null +++ b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/__main__.py @@ -0,0 +1,3 @@ +from .server import main + +main() diff --git a/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_client.py b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_client.py new file mode 100644 index 0000000..61c4df7 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_client.py @@ -0,0 +1,100 @@ +"""HTTP transport. Never echo credentials, upstream error bodies, or URLs.""" + +from __future__ import annotations + +import json +import os +import re +from pathlib import Path +from typing import Any +from urllib.parse import urlsplit + +import httpx + +_SECRET_KEYS = { + "password", "passwd", "pwd", "secret", "token", "api_token", "api_key", + "authorization", "access_token", "refresh_token", "private_key", "client_secret", + "ssh_password", "enable_password", "hop_password", "credentials", +} + + +def api_token() -> str: + """Use the shared MCP credential convention; never log paths or file contents.""" + token = os.getenv("NETX_API_TOKEN", "").strip() + if token: + return token + for candidate in (os.getenv("NETX_MCP_TOKEN_FILE", "").strip(), "data/auth/mcp_token"): + if not candidate: + continue + try: + token = Path(candidate).read_text(encoding="utf-8").strip() + except (OSError, UnicodeError, ValueError): + continue + if token: + return token + return "" + + +def redact(value: Any, *, tokens: tuple[str, ...] | None = None) -> Any: + """Remove credential fields while retaining device raw A/B and CLI evidence.""" + if tokens is None: + tokens = (api_token(),) + if isinstance(value, dict): + return {k: redact(v, tokens=tokens) for k, v in value.items() if str(k).lower() not in _SECRET_KEYS} + if isinstance(value, list): + return [redact(v, tokens=tokens) for v in value] + if isinstance(value, str): + for token in tokens: + if token: + value = value.replace(token, "[REDACTED]") + value = re.sub(r"(?i)\bBearer\s+[^\s\"']+", "Bearer [REDACTED]", value) + value = re.sub( + r"(?im)(\b(?:password|passwd|secret|api[_-]?key|access[_-]?token)\s*[:=]\s*)([^\s,;]+)", + r"\1[REDACTED]", value, + ) + return value + + +def http_json(method: str, path: str, *, params: dict[str, Any] | None = None, + timeout: float = 45.0) -> dict[str, Any]: + if method != "GET": + return {"ok": False, "error": "read_only_transport"} + base = os.getenv("NETX_API_URL", "http://127.0.0.1:8890").strip().rstrip("/") + try: + parsed = urlsplit(base) + if (parsed.scheme not in {"http", "https"} or not parsed.hostname + or parsed.username or parsed.password or parsed.query or parsed.fragment): + return {"ok": False, "error": "netx_api_url_invalid"} + except ValueError: + return {"ok": False, "error": "netx_api_url_invalid"} + token = api_token() + if not token: + return {"ok": False, "error": "netx_token_missing", + "hint": "设置 NETX_API_TOKEN、NETX_MCP_TOKEN_FILE，或工作目录 data/auth/mcp_token。"} + query: dict[str, Any] = {} + if os.getenv("NETX_LANG", "zh").strip().lower().startswith("en"): + query["lang"] = "en" + query.update(params or {}) + try: + with httpx.Client(timeout=timeout, trust_env=False, follow_redirects=False) as client: + response = client.request("GET", base + path, params=query or None, headers={ + "accept": "application/json", "authorization": f"Bearer {token}", + }) + if not response.is_success: + return {"ok": False, "error": f"netx_http_{response.status_code}", + "hint": "检查 API 地址、令牌 scope 和资源 ID。"} + data = response.json() if response.content else {} + # Use the actual request token even if the credential file rotates mid-request. + return {"ok": True, "data": redact(data if isinstance(data, dict) else {"raw": data}, tokens=(token,))} + except (ValueError, httpx.HTTPError): + return {"ok": False, "error": "netx_request_failed", + "hint": "检查 API 连通性及 JSON 响应。"} + + +def mcp_from_handler_result(result: dict[str, Any]) -> dict[str, Any]: + result = redact(result) + out: dict[str, Any] = {"content": [{"type": "text", "text": json.dumps(result, ensure_ascii=False)}]} + if result.get("ok") is False or (isinstance(result.get("data"), dict) + and result["data"].get("ok") is False): + out["isError"] = True + return out diff --git a/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_tools.py b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_tools.py new file mode 100644 index 0000000..c5d4a0f --- /dev/null +++ b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/http_tools.py @@ -0,0 +1,268 @@ +"""The eight bizMonitor tools, aligned with netxops and backend HTTP contracts.""" + +from __future__ import annotations + +import math +from typing import Any +from urllib.parse import quote + +from .http_client import http_json, mcp_from_handler_result + + +class _InvalidArgument(ValueError): + """Only messages constructed from known schema fields; never include values.""" + + +def _str(description: str = "") -> dict[str, Any]: + return {"type": "string", "description": description} + + +def _int(default: int, maximum: int, minimum: int = 1) -> dict[str, Any]: + return {"type": "integer", "default": default, "minimum": minimum, "maximum": maximum} + + +def _tool(name: str, description: str, properties: dict[str, Any], + required: tuple[str, ...] = ()) -> dict[str, Any]: + return {"name": name, "description": description, "inputSchema": { + "type": "object", "properties": properties, "required": list(required), + "additionalProperties": False, + }, "annotations": {"readOnlyHint": True, "destructiveHint": False, "idempotentHint": True}} + + +HTTP_MCP_TOOLS = [ + _tool("listBizMonitors", "入口：列出割接项目和业务状态任务，无需 ID。取得 project_id/task_id 后查上下文或批次。", { + "kind": {"type": "string", "enum": ["all", "projects", "tasks"], "default": "all"}, + "purpose": _str("任务用途：portrait / cutover_hf；空为全部"), + "status": _str("本地过滤状态，如 active/paused"), "q": _str("名称、设备、备注或 ID 关键词"), + "limit": _int(100, 500), + }), + _tool("listBizMonitorBatches", "列批次/评估运行 ID：project_id→割接批次，task_id→采集批次，batch_id+kind=runs→评估运行。include_runs 最多展开前 5 个割接批次。", { + "kind": {"type": "string", "enum": ["auto", "all", "cutover", "collect", "runs"], "default": "auto"}, + "project_id": _str(), "task_id": _str(), "batch_id": _str("割接批次 ID，非采集批次 ID"), + "include_runs": {"type": "boolean", "default": False}, "status": _str(), + "limit": _int(50, 200), "runs_limit": _int(10, 50), + }), + _tool("getBizMonitorContext", "读取项目、监控/对比模板、端口映射及关联任务命令。project_id/task_id 至少一个；原始 A/B 键保留。", { + "project_id": _str(), "task_id": _str(), + }), + _tool("getBizMonitorBoard", "读取割接批次看板、最新或指定评估运行摘要、sheet 卡片及 verdict 统计。", { + "batch_id": _str("割接批次 ID"), "run_id": _str("可选评估运行 ID"), + }, ("batch_id",)), + _tool("listBizMonitorReds", "读取项目红票及证据（设备、采集时间、show 命令、接口原始/归一/映射键）；status 空为全部。", { + "project_id": _str(), "status": _str("open / carried / resolved；空为全部"), "limit": _int(200, 500), + }, ("project_id",)), + _tool("getBizMonitorDiffs", "分页读取评估差异，保留原始 A/B、match_* 键与 show 命令证据。q 转发后端 kw；用 offset/limit 翻页。", { + "run_id": _str(), "metric_id": _str(), "sheet_id": _str(), "verdict": _str(), + "color": _str("如 red"), "q": _str("差异关键词，映射 HTTP kw"), + "limit": _int(100, 500), "offset": _int(0, 9007199254740991, 0), + }, ("run_id",)), + _tool("getBizCollectBatch", "读取采集批次工作簿摘要、commands 和 sheet 目录。后端不返回全量指标行及 CLI 原文；以命令 ID 查原文。", { + "batch_id": _str("业务状态采集批次 ID"), + }, ("batch_id",)), + _tool("getBizCollectCommandRaw", "深挖一个采集命令：返回完整 raw_text、raw_command、设备和采集时间；仅凭据脱敏，不截断原文。", { + "batch_id": _str("采集批次 ID"), "command_id": _str("getBizCollectBatch 返回的命令 ID"), + }, ("batch_id", "command_id")), +] +TOOL_REQUIRED_SCOPE = {t["name"]: "biz-monitor:read" for t in HTTP_MCP_TOOLS} +_SCHEMAS = {t["name"]: t["inputSchema"] for t in HTTP_MCP_TOOLS} + + +def tools_for_scopes(scopes: list[str] | None) -> list[dict[str, Any]]: + # If /auth/me is unavailable, let the backend enforce authorization per GET. + return list(HTTP_MCP_TOOLS) if scopes is None or "biz-monitor:read" in scopes else [] + + +def _s(args: dict[str, Any], key: str, default: str = "") -> str: + return str(args.get(key, default)).strip() + + +def _n(args: dict[str, Any], key: str, default: int, low: int, high: int) -> int: + value = args.get(key, default) + if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value): + raise _InvalidArgument(f"{key} 必须为有限数值") + return max(low, min(high, math.trunc(value))) + + +def _pick(row: dict[str, Any], keys: str) -> dict[str, Any]: + return {k: row[k] for k in keys.split() if k in row} + + +def _items(result: dict[str, Any]) -> list[dict[str, Any]]: + rows = result.get("data", {}).get("items", []) + return [row for row in rows if isinstance(row, dict)] if isinstance(rows, list) else [] + + +def _failed(result: dict[str, Any]) -> bool: + return result.get("ok") is False or (isinstance(result.get("data"), dict) + and result["data"].get("ok") is False) + + +def _error(code: str, detail: str = "") -> dict[str, Any]: + return {"ok": False, "error": code, **({"detail": detail} if detail else {})} + + +def _project(row: dict[str, Any]) -> dict[str, Any]: + out = _pick(row, "id name status note monitor_template_id old_task_id new_task_id old_hf_task_id new_hf_task_id hf_interval_sec hf_start_at hf_end_at updated_at") + out["monitor_template_name"] = (row.get("monitor_template") or {}).get("name", "") + for side in ("old", "new"): + task = row.get(f"{side}_task") or {} + name, ip = str(task.get("ne_name") or "").strip(), str(task.get("ne_ip") or "").strip() + out[f"{side}_ne"] = f"{name} ({ip})" if name and ip else name or ip + return out + + +def _run(row: dict[str, Any]) -> dict[str, Any]: + out = _pick(row, "id batch_id purpose old_batch_id new_batch_id created_at") + summary = row.get("summary") or {} + progress = summary.get("progress") or {} + if "anomaly" in summary: + out["anomaly"] = summary["anomaly"] + for key in ("ok", "total"): + value = progress.get(key, summary.get(f"progress_{key}")) + if value is not None: + out[f"progress_{key}"] = value + return out + + +def _catalog(args: dict[str, Any]) -> dict[str, Any]: + kind = _s(args, "kind", "all").lower() or "all" + if kind not in {"all", "projects", "tasks"}: + return _error("kind_invalid", "kind must be all|projects|tasks") + limit = _n(args, "limit", 100, 1, 500) + status, query = _s(args, "status").lower(), _s(args, "q").lower() + data: dict[str, Any] = {"kind": kind} + counts = {"projects": 0, "tasks": 0, "projects_returned": 0, "tasks_returned": 0} + for group, path in (("projects", "/v1/biz-migration/projects"), ("tasks", "/v1/biz-state/tasks")): + if kind not in {"all", group}: + continue + params = {"purpose": _s(args, "purpose")} if group == "tasks" and _s(args, "purpose") else {} + result = http_json("GET", path, params=params) + if _failed(result): + return result + rows = [_project(row) for row in _items(result)] if group == "projects" else _items(result) + rows = [row for row in rows if (not status or str(row.get("status", "")).lower() == status) + and (not query or query in " ".join(str(row.get(k) or "").lower() for k in + "id name note status ne_name ne_ip ne_id old_ne new_ne monitor_template_name purpose vendor".split()))] + counts[group] = len(rows) + data[group] = rows[:limit] + counts[f"{group}_returned"] = len(data[group]) + data.update(counts=counts, next="将 project_id/task_id 传给 getBizMonitorContext；用 listBizMonitorBatches 获取 batch_id/run_id。") + return {"ok": True, "data": data} + + +def _batches(args: dict[str, Any]) -> dict[str, Any]: + pid, tid, bid = (_s(args, key) for key in ("project_id", "task_id", "batch_id")) + kind = _s(args, "kind", "auto").lower() or "auto" + include = args.get("include_runs") is True + limit, run_limit = _n(args, "limit", 50, 1, 200), _n(args, "runs_limit", 10, 1, 50) + status = _s(args, "status").lower() + if kind == "auto": + kind = "all" if pid and tid else "cutover" if pid else "collect" if tid else "runs" if bid else "" + if not kind: + return _error("project_id_or_task_id_or_batch_id_required") + if kind not in {"all", "cutover", "collect", "runs"}: + return _error("kind_invalid", "kind must be auto|all|cutover|collect|runs") + cutover, collect = kind in {"all", "cutover"}, kind in {"all", "collect"} + for needed, value, key in ((cutover, pid, "project_id"), (collect, tid, "task_id"), (kind == "runs", bid, "batch_id")): + if needed and not value: + return _error(f"{key}_required") + cb: list[dict[str, Any]] = [] + cl: list[dict[str, Any]] = [] + runs: list[dict[str, Any]] = [] + if cutover: + result = http_json("GET", f"/v1/biz-migration/projects/{quote(pid, safe='')}/batches", params={}) + if _failed(result): + return result + cb = [_pick(row, "id project_id batch_label status accept_status accept_run_id started_at ended_at note created_at updated_at") + for row in _items(result) if (not bid or str(row.get("id") or "") == bid) + and (not status or str(row.get("status") or "").lower() == status)][:limit] + if collect: + result = http_json("GET", f"/v1/biz-state/tasks/{quote(tid, safe='')}/batches", params={"limit": limit}) + if _failed(result): + return result + cl = [_pick(row, "id task_id status ne_name command_count row_count is_baseline protected message started_at ended_at") + for row in _items(result) if not status or str(row.get("status") or "").lower() == status][:limit] + for row in cl: + row["task_id"] = row.get("task_id") or tid + run_bids = [bid] if kind == "runs" else [str(row.get("id") or "").strip() for row in cb if row.get("id")][:5] if include else [] + for run_bid in run_bids: + result = http_json("GET", f"/v1/biz-migration/batches/{quote(run_bid, safe='')}/runs", params={"limit": run_limit}) + target = next((row for row in cb if row.get("id") == run_bid), None) + if _failed(result): + if kind == "runs": + return result + if target is not None: + target["runs_error"] = result.get("error") or "runs_fetch_failed" + continue + items = [_run(row) for row in _items(result)][:run_limit] + runs.extend(items) + if target is not None: + target["runs"] = items + data: dict[str, Any] = {"kind": kind, "counts": {"cutover_batches": len(cb), "collect_batches": len(cl), "runs": len(runs)}} + data.update({key: val for key, val in (("project_id", pid), ("task_id", tid), ("batch_id", bid)) if val}) + if cutover: + data["cutover_batches"] = cb + if collect: + data["collect_batches"] = cl + if kind == "runs": + data["runs"] = runs + data["next"] = "割接 batch_id→getBizMonitorBoard/listBizMonitorBatches(kind=runs)；采集 batch_id→getBizCollectBatch；run_id→getBizMonitorDiffs。" + return {"ok": True, "data": data} + + +def call_http_tool(name: str, args: dict[str, Any]) -> dict[str, Any]: + try: + schema = _SCHEMAS.get(name) + if schema is None: + return mcp_from_handler_result(_error("unknown_tool")) + for key, value in args.items(): + prop = schema["properties"].get(key) + if prop is None: + raise _InvalidArgument("存在未知字段；允许字段：" + ", ".join(schema["properties"])) + if prop["type"] == "string" and not isinstance(value, str): + raise _InvalidArgument(f"{key} 必须为字符串") + if prop["type"] == "boolean" and not isinstance(value, bool): + raise _InvalidArgument(f"{key} 必须为布尔值") + if name == "listBizMonitors": + result = _catalog(args) + elif name == "listBizMonitorBatches": + result = _batches(args) + else: + pid, tid, bid, rid, cid = (_s(args, key) for key in ("project_id", "task_id", "batch_id", "run_id", "command_id")) + params: dict[str, Any] = {} + if name == "getBizMonitorContext": + if not pid and not tid: + return mcp_from_handler_result(_error("project_id_or_task_id_required")) + path = "/v1/biz-migration/monitor-context" + params = {k: v for k, v in (("project_id", pid), ("task_id", tid)) if v} + else: + missing = [key for key in schema["required"] if not _s(args, key)] + if missing: + code = "batch_id_and_command_id_required" if name == "getBizCollectCommandRaw" else f"{missing[0]}_required" + return mcp_from_handler_result(_error(code)) + if name == "getBizMonitorBoard": + path = f"/v1/biz-migration/batches/{quote(bid, safe='')}/board" + if rid: + params["run_id"] = rid + elif name == "listBizMonitorReds": + path = f"/v1/biz-migration/projects/{quote(pid, safe='')}/red-tickets" + if _s(args, "status"): + params["status"] = _s(args, "status") + elif name == "getBizMonitorDiffs": + path = f"/v1/biz-migration/runs/{quote(rid, safe='')}/diffs" + params = {("kw" if k == "q" else k): _s(args, k) for k in ("metric_id", "sheet_id", "verdict", "color", "q") if _s(args, k)} + if "offset" in args: + params["offset"] = _n(args, "offset", 0, 0, 9007199254740991) + elif name == "getBizCollectBatch": + path = f"/v1/biz-state/batches/{quote(bid, safe='')}" + else: + path = f"/v1/biz-state/batches/{quote(bid, safe='')}/commands/{quote(cid, safe='')}" + if "limit" in args: + params["limit"] = _n(args, "limit", 200, 1, 500) + result = http_json("GET", path, params=params) + return mcp_from_handler_result(result) + except _InvalidArgument as exc: + return mcp_from_handler_result(_error("invalid_argument", str(exc))) + except (ValueError, TypeError, OverflowError): + # Avoid echoing user-provided values (which might contain credentials). + return mcp_from_handler_result(_error("invalid_argument", "参数类型或字段不合法；请参照 tools/list 的 inputSchema。")) diff --git a/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/server.py b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/server.py new file mode 100644 index 0000000..ba8bf49 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/src/netx_bizmonitor_mcp/server.py @@ -0,0 +1,97 @@ +"""Newline-delimited UTF-8 JSON-RPC MCP on stdio; stdout is protocol-only.""" + +from __future__ import annotations + +import json +import sys +from typing import Any + +from . import __version__ +from .http_client import http_json, mcp_from_handler_result +from .http_tools import TOOL_REQUIRED_SCOPE, call_http_tool, tools_for_scopes + + +def _fetch_scopes() -> list[str] | None: + result = http_json("GET", "/v1/auth/me") + if result.get("ok") is not True or not isinstance(result.get("data"), dict): + return None + data = result["data"] + scopes = data.get("scopes") + if not isinstance(scopes, list) and isinstance(data.get("user"), dict): + scopes = data["user"].get("scopes") + return [str(scope) for scope in scopes] if isinstance(scopes, list) else None + + +def _result(rid: Any, result: dict[str, Any]) -> dict[str, Any]: + return {"jsonrpc": "2.0", "id": rid, "result": result} + + +def _error(rid: Any, code: int, message: str) -> dict[str, Any]: + return {"jsonrpc": "2.0", "id": rid, "error": {"code": code, "message": message}} + + +def handle_request(request: Any) -> dict[str, Any] | None: + if (not isinstance(request, dict) or request.get("jsonrpc") != "2.0" + or not isinstance(request.get("method"), str) + or ("id" in request and (isinstance(request["id"], bool) + or not isinstance(request["id"], (str, int, type(None)))))): + return _error(None, -32600, "Invalid Request") + # Includes initialize/ping/unknown-method notifications: never reply or execute tools. + if "id" not in request: + return None + rid, method = request["id"], request["method"] + params = request.get("params", {}) + if not isinstance(params, dict): + return _error(rid, -32602, "params must be an object") + if method == "initialize": + return _result(rid, {"protocolVersion": "2024-11-05", "capabilities": {"tools": {}}, + "serverInfo": {"name": "netx-bizmonitor-mcp", "version": __version__}}) + if method == "ping": + return _result(rid, {}) + if method == "tools/list": + return _result(rid, {"tools": tools_for_scopes(_fetch_scopes())}) + if method == "tools/call": + name, args = params.get("name"), params.get("arguments", {}) + if not isinstance(name, str) or name not in TOOL_REQUIRED_SCOPE: + return _error(rid, -32602, "Unknown tool name") + if not isinstance(args, dict): + return _error(rid, -32602, "arguments must be an object") + # Refresh per call so revocation is not hidden by a cached permission list. + granted = _fetch_scopes() + required = TOOL_REQUIRED_SCOPE[name] + if granted is not None and required not in granted: + return _result(rid, mcp_from_handler_result({"ok": False, "error": "insufficient_scope", + "required_scope": required, "hint": "请管理员授予 API 令牌 biz-monitor:read。"})) + return _result(rid, call_http_tool(name, args)) + return _error(rid, -32601, "Method not found") + + +def run_stdio_loop() -> None: + for line in sys.stdin: + if not line.strip(): + continue + try: + request = json.loads(line) + except (ValueError, RecursionError): + response = _error(None, -32700, "Parse error") + else: + try: + response = handle_request(request) + except Exception: + # Never expose traceback, connection URL, headers or exception text. + response = None if isinstance(request, dict) and "id" not in request else _error( + request.get("id") if isinstance(request, dict) else None, -32603, "Internal error") + if response is not None: + sys.stdout.write(json.dumps(response, ensure_ascii=False) + "\n") + sys.stdout.flush() + + +def main() -> None: + for stream in (sys.stdin, sys.stdout, sys.stderr): + if hasattr(stream, "reconfigure"): + stream.reconfigure(encoding="utf-8", errors="replace") + run_stdio_loop() + + +if __name__ == "__main__": + main() diff --git a/packages/netx-bizmonitor-mcp/tests/conftest.py b/packages/netx-bizmonitor-mcp/tests/conftest.py new file mode 100644 index 0000000..883dfde --- /dev/null +++ b/packages/netx-bizmonitor-mcp/tests/conftest.py @@ -0,0 +1,6 @@ +"""Always import this checkout's standalone package, not the main repository.""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) diff --git a/packages/netx-bizmonitor-mcp/tests/test_mcp_bizmonitor.py b/packages/netx-bizmonitor-mcp/tests/test_mcp_bizmonitor.py new file mode 100644 index 0000000..c697b28 --- /dev/null +++ b/packages/netx-bizmonitor-mcp/tests/test_mcp_bizmonitor.py @@ -0,0 +1,340 @@ +from __future__ import annotations + +import io +import json +import os +from pathlib import Path +import subprocess +import sys +from unittest.mock import patch + +import httpx +import pytest + +from netx_bizmonitor_mcp.http_client import api_token, http_json, redact +from netx_bizmonitor_mcp.http_tools import HTTP_MCP_TOOLS, TOOL_REQUIRED_SCOPE, call_http_tool, tools_for_scopes +from netx_bizmonitor_mcp.server import _fetch_scopes, handle_request, run_stdio_loop + + +def payload(result): + return json.loads(result["content"][0]["text"]) + + +def request(method, params=None, rid=1): + return {"jsonrpc": "2.0", "id": rid, "method": method, "params": params or {}} + + +def test_exact_eight_tools_read_only_and_canonical_scope(): + assert {t["name"] for t in HTTP_MCP_TOOLS} == { + "listBizMonitors", "listBizMonitorBatches", "getBizMonitorContext", "getBizMonitorBoard", + "listBizMonitorReds", "getBizMonitorDiffs", "getBizCollectBatch", "getBizCollectCommandRaw", + } + assert len(HTTP_MCP_TOOLS) == 8 + assert all(t["annotations"]["readOnlyHint"] for t in HTTP_MCP_TOOLS) + assert set(TOOL_REQUIRED_SCOPE.values()) == {"biz-monitor:read"} + assert tools_for_scopes(["ne:read", "ops:write"]) == [] + assert len(tools_for_scopes(["biz-monitor:read"])) == 8 + assert len(tools_for_scopes(None)) == 8 + + +def test_catalog_filters_slims_projects_and_keeps_totals(): + def fake(method, path, *, params): + assert method == "GET" + if path == "/v1/biz-migration/projects": + assert params == {} + return {"ok": True, "data": {"items": [ + {"id": f"p{i}", "status": "active", "monitor_template": {"name": "Core"}, + "old_task": {"ne_name": "R1", "ne_ip": "10.0.0.1", "commands": "large"}, "large_template": "drop"} + for i in range(3)]}} + assert path == "/v1/biz-state/tasks" and params == {"purpose": "portrait"} + return {"ok": True, "data": {"items": [{"id": "t", "status": "paused", "ne_name": "R1"}]}} + with patch("netx_bizmonitor_mcp.http_tools.http_json", side_effect=fake): + result = payload(call_http_tool("listBizMonitors", {"q": "r1", "status": "ACTIVE", "purpose": "portrait", "limit": 1})) + data = result["data"] + assert data["counts"] == {"projects": 3, "tasks": 0, "projects_returned": 1, "tasks_returned": 0} + assert data["projects"][0]["old_ne"] == "R1 (10.0.0.1)" + assert "old_task" not in data["projects"][0] and "large_template" not in data["projects"][0] + assert "netx__" not in data["next"] + + +def test_batch_cutover_expand_runs_and_partial_error(): + def fake(method, path, *, params): + if path.endswith("/batches"): + return {"ok": True, "data": {"items": [{"id": "b1", "status": "done", "large": "drop"}, {"id": "b2", "status": "done"}]}} + assert params == {"limit": 3} + if "/b2/" in path: + return {"ok": False, "error": "netx_http_404"} + return {"ok": True, "data": {"items": [{"id": "r1", "summary": {"anomaly": 2, "progress": {"ok": 4, "total": 5}, "large": "drop"}}]}} + with patch("netx_bizmonitor_mcp.http_tools.http_json", side_effect=fake) as mock: + out = call_http_tool("listBizMonitorBatches", {"project_id": "p/1", "include_runs": True, "runs_limit": 3}) + data = payload(out)["data"] + assert "isError" not in out and mock.call_count == 3 + assert mock.call_args_list[0].args == ("GET", "/v1/biz-migration/projects/p%2F1/batches") + assert data["cutover_batches"][1]["runs_error"] == "netx_http_404" + assert data["cutover_batches"][0]["runs"] == [{"id": "r1", "anomaly": 2, "progress_ok": 4, "progress_total": 5}] + assert data["counts"]["runs"] == 1 and "runs" not in data + assert "large" not in data["cutover_batches"][0] + + +def test_batch_collect_and_runs_modes(): + with patch("netx_bizmonitor_mcp.http_tools.http_json") as mock: + mock.return_value = {"ok": True, "data": {"items": [{"id": "c1", "status": "done", "raw_text": "drop"}]}} + out = payload(call_http_tool("listBizMonitorBatches", {"task_id": "t", "limit": 20})) + mock.assert_called_once_with("GET", "/v1/biz-state/tasks/t/batches", params={"limit": 20}) + assert out["data"]["collect_batches"] == [{"id": "c1", "status": "done", "task_id": "t"}] + mock.reset_mock() + mock.return_value = {"ok": True, "data": {"items": [{"id": "r1", "summary": {"progress_ok": 2}}]}} + out = payload(call_http_tool("listBizMonitorBatches", {"batch_id": "b"})) + mock.assert_called_once_with("GET", "/v1/biz-migration/batches/b/runs", params={"limit": 10}) + assert out["data"]["runs"] == [{"id": "r1", "progress_ok": 2}] + + +def test_batch_all_caps_run_expansion_at_five(): + def fake(method, path, *, params): + if path == "/v1/biz-migration/projects/p/batches": + return {"ok": True, "data": {"items": [{"id": f"b{i}"} for i in range(8)]}} + if path == "/v1/biz-state/tasks/t/batches": + return {"ok": True, "data": {"items": [{"id": "c"}]}} + assert path.endswith("/runs") + return {"ok": True, "data": {"items": [{"id": "r"}]}} + with patch("netx_bizmonitor_mcp.http_tools.http_json", side_effect=fake) as mock: + data = payload(call_http_tool("listBizMonitorBatches", { + "project_id": "p", "task_id": "t", "include_runs": True, + }))["data"] + assert mock.call_count == 7 # two catalogs plus five run lookups + assert data["kind"] == "all" + assert data["counts"] == {"cutover_batches": 8, "collect_batches": 1, "runs": 5} + assert sum("runs" in row for row in data["cutover_batches"]) == 5 + + +@pytest.mark.parametrize("name,args,path,params", [ + ("getBizMonitorContext", {"project_id": " p/1 ", "task_id": "t"}, "/v1/biz-migration/monitor-context", {"project_id": "p/1", "task_id": "t"}), + ("getBizMonitorBoard", {"batch_id": "b/1", "run_id": "r"}, "/v1/biz-migration/batches/b%2F1/board", {"run_id": "r"}), + ("listBizMonitorReds", {"project_id": "p", "status": "open", "limit": 700}, "/v1/biz-migration/projects/p/red-tickets", {"status": "open", "limit": 500}), + ("getBizMonitorDiffs", {"run_id": "r/1", "metric_id": "m", "sheet_id": "s", "verdict": "changed", "color": "red", "q": "xe-1", "offset": -2, "limit": 10}, "/v1/biz-migration/runs/r%2F1/diffs", {"metric_id": "m", "sheet_id": "s", "verdict": "changed", "color": "red", "kw": "xe-1", "offset": 0, "limit": 10}), + ("getBizCollectBatch", {"batch_id": "c/1"}, "/v1/biz-state/batches/c%2F1", {}), + ("getBizCollectCommandRaw", {"batch_id": "c", "command_id": "cmd/1"}, "/v1/biz-state/batches/c/commands/cmd%2F1", {}), +]) +def test_each_read_tool_forwards_exact_http(name, args, path, params): + evidence = {"old_key": "xe-1", "new_key": "ge-2", "evidence": {"raw_command": "show interface"}} + with patch("netx_bizmonitor_mcp.http_tools.http_json", return_value={"ok": True, "data": evidence}) as mock: + result = call_http_tool(name, args) + mock.assert_called_once_with("GET", path, params=params) + assert payload(result)["data"] == evidence + + +def test_diff_default_limit_not_forwarded_and_only_kw_used(): + with patch("netx_bizmonitor_mcp.http_tools.http_json", return_value={"ok": True, "data": {}}) as mock: + call_http_tool("getBizMonitorDiffs", {"run_id": "r", "q": " A B "}) + assert mock.call_args.kwargs["params"] == {"kw": "A B"} + + +def test_raw_not_truncated_and_credentials_removed(monkeypatch): + monkeypatch.setenv("NETX_API_TOKEN", "nxt_do_not_leak") + raw = "show interfaces\n" + "interface xe-1 up\n" * 10000 + with patch("netx_bizmonitor_mcp.http_tools.http_json", return_value={"ok": True, "data": { + "raw_text": raw, "device": {"password": "hidden", "ne_name": "R1"}, + "api_token": "hidden", "note": "Bearer abcd password=secret nxt_do_not_leak", + }}): + result = call_http_tool("getBizCollectCommandRaw", {"batch_id": "b", "command_id": "c"}) + data = payload(result)["data"] + assert data["raw_text"] == raw and data["device"] == {"ne_name": "R1"} + assert "api_token" not in data + assert "abcd" not in data["note"] and "secret" not in data["note"] and "nxt_do_not_leak" not in data["note"] + + +@pytest.mark.parametrize("name,args,error", [ + ("getBizMonitorContext", {}, "project_id_or_task_id_required"), + ("getBizMonitorBoard", {}, "batch_id_required"), + ("listBizMonitorReds", {}, "project_id_required"), + ("getBizMonitorDiffs", {}, "run_id_required"), + ("getBizCollectBatch", {}, "batch_id_required"), + ("getBizCollectCommandRaw", {"batch_id": "b"}, "batch_id_and_command_id_required"), + ("listBizMonitorBatches", {}, "project_id_or_task_id_or_batch_id_required"), + ("listBizMonitorBatches", {"kind": "all", "project_id": "p"}, "task_id_required"), + ("listBizMonitors", {"kind": "project"}, "kind_invalid"), + ("listBizMonitors", {"limit": "two"}, "invalid_argument"), + ("getBizMonitorDiffs", {"run_id": "r", "offset": True}, "invalid_argument"), + ("getBizMonitorDiffs", {"run_id": "r", "kw": "x"}, "invalid_argument"), +]) +def test_bad_arguments_never_make_http_call(name, args, error): + with patch("netx_bizmonitor_mcp.http_tools.http_json") as mock: + result = call_http_tool(name, args) + mock.assert_not_called() + assert result["isError"] is True and payload(result)["error"] == error + + +def test_argument_error_names_field_without_echoing_value(): + result = payload(call_http_tool("listBizMonitors", {"limit": "password=hidden"})) + assert "limit" in result["detail"] + assert "hidden" not in json.dumps(result) + + +@pytest.mark.parametrize("failure", [ + {"ok": False, "error": "netx_http_404"}, + {"ok": True, "data": {"ok": False, "error": "not_found"}}, +]) +def test_upstream_failures_are_tool_errors(failure): + with patch("netx_bizmonitor_mcp.http_tools.http_json", return_value=failure): + result = call_http_tool("getBizCollectBatch", {"batch_id": "b"}) + assert result["isError"] is True + + +def test_fetch_scopes_and_server_scope_denial(): + with patch("netx_bizmonitor_mcp.server.http_json", return_value={"ok": True, "data": {"scopes": ["ne:read"]}}): + assert _fetch_scopes() == ["ne:read"] + assert handle_request(request("tools/list"))["result"]["tools"] == [] + with patch("netx_bizmonitor_mcp.server.call_http_tool") as call: + denied = handle_request(request("tools/call", {"name": "getBizCollectBatch", "arguments": {"batch_id": "b"}})) + call.assert_not_called() + assert "error" not in denied and denied["result"]["isError"] + assert payload(denied["result"])["required_scope"] == "biz-monitor:read" + with patch("netx_bizmonitor_mcp.server.http_json", return_value={"ok": True, "data": {"user": {"scopes": ["biz-monitor:read"]}}}): + assert _fetch_scopes() == ["biz-monitor:read"] + with patch("netx_bizmonitor_mcp.server.http_json", return_value={"ok": False}): + assert _fetch_scopes() is None + + +def test_protocol_errors_and_notifications(): + for obj in ([], "x", {}, {"jsonrpc": "2.0", "id": True, "method": "ping"}): + assert handle_request(obj)["error"]["code"] == -32600 + assert handle_request(request("unknown"))["error"]["code"] == -32601 + assert handle_request(request("tools/call", {"name": "netx__getBizCollectBatch"}))["error"]["code"] == -32602 + assert handle_request(request("tools/call", {"name": "getBizCollectBatch", "arguments": []}))["error"]["code"] == -32602 + with patch("netx_bizmonitor_mcp.server.http_json") as mock: + for method in ("initialize", "notifications/initialized", "notifications/cancelled", "ping", "tools/call"): + assert handle_request({"jsonrpc": "2.0", "method": method}) is None + mock.assert_not_called() + stdin, stdout = io.StringIO('{bad\n' + json.dumps(request("ping", rid=2)) + '\n'), io.StringIO() + with patch("sys.stdin", stdin), patch("sys.stdout", stdout): + run_stdio_loop() + lines = [json.loads(line) for line in stdout.getvalue().splitlines()] + assert lines[0]["error"]["code"] == -32700 and lines[1]["id"] == 2 + + +def test_http_auth_lang_redaction_and_error_sanitization(monkeypatch): + monkeypatch.setenv("NETX_API_TOKEN", "nxt_secret") + monkeypatch.setenv("NETX_API_URL", "http://example.test/netx") + monkeypatch.setenv("NETX_LANG", "en") + with patch("netx_bizmonitor_mcp.http_client.httpx.Client") as client: + response = httpx.Response(200, json={"id": "b", "password": "secret", "nested": {"api_key": "bad"}}) + client.return_value.__enter__.return_value.request.return_value = response + result = http_json("GET", "/v1/biz-state/batches/b", params={"limit": 2}) + client.assert_called_once_with(timeout=45.0, trust_env=False, follow_redirects=False) + kwargs = client.return_value.__enter__.return_value.request.call_args.kwargs + assert kwargs["headers"]["authorization"] == "Bearer nxt_secret" + assert kwargs["params"] == {"lang": "en", "limit": 2} + assert result["data"] == {"id": "b", "nested": {}} + client.return_value.__enter__.return_value.request.return_value = httpx.Response(500, text="password=leak nxt_secret") + result = http_json("GET", "/v1/test") + assert result["error"] == "netx_http_500" and "leak" not in json.dumps(result) + client.return_value.__enter__.return_value.request.side_effect = httpx.ConnectError("Bearer leak") + assert "leak" not in json.dumps(http_json("GET", "/v1/test")) + + +def test_invalid_url_token_and_write_never_open_client(monkeypatch, tmp_path): + monkeypatch.delenv("NETX_API_TOKEN", raising=False) + monkeypatch.delenv("NETX_MCP_TOKEN_FILE", raising=False) + monkeypatch.chdir(tmp_path) + with patch("netx_bizmonitor_mcp.http_client.httpx.Client") as client: + monkeypatch.setenv("NETX_API_URL", "https://user:pass@example.test") + assert http_json("GET", "/x")["error"] == "netx_api_url_invalid" + monkeypatch.setenv("NETX_API_URL", "http://example.test") + assert http_json("GET", "/x")["error"] == "netx_token_missing" + assert http_json("POST", "/x")["error"] == "read_only_transport" + client.assert_not_called() + + +def test_stdio_subprocess_initialize_list_notifications_errors(tmp_path): + env = os.environ.copy() + env["PYTHONPATH"] = str(Path(__file__).resolve().parents[1] / "src") + env["NETX_API_URL"] = "http://127.0.0.1:1" + env.pop("NETX_API_TOKEN", None) # no network attempted when token is absent + env.pop("NETX_MCP_TOKEN_FILE", None) + messages = [request("initialize"), {"jsonrpc": "2.0", "method": "initialize"}, + {"jsonrpc": "2.0", "method": "notifications/initialized"}, request("ping", rid=2), + request("tools/list", rid=3), request("tools/call", {"name": "getBizCollectBatch", "arguments": {}}, rid=4)] + run = subprocess.run([sys.executable, "-m", "netx_bizmonitor_mcp"], + input="\n".join(json.dumps(item) for item in messages) + "\n{bad\n", + text=True, encoding="utf-8", capture_output=True, env=env, cwd=tmp_path, timeout=10) + assert run.returncode == 0 and run.stderr == "" + replies = [json.loads(line) for line in run.stdout.splitlines()] + assert len(replies) == 5 + assert replies[0]["result"]["serverInfo"]["name"] == "netx-bizmonitor-mcp" + assert replies[1] == {"jsonrpc": "2.0", "id": 2, "result": {}} + assert len(replies[2]["result"]["tools"]) == 8 + assert replies[3]["result"]["isError"] is True + assert replies[4]["error"]["code"] == -32700 + + +def test_token_priority_env_then_explicit_file_then_cwd(monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + default = tmp_path / "data" / "auth" / "mcp_token" + default.parent.mkdir(parents=True) + default.write_text(" default-token\n", encoding="utf-8") + explicit = tmp_path / "custom-token" + explicit.write_text(" explicit-token\n", encoding="utf-8") + monkeypatch.setenv("NETX_API_TOKEN", " env-token ") + monkeypatch.setenv("NETX_MCP_TOKEN_FILE", str(explicit)) + assert api_token() == "env-token" + monkeypatch.setenv("NETX_API_TOKEN", " ") + assert api_token() == "explicit-token" + explicit.write_text(" \n", encoding="utf-8") + assert api_token() == "default-token" + monkeypatch.delenv("NETX_MCP_TOKEN_FILE") + assert api_token() == "default-token" + + +@pytest.mark.parametrize("file_error", ["missing", "directory", "invalid_utf8", "permission"]) +def test_token_file_errors_fall_back_without_exposing_contents(monkeypatch, tmp_path, capsys, file_error): + monkeypatch.chdir(tmp_path) + monkeypatch.delenv("NETX_API_TOKEN", raising=False) + default = tmp_path / "data" / "auth" / "mcp_token" + default.parent.mkdir(parents=True) + default.write_text("fallback-token", encoding="utf-8") + explicit = tmp_path / "sensitive-file" + if file_error == "directory": + explicit.mkdir() + elif file_error == "invalid_utf8": + explicit.write_bytes(b"secret\xff") + monkeypatch.setenv("NETX_MCP_TOKEN_FILE", str(explicit)) + if file_error == "permission": + original = Path.read_text + def read(path, *args, **kwargs): + if path == explicit: + raise PermissionError("sensitive exception text") + return original(path, *args, **kwargs) + monkeypatch.setattr(Path, "read_text", read) + assert api_token() == "fallback-token" + default.unlink() + assert api_token() == "" + with patch("netx_bizmonitor_mcp.http_client.httpx.Client") as client: + result = http_json("GET", "/v1/auth/me") + client.assert_not_called() + assert result["error"] == "netx_token_missing" + assert "sensitive-file" not in json.dumps(result) and "sensitive exception" not in json.dumps(result) + captured = capsys.readouterr() + assert captured.out == captured.err == "" + + +@pytest.mark.parametrize("use_explicit", [True, False]) +def test_file_token_auth_and_redaction_survives_rotation(monkeypatch, tmp_path, use_explicit): + monkeypatch.chdir(tmp_path) + monkeypatch.delenv("NETX_API_TOKEN", raising=False) + monkeypatch.delenv("NETX_MCP_TOKEN_FILE", raising=False) + monkeypatch.setenv("NETX_API_URL", "http://example.test") + token_file = tmp_path / "custom-token" if use_explicit else tmp_path / "data" / "auth" / "mcp_token" + token_file.parent.mkdir(parents=True, exist_ok=True) + token_file.write_text("nxt_file_secret", encoding="utf-8") + if use_explicit: + monkeypatch.setenv("NETX_MCP_TOKEN_FILE", str(token_file)) + assert redact({"nested": ["raw nxt_file_secret"]}) == {"nested": ["raw [REDACTED]"]} + def response(*args, **kwargs): + assert kwargs["headers"]["authorization"] == "Bearer nxt_file_secret" + token_file.write_text("nxt_rotated", encoding="utf-8") + return httpx.Response(200, json={"raw_text": "prefix nxt_file_secret suffix"}) + with patch("netx_bizmonitor_mcp.http_client.httpx.Client") as client: + client.return_value.__enter__.return_value.request.side_effect = response + result = http_json("GET", "/v1/biz-state/batches/b/commands/c") + assert result["data"]["raw_text"] == "prefix [REDACTED] suffix" + assert "nxt_file_secret" not in json.dumps(result) diff --git a/packages/netx-mcp/README.md b/packages/netx-mcp/README.md index 009c1ec..7134d61 100644 --- a/packages/netx-mcp/README.md +++ b/packages/netx-mcp/README.md @@ -48,14 +48,24 @@ python -m netx_mcp 0.3.1：`getNmsNe` 可直接传 host_name/IP；输出统一 `nms_ne_id` / `source=nms`；参数类型错误返回工具错误而非协议崩溃。 -参数优先 `nms_ne_id` / `nms_ne_ids`（保留 `ume_*` 别名）。 +参数使用 `nms_ne_id` / `nms_ne_ids`，路径端点使用 `from_nms_ne_id` / `to_nms_ne_id`；`getManagedNe` 只接受 `ne_id`。旧 `ume_*`、`managed_ne_id`、`id`、告警 `ne_name` 别名已删除，未知参数会返回工具错误。REST 适配器内部的 `ume_*` 字段不变。 + +NMS 查询（含 inventory）按当前 HTTP 权限规则需要 `alarms:read`；托管资源、CLI 目标和路径查询需要 `ne:read`；执行与轮询需要 `ne:exec`；SQL 需要 `sql:query`。所有工具返回递归裁剪结构化口令、令牌等凭据，但 CLI 原文输出仍需调用者判断是否敏感。 拓扑画布 / Fabric → skill **topology** / [`netx-topology-mcp`](../netx-topology-mcp)（见 [docs/MCP_TOPOLOGY.md](../../docs/MCP_TOPOLOGY.md)）。 +业务监控 / 割接 → [`netx-bizmonitor-mcp`](../netx-bizmonitor-mcp)。三组统一配置见根 `mcp.all.json`，均不依赖 netxops。 + +本包的 `execManagedNe` 独立强制只读 show/display；即使后端设备开放 shell，也拒绝配置、命令拼接、重定向和多行脚本。getManagedNe.capability 描述后端能力，不能作为 MCP 脚本许可；Linux/RouterOS 原生 CLI 当前未暴露。异步 ack 返回 `poll_after_ms=5000`；单台核对 terminal/result.ok，批量还须核对 summary.failed=0 和每个 results[].ok。作业 succeeded 仍可能有单台失败，须分别报告，勿整批重跑已成功设备。失败 job 作为工具错误返回，保留原始失败证据。 + +同 IP 多条纳管记录须按厂商/驱动选 ID。connect_status 为历史连接测试状态，不替代当前执行结果；有正常业务正文才算查询通过，不能把空正文或仅命令头当作成功。后端的 Cisco IOS SSH 驱动使用 CR 换行并等待命令回显和完整提示符；不影响其它厂商或 Telnet 的默认换行。 + ## Breaking (0.3.0) 模型工具名从 `*Ume*` 改为 `*Nms*`（与 dsh-netxops 对齐）。请更新 skills / 提示词并重启 MCP 宿主。 -## 兼容 +## 入口 -全量 `netx-ops` 开发安装下 `python -m netx_api.mcp` 仍会转到本包；新环境请只装 **netx-mcp**。 +只保留独立 **netx-mcp** 包入口：`python -m netx_mcp` / `netx-mcp`。旧 `netx_api.mcp`、`netx_api/mcp_server.py` 和直连数据库模式已删除；配置需迁移到本包。 + +旧 DB 模式的导入告警查询/聚合/批次/诊断、同步触发和历史告警查询未纳入本包 15 工具，相关 HTTP API 仍保留。 diff --git a/packages/netx-mcp/src/netx_mcp/cli_guard.py b/packages/netx-mcp/src/netx_mcp/cli_guard.py new file mode 100644 index 0000000..aadbed1 --- /dev/null +++ b/packages/netx-mcp/src/netx_mcp/cli_guard.py @@ -0,0 +1,26 @@ +"""MCP 独立只读边界：后端开放 shell 策略也不能扩大模型的执行权限。""" +from __future__ import annotations + +import re + +_PREFIX = re.compile(r"(?i)^(show|display)[ \t]+\S") +_CONTROL = re.compile(r"[\x00-\x08\x0a-\x1f\x7f-\x9f\u2028\u2029]") +_SHELL = re.compile(r"[;&`$<>]") +_DESTRUCTIVE = re.compile( + r"(?i)\b(reboot|reload|shutdown|erase|format|delete|commit|rollback|redirect|append|tee)\b" + r"|\b(?:configure\s+terminal|conf\s+t|write\s+memory|system-view|undo\s+)" +) +_FILTER = re.compile(r"(?i)^(include|exclude|begin|section|count|match|grep|one-line|no-more)(?:[ \t]|$)") + + +def validate_readonly_cli(command: str) -> None: + if not isinstance(command, str) or not command.strip(): + raise ValueError("MCP CLI requires a nonempty show/display command") + if len(command) > 500 or _CONTROL.search(command) or _SHELL.search(command): + raise ValueError("MCP CLI rejects control characters, shell operators and commands longer than 500 characters") + cmd = command.strip() + if not _PREFIX.match(cmd) or _DESTRUCTIVE.search(cmd): + raise ValueError("MCP CLI allows read-only show/display only; open device policies do not override this boundary") + parts = cmd.split("|") + if any(not _FILTER.match(part.strip()) for part in parts[1:]): + raise ValueError("MCP CLI pipes allow output filters only") diff --git a/packages/netx-mcp/src/netx_mcp/http_tools.py b/packages/netx-mcp/src/netx_mcp/http_tools.py index e604cbf..faa65b9 100644 --- a/packages/netx-mcp/src/netx_mcp/http_tools.py +++ b/packages/netx-mcp/src/netx_mcp/http_tools.py @@ -1,7 +1,7 @@ """MCP tool schemas and HTTP-backed handlers (NMS adapter + managed NE). Model-facing tool names use generic Nms*; REST paths remain /v1/ume/* for the -current zte-ume provider. Params prefer nms_ne_id with ume_ne_id aliases. +current zte-ume provider. Model-facing params use nms_ne_id exclusively. """ from __future__ import annotations @@ -9,6 +9,7 @@ from __future__ import annotations import os from typing import Any, Callable +from .cli_guard import validate_readonly_cli from .http_client import http_json, http_post_json, mcp_from_handler_result, quote_ne_id _EXEC_MAX_COMMANDS_CAP = 50 @@ -47,17 +48,16 @@ def _truthy_async_flag(raw: Any) -> bool | None: def _count_exec_ne_targets(args: dict[str, Any]) -> int: n = 0 - for key in ("ne_ids", "nms_ne_ids", "ume_ne_ids"): + for key in ("ne_ids", "nms_ne_ids"): val = args.get(key) if isinstance(val, list): - n = max(n, len([x for x in val if str(x or "").strip()])) + n += len([x for x in val if str(x or "").strip()]) targets = args.get("targets") if isinstance(targets, list): - n = max(n, len([t for t in targets if isinstance(t, dict)])) + n += len([t for t in targets if isinstance(t, dict)]) if n == 0 and ( str(args.get("ne_id") or "").strip() or str(args.get("nms_ne_id") or "").strip() - or str(args.get("ume_ne_id") or "").strip() ): return 1 return int(n) @@ -243,11 +243,8 @@ def _query_ume_alarms(args: dict[str, Any]) -> dict[str, Any]: if str(args.get("is_cleared") or "").strip(): params["is_cleared"] = str(args.get("is_cleared")).strip() keyword = str(args.get("keyword") or "").strip() - ne_name = str(args.get("ne_name") or "").strip() if keyword: params["keyword"] = keyword - elif ne_name: - params["keyword"] = ne_name if str(args.get("ne_id") or "").strip(): params["ne_id"] = str(args.get("ne_id")).strip() if str(args.get("host_name") or "").strip(): @@ -264,7 +261,7 @@ def _aggregate_ume_alarms(args: dict[str, Any]) -> dict[str, Any]: if group_by: return _aggregate_ume_alarms_raw(args) # Default top 50 named NEs — missing host buckets reported separately. - params: dict[str, Any] = {"top_ne": _int_arg(args, "top_ne", 50, 0, 500)} + params: dict[str, Any] = {"top_ne": _int_arg(args, "top_ne", 50, 0, 5000)} if "exclude_missing_host" in args: params["exclude_missing_host"] = bool(args.get("exclude_missing_host")) if _severity(args): @@ -357,6 +354,8 @@ def _query_ume_alarms_raw(args: dict[str, Any]) -> dict[str, Any]: def _aggregate_ume_alarms_raw(args: dict[str, Any]) -> dict[str, Any]: + if not str(args.get("group_by") or "").strip(): + raise ToolArgError("group_by is required; use listNmsAlarmFields to choose a field") params: dict[str, Any] = {} for k in ( "group_by", @@ -442,9 +441,7 @@ def _first_str(*vals: Any) -> str: def _get_managed_ne(args: dict[str, Any]) -> dict[str, Any]: - ne_id = str( - args.get("ne_id") or args.get("managed_ne_id") or args.get("id") or "" - ).strip() + ne_id = str(args.get("ne_id") or "").strip() if not ne_id: return { "ok": False, @@ -473,12 +470,46 @@ def _get_managed_ne(args: dict[str, Any]) -> dict[str, Any]: return out +def _exec_ack(out: dict[str, Any]) -> dict[str, Any]: + if not out.get("ok"): + return out + data = out.get("data") if isinstance(out.get("data"), dict) else out + if isinstance(data, dict) and data.get("job_id"): + data = dict(data) + data["poll_after_ms"] = 5000 + data["hint"] = ( + "Execution started; it has not succeeded yet. Continue independent work, then poll " + "getNeExecJob with this job_id after about 5 seconds. Do not resubmit commands while " + "the job is running. Inspect terminal, result.ok, summary.failed and every target result before reporting success." + ) + return data if isinstance(data, dict) else {"ok": True, "data": data} + + +def _exec_result_failed(data: Any) -> bool: + """Batch ok means the request ran; individual targets may still all fail.""" + if not isinstance(data, dict): + return False + if data.get("ok") is False: + return True + summary = data.get("summary") + if isinstance(summary, dict) and isinstance(summary.get("failed"), int) and summary["failed"] > 0: + return True + results = data.get("results") + return isinstance(results, list) and any(isinstance(row, dict) and row.get("ok") is False for row in results) + + def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: + supplied_commands = list(args.get("commands") or []) + for target in args.get("targets") or []: + supplied_commands.extend(target.get("commands") or []) + for command in supplied_commands: + try: + validate_readonly_cli(command) + except ValueError as exc: + raise ToolArgError(str(exc)) from None targets_raw = args.get("targets") ne_ids_raw = args.get("ne_ids") nms_ne_ids_raw = args.get("nms_ne_ids") - if not isinstance(nms_ne_ids_raw, list) or not nms_ne_ids_raw: - nms_ne_ids_raw = args.get("ume_ne_ids") shared_cmds_raw = args.get("commands") shared_commands = ( [str(c).strip() for c in shared_cmds_raw if str(c).strip()] @@ -492,6 +523,13 @@ def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: ) want_async = _should_run_exec_async(args) + if multi and (args.get("ne_id") or args.get("nms_ne_id")): + raise ToolArgError("Do not mix single NE identifiers with batch targets") + if multi and _count_exec_ne_targets(args) == 0: + raise ToolArgError("targets_required") + if _count_exec_ne_targets(args) > 20: + raise ToolArgError("too_many_targets (max 20 across targets/ne_ids/nms_ne_ids)") + if multi: body: dict[str, Any] = {} if isinstance(targets_raw, list) and targets_raw: @@ -502,13 +540,20 @@ def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: row: dict[str, Any] = {} if str(t.get("ne_id") or "").strip(): row["ne_id"] = str(t.get("ne_id")).strip() - # Wire still uses ume_ne_id on REST; accept nms_ne_id as preferred model param. - nms_id = _first_str(t.get("nms_ne_id"), t.get("ume_ne_id")) + # Wire still uses ume_ne_id on REST. + nms_id = _first_str(t.get("nms_ne_id")) + if bool(row.get("ne_id")) == bool(nms_id): + raise ToolArgError("Each target requires exactly one of ne_id or nms_ne_id") if nms_id: row["ume_ne_id"] = nms_id cmds = t.get("commands") if isinstance(cmds, list) and cmds: row["commands"] = [str(c).strip() for c in cmds if str(c).strip()] + effective_commands = row.get("commands", shared_commands) + if not effective_commands: + raise ToolArgError("commands_required for every target") + if len(effective_commands) > exec_max_commands(): + raise ToolArgError("too_many_commands for target") if row: cleaned_targets.append(row) body["targets"] = cleaned_targets @@ -516,6 +561,8 @@ def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: body["ne_ids"] = [str(x).strip() for x in ne_ids_raw if str(x).strip()] if isinstance(nms_ne_ids_raw, list) and nms_ne_ids_raw: body["ume_ne_ids"] = [str(x).strip() for x in nms_ne_ids_raw if str(x).strip()] + if (body.get("ne_ids") or body.get("ume_ne_ids")) and not shared_commands: + raise ToolArgError("commands_required for ne_ids/nms_ne_ids") if shared_commands: if len(shared_commands) > exec_max_commands(): return {"ok": False, "error": "too_many_commands", "error_code": "too_many_commands"} @@ -525,28 +572,25 @@ def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: body["concurrency"] = _int_arg(args, "concurrency", 4, 1, 8) if want_async: out = http_post_json("/v1/managed-ne/exec-jobs", body, timeout=60.0) - if not out.get("ok"): - return out - data = out.get("data") if isinstance(out.get("data"), dict) else out - return data if isinstance(data, dict) else {"ok": True, "data": data} + return _exec_ack(out) # Wall clock: many NEs × per-cmd timeout; keep below oclaw MCP override. out = http_post_json("/v1/managed-ne/exec-batch", body, timeout=600.0) if not out.get("ok"): return out data = out.get("data") or {} - if isinstance(data, dict) and data.get("ok") is False: + if _exec_result_failed(data): return {"ok": False, "data": data, "error": str(data.get("error") or "exec_batch_failed")} return {"ok": True, "data": data} ne_id = str(args.get("ne_id") or "").strip() - nms_ne_id = _first_str(args.get("nms_ne_id"), args.get("ume_ne_id")) + nms_ne_id = _first_str(args.get("nms_ne_id")) if bool(ne_id) == bool(nms_ne_id): return { "ok": False, "error": "exactly_one_of_ne_id_or_nms_ne_id_required", "error_code": "exactly_one_of_ne_id_or_nms_ne_id_required", "hint": ( - "For one NE pass ne_id OR nms_ne_id (alias ume_ne_id). For many NEs pass " + "For one NE pass ne_id OR nms_ne_id. For many NEs pass " "ne_ids / nms_ne_ids with shared commands, or targets[] with per-NE commands — " "one call, concurrent on server." ), @@ -564,10 +608,7 @@ def _exec_managed_ne(args: dict[str, Any]) -> dict[str, Any]: body["read_timeout_sec"] = _int_arg(args, "read_timeout_sec", 60, 10, 120) if want_async: out = http_post_json("/v1/managed-ne/exec-jobs", body, timeout=60.0) - if not out.get("ok"): - return out - data = out.get("data") if isinstance(out.get("data"), dict) else out - return data if isinstance(data, dict) else {"ok": True, "data": data} + return _exec_ack(out) out = http_post_json("/v1/managed-ne/exec", body, timeout=300.0) if not out.get("ok"): return out @@ -586,7 +627,16 @@ def _get_ne_exec_job(args: dict[str, Any]) -> dict[str, Any]: "error_code": "job_id_required", "hint": "Pass job_id from execManagedNe async ack.", } - return http_json("GET", f"/v1/managed-ne/exec-jobs/{quote_ne_id(job_id)}", params=None) + out = http_json("GET", f"/v1/managed-ne/exec-jobs/{quote_ne_id(job_id)}", params=None) + data = out.get("data") + if out.get("ok") and isinstance(data, dict): + if data.get("terminal") and (data.get("status") in {"failed", "timeout"} + or _exec_result_failed(data.get("result"))): + return {"ok": False, "error": "ne_exec_job_failed", "data": data} + if not data.get("terminal") and data.get("status") == "running": + data["poll_after_ms"] = 5000 + data["hint"] = "Still running. Continue independent work; poll this job_id after about 5 seconds. Do not resubmit." + return out def _list_cli_targets(args: dict[str, Any]) -> dict[str, Any]: @@ -610,9 +660,9 @@ def _list_cli_targets(args: dict[str, Any]) -> dict[str, Any]: def _find_topology_paths(args: dict[str, Any]) -> dict[str, Any]: - from_uid = _first_str(args.get("from_nms_ne_id"), args.get("from_ume_ne_id")) + from_uid = _first_str(args.get("from_nms_ne_id")) from_mid = str(args.get("from_managed_ne_id") or "").strip() - to_uid = _first_str(args.get("to_nms_ne_id"), args.get("to_ume_ne_id")) + to_uid = _first_str(args.get("to_nms_ne_id")) to_mid = str(args.get("to_managed_ne_id") or "").strip() if bool(from_uid) == bool(from_mid): return {"ok": False, "error": "exactly_one_of_from_nms_ne_id_or_from_managed_ne_id_required"} @@ -664,7 +714,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "is_cleared": {"type": "string", "description": "\"False\" = active only; \"True\" = cleared."}, "ne_id": {"type": "string", "description": "Filter only; do not show UUID to users"}, "host_name": {"type": "string", "description": "Filter by NE host_name"}, - "ne_name": {"type": "string", "description": "Legacy alias mapped to keyword"}, "keyword": { "type": "string", "description": ( @@ -703,9 +752,9 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "top_ne": { "type": "integer", "minimum": 0, - "maximum": 500, + "maximum": 5000, "default": 50, - "description": "Max NE buckets to return (0 = all, capped at API 5000). Ignored when group_by is set.", + "description": "Max NE buckets to return (0 = all; positive limit up to 5000). Ignored when group_by is set.", }, "exclude_missing_host": { "type": "boolean", @@ -883,8 +932,9 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "name": "getManagedNe", "description": ( "Get one **managed** NE by managed ne_id (from listManagedNe / listCliTargets source=managed). " - "Response includes capability (device_family, exec_policy_effective, recommended_mode, hints) " - "— read it before complex execManagedNe. " + "Response includes BACKEND capability (device_family, exec_policy_effective, recommended_mode, hints). " + "This MCP independently permits show/display only: backend shell/script capability does not " + "authorize scripts here. Native Linux/RouterOS CLI is not exposed by this MCP. " "Do NOT pass NMS inventory UUID here — use getNmsNe or execManagedNe(nms_ne_id=...) instead." ), "inputSchema": { @@ -894,23 +944,21 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "type": "string", "description": "Managed NE id (not NMS host_name / not NMS ne_id unless they coincide).", }, - "managed_ne_id": {"type": "string", "description": "Alias for ne_id."}, - "id": {"type": "string", "description": "Alias for ne_id."}, }, - "required": [], + "required": ["ne_id"], "additionalProperties": False, }, }, { "name": "execManagedNe", "description": ( - f"Run CLI via netx (default read-only: show/display/ping/traceroute; " + f"Run single-line read-only show/display CLI via netx (" f"max {exec_max_commands()} commands per NE, NETX_NE_EXEC_MAX_COMMANDS). " - "Managed NE exec_policy=linux_shell|unrestricted allows shell/script on Linux or " - "MikroTik (routeros/switchos) hosts " - "(pipes/&&/;/quotes/heredoc / RouterOS multiline OK; check getManagedNe / listManagedNe). " - "Single NE: ne_id OR nms_ne_id (+ alias ume_ne_id) + commands. " - "Many NEs (batch-first, server concurrency default 4, max 20): " + "MCP independently allows single-line read-only show/display only, even when the " + "device has an open exec_policy. Shell operators, configuration commands and multiline " + "scripts are rejected before HTTP; only read-only output-filter pipes are allowed. " + "Single NE: ne_id OR nms_ne_id + commands. " + "Many NEs (batch-first, max 20 targets, concurrency default 4, max 8): " "(1) same CLI on all → ne_ids[]/nms_ne_ids[] + shared commands; " "(2) different CLI per NE (vendor/role) → ONE targets=[" "{nms_ne_id|ne_id, commands:[…]}, …] — do NOT fall back to one-NE loops. " @@ -918,14 +966,15 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "Default read_timeout_sec=60; on timeout raise to 90–120 — do not blind-retry. " "Long or multi-NE work: async=true (or auto when ≥4 NEs) returns job_id immediately; " "poll getNeExecJob. Pass async=false to force sync. " - "Read getManagedNe.capability before choosing show_only vs script_on_device." + "Use getManagedNe.capability to identify the vendor and backend policy; open backend " + "policies never expand this MCP's show/display restriction. For batches inspect " + "summary.failed and every result: job succeeded/result.ok alone does not mean all targets passed." ), "inputSchema": { "type": "object", "properties": { "ne_id": {"type": "string"}, "nms_ne_id": {"type": "string", "description": "NMS inventory id (preferred)."}, - "ume_ne_id": {"type": "string", "description": "Legacy alias of nms_ne_id."}, "ne_ids": { "type": "array", "items": {"type": "string"}, @@ -938,12 +987,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "maxItems": 20, "description": "NMS inventory ne_ids for concurrent batch (shared commands).", }, - "ume_ne_ids": { - "type": "array", - "items": {"type": "string"}, - "maxItems": 20, - "description": "Legacy alias of nms_ne_ids.", - }, "targets": { "type": "array", "maxItems": 20, @@ -952,7 +995,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "properties": { "ne_id": {"type": "string"}, "nms_ne_id": {"type": "string"}, - "ume_ne_id": {"type": "string", "description": "Legacy alias of nms_ne_id."}, "commands": { "type": "array", "items": {"type": "string"}, @@ -1010,7 +1052,10 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "description": ( "Poll a background execManagedNe job (job_id from async ack). " "When terminal=true, result holds the exec/exec-batch payload. " - "Do not busy-wait; end the turn and poll later if still running." + "Use poll_after_ms (about 5 seconds); continue independent work while waiting. " + "Terminal failed/timeout is a tool error and retains the original result for diagnosis." + " Batch jobs can finish succeeded with individual failures: inspect result.summary.failed " + "and each result.results[].ok before reporting success." ), "inputSchema": { "type": "object", @@ -1033,9 +1078,9 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "properties": { "source": { "type": "string", - "enum": ["managed", "nms", "ume", "all"], + "enum": ["managed", "nms", "all"], "default": "all", - "description": "nms preferred; ume is a legacy alias mapped to the same inventory.", + "description": "nms selects the configured NMS inventory; managed selects managed NEs.", }, "keyword": {"type": "string"}, "page": {"type": "integer", "minimum": 1, "default": 1}, @@ -1050,8 +1095,8 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "description": ( "Find up to max_paths simple paths between two fabric nodes for troubleshooting " "(parallel links between the same NEs count as one hop; see edges[].parallel_links). " - "For each endpoint provide exactly one of nms_ne_id (from NMS alarm ne_id; alias " - "from_ume_ne_id) or managed_ne_id — resolved to fabric node internally. Returns " + "For each endpoint provide exactly one of nms_ne_id (from NMS alarm ne_id) or " + "managed_ne_id — resolved to fabric node internally. Returns " "shortest paths first with compact label + node/edge summary (detail=summary default). " "Use after critical alarms to correlate neighboring NEs before CLI login." ), @@ -1062,10 +1107,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "type": "string", "description": "Source NMS ne_id (from alarm ne_id); mutually exclusive with from_managed_ne_id", }, - "from_ume_ne_id": { - "type": "string", - "description": "Legacy alias of from_nms_ne_id", - }, "from_managed_ne_id": { "type": "string", "description": "Source managed NE id; mutually exclusive with from_nms_ne_id", @@ -1074,10 +1115,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "type": "string", "description": "Target NMS ne_id; mutually exclusive with to_managed_ne_id", }, - "to_ume_ne_id": { - "type": "string", - "description": "Legacy alias of to_nms_ne_id", - }, "to_managed_ne_id": { "type": "string", "description": "Target managed NE id; mutually exclusive with to_nms_ne_id", @@ -1121,8 +1158,8 @@ TOOL_REQUIRED_SCOPE: dict[str, str] = { "queryNmsAlarms": "alarms:read", "aggregateNmsAlarms": "alarms:read", "runNmsDiagnostics": "alarms:read", - "queryNmsNeInventory": "ne:read", - "getNmsNe": "ne:read", + "queryNmsNeInventory": "alarms:read", + "getNmsNe": "alarms:read", "queryNmsAlarmsRaw": "alarms:read", "aggregateNmsAlarmsRaw": "alarms:read", "listNmsAlarmFields": "alarms:read", @@ -1137,7 +1174,7 @@ TOOL_REQUIRED_SCOPE: dict[str, str] = { def tools_for_scopes(scopes: list[str] | set[str] | frozenset[str] | None) -> list[dict[str, Any]]: - """Filter MCP tool list by granted scopes. Empty/None => return all (offline / unauthenticated listing).""" + """Filter by granted scopes. None advertises all offline; an empty grant advertises none.""" if scopes is None: return list(HTTP_MCP_TOOLS) granted = {str(s).strip().lower() for s in scopes if str(s).strip()} @@ -1152,12 +1189,56 @@ def tools_for_scopes(scopes: list[str] | set[str] | frozenset[str] | None) -> li return out +def _validate_arguments(args: Any, schema: dict[str, Any], path: str = "arguments") -> None: + """Enforce advertised shapes even when a host skips JSON Schema validation.""" + kind = schema.get("type") + expected = {"object": dict, "array": list, "string": str, "boolean": bool, "integer": int} + if kind in expected and (not isinstance(args, expected[kind]) or (kind == "integer" and isinstance(args, bool))): + raise ToolArgError(f"{path} must be {kind}") + if "enum" in schema and args not in schema["enum"]: + raise ToolArgError(f"{path} must be one of {schema['enum']}") + if kind == "object": + props = schema.get("properties", {}) + unknown = set(args) - set(props) + if schema.get("additionalProperties") is False and unknown: + raise ToolArgError(f"{path}: unsupported parameter(s): {', '.join(sorted(unknown))}") + for key, value in args.items(): + if key in props: + _validate_arguments(value, props[key], f"{path}.{key}") + elif kind == "array": + # Command limits are read at invocation time by the handler (env may change). + if not path.endswith(".commands") and len(args) > schema.get("maxItems", float("inf")): + raise ToolArgError(f"{path} has too many items (max {schema['maxItems']})") + if len(args) < schema.get("minItems", 0): + raise ToolArgError(f"{path} requires at least {schema['minItems']} item(s)") + for i, value in enumerate(args): + _validate_arguments(value, schema.get("items", {}), f"{path}[{i}]") + + +_SECRET_KEYS = frozenset({ + "password", "hop_password", "password_enc", "hop_password_enc", "password_ciphertext", + "hop_password_ciphertext", "credential_secret", "secret_key", "token", "access_token", + "refresh_token", "authorization", "private_key", +}) + + +def _redact_secrets(value: Any) -> Any: + """Keep capability/has_password flags, but never expose structured credentials to a model.""" + if isinstance(value, dict): + return {key: _redact_secrets(item) for key, item in value.items() if str(key).lower() not in _SECRET_KEYS} + if isinstance(value, list): + return [_redact_secrets(item) for item in value] + return value + + def call_http_tool(name: str, args: dict[str, Any]) -> dict[str, Any]: fn = _HANDLERS.get(str(name or "").strip()) if not fn: raise ValueError(f"unknown tool: {name}") try: + schema = next(tool["inputSchema"] for tool in HTTP_MCP_TOOLS if tool["name"] == name) + _validate_arguments(args, schema) result = fn(dict(args or {})) except ToolArgError as exc: result = {"ok": False, "error": "invalid_argument", "error_code": "invalid_argument", "detail": str(exc)} - return mcp_from_handler_result(result) + return mcp_from_handler_result(_redact_secrets(result)) diff --git a/packages/netx-mcp/tests/test_cli_safety.py b/packages/netx-mcp/tests/test_cli_safety.py new file mode 100644 index 0000000..9f0cc99 --- /dev/null +++ b/packages/netx-mcp/tests/test_cli_safety.py @@ -0,0 +1,77 @@ +"""无设备/网络副作用地验证 MCP 执行边界与终态失败语义。""" +import json +from unittest.mock import patch + +import pytest + +from netx_mcp.http_tools import call_http_tool + + +@pytest.mark.parametrize("command", [ + "reboot", "configure terminal", "show version; reboot", "show version && reboot", + "show version | reboot", "show version | redirect flash:config", "show $(reboot)", + "show `reboot`", "display version\nreboot", "show version\u2028reboot", + "show version > config", "show version\x00", "/system reboot", +]) +@pytest.mark.parametrize("batch", [False, True]) +def test_unsafe_cli_never_reaches_http(command, batch): + args = {"targets": [{"ne_id": "test-only", "commands": [command]}]} if batch else { + "ne_id": "test-only", "commands": [command], "async": True} + with patch("netx_mcp.http_tools.http_post_json") as post: + result = call_http_tool("execManagedNe", args) + assert result.get("isError") is True + post.assert_not_called() + + +@pytest.mark.parametrize("command", ["show version", "display version", "show bgp summary | one-line"]) +def test_readonly_cli_still_executes(command): + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": {"ok": True}}) as post: + result = call_http_tool("execManagedNe", {"ne_id": "test-only", "commands": [command]}) + assert not result.get("isError") + assert post.call_args.args[0] == "/v1/managed-ne/exec" + assert post.call_args.args[1]["commands"] == [command] + + +def test_async_ack_explains_polling_without_forcing_turn_end(): + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": { + "ok": True, "job_id": "test-job", "status": "running", "hint": "end the turn"}}): + result = call_http_tool("execManagedNe", {"ne_id": "test-only", "commands": ["show version"], "async": True}) + data = json.loads(result["content"][0]["text"]) + assert data["poll_after_ms"] == 5000 and data["job_id"] == "test-job" + assert "end the turn" not in data["hint"] + + +def test_failed_exec_job_is_tool_error_and_preserves_evidence(): + terminal = {"ok": True, "terminal": True, "status": "failed", "result": { + "ok": False, "detail": "SSH connection failed", "commands": ["display version"]}} + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": terminal}): + result = call_http_tool("getNeExecJob", {"job_id": "test-job"}) + assert result.get("isError") is True + data = json.loads(result["content"][0]["text"]) + assert data["data"]["result"]["detail"] == "SSH connection failed" + + +@pytest.mark.parametrize("failed", [1, 2]) +@pytest.mark.parametrize("async_job", [False, True]) +def test_batch_target_failures_are_tool_errors_even_when_backend_ok(failed, async_job): + result = {"ok": True, "summary": {"total": 2, "ok": 2-failed, "failed": failed}, + "results": [{"ok": failed == 1, "output": "version"}, {"ok": False, "detail": "SSH banner timeout"}]} + terminal = {"ok": True, "terminal": True, "status": "succeeded", "result": result} + if async_job: + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": terminal}): + out = call_http_tool("getNeExecJob", {"job_id": "test-job"}) + assert json.loads(out["content"][0]["text"])["data"] == terminal + else: + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": result}): + out = call_http_tool("execManagedNe", {"ne_ids": ["m1", "m2"], "commands": ["show version"], "async": False}) + assert json.loads(out["content"][0]["text"])["data"] == result + assert out["isError"] is True + + +@pytest.mark.parametrize("target_ok", [False, True]) +def test_batch_checks_individual_results_without_summary(target_ok): + terminal = {"ok": True, "terminal": True, "status": "succeeded", "result": { + "ok": True, "results": [{"ok": target_ok, "output": "version"}]}} + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": terminal}): + out = call_http_tool("getNeExecJob", {"job_id": "test-job"}) + assert bool(out.get("isError")) is not target_ok diff --git a/packages/netx-mcp/tests/test_mcp_http.py b/packages/netx-mcp/tests/test_mcp_http.py index 5b80df7..6393825 100644 --- a/packages/netx-mcp/tests/test_mcp_http.py +++ b/packages/netx-mcp/tests/test_mcp_http.py @@ -103,18 +103,6 @@ def test_call_find_topology_paths_defaults_summary_detail() -> None: assert payload["ok"] is True -def test_call_find_topology_paths_accepts_legacy_ume_params() -> None: - with patch("netx_mcp.http_tools.http_post_json") as mock_post: - mock_post.return_value = {"ok": True, "data": {"path_count": 0, "paths": []}} - call_http_tool( - "findTopologyPaths", - {"from_ume_ne_id": "a", "to_ume_ne_id": "b"}, - ) - body = mock_post.call_args[0][1] - assert body["from_ume_ne_id"] == "a" - assert body["to_ume_ne_id"] == "b" - - def test_call_exec_managed_ne_defaults_read_timeout() -> None: with patch("netx_mcp.http_tools.http_post_json") as mock_post: mock_post.return_value = {"ok": True, "data": {"ok": True, "output": "hi"}} @@ -138,16 +126,6 @@ def test_get_managed_ne_requires_id_with_hint() -> None: assert "hint" in payload -def test_get_managed_ne_accepts_managed_ne_id_alias() -> None: - with patch("netx_mcp.http_tools.http_json") as mock_http: - mock_http.return_value = {"ok": True, "data": {"ne_id": "m1"}} - out = call_http_tool("getManagedNe", {"managed_ne_id": "m1"}) - mock_http.assert_called_once() - assert mock_http.call_args[0][1].endswith("/m1") - payload = json.loads(out["content"][0]["text"]) - assert payload["ok"] is True - - def test_call_get_nms_ne_requires_id() -> None: out = call_http_tool("getNmsNe", {}) assert out.get("isError") is True @@ -202,7 +180,7 @@ def test_exec_managed_ne_schema_documents_batch() -> None: props = tool["inputSchema"]["properties"] assert "ne_ids" in props assert "nms_ne_ids" in props - assert "ume_ne_ids" in props + assert "ume_ne_ids" not in props assert "targets" in props assert "concurrency" in props assert "Many NEs" in tool["description"] or "ne_ids" in tool["description"] @@ -394,3 +372,194 @@ def test_managed_ne_list_drops_disabled_hop_noise() -> None: out = call_http_tool("listManagedNe", {"connect_status": "pass"}) row = json.loads(out["content"][0]["text"])["data"]["items"][0] assert row == {"id": "m1", "hop_enabled": False} + + +@pytest.mark.parametrize(("name", "args"), [ + ("queryNmsAlarms", {"ne_name": "old"}), + ("getManagedNe", {"managed_ne_id": "old"}), + ("getManagedNe", {"id": "old"}), + ("execManagedNe", {"ume_ne_id": "old", "commands": ["show version"]}), + ("execManagedNe", {"ume_ne_ids": ["old"], "commands": ["show version"]}), + ("execManagedNe", {"targets": [{"ume_ne_id": "old", "commands": ["show version"]}]}), + ("findTopologyPaths", {"from_ume_ne_id": "old", "to_nms_ne_id": "b"}), + ("findTopologyPaths", {"from_nms_ne_id": "a", "to_ume_ne_id": "old"}), + ("listCliTargets", {"source": "ume"}), +]) +def test_obsolete_aliases_are_rejected_before_http(name: str, args: dict) -> None: + with patch("netx_mcp.http_tools.http_json") as get, patch("netx_mcp.http_tools.http_post_json") as post: + out = call_http_tool(name, args) + get.assert_not_called() + post.assert_not_called() + assert out["isError"] is True + assert json.loads(out["content"][0]["text"])["error"] == "invalid_argument" + + +def test_all_fifteen_tools_match_backend_scope_policy() -> None: + from netx_api.auth_scopes import required_scope_for_request + from netx_mcp.http_tools import TOOL_REQUIRED_SCOPE + + routes = { + "queryNmsAlarms": ("GET", "/v1/ume/alarms"), + "aggregateNmsAlarms": ("GET", "/v1/ume/alarms/aggregate"), + "runNmsDiagnostics": ("GET", "/v1/ume/diagnostics"), + "queryNmsNeInventory": ("GET", "/v1/ume/inventory/ne"), + "getNmsNe": ("GET", "/v1/ume/inventory/ne/id"), + "queryNmsAlarmsRaw": ("GET", "/v1/ume/alarms/raw"), + "aggregateNmsAlarmsRaw": ("GET", "/v1/ume/alarms/aggregate/raw"), + "listNmsAlarmFields": ("GET", "/v1/ume/alarms/fields"), + "sqlQueryNms": ("POST", "/v1/sql/ume_query"), + "listManagedNe": ("GET", "/v1/managed-ne"), + "getManagedNe": ("GET", "/v1/managed-ne/id"), + "execManagedNe": ("POST", "/v1/managed-ne/exec"), + "getNeExecJob": ("GET", "/v1/managed-ne/exec-jobs/id"), + "listCliTargets": ("GET", "/v1/cli/targets"), + "findTopologyPaths": ("POST", "/v1/topology/fabric/paths"), + } + assert set(routes) == {t["name"] for t in HTTP_MCP_TOOLS} == set(TOOL_REQUIRED_SCOPE) + for name, (method, path) in routes.items(): + assert TOOL_REQUIRED_SCOPE[name] == required_scope_for_request(method, path) + assert name in {t["name"] for t in tools_for_scopes([TOOL_REQUIRED_SCOPE[name]])} + assert tools_for_scopes([]) == [] + + +@pytest.mark.parametrize(("name", "args", "path", "expected"), [ + ("aggregateNmsAlarms", {"top_ne": 3000}, "/v1/ume/alarms/aggregate", {"top_ne": 3000}), + ("runNmsDiagnostics", {}, "/v1/ume/diagnostics", None), + ("queryNmsNeInventory", {"keyword": "edge", "page": 9, "page_size": 900}, "/v1/ume/inventory/ne", {"keyword": "edge", "page": 9, "page_size": 500}), + ("listNmsAlarmFields", {}, "/v1/ume/alarms/fields", None), + ("getNeExecJob", {"job_id": "job /?"}, "/v1/managed-ne/exec-jobs/job%20%2F%3F", None), + ("listCliTargets", {"source": "nms", "keyword": "edge"}, "/v1/cli/targets", {"source": "ume", "keyword": "edge", "page": 1, "page_size": 50}), +]) +def test_ops_read_routes_and_backend_limits(name: str, args: dict, path: str, expected: dict | None) -> None: + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": {"items": []}}) as get: + out = call_http_tool(name, args) + get.assert_called_once_with("GET", path, params=expected) + assert not out.get("isError") + + +def test_raw_aggregation_filters_and_compaction() -> None: + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": {"buckets": [{"key": "edge", "count": 4}], "meta": {"group_by_allowed": ["large"], "time_filter_field": "last_seen_at"}}}) as get: + out = call_http_tool("aggregateNmsAlarmsRaw", {"group_by": "alarm_host_name", "severity": "CRITICAL", "exclude_missing_host": False, "limit": 9999}) + assert get.call_args.kwargs["params"] == {"group_by": "alarm_host_name", "severity": "critical", "exclude_missing_host": False, "limit": 2000} + data = json.loads(out["content"][0]["text"])["data"] + assert data["buckets"][0]["count"] == 4 + assert data["meta"] == {"time_filter_field": "last_seen_at"} + + +def test_sql_limits_are_forwarded_and_denial_is_tool_error() -> None: + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": False, "error": "netx_http_403", "detail": "insufficient_scope:sql:query"}) as post: + out = call_http_tool("sqlQueryNms", {"sql": " SELECT ne_id FROM ume_inventory_ne ", "limit": 9999, "statement_timeout_ms": 99999}) + post.assert_called_once_with("/v1/sql/ume_query", {"sql": "SELECT ne_id FROM ume_inventory_ne", "limit": 2000, "statement_timeout_ms": 30000}, timeout=60.0) + assert out["isError"] is True + assert "insufficient_scope" in out["content"][0]["text"] + + +@pytest.mark.parametrize("name,args", [ + ("listManagedNe", {"keyword": "edge"}), + ("getManagedNe", {"ne_id": "m1"}), + ("listCliTargets", {"source": "managed"}), + ("getNmsNe", {"ne_id": "u1"}), +]) +def test_structured_credentials_are_removed_but_capabilities_remain(name: str, args: dict) -> None: + row = {"id": "m1", "password": "secret", "has_password": True, "capability": {"recommended_mode": "show_only", "token": "secret"}, "hop_password_enc": "secret"} + data = {"items": [row]} if name.startswith("list") else row + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": data}): + out = call_http_tool(name, args) + text = out["content"][0]["text"] + assert "secret" not in text + assert "show_only" in text + assert "has_password" in text + + +@pytest.mark.parametrize("args", [ + {"targets": [{"ne_id": "m1", "nms_ne_id": "u1", "commands": ["show version"]}]}, + {"targets": [{"ne_id": "m1"}]}, + {"targets": [{"ne_id": "m1", "commands": ["show version"] * 6}]}, + {"ne_ids": ["m1"]}, + {"ne_ids": ["m1"] * 11, "nms_ne_ids": ["u1"] * 10, "commands": ["show version"]}, + {"ne_id": "m1", "ne_ids": ["m2"], "commands": ["show version"]}, + {"targets": ["m1"], "commands": ["show version"]}, +]) +def test_invalid_exec_batches_do_not_reach_http(args: dict, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("NETX_NE_EXEC_MAX_COMMANDS", "5") + with patch("netx_mcp.http_tools.http_post_json") as post: + out = call_http_tool("execManagedNe", args) + post.assert_not_called() + assert out["isError"] is True + + +def test_async_auto_counts_all_target_forms_and_poll_preserves_terminal_result(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("NETX_NE_EXEC_ASYNC_MIN_NES", "4") + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": {"ok": True, "job_id": "j1", "terminal": False}}) as post: + ack = call_http_tool("execManagedNe", {"ne_ids": ["m1", "m2"], "nms_ne_ids": ["u1", "u2"], "commands": ["show version"], "concurrency": 20}) + assert post.call_args.args[0] == "/v1/managed-ne/exec-jobs" + assert post.call_args.args[1]["concurrency"] == 8 + assert json.loads(ack["content"][0]["text"])["job_id"] == "j1" + terminal = {"ok": True, "terminal": True, "result": {"ok": True, "results": [{"ok": False, "error": "timeout"}]}} + with patch("netx_mcp.http_tools.http_json", return_value={"ok": True, "data": terminal}): + out = call_http_tool("getNeExecJob", {"job_id": "j1"}) + assert json.loads(out["content"][0]["text"])["data"] == terminal + assert out["isError"] is True + + +def test_explicit_sync_overrides_auto_async(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("NETX_NE_EXEC_ASYNC_MIN_NES", "1") + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": {"ok": True}}) as post: + call_http_tool("execManagedNe", {"nms_ne_ids": ["u1", "u2"], "commands": ["show version"], "async": False}) + assert post.call_args.args[0] == "/v1/managed-ne/exec-batch" + + +def test_stdio_scope_denial_never_invokes_handler() -> None: + import io + from netx_mcp.server import run_stdio_loop + + request = {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "execManagedNe", "arguments": {"ne_id": "m1", "commands": ["show version"]}}} + output = io.StringIO() + with patch("netx_mcp.server.sys.stdin", io.StringIO(json.dumps(request) + "\n")), patch("netx_mcp.server.sys.stdout", output), patch("netx_mcp.server._fetch_scopes", return_value=["ne:read"]), patch("netx_mcp.server.call_http_tool") as handler: + run_stdio_loop() + handler.assert_not_called() + response = json.loads(output.getvalue()) + assert response["error"]["code"] == -32001 + assert "ne:exec" in response["error"]["message"] + + +@pytest.mark.parametrize("method", ["GET", "POST"]) +@pytest.mark.parametrize("status", [401, 403, 404, 500]) +def test_http_errors_preserve_status_and_bound_detail(method: str, status: int) -> None: + import httpx + from netx_mcp.http_client import http_json, http_post_json + + transport = httpx.MockTransport(lambda request: httpx.Response(status, text="denied" * 300)) + client = httpx.Client(transport=transport) + with patch("netx_mcp.http_client.httpx.Client", return_value=client): + result = http_json("GET", "/v1/ume/alarms") if method == "GET" else http_post_json("/v1/managed-ne/exec", {}) + assert result["ok"] is False + assert result["error"] == f"netx_http_{status}" + assert len(result["detail"]) == 800 + + + +def test_mixed_exec_targets_translate_nms_ids_and_preserve_per_target_commands() -> None: + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": {"ok": True, "items": [{"ok": True, "password": "secret", "output": "version"}]}}) as post: + out = call_http_tool("execManagedNe", {"targets": [{"ne_id": "m1"}, {"nms_ne_id": "u1", "commands": ["display version"]}], "commands": ["show version"], "async": False}) + assert post.call_args.args[0] == "/v1/managed-ne/exec-batch" + assert post.call_args.args[1]["targets"] == [{"ne_id": "m1"}, {"ume_ne_id": "u1", "commands": ["display version"]}] + text = out["content"][0]["text"] + assert "secret" not in text + assert json.loads(text)["data"]["items"][0]["output"] == "version" + + +def test_async_job_queue_failure_is_not_a_successful_ack() -> None: + with patch("netx_mcp.http_tools.http_post_json", return_value={"ok": True, "data": {"ok": False, "error": "ne_exec_job_busy"}}): + out = call_http_tool("execManagedNe", {"ne_id": "m1", "commands": ["show version"], "async": True}) + assert out["isError"] is True + assert json.loads(out["content"][0]["text"])["error"] == "ne_exec_job_busy" + + +@pytest.mark.parametrize("args", [{}, {"group_by": ""}]) +def test_raw_aggregation_missing_dimension_is_actionable(args: dict) -> None: + with patch("netx_mcp.http_tools.http_json") as get: + out = call_http_tool("aggregateNmsAlarmsRaw", args) + get.assert_not_called() + assert out["isError"] is True + assert "group_by" in out["content"][0]["text"] diff --git a/packages/netx-topology-mcp/README.md b/packages/netx-topology-mcp/README.md index b5b6405..0f6cbea 100644 --- a/packages/netx-topology-mcp/README.md +++ b/packages/netx-topology-mcp/README.md @@ -8,7 +8,7 @@ MCP 宿主 → stdio netx_topology_mcp → HTTP NETX_API_URL → netx AP 详细说明 → **[docs/MCP_TOPOLOGY.md](../../docs/MCP_TOPOLOGY.md)** -配套 Skill → **[`.cursor/skills/netx-topology`](../../.cursor/skills/netx-topology/SKILL.md)**（画图流水线与硬规则） +配套 Skill → **[`skills/topology/netx-topology`](../../skills/topology/netx-topology/SKILL.md)**（画图流水线与硬规则） ## 目录模型 @@ -38,14 +38,35 @@ pip install "git+https://github.com/hansjone/netx.git#subdirectory=packages/netx 画图请在 env 里设带 `ne:write` 的 `NETX_API_TOKEN`（不要只靠仓库 `mcp_token`）。`getTopologyView` / `projectTopologyNeighbors` 默认 `detail=summary`（坐标抽样 + `links[]` 邻接）；`queryTopologyEdges` 默认 adjacency（一对网元一条线），避免整图/端口边占满上下文。 -## 工具（14） +## 工具（16） + +覆盖 netxops `topology` 清单全部 15 项，另提供 `sinkTopologyDualUnits`。netxops 工具名前缀为 `netx__`，独立 MCP 不带此前缀。 | 类别 | 工具 | |------|------| | 树/区域/画布 | `getTopologyTree`, `createTopologyFolder`（唯一建图，返回 view_id）, `getTopologyView` | | 画图 | `addTopologyViewNodes` / `remove…` / `sinkTopologyDualUnits`（分批沉入+默认可先 layout_dual_unit） / `copyTopologyViewNodes`（一键克隆测沙箱） / `update…Positions`（**优先筛选**，API 自选 id；也可 id 列表）, `projectTopologyNeighbors` | | Fabric 只读 | `queryTopologyFabricNodes`（summary\|list\|search）, `queryTopologyNeighborhood`, `queryTopologyEdges` | +| Fabric 分类 | `classifyTopologyFabricNodes`（match/tag/patch/unmatched/preview_rules/apply_rules/list_rules） | | 自动布图 | `layoutTopologyView`（小图 `compact`/`corridor`/`rings`；巨图：`sinkTopologyDualUnits`→`orbit_sweep`→`polish_crossings`→`clear_edge_hits`） | | 布图验收/规划 | `analyzeTopologyViewLayout`（score；`detail=structure` 定重心；`both`=structure+手拖 sight） | **安全约束：** MCP **不会**创建 Fabric 占位节点、**不会**写手工链路；画布只能引用已存在的 fabric 节点。 + +画布节点增删和位置布局只接受 `keyword/role/vendor/link_status` 筛选。按 `region_folder_id/level/level_major` 画图时，先用 `queryTopologyFabricNodes(mode=list)` 分页查询，再将返回的 `id` 作为 `fabric_node_ids` 传入画布工具。`projectTopologyNeighbors(dry_run=true)` 可预览邻居投影而不保存。 + +## 自动布局与异步任务 + +1. `layoutTopologyView({view_id, action:"layout", recipe:"compact", mode:"preview"})` 预览；`mode:"apply"` 才保存坐标。 +2. 需要后台计算时，调用 `layoutTopologyView({view_id, action:"layout", recipe:"compact", mode:"apply", params:{background:true}})`，记录返回的 `job_id`。布局/局部优化计算支持显式后台；`move_nodes` 和 `align_reference` 按各自流程同步执行。 +3. 调用 `layoutTopologyView({action:"job_status", params:{job_id}})`，直到 `status` 为 `done/error/cancelled`。`running/cancelling` 仍需轮询；`stale=true` 仅表示心跳过期，不能视为完成。成功必须检查 `ok=true`、`status=done`、`result.applied=true`。 +4. `getTopologyView({view_id, detail:"full"})` 回读 HTTP API 持久化后的坐标。`job_status` 的结果会裁剪大字段，不替代完整画布回读。 +5. 需要取消时调用 `layoutTopologyView({action:"job_cancel", params:{job_id}})`；取消为协作式，仍需轮询终态。已经写入的坐标不会因取消自动回滚。 + +部分大图优化会自动转后台：`polish_crossings/orbit_sweep/clear_edge_hits/compact_bbox/pull_far_chains/layout_dual_unit` 在节点数达到 600 时；`orbit_sweep(until_limit=true)` 的 apply 总是后台。`params.sync=true` 可强制同步。 + +任务 JSON 默认位于仓库 `data/runtime/layout_jobs`；用绝对路径 `NETX_LAYOUT_JOB_DIR` 可指定共享目录。后台子进程和任务文件能跨 MCP 主进程重启，轮询方必须访问同一目录；这不是 HTTP 后端任务接口，也不支持跨机器自动共享。历史回收仅清理终态，保留运行中的任务。 + +HTTP 200 内的 `ok=false` 会作为 MCP `isError=true` 返回。分批写入遇错即停；复制/移动/下沉若保存目标坐标失败，会保留源成员并报告已增加的目标成员。最终回读失败返回 `view_readback_failed` 和 `write_completed=true`，此时应先检查画布再重试。多次 HTTP 写入不构成事务，已完成的部分不会自动撤销。 + +只使用工具 schema 中公开的动作。旧动作别名 `resolve_overlaps/sink_nodes/align_to_reference`、查询模式 `stats/count/find`、分类动作 `find/bulk_tag/assign/preview/apply/rules`、详情别名 `raw/graph/full`（边查询的 full）已移除；规范动作分别见 `layoutTopologyView(catalog=true)` 与各工具 schema。 diff --git a/packages/netx-topology-mcp/src/netx_topology_mcp/http_tools.py b/packages/netx-topology-mcp/src/netx_topology_mcp/http_tools.py index 17dfe37..7caf3a5 100644 --- a/packages/netx-topology-mcp/src/netx_topology_mcp/http_tools.py +++ b/packages/netx-topology-mcp/src/netx_topology_mcp/http_tools.py @@ -28,7 +28,10 @@ def _data(out: dict[str, Any]) -> dict[str, Any]: data = out.get("data") if isinstance(data, dict): merged = dict(data) - merged["ok"] = True + # HTTP 200 can carry a failed mutation (for example membership_max_nodes). + merged.setdefault("ok", True) + if merged.get("ok") is False and not merged.get("error"): + merged["error"] = "netx_operation_failed" return merged return {"ok": True, "data": data} @@ -233,10 +236,12 @@ def _get_topology_view(args: dict[str, Any]) -> dict[str, Any]: if not out.get("ok"): return out detail = str(args.get("detail") or "summary").strip().lower() - if detail in {"full", "raw", "graph"}: + if detail not in {"summary", "full"}: + return {"ok": False, "error": "detail_invalid", "hint": "detail=summary|full"} + if detail == "full": out["detail"] = "full" return out - sample = int(args.get("sample") or 20) + sample = int(args["sample"]) if args.get("sample") is not None else 20 return _summarize_view_graph(out, sample=sample) @@ -359,7 +364,7 @@ def _patch_positions_chunked(view_id: str, positions: list[dict[str, Any]]) -> d "updated": updated, "detail": out, } - updated += int(out.get("updated") or len(chunk)) + updated += int(out.get("updated", len(chunk))) chunks += 1 return {"ok": True, "updated": updated, "chunks": chunks} @@ -469,7 +474,6 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: _public = { "layout", "fix_overlaps", - "resolve_overlaps", "untangle", "straighten_channels", "layout_dual_unit", @@ -478,11 +482,9 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: "compact_bbox", "pull_far_chains", "align_reference", - "align_to_reference", "orbit_sweep", "level_bands", "move_nodes", - "sink_nodes", "job_status", "job_cancel", } @@ -503,16 +505,12 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: ), **list_layout_catalog(), } - if action == "resolve_overlaps": - action = "fix_overlaps" - args = {**args, "action": "fix_overlaps"} - # Bidirectional membership move: view_id=TO, source_view_id=FROM. - if action in {"move_nodes", "sink_nodes"}: + if action == "move_nodes": return _move_topology_view_nodes(args) # Reference alignment: view_id=TO, source_view_id|params.reference_view_id=FROM. - if action in {"align_reference", "align_to_reference"}: + if action == "align_reference": return _align_topology_reference(args) # Background job poll / cancel (no view_id required). @@ -544,8 +542,8 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: "error": "job_not_found", "job_id": job_id, "hint": ( - "Unknown job_id. Jobs are in-process on this MCP worker; " - "restart or another instance cannot see them." + "Unknown job_id. Poll from a worker sharing NETX_LAYOUT_JOB_DIR; " + "records survive MCP restarts but may have been pruned." ), } status = str(job.get("status") or "unknown") @@ -870,9 +868,7 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: force_sync = _truthy((overrides or {}).get("sync")) or _truthy( (overrides or {}).get("_force_sync") ) - until_limit = _truthy((overrides or {}).get("until_limit")) or _truthy( - (overrides or {}).get("until_stall") - ) + until_limit = _truthy((overrides or {}).get("until_limit")) heavy_bg = action in { "polish_crossings", "orbit_sweep", @@ -885,10 +881,9 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: bg_n_thr = 80 if (action == "orbit_sweep" and until_limit) else 600 if ( mode == "apply" - and heavy_bg and not force_sync and ( - len(nodes) >= bg_n_thr + (heavy_bg and len(nodes) >= bg_n_thr) or _truthy((overrides or {}).get("background")) or (until_limit and action == "orbit_sweep") ) @@ -912,6 +907,9 @@ def _layout_topology_view(args: dict[str, Any]) -> dict[str, Any]: tool_args=sync_args, meta={"node_count": len(nodes), "until_limit": until_limit}, ) + started = job_public(job_id) + if started and started.get("status") in {"done", "error", "cancelled"}: + return _layout_topology_view({"action": "job_status", "params": {"job_id": job_id}}) print( f"[netx-topology] {action} background job_id={job_id} n={len(nodes)}" + (" until_limit" if until_limit else ""), @@ -1870,10 +1868,6 @@ def _add_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: "error": "fabric_nodes_only", "detail": "Use keyword/role/vendor/link_status or fabric_node_ids; never managed/UME ids.", } - if args.get("max_nodes") is not None: - bump = _ensure_view_max_nodes(view_id, int(args.get("max_nodes") or 2000)) - if isinstance(bump, dict) and bump.get("ok") is False: - return bump filters = _filter_fields(args) fabric_ids = [str(x) for x in (args.get("fabric_node_ids") or []) if str(x).strip()] if not filters and not fabric_ids: @@ -1883,6 +1877,11 @@ def _add_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: "detail": "Pass keyword/role/vendor/link_status (preferred) or fabric_node_ids.", } + if args.get("max_nodes") is not None: + bump = _ensure_view_max_nodes(view_id, int(args.get("max_nodes") or 2000)) + if bump.get("ok") is False: + return bump + layout = str(args.get("layout") or "grid").strip() or "grid" if filters: body: dict[str, Any] = { @@ -1909,6 +1908,8 @@ def _add_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: "return_graph": False, } parts.append(_data(http_json("POST", f"/v1/topology/views/{view_id}/nodes", body=body))) + if not parts[-1].get("ok"): + break return _merge_mutation_summaries(parts) @@ -1935,6 +1936,8 @@ def _remove_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: ) ) ) + if not parts[-1].get("ok"): + break return _merge_mutation_summaries(parts) @@ -2002,6 +2005,8 @@ def _update_topology_view_positions(args: dict[str, Any]) -> dict[str, Any]: ) ) ) + if not parts[-1].get("ok"): + break return _merge_mutation_summaries(parts) @@ -2009,8 +2014,11 @@ def _project_topology_neighbors(args: dict[str, Any]) -> dict[str, Any]: view_id = str(args.get("view_id") or "").strip() if not view_id: return {"ok": False, "error": "view_id_required"} + detail = str(args.get("detail") or "summary").strip().lower() + if detail not in {"summary", "full"}: + return {"ok": False, "error": "detail_invalid", "hint": "detail=summary|full"} body: dict[str, Any] = {} - seeds = args.get("seed_fabric_node_ids") or args.get("fabric_node_ids") or [] + seeds = args.get("seed_fabric_node_ids") or [] if isinstance(seeds, list) and seeds: body["seed_fabric_node_ids"] = [str(x).strip() for x in seeds if str(x).strip()] mids = args.get("managed_ne_ids") or [] @@ -2019,6 +2027,8 @@ def _project_topology_neighbors(args: dict[str, Any]) -> dict[str, Any]: region_folder_id = str(args.get("region_folder_id") or "").strip() if region_folder_id: body["region_folder_id"] = region_folder_id + if "dry_run" in args: + body["dry_run"] = _truthy(args["dry_run"]) # API still returns the full graph; large canvases easily exceed the 60s default. out = _data( http_json( @@ -2030,12 +2040,11 @@ def _project_topology_neighbors(args: dict[str, Any]) -> dict[str, Any]: ) if not out.get("ok"): return out - detail = str(args.get("detail") or "summary").strip().lower() skipped = int(out.get("out_of_region_skipped") or 0) skipped_sample = out.get("out_of_region_sample") or [] if not isinstance(skipped_sample, list): skipped_sample = [] - if detail in {"full", "raw", "graph"}: + if detail == "full": out["detail"] = "full" out["region_folder_id"] = region_folder_id or None out["out_of_region_skipped"] = skipped @@ -2046,9 +2055,11 @@ def _project_topology_neighbors(args: dict[str, Any]) -> dict[str, Any]: f"skipped {skipped} out-of-region neighbors (not added)." ) return out - summary = _summarize_view_graph(out, sample=int(args.get("sample") or 20)) + sample = int(args["sample"]) if args.get("sample") is not None else 20 + summary = _summarize_view_graph(out, sample=sample) summary["view_id"] = view_id summary["projected"] = True + summary["dry_run"] = bool(body.get("dry_run")) summary["region_folder_id"] = region_folder_id or None summary["out_of_region_skipped"] = skipped summary["out_of_region_sample"] = [ @@ -2076,13 +2087,15 @@ def _query_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: mode = "search" else: mode = "list" - if mode in {"summary", "stats", "count"}: + if mode not in {"summary", "search", "list"}: + return {"ok": False, "error": "mode_invalid", "hint": "mode=summary|list|search"} + if mode == "summary": out = _data(http_json("GET", "/v1/topology/fabric/summary")) if isinstance(out, dict) and out.get("ok") is not False: out = dict(out) out["mode"] = "summary" return out - if mode in {"search", "find"}: + if mode == "search": needle = q or keyword if not needle: return { @@ -2141,11 +2154,11 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: "hint": ( "action=match|tag|patch|unmatched|preview_rules|apply_rules|list_rules. " "Typical: match → tag(level|role, dry_run) → tag; " - "then addTopologyViewNodes by role/level_major/region." + "then queryTopologyFabricNodes by level/region and add returned fabric_node_ids." ), } - if action in {"match", "find"}: + if action == "match": pattern = str(args.get("pattern") or args.get("q") or "").strip() if not pattern: return {"ok": False, "error": "pattern_required"} @@ -2180,7 +2193,7 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: ) return out - if action in {"tag", "bulk_tag", "assign"}: + if action == "tag": level = args.get("level") if "level" in args else Ellipsis role = args.get("role") if "role" in args else Ellipsis region = args.get("region_folder_id") if "region_folder_id" in args else Ellipsis @@ -2232,7 +2245,7 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: else: out["hint"] = ( "Tagged. Use queryTopologyFabricNodes(role|level_major|region_folder_id) or " - "addTopologyViewNodes filters to place on region canvas." + "pass returned fabric_node_ids to addTopologyViewNodes to place on region canvas." ) return out @@ -2286,7 +2299,7 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: page["action"] = "unmatched" return page - if action in {"preview_rules", "preview"}: + if action == "preview_rules": out = _data(http_json("POST", "/v1/topology/classify/preview")) if isinstance(out, dict) and out.get("ok") is not False: out = dict(out) @@ -2294,7 +2307,7 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: out["hint"] = "Rule engine dry-run. Re-call action=apply_rules to write." return out - if action in {"apply_rules", "apply"}: + if action == "apply_rules": skip_manual = True if "skip_manual" in args: skip_manual = _truthy(args.get("skip_manual")) @@ -2310,7 +2323,7 @@ def _classify_topology_fabric_nodes(args: dict[str, Any]) -> dict[str, Any]: out["action"] = "apply_rules" return out - if action in {"list_rules", "rules"}: + if action == "list_rules": out = _data(http_json("GET", "/v1/topology/classify/rules")) if isinstance(out, dict) and out.get("ok") is not False: out = dict(out) @@ -2362,7 +2375,9 @@ def _query_topology_edges(args: dict[str, Any]) -> dict[str, Any]: page_size = min(500, max(1, int(args.get("page_size") or 100))) node_id = str(args.get("node_id") or "").strip() detail = str(args.get("detail") or "adjacency").strip().lower() - want_ports = detail in {"ports", "raw", "full"} + if detail not in {"adjacency", "ports"}: + return {"ok": False, "error": "detail_invalid", "hint": "detail=adjacency|ports"} + want_ports = detail == "ports" params: dict[str, Any] = { "page": page, "page_size": page_size, @@ -2374,16 +2389,13 @@ def _query_topology_edges(args: dict[str, Any]) -> dict[str, Any]: params["status"] = str(args.get("status")).strip() if str(args.get("source") or "").strip(): src = str(args.get("source")).strip().lower() - if src == "stale": - src = "lldp" params["source"] = src if str(args.get("keyword") or "").strip(): params["keyword"] = str(args.get("keyword")).strip() - out = http_json("GET", "/v1/topology/fabric/edges", params=params) - if not isinstance(out, dict) or not out.get("ok"): - return out if isinstance(out, dict) else {"ok": False, "error": "invalid_response"} - data = out.get("data") if isinstance(out.get("data"), dict) else {} + data = _data(http_json("GET", "/v1/topology/fabric/edges", params=params)) + if not data.get("ok"): + return data items = [e for e in (data.get("items") or []) if isinstance(e, dict)] edge_total = int(data.get("total") or len(items)) links = _collapse_edges_to_links(items, include_names=True) @@ -2528,12 +2540,8 @@ def _sink_topology_dual_units(args: dict[str, Any]) -> dict[str, Any]: units_as_batch_rows, ) - source_view_id = str( - args.get("source_view_id") or args.get("root_view_id") or "" - ).strip() - sink_view_id = str( - args.get("sink_view_id") or args.get("target_view_id") or "" - ).strip() + source_view_id = str(args.get("source_view_id") or "").strip() + sink_view_id = str(args.get("sink_view_id") or "").strip() if not source_view_id or not sink_view_id: return { "ok": False, @@ -2883,6 +2891,17 @@ def _sink_topology_dual_units(args: dict[str, Any]) -> dict[str, Any]: patch_out: dict[str, Any] = {"ok": True, "updated": 0} if pos_patch: patch_out = _patch_positions_chunked(sink_view_id, pos_patch) + if not patch_out.get("ok"): + return { + "ok": False, + "error": patch_out.get("error") or "positions_patch_failed", + "detail": patch_out, + "source_preserved": True, + "added": int(add_out.get("added", 0)), + "batches": batches, + "batch": batch_row, + "hint": "Sink membership may already be added; source nodes were kept. Inspect both views before retrying.", + } rm_out = _remove_topology_view_nodes( {"view_id": source_view_id, "fabric_node_ids": move_ids} @@ -2900,6 +2919,16 @@ def _sink_topology_dual_units(args: dict[str, Any]) -> dict[str, Any]: # Refresh counts src2 = _data(http_json("GET", f"/v1/topology/views/{source_view_id}", timeout=120.0)) snk2 = _data(http_json("GET", f"/v1/topology/views/{sink_view_id}", timeout=120.0)) + if not src2.get("ok") or not snk2.get("ok"): + return { + "ok": False, + "error": "view_readback_failed", + "write_completed": True, + "detail": {"source": src2, "sink": snk2}, + "batches": batches, + "batch": batch_row, + "hint": "Writes completed but verification failed. Read both views before retrying this batch.", + } src_ids2, _ = _view_node_id_pos(src2 if src2.get("ok") else {}) snk_ids2, _ = _view_node_id_pos(snk2 if snk2.get("ok") else {}) source_remaining = len(src_ids2) if src2.get("ok") else max(0, source_remaining - len(move_ids)) @@ -2908,8 +2937,8 @@ def _sink_topology_dual_units(args: dict[str, Any]) -> dict[str, Any]: batch_row.update( { "added": int(add_out.get("added") or len(add_ids)), - "positions_updated": int(patch_out.get("updated") or len(pos_patch)), - "removed_from_source": int(rm_out.get("removed") or len(move_ids)), + "positions_updated": int(patch_out.get("updated", 0)), + "removed_from_source": int(rm_out.get("removed", 0)), "source_after": source_remaining, "sink_after": sink_count, "max_nodes_bump": bump, @@ -2963,19 +2992,8 @@ def _move_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: Bidirectional: swap view_id / source_view_id to send nodes back. Caller supplies fabric_node_ids — no auto CN/dual-unit selection. """ - dest_view_id = str( - args.get("view_id") - or args.get("sink_view_id") - or args.get("target_view_id") - or args.get("to_view_id") - or "" - ).strip() - source_view_id = str( - args.get("source_view_id") - or args.get("from_view_id") - or args.get("root_view_id") - or "" - ).strip() + dest_view_id = str(args.get("view_id") or "").strip() + source_view_id = str(args.get("source_view_id") or "").strip() if not dest_view_id or not source_view_id: return { "ok": False, @@ -3010,6 +3028,8 @@ def _move_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: return {"ok": False, "error": "fabric_node_ids_empty"} mode = str(args.get("mode") or "preview").strip().lower() or "preview" + if mode not in {"preview", "apply"}: + return {"ok": False, "error": "mode_invalid", "hint": "mode=preview|apply"} dry_run = bool(overrides.get("dry_run") if "dry_run" in overrides else args.get("dry_run")) if mode != "apply": dry_run = True @@ -3109,7 +3129,7 @@ def _move_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: bump = _ensure_view_max_nodes(dest_view_id, min(2000, max(need_cap + 20, 200))) if isinstance(bump, dict) and bump.get("ok") is False: - return {**bump, **base} + return {**base, **bump} add_out: dict[str, Any] = {"ok": True, "added": 0, "skipped_existing": len(already)} if add_ids: @@ -3158,6 +3178,16 @@ def _move_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: patch_out: dict[str, Any] = {"ok": True, "updated": 0} if pos_patch: patch_out = _patch_positions_chunked(dest_view_id, pos_patch) + if not patch_out.get("ok"): + return { + **base, + "ok": False, + "error": patch_out.get("error") or "positions_patch_failed", + "detail": patch_out, + "added": int(add_out.get("added", 0)), + "source_preserved": True, + "hint": "Destination membership may already be added; source nodes were kept. Inspect both views before retrying.", + } rm_out: dict[str, Any] = {"ok": True, "removed": 0} if remove_from_source: @@ -3176,14 +3206,23 @@ def _move_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: src2 = _data(http_json("GET", f"/v1/topology/views/{source_view_id}", timeout=120.0)) dst2 = _data(http_json("GET", f"/v1/topology/views/{dest_view_id}", timeout=120.0)) + if not src2.get("ok") or not dst2.get("ok"): + return { + **base, + "ok": False, + "error": "view_readback_failed", + "write_completed": True, + "detail": {"source": src2, "destination": dst2}, + "hint": "Writes completed but verification failed. Read both views before retrying the move.", + } src_after, _ = _view_node_id_pos(src2 if src2.get("ok") else {}) dst_after, _ = _view_node_id_pos(dst2 if dst2.get("ok") else {}) return { **base, "dry_run": False, "added": int(add_out.get("added") or len(add_ids)), - "positions_updated": int(patch_out.get("updated") or len(pos_patch)), - "removed_from_source": int(rm_out.get("removed") or (len(move_ids) if remove_from_source else 0)), + "positions_updated": int(patch_out.get("updated", 0)), + "removed_from_source": int(rm_out.get("removed", 0)), "source_after": len(src_after) if src2.get("ok") else None, "dest_after": len(dst_after) if dst2.get("ok") else None, "max_nodes_bump": bump, @@ -3201,12 +3240,8 @@ def _copy_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: Source membership is unchanged. For test sandboxes: clone a known-good canvas onto a fresh child region without re-adding ids by hand. """ - source_view_id = str( - args.get("source_view_id") or args.get("from_view_id") or "" - ).strip() - target_view_id = str( - args.get("target_view_id") or args.get("to_view_id") or "" - ).strip() + source_view_id = str(args.get("source_view_id") or "").strip() + target_view_id = str(args.get("target_view_id") or "").strip() if not source_view_id or not target_view_id: return { "ok": False, @@ -3322,8 +3357,29 @@ def _copy_topology_view_nodes(args: dict[str, Any]) -> dict[str, Any]: ] if positions: patch_out = _patch_positions_chunked(target_view_id, positions) + if not patch_out.get("ok"): + return { + "ok": False, + "error": patch_out.get("error") or "positions_patch_failed", + "detail": patch_out, + "source_view_id": source_view_id, + "target_view_id": target_view_id, + "cleared": cleared, + "added": int(add_out.get("added", 0)), + "hint": "Target membership may already be changed; source is unchanged. Inspect target before retrying.", + } dst2 = _data(http_json("GET", f"/v1/topology/views/{target_view_id}", timeout=120.0)) + if not dst2.get("ok"): + return { + "ok": False, + "error": "view_readback_failed", + "write_completed": True, + "source_view_id": source_view_id, + "target_view_id": target_view_id, + "detail": dst2, + "hint": "Writes completed but verification failed. Read target before retrying the copy.", + } dst_after, _ = _view_node_id_pos(dst2 if dst2.get("ok") else {}) return { "ok": True, @@ -3779,6 +3835,11 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "skipped peers are counted in out_of_region_skipped." ), }, + "dry_run": { + "type": "boolean", + "default": False, + "description": "Preview projected graph without saving placements.", + }, "detail": { "type": "string", "enum": ["summary", "full"], @@ -3797,8 +3858,7 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "Default: q→search, else list. " "summary=node/edge counts; list=paged filters " "(keyword/role/link_status/region_folder_id); " - "search=quick name/IP/id (needs q). Replaces getTopologyFabricSummary / " - "listTopologyFabricNodes / searchTopologyFabricNodes." + "search=quick name/IP/id (needs q)." ), "inputSchema": { "type": "object", @@ -3853,7 +3913,8 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "level: 0=external, 1=core, 2=agg, 3=access; use 1.1 / 2.1 for sub-tiers. " "action=match → tag(level|role preset, dry_run) → tag. " "role preset still accepted (maps to *.0). " - "After tagging, addTopologyViewNodes by role/region. No slice maps. " + "After tagging, queryTopologyFabricNodes by region/level, then place returned " + "fabric_node_ids with addTopologyViewNodes (canvas filters do not support region/level). " "Requires ne:write." ), "inputSchema": { @@ -4030,8 +4091,8 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ }, "use_source_dual_portals": { "type": "boolean", - "default": True, - "description": "Exclude portals from source dual_units (default true).", + "default": False, + "description": "Exclude portals from source dual_units (default false).", }, "use_sink_dual_portals": { "type": "boolean", @@ -4156,7 +4217,7 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "action=layout: recipe=rings|corridor|compact|unstick (small graphs). " "action=layout_dual_unit: CN/core dual-portal eye (max membership; " "residual crossings OK unless params.require_zero_cross=true). " - "action=move_nodes (alias sink_nodes): move fabric_node_ids from " + "action=move_nodes: move fabric_node_ids from " "source_view_id→view_id; park=true for orbit attach; swap views to reverse. " "Prefer sinkTopologyDualUnits for dual_units batches. " "action=orbit_sweep: crossing orbit; preview+node_id / apply+pick; " @@ -4170,9 +4231,12 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "action=pull_far_chains: scale far corridors/isolates toward portal mid-point. " "action=align_reference: SAME-network A/B debug only " "(shared fabric_node_ids); not a cross-network template. " - "action=fix_overlaps|resolve_overlaps: pull apart overlaps (not on eye sinks). " + "action=fix_overlaps: pull apart overlaps (not on eye sinks). " "action=untangle / straighten_channels: surgical polish (not on eye sinks). " "action=job_status|job_cancel: poll/cancel background jobs. " + "params.background=true starts an apply action in a durable subprocess; " + "poll params.job_id until done|error|cancelled, then inspect result. " + "Successful apply persists positions; getTopologyView reads them back. " "preset: loose|balanced|dense. mode: preview|apply. " "Workflow: analyze(structure) → sinkTopologyDualUnits(max_units=1) → " "suggestSinkHubs/move_nodes(park) → until_limit(crossing→total) → " @@ -4192,7 +4256,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "enum": [ "layout", "fix_overlaps", - "resolve_overlaps", "layout_dual_unit", "straighten_channels", "untangle", @@ -4204,7 +4267,6 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "orbit_sweep", "level_bands", "move_nodes", - "sink_nodes", "job_status", "job_cancel", ], @@ -4212,13 +4274,13 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "description": ( "layout=full recipe; layout_dual_unit=dual-portal eye; " "level_bands=horizontal layer bands; " - "move_nodes|sink_nodes=migrate fabric_node_ids; " + "move_nodes=migrate fabric_node_ids; " "orbit_sweep=polar sweep; polish_crossings=one-shot cut crossings; " "clear_edge_hits=eject edge hits (no crossing rise); " "pull_far_chains=scale far corridors toward portal mid; " "compact_bbox=gated farthest-K shrink; " "align_reference=same-network A/B debug only; " - "fix_overlaps|resolve_overlaps; " + "fix_overlaps; " "untangle/straighten_channels; job_status|job_cancel." ), }, @@ -4281,10 +4343,15 @@ HTTP_MCP_TOOLS: list[dict[str, Any]] = [ "max_jump/angle_step/nn_floor; until_limit defaults " "protect_rigid=portals + objective=crossing; " "single-node/round default protect_rigid=off. " - "move_nodes|sink_nodes: fabric_node_ids[] (required), " + "move_nodes: fabric_node_ids[] (required), " "copy_positions (default true), park, remove_from_source " "(default true), pad/offset_x/offset_y." ), + "properties": { + "job_id": {"type": "string", "description": "Required for job_status/job_cancel; returned by background start."}, + "background": {"type": "boolean", "description": "Run apply computation in a durable subprocess and return job_id immediately."}, + "sync": {"type": "boolean", "description": "Force inline computation, disabling automatic background execution."}, + }, "additionalProperties": True, }, "catalog": { diff --git a/packages/netx-topology-mcp/src/netx_topology_mcp/layout_jobs.py b/packages/netx-topology-mcp/src/netx_topology_mcp/layout_jobs.py index 39ae76d..265adea 100644 --- a/packages/netx-topology-mcp/src/netx_topology_mcp/layout_jobs.py +++ b/packages/netx-topology-mcp/src/netx_topology_mcp/layout_jobs.py @@ -89,7 +89,12 @@ def _prune_disk() -> None: files = [p for p in files if not p.name.endswith(".args.json")] if len(files) <= _MAX_JOBS: return - for path in files[: max(0, len(files) - _MAX_JOBS)]: + # Active jobs must remain pollable and cancellable even when history fills up. + removable = [ + path for path in files + if (_read_job_disk(path.stem) or {}).get("status") in {"done", "error", "cancelled"} + ] + for path in removable[: max(0, len(files) - _MAX_JOBS)]: try: path.unlink(missing_ok=True) path.with_name(path.stem + ".args.json").unlink(missing_ok=True) @@ -400,25 +405,33 @@ def start_job( subprocess, "DETACHED_PROCESS", 0x00000008 ) - proc = subprocess.Popen( # noqa: S603 — controlled argv - [sys.executable, "-m", "netx_topology_mcp.layout_job_worker", "--job-id", job_id], - env=env, - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - creationflags=creationflags, - close_fds=True, - ) + try: + proc = subprocess.Popen( # noqa: S603 — controlled argv + [sys.executable, "-m", "netx_topology_mcp.layout_job_worker", "--job-id", job_id], + env=env, + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + creationflags=creationflags, + close_fds=True, + ) + except OSError as exc: + finish_job(job_id, error=f"worker_start_failed:{exc}") + _args_path(job_id).unlink(missing_ok=True) + # The caller receives a pollable terminal job instead of an orphaned running record. + return job_id with _LOCK: job = _read_job_disk(job_id) or job job["pid"] = proc.pid - job["progress"] = { - **(job.get("progress") or {}), - "phase": "running", - "message": f"worker pid={proc.pid}", - "updated_at": time.time(), - } - job["heartbeat_at"] = time.time() + # A short computation can finish before Popen returns to the parent. + # Keep its terminal progress/result intact. + if job.get("status") in {"running", "cancelling"}: + job["progress"] = { + **(job.get("progress") or {}), + "message": f"worker pid={proc.pid}", + "updated_at": time.time(), + } + job["heartbeat_at"] = time.time() _JOBS[job_id] = job _write_job_disk(job) return job_id diff --git a/packages/netx-topology-mcp/src/netx_topology_mcp/layout_tool.py b/packages/netx-topology-mcp/src/netx_topology_mcp/layout_tool.py index fc30960..e589a1b 100644 --- a/packages/netx-topology-mcp/src/netx_topology_mcp/layout_tool.py +++ b/packages/netx-topology-mcp/src/netx_topology_mcp/layout_tool.py @@ -144,7 +144,6 @@ _PARAM_KEYS = ( ACTIONS = ( "layout", "fix_overlaps", - "resolve_overlaps", # alias of fix_overlaps "untangle", "straighten_channels", "layout_dual_unit", @@ -328,8 +327,6 @@ def run_layout_on_graph( f"unknown_action:{action_key}; " f"allowed={','.join(ACTIONS)}" ) - if action_key == "resolve_overlaps": - action_key = "fix_overlaps" base_params = build_params(preset=preset, overrides=params) st0 = build_state_from_nodes_edges(nodes, edges) park_phantom_nodes(st0) @@ -899,14 +896,13 @@ def list_layout_catalog() -> dict[str, Any]: "actions": { "layout": "全图配方（骨架）；块内 pack，结束时局部解叠", "fix_overlaps": "只拉开当前重叠点（+1 跳邻居），不重排全图", - "resolve_overlaps": "fix_overlaps 别名", "straighten_channels": ( "拉直 deg≤2 通道（弦/横/纵）；仅全局交叉下降才接受该通道" "（params: step/min_len）" ), "layout_dual_unit": ( "双门户单元美化（多走廊→平行 H/V 道；链→拉直）；" - "单元内交叉必须为 0 才接受（params: unit_id）" + "默认允许残留交叉；params.require_zero_cross=true 时要求为 0（params: unit_id）" ), "untangle": ( "贪心挪低度数点降交叉；默认 protect_rigid=portals " @@ -963,7 +959,6 @@ def list_layout_catalog() -> dict[str, Any]: "默认 remove_from_source；对调两 view 即回迁；" "copy_positions|park；mode=preview|apply" ), - "sink_nodes": "move_nodes 别名", }, "recipes": { "rings": "环+链花瓣；core_bar 时梁优先（多 CN 不塞进 AN 列）", diff --git a/packages/netx-topology-mcp/tests/conftest.py b/packages/netx-topology-mcp/tests/conftest.py new file mode 100644 index 0000000..dbcc194 --- /dev/null +++ b/packages/netx-topology-mcp/tests/conftest.py @@ -0,0 +1,8 @@ +"""Keep layout-job persistence inside pytest temporary directories.""" + +import pytest + + +@pytest.fixture(autouse=True) +def isolated_layout_job_dir(tmp_path, monkeypatch): + monkeypatch.setenv("NETX_LAYOUT_JOB_DIR", str(tmp_path / "layout-jobs")) diff --git a/packages/netx-topology-mcp/tests/test_layout_jobs.py b/packages/netx-topology-mcp/tests/test_layout_jobs.py index 0c5ca70..d2be16f 100644 --- a/packages/netx-topology-mcp/tests/test_layout_jobs.py +++ b/packages/netx-topology-mcp/tests/test_layout_jobs.py @@ -16,6 +16,60 @@ from netx_topology_mcp.layout_jobs import ( ) +def test_terminal_job_survives_cache_reset_and_reports_failure(tmp_path) -> None: + from netx_topology_mcp import layout_jobs as lj + + lj._write_job_disk({"job_id": "persisted", "status": "running", "started_at": time.time()}) + lj.finish_job("persisted", result={"ok": False, "error": "backend_write_failed", "applied": False}) + lj._JOBS.clear() + out = _layout_topology_view({"action": "job_status", "params": {"job_id": "persisted"}}) + assert out["ok"] is False + assert out["status"] == "error" + assert out["error"] == "backend_write_failed" + assert out["result"]["applied"] is False + + +def test_spawn_failure_is_persisted_as_terminal_error() -> None: + from netx_topology_mcp import layout_jobs as lj + + with patch.object(lj.subprocess, "Popen", side_effect=OSError("spawn denied")): + jid = start_job(action="layout", view_id="v", tool_args={"view_id": "v"}) + lj._JOBS.clear() + out = _layout_topology_view({"action": "job_status", "params": {"job_id": jid}}) + assert out["ok"] is False + assert out["status"] == "error" + assert out["error"].startswith("worker_start_failed:") + assert not lj._args_path(jid).exists() + + +def test_fast_worker_terminal_progress_is_not_overwritten() -> None: + from types import SimpleNamespace + from netx_topology_mcp import layout_jobs as lj + + def spawn(argv, **kwargs): + jid = argv[-1] + lj.finish_job(jid, result={"ok": True, "applied": True}) + return SimpleNamespace(pid=123) + + with patch.object(lj.subprocess, "Popen", side_effect=spawn): + jid = start_job(action="layout", view_id="v", tool_args={"view_id": "v"}) + out = lj.job_public(jid) + assert out["status"] == "done" + assert out["progress"]["phase"] == "done" + assert out["progress"]["pct"] == 100.0 + + +def test_pruning_preserves_active_jobs() -> None: + from netx_topology_mcp import layout_jobs as lj + + lj._write_job_disk({"job_id": "active", "status": "running"}) + lj._write_job_disk({"job_id": "completed", "status": "done"}) + with patch.object(lj, "_MAX_JOBS", 1): + lj._prune_disk() + assert lj._read_job_disk("active") is not None + assert lj._read_job_disk("completed") is None + + def test_start_job_and_poll() -> None: def runner() -> dict: report_progress("mid", pct=50.0, message="halfway", step=1, total_steps=2) diff --git a/packages/netx-topology-mcp/tests/test_move_nodes.py b/packages/netx-topology-mcp/tests/test_move_nodes.py index 35e7382..00ded2d 100644 --- a/packages/netx-topology-mcp/tests/test_move_nodes.py +++ b/packages/netx-topology-mcp/tests/test_move_nodes.py @@ -10,10 +10,10 @@ def test_move_nodes_in_layout_schema_and_catalog() -> None: tool = next(t for t in HTTP_MCP_TOOLS if t["name"] == "layoutTopologyView") enum = tool["inputSchema"]["properties"]["action"]["enum"] assert "move_nodes" in enum - assert "sink_nodes" in enum + assert "sink_nodes" not in enum cat = list_layout_catalog() assert "move_nodes" in cat["actions"] - assert "sink_nodes" in cat["actions"] + assert "sink_nodes" not in cat["actions"] def test_move_nodes_requires_ids_and_views() -> None: diff --git a/packages/netx-topology-mcp/tests/test_topology_contract.py b/packages/netx-topology-mcp/tests/test_topology_contract.py new file mode 100644 index 0000000..0de9d6c --- /dev/null +++ b/packages/netx-topology-mcp/tests/test_topology_contract.py @@ -0,0 +1,263 @@ +"""HTTP contract and partial-failure regression tests; no backend required.""" + +from __future__ import annotations + +import json +from unittest.mock import patch + +import pytest + +from netx_topology_mcp import http_tools as tools + + +def payload(name, args): + result = tools.call_http_tool(name, args) + return result, json.loads(result["content"][0]["text"]) + + +@pytest.mark.parametrize("name,args", [ + ("getTopologyTree", {}), + ("getTopologyView", {"view_id": "v"}), + ("createTopologyFolder", {"name": "测试"}), + ("addTopologyViewNodes", {"view_id": "v", "fabric_node_ids": ["n"]}), + ("removeTopologyViewNodes", {"view_id": "v", "fabric_node_ids": ["n"]}), + ("copyTopologyViewNodes", {"source_view_id": "s", "target_view_id": "t"}), + ("updateTopologyViewPositions", {"view_id": "v", "layout": "grid"}), + ("projectTopologyNeighbors", {"view_id": "v"}), + ("queryTopologyFabricNodes", {"mode": "list"}), + ("classifyTopologyFabricNodes", {"action": "apply_rules"}), + ("queryTopologyNeighborhood", {"node_id": "n"}), + ("queryTopologyEdges", {}), + ("layoutTopologyView", {"view_id": "v"}), + ("suggestSinkHubs", {"source_view_id": "s"}), + ("analyzeTopologyViewLayout", {"view_id": "v"}), + ("sinkTopologyDualUnits", {"source_view_id": "s", "sink_view_id": "t"}), +]) +def test_business_failure_is_mcp_error(name, args): + with patch.object(tools, "http_json", return_value={ + "ok": True, "data": {"ok": False, "error": "backend_rejected"}, + }): + result, data = payload(name, args) + assert result.get("isError") is True, (name, data) + assert data["ok"] is False + assert data["error"] == "backend_rejected" + + +@pytest.mark.parametrize("name,args,method", [ + ("addTopologyViewNodes", {"view_id": "v", "fabric_node_ids": [f"n{i}" for i in range(501)]}, "POST"), + ("removeTopologyViewNodes", {"view_id": "v", "fabric_node_ids": [f"n{i}" for i in range(501)]}, "POST"), + ("updateTopologyViewPositions", {"view_id": "v", "positions": [ + {"fabric_node_id": f"n{i}", "x": i, "y": 0} for i in range(501) + ]}, "PATCH"), +]) +def test_chunk_failure_stops_later_mutations(name, args, method): + with patch.object(tools, "http_json", side_effect=[ + {"ok": True, "data": {"ok": True, "added": 200, "updated": 200, "removed": 200}}, + {"ok": True, "data": {"ok": False, "error": "write_failed"}}, + ]) as call: + result, data = payload(name, args) + assert result.get("isError") is True + assert call.call_count == 2 + assert call.call_args.args[0] == method + assert data["error"] == "write_failed" + + +def graph(node="n"): + return {"ok": True, "nodes": [{"fabric_node_id": node, "x": 1, "y": 2}], "edges": []} + + +@pytest.mark.parametrize("name,args", [ + ("copyTopologyViewNodes", {"source_view_id": "s", "target_view_id": "t"}), + ("layoutTopologyView", {"view_id": "t", "source_view_id": "s", "action": "move_nodes", "mode": "apply", "params": {"fabric_node_ids": ["n"]}}), + ("sinkTopologyDualUnits", {"source_view_id": "s", "sink_view_id": "t", "layout_batch": False}), +]) +def test_position_failure_preserves_source(name, args): + def http(method, path, **kwargs): + if method == "GET": + return {"ok": True, "data": graph() if path.endswith("/s") else {"nodes": [], "edges": []}} + raise AssertionError(f"Unexpected mutation {method} {path}") + + with patch.object(tools, "http_json", side_effect=http), patch.object( + tools, "_ensure_view_max_nodes", return_value={"ok": True} + ), patch.object(tools, "_add_topology_view_nodes", return_value={"ok": True, "added": 1}), patch.object( + tools, "_patch_positions_chunked", return_value={"ok": False, "error": "patch_denied", "updated": 0} + ), patch.object(tools, "_fabric_bridges_into", return_value=[]), patch.object( + tools, "_remove_topology_view_nodes" + ) as remove: + result, data = payload(name, args) + assert result.get("isError") is True, data + assert data["error"] == "patch_denied" + assert data["added"] == 1 + remove.assert_not_called() + + +def test_move_capacity_error_stays_error(): + with patch.object(tools, "http_json", return_value={"ok": True, "data": graph()}), patch.object( + tools, "_ensure_view_max_nodes", return_value={"ok": False, "error": "cap_denied"} + ), patch.object(tools, "_add_topology_view_nodes") as add: + result, data = payload("layoutTopologyView", { + "view_id": "t", "source_view_id": "s", "action": "move_nodes", "mode": "apply", + "params": {"fabric_node_ids": ["n"]}, + }) + assert result.get("isError") is True + assert data["error"] == "cap_denied" + add.assert_not_called() + + +def test_invalid_add_does_not_change_capacity(): + with patch.object(tools, "http_json") as http: + result, _ = payload("addTopologyViewNodes", {"view_id": "v", "max_nodes": 2000}) + assert result.get("isError") is True + http.assert_not_called() + + +@pytest.mark.parametrize("name,args", [ + ("getTopologyView", {"view_id": "v", "sample": 0}), + ("projectTopologyNeighbors", {"view_id": "v", "sample": 0, "dry_run": True}), +]) +def test_zero_sample_and_projection_preview(name, args): + with patch.object(tools, "http_json", return_value={"ok": True, "data": graph()}) as http: + result, data = payload(name, args) + assert result.get("isError") is not True + assert data["sample_nodes"] == [] + if name == "projectTopologyNeighbors": + assert http.call_args.kwargs["body"]["dry_run"] is True + assert data["dry_run"] is True + + +def test_invalid_projection_detail_does_not_write(): + with patch.object(tools, "http_json") as http: + result, data = payload("projectTopologyNeighbors", {"view_id": "v", "detail": "raw"}) + assert result.get("isError") is True + assert data["error"] == "detail_invalid" + http.assert_not_called() + + +def test_zero_updated_is_not_reported_as_successful_count(): + with patch.object(tools, "http_json", return_value={"ok": True, "data": {"ok": True, "updated": 0, "skipped_locked": 1}}): + assert tools._patch_positions_chunked("v", [{"fabric_node_id": "n", "x": 1, "y": 2}])["updated"] == 0 + + +def test_apply_rules_uses_query_flags(): + with patch.object(tools, "http_json", return_value={"ok": True, "data": {"updated": 1}}) as http: + payload("classifyTopologyFabricNodes", {"action": "apply_rules", "skip_manual": False, "fill_empty_only": True}) + assert http.call_args.args == ("POST", "/v1/topology/classify/apply") + assert http.call_args.kwargs["params"] == {"skip_manual": False, "fill_empty_only": True} + + +@pytest.mark.parametrize("action", ["resolve_overlaps", "sink_nodes", "align_to_reference", "compose_views", "compose_orbit"]) +def test_removed_layout_actions_are_not_advertised_or_called(action): + with patch.object(tools, "http_json") as http: + result, data = payload("layoutTopologyView", {"view_id": "v", "action": action}) + assert result.get("isError") is True + assert data["error"] == f"unknown_action:{action}" + assert action not in data["actions"] + schema = next(t for t in tools.HTTP_MCP_TOOLS if t["name"] == "layoutTopologyView")["inputSchema"] + assert action not in schema["properties"]["action"]["enum"] + http.assert_not_called() + + +@pytest.mark.parametrize("name,args", [ + ("queryTopologyFabricNodes", {"mode": "find"}), + ("classifyTopologyFabricNodes", {"action": "apply"}), + ("queryTopologyEdges", {"detail": "raw"}), +]) +def test_removed_query_aliases_do_not_call_backend(name, args): + with patch.object(tools, "http_json") as http: + result, _ = payload(name, args) + assert result.get("isError") is True + http.assert_not_called() + + +def test_explicit_background_layout_starts_worker(): + canvas = graph() + canvas["nodes"].append({"fabric_node_id": "n2", "x": 100, "y": 200}) + with patch.object(tools, "http_json", return_value={"ok": True, "data": canvas}), patch.object( + tools, "start_job", return_value="test-job" + ) as start, patch.object(tools, "job_public", return_value=None), patch.object(tools, "run_layout_on_graph") as compute: + result, data = payload("layoutTopologyView", {"view_id": "v", "action": "layout", "mode": "apply", "params": {"background": True}}) + assert result.get("isError") is not True + assert data["job_id"] == "test-job" + assert data["applied"] is False + assert start.call_args.kwargs["tool_args"]["params"]["_force_sync"] is True + compute.assert_not_called() + + +def test_background_spawn_error_is_immediately_reported(): + from netx_topology_mcp import layout_jobs as jobs + canvas = graph() + canvas["nodes"].append({"fabric_node_id": "n2", "x": 100, "y": 200}) + with patch.object(tools, "http_json", return_value={"ok": True, "data": canvas}), patch.object( + jobs.subprocess, "Popen", side_effect=OSError("worker unavailable") + ): + result, data = payload("layoutTopologyView", {"view_id": "v", "action": "layout", "mode": "apply", "params": {"background": True}}) + assert result.get("isError") is True + assert data["status"] == "error" + assert data["error"].startswith("worker_start_failed:") + assert data["job_id"] + + +def test_background_apply_poll_and_persistent_view_readback(): + """Execute worker orchestration inline against an in-memory HTTP backend.""" + from types import SimpleNamespace + from netx_topology_mcp import layout_jobs as jobs + from netx_topology_mcp.layout_job_worker import main as worker_main + + canvas = {"nodes": [ + {"fabric_node_id": "n", "x": 0, "y": 0}, + {"fabric_node_id": "n2", "x": 100, "y": 200}, + ], "edges": []} + positions = [ + {"fabric_node_id": "n", "x": 20, "y": 30}, + {"fabric_node_id": "n2", "x": 220, "y": 330}, + ] + + def http(method, path, **kwargs): + if method == "GET": + return {"ok": True, "data": canvas} + assert method == "PATCH" and path == "/v1/topology/views/v/positions" + canvas["nodes"] = kwargs["body"]["positions"] + return {"ok": True, "data": {"ok": True, "updated": 2}} + + def spawn(argv, **kwargs): + assert worker_main(["--job-id", argv[-1]]) == 0 + return SimpleNamespace(pid=123) + + with patch.object(tools, "http_json", side_effect=http), patch.object( + jobs.subprocess, "Popen", side_effect=spawn + ), patch.object(tools, "run_layout_on_graph", return_value={"ok": True, "positions": positions, "overlap": {}}): + _, started = payload("layoutTopologyView", { + "view_id": "v", "action": "layout", "mode": "apply", "params": {"background": True}, + }) + jobs._JOBS.clear() + _, polled = payload("layoutTopologyView", {"action": "job_status", "params": {"job_id": started["job_id"]}}) + _, view = payload("getTopologyView", {"view_id": "v", "detail": "full"}) + assert polled["status"] == "done" + assert polled["result"]["applied"] is True + assert polled["result"]["updated"] == 2 + assert view["nodes"] == positions + + +def test_hub_schema_matches_default_exclusion_behavior(): + tool = next(t for t in tools.HTTP_MCP_TOOLS if t["name"] == "suggestSinkHubs") + assert tool["inputSchema"]["properties"]["use_source_dual_portals"]["default"] is False + + +@pytest.mark.parametrize("name,args", [ + ("copyTopologyViewNodes", {"source_view_id": "s", "target_view_id": "t"}), + ("layoutTopologyView", {"view_id": "t", "source_view_id": "s", "action": "move_nodes", "mode": "apply", "params": {"fabric_node_ids": ["n"]}}), + ("sinkTopologyDualUnits", {"source_view_id": "s", "sink_view_id": "t", "layout_batch": False}), +]) +def test_failed_final_readback_reports_completed_writes(name, args): + with patch.object(tools, "http_json", side_effect=[ + {"ok": True, "data": graph()}, {"ok": True, "data": {"nodes": [], "edges": []}}, + {"ok": False, "error": "read_timeout"}, {"ok": False, "error": "read_timeout"}, + ]), patch.object(tools, "_ensure_view_max_nodes", return_value={"ok": True}), patch.object( + tools, "_add_topology_view_nodes", return_value={"ok": True, "added": 1} + ), patch.object(tools, "_patch_positions_chunked", return_value={"ok": True, "updated": 1}), patch.object( + tools, "_remove_topology_view_nodes", return_value={"ok": True, "removed": 1} + ), patch.object(tools, "_fabric_bridges_into", return_value=[]): + result, data = payload(name, args) + assert result.get("isError") is True + assert data["error"] == "view_readback_failed" + assert data["write_completed"] is True diff --git a/pyproject.toml b/pyproject.toml index 405aeab..d763bee 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -34,10 +34,7 @@ dependencies = [ [project.optional-dependencies] mcp = ["netx-mcp @ file:packages/netx-mcp"] mcp-topology = ["netx-topology-mcp @ file:packages/netx-topology-mcp"] - -[project.scripts] -netx-mcp = "netx_mcp.server:main" -netx-topology-mcp = "netx_topology_mcp.server:main" +mcp-bizmonitor = ["netx-bizmonitor-mcp @ file:packages/netx-bizmonitor-mcp"] [tool.setuptools.packages.find] where = ["."] diff --git a/scripts/mcp_acceptance.py b/scripts/mcp_acceptance.py new file mode 100644 index 0000000..4748bab --- /dev/null +++ b/scripts/mcp_acceptance.py @@ -0,0 +1,166 @@ +"""通过真实 stdio MCP 留存本地验收证据；不安装插件、不导入 handler。 + +使用已有本机 MCP 配置中的凭据，只在子进程内存中传递，证据自动去敏。 +示例：python scripts/mcp_acceptance.py ops runNmsDiagnostics '{}' +""" +from __future__ import annotations + +import argparse +import datetime as dt +import json +import os +from pathlib import Path +import queue +import re +import subprocess +import sys +import threading +import time + +ROOT = Path(__file__).resolve().parents[1] +GROUPS = { + "ops": ("netx", "netx_mcp", "netx-mcp"), + "topology": ("netx-topology", "netx_topology_mcp", "netx-topology-mcp"), + "biz": ("netx-bizmonitor", "netx_bizmonitor_mcp", "netx-bizmonitor-mcp"), +} +SENSITIVE = re.compile(r"password|passwd|secret|token|authorization|cookie|private_key", re.I) + + +def redact(value): + if isinstance(value, dict): + return {k: "[已去敏]" if SENSITIVE.search(k) else redact(v) for k, v in value.items()} + if isinstance(value, list): + return [redact(x) for x in value] + if isinstance(value, str): + # MCP 的 JSON 位于 content.text 内；递归处理，避免只净化外层。 + try: + decoded = json.loads(value) + except (ValueError, TypeError): + return re.sub(r"\bnxt_[\w-]+", "[已去敏]", value) + if isinstance(decoded, (dict, list)): + return json.dumps(redact(decoded), ensure_ascii=False) + return value + + +def payload(response): + result = response.get("result", {}) + content = result.get("content", []) + if content and content[0].get("type") == "text": + try: + return json.loads(content[0]["text"]) + except ValueError: + pass + return result or response.get("error", {}) + + +class McpSession: + def __init__(self, group, config, evidence, source_root=ROOT): + source_root = Path(source_root).resolve() + server_id, module, package = GROUPS[group] + cfg = json.loads(Path(config).read_text(encoding="utf-8-sig")) + settings = cfg["mcpServers"][server_id].get("env", {}) + env = os.environ.copy() + env.update({k: str(v) for k, v in settings.items()}) + # 只允许本机 netx；不能由配置悄悄指向生产服务。 + actual_url = (env.get("NETX_API_URL") or env.get("OCLAW_NETX_BASE_URL") + or "http://127.0.0.1:8890").rstrip("/") + if actual_url not in ( + "http://127.0.0.1:8890", "http://localhost:8890", + "http://127.0.0.1:8891", "http://localhost:8891" + ): + raise ValueError("验收仅允许本机 8890 API 或隔离样例 8891 API") + env["NETX_API_URL"] = actual_url + env.pop("OCLAW_NETX_BASE_URL", None) + env["PYTHONPATH"] = str(source_root / "packages" / package / "src") + env["PYTHONUTF8"] = "1" + self.group, self.evidence, self.seq = group, Path(evidence), 0 + self.stderr = [] + self.proc = subprocess.Popen( + [sys.executable, "-m", module], cwd=source_root, env=env, + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, + encoding="utf-8", bufsize=1, + ) + self.lines = queue.Queue() + threading.Thread(target=self._read_stdout, daemon=True).start() + threading.Thread(target=self._read_stderr, daemon=True).start() + self.rpc("initialize", {"protocolVersion": "2024-11-05", "capabilities": {}, + "clientInfo": {"name": "netx-mcp-acceptance", "version": "1"}}) + self.proc.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n") + self.proc.stdin.flush() + + def _read_stdout(self): + for line in self.proc.stdout: + self.lines.put(line) + + def _read_stderr(self): + for line in self.proc.stderr: + self.stderr.append(line) + + def rpc(self, method, params=None, timeout=180): + self.seq += 1 + request = {"jsonrpc": "2.0", "id": self.seq, "method": method, "params": params or {}} + start = time.monotonic() + self.proc.stdin.write(json.dumps(request, ensure_ascii=False) + "\n") + self.proc.stdin.flush() + deadline = start + timeout + while True: + try: + line = self.lines.get(timeout=max(0.01, deadline - time.monotonic())) + except queue.Empty as exc: + raise TimeoutError(f"{self.group} {method} 超时；stderr={''.join(self.stderr)[-500:]}") from exc + response = json.loads(line) + if response.get("id") == self.seq: + break + if "id" in response: + raise RuntimeError("服务器对通知回包或返回了错误的请求 id") + record = {"at": dt.datetime.now(dt.timezone(dt.timedelta(hours=8))).isoformat(), + "group": self.group, "request": request, "response": response, + "elapsed_ms": round((time.monotonic() - start) * 1000)} + self.evidence.parent.mkdir(parents=True, exist_ok=True) + with self.evidence.open("a", encoding="utf-8") as f: + f.write(json.dumps(redact(record), ensure_ascii=False) + "\n") + return response + + def call(self, name, args): + if name == "execManagedNe": + commands = list(args.get("commands") or []) + for target in args.get("targets") or []: + commands.extend(target.get("commands") or []) + for cmd in commands: + if not re.match(r"(?i)^(show|display)\s+", cmd) or re.search(r"[;\r\n`$&<>|]", cmd): + raise ValueError("验收 CLI 只允许单行 show/display") + return self.rpc("tools/call", {"name": name, "arguments": args}) + + def close(self): + self.proc.stdin.close() + try: + self.proc.wait(timeout=8) + except subprocess.TimeoutExpired: + self.proc.terminate() + self.proc.wait(timeout=5) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("group", choices=GROUPS) + parser.add_argument("tool", help="工具名，或 tools/list") + parser.add_argument("arguments", nargs="?", default="{}") + parser.add_argument("--config", default=str(ROOT / "mcp.all.json")) + parser.add_argument("--source-root", default=str(ROOT), help="被验收包所在 worktree 根目录") + parser.add_argument("--evidence", default=str(ROOT / "artifacts/mcp-evidence.jsonl")) + opts = parser.parse_args() + session = McpSession(opts.group, opts.config, opts.evidence, opts.source_root) + try: + if opts.tool == "tools/list": + response = session.rpc("tools/list") + names = [t["name"] for t in response.get("result", {}).get("tools", [])] + print(json.dumps({"count": len(names), "tools": names}, ensure_ascii=False)) + else: + response = session.call(opts.tool, json.loads(opts.arguments)) + print(json.dumps(redact(payload(response)), ensure_ascii=False)) + finally: + session.close() + + +if __name__ == "__main__": + main() diff --git a/scripts/seed_mcp_acceptance.py b/scripts/seed_mcp_acceptance.py new file mode 100644 index 0000000..44f0e43 --- /dev/null +++ b/scripts/seed_mcp_acceptance.py @@ -0,0 +1,264 @@ +"""离线生成验收样例/非现场故障；评价结论必须来自真实 netx 业务逻辑。 + +用法（在 worktree 根目录，使用已有 netx Python 环境）： + python scripts/seed_mcp_acceptance.py --database <绝对路径>/artifacts/acceptance.sqlite \ + --metadata <绝对路径>/artifacts/acceptance.json + +只接受本工人/组长 worktree artifacts 下带 acceptance 的新文件。 +不启动服务、不导入应用入口、不使用网络、不连接应用数据库。 +""" + +from __future__ import annotations + +import argparse +from datetime import datetime, timedelta, timezone +import json +import os +from pathlib import Path +import sys +from typing import Any + +ROOT = Path(__file__).resolve().parents[1] +ALLOWED_WORKTREES = frozenset({"netx-mcp-bizmonitor-codex", "netx-mcp-lead"}) +DECLARATION = "验收样例/非现场故障：离线合成接口采集输入，真实执行 netx 业务评价；无真实设备或凭据。" +PREFIX = "acceptance-" + + +def artifact_path(raw: str, *, metadata: bool = False) -> Path: + if ROOT.parent.name != "agent-wt" or ROOT.name not in ALLOWED_WORKTREES: + raise ValueError("脚本必须位于允许的 agent-wt worktree 内") + path = Path(raw) + if not path.is_absolute() or "://" in raw: + raise ValueError("必须指定 SQLite 文件绝对路径，不能使用数据库 URL") + path = path.resolve() + allowed = [] + for name in ALLOWED_WORKTREES: + worktree = ROOT.parent / name + base = worktree / "artifacts" + # Reparse points/symlinks must not turn the allowlist into an external directory. + if worktree.resolve() != worktree or base.resolve() != base: + continue + allowed.append(base) + if not any(path.is_relative_to(base) for base in allowed): + raise ValueError("仅允许本工人/组长 worktree 的 artifacts 目录") + suffixes = {".json"} if metadata else {".sqlite", ".sqlite3", ".db"} + if "acceptance" not in path.name.lower() or path.suffix.lower() not in suffixes: + raise ValueError("文件名必须含 acceptance，扩展名须为 SQLite 文件或元数据 JSON") + if path.exists(): + raise ValueError("目标已存在，拒绝覆盖；请使用新的验收文件名") + return path + + +def _offline_audit(event: str, _args: tuple[Any, ...]) -> None: + if event in {"socket.connect", "socket.getaddrinfo", "socket.bind"}: + raise RuntimeError("离线样例脚本禁止网络/监听端口") + + +def seed(database: Path) -> dict[str, Any]: + # Environment must precede imports; explicit sys.path wins over editable main repo. + if any(name == "netx_api" or name.startswith("netx_api.") for name in sys.modules): + raise RuntimeError("必须通过独立 Python 进程运行，不能复用已导入 netx_api 的进程") + url = "sqlite:///" + database.as_posix() + os.environ["NETX_DATABASE_URL"] = url + os.environ["NETX_CREDENTIAL_SECRET_FILE"] = str(database.with_suffix(".unused-secret")) + for setting in ("NE_COLLECT_SCHEDULER_ENABLED", "CONFIG_SYNC_SCHEDULER_ENABLED", + "LLDP_COLLECT_SCHEDULER_ENABLED", "FABRIC_RECONCILE_SCHEDULER_ENABLED", + "PORT_TRAFFIC_SCHEDULER_ENABLED", "BIZ_STATE_SCHEDULER_ENABLED", + "UME_KEEPALIVE_ENABLED", "UME_ALARM_WS_ENABLED", + "UME_SYNC_ALARMS_CURRENT_ENABLED", "UME_SYNC_INVENTORY_AUTO_ENABLED", + "UME_SYNC_TOPOLOGY_AUTO_ENABLED"): + os.environ["NETX_" + setting] = "false" + sys.path.insert(0, str(ROOT)) + sys.addaudithook(_offline_audit) + + import netx_api + from sqlalchemy.engine import make_url + from netx_api.db import Base, SessionLocal, engine + from netx_api.biz_migration import service as mig + from netx_api.biz_state import service as biz + from netx_api.models import ( + BizCompareTemplate, BizMigrationProject, BizMonitorTemplate, + BizPortMapping, BizPortMappingRow, BizStateBatch, BizStateBatchCommand, + BizStateMetricRow, BizStateTask, BizStateTaskItem, + ) + + if Path(netx_api.__file__).resolve().parent != ROOT / "netx_api": + raise RuntimeError("netx_api 未从当前 worktree 导入，拒绝运行") + if engine.url != make_url(url): + raise RuntimeError("数据库配置未指向隔离 SQLite 文件，拒绝运行") + Base.metadata.create_all(engine) + now = datetime.now(timezone.utc).replace(tzinfo=None) + ids = { + "project_id": PREFIX + "project", + "old_portrait_task_id": PREFIX + "old-portrait", + "new_portrait_task_id": PREFIX + "new-portrait", + "old_current_task_id": PREFIX + "old-current", + "new_current_task_id": PREFIX + "new-current", + } + batch_ids: dict[str, str] = {} + command_ids: dict[str, str] = {} + try: + with SessionLocal() as db: + sheet = {"sheet_id": "interface_brief", "title": "验收样例接口", + "metric_id": "interface_brief", "key_fields": ["interface"], + "iface_fields": ["interface"], "compare_fields": ["admin", "phy", "prot"], + "row_filters": [], "field_rules": []} + db.add(BizCompareTemplate(id=PREFIX + "compare", name="验收样例/非现场故障-接口对比", + metric_id="interface_brief", key_fields=["interface"], + iface_fields=["interface"], compare_fields=["admin", "phy", "prot"], + metrics_json=[sheet], iface_normalize_json=[{"from": "GE", "to": "gei"}])) + db.add(BizMonitorTemplate(id=PREFIX + "monitor", name="验收样例/非现场故障-接口监控", + compare_template_id=PREFIX + "compare", + collect_metric_ids_json=["interface_brief"], + defaults_json={"out_of_expect": "strict"}, sheet_overrides_json=[{ + "metric_id": "interface_brief", "sheet_id": "interface_brief", + "status_fields": ["admin", "phy", "prot"], + "down_values": ["down"], "up_values": ["up"], + "success": [{"old": ["removed", "down"], "new": ["added", "up", "unchanged"]}], + "anomaly": [], + }])) + db.add(BizPortMapping(id=PREFIX + "mapping", name="验收样例端口映射", note=DECLARATION)) + db.add(BizPortMappingRow(id=PREFIX + "mapping-row", mapping_id=PREFIX + "mapping", + before_if="gei-old", after_if="xgei-new")) + for side in ("old", "new"): + for mode in ("portrait", "current"): + tid = ids[f"{side}_{mode}_task_id"] + task = BizStateTask( + id=tid, source="import", ne_id=PREFIX + side + "-offline", + ne_name=f"验收样例-{'旧' if side == 'old' else '新'}侧（非现场）", ne_ip="", + vendor="zte", device_type="router", note=DECLARATION, + purpose="portrait" if mode == "portrait" else "cutover_hf", + status="draft", collect_manual_only=True, collect_running=False, + interval_sec=3600, retention_days=30, + ) + db.add(task) + db.add(BizStateTaskItem(id=PREFIX + side + "-" + mode + "-item", task_id=tid, + source_profile_id="zte.interface_brief", kind="catalog", + enabled=False, title="验收样例离线输入；禁用采集")) + bid = PREFIX + side + "-" + mode + "-collect" + cid = PREFIX + side + "-" + mode + "-command" + batch_ids[f"{side}_{mode}"] = bid + command_ids[f"{side}_{mode}"] = cid + at = now - timedelta(minutes=10) if mode == "portrait" else now - timedelta(seconds=2) + interface = "GE-old" if side == "old" else "xgei-new" + state = "up" if side == "old" else "down" + raw = ("# 验收样例/非现场故障；离线合成输出\n" + "show interface brief\nInterface Admin Phy Prot\n" + f"{interface} {state} {state} {state}\n") + db.add(BizStateBatch(id=bid, task_id=tid, source="import", ne_id=task.ne_id, + ne_name=task.ne_name, vendor="zte", status="success", + started_at=at, ended_at=at + timedelta(seconds=1), + row_count=1, command_count=1, message=DECLARATION, + is_baseline=mode == "portrait")) + db.add(BizStateBatchCommand(id=cid, batch_id=bid, + profile_id="zte.interface_brief", parser_id="zte.interface_brief", + metric_id="interface_brief", raw_command="show interface brief", + parse_status="ok", row_count=1, raw_line_count=len(raw.splitlines()), + raw_text=raw, message=DECLARATION, created_at=at)) + db.add(BizStateMetricRow(id=PREFIX + side + "-" + mode + "-row", batch_id=bid, + batch_command_id=cid, task_id=tid, ne_id=task.ne_id, + metric_id="interface_brief", seq=0, collected_at=at, + data_json={"interface": interface, "admin": state, "phy": state, "prot": state})) + db.add(BizMigrationProject( + id=ids["project_id"], name="验收样例/非现场故障-旧口仍UP新口DOWN", note=DECLARATION, + old_task_id=ids["old_portrait_task_id"], new_task_id=ids["new_portrait_task_id"], + old_hf_task_id=ids["old_current_task_id"], new_hf_task_id=ids["new_current_task_id"], + old_baseline_batch_id=batch_ids["old_portrait"], new_baseline_batch_id=batch_ids["new_portrait"], + mapping_id=PREFIX + "mapping", monitor_template_id=PREFIX + "monitor", + collect_metric_ids_json=["interface_brief"], status="draft", hf_interval_sec=3600, + )) + db.commit() + cutover = mig.create_batch(db, ids["project_id"], { + "batch_label": "验收样例/非现场故障-第一批", "status": "active", + "expect_set": {"ports": ["gei-old"]}, "note": DECLARATION, + }) + # This executes real run_evaluate and real red-ticket persistence. + finished = mig.finish_batch(db, cutover["id"], mark_done=False) + run_id = finished["run"]["id"] + diffs = mig.list_run_diffs(db, run_id, color="red", limit=50) + reds = mig.list_red_tickets(db, ids["project_id"]) + if diffs["total"] < 1 or reds["open_count"] < 1 or finished["accept_summary"]["passed"]: + raise RuntimeError("真实评价未生成预期红项，样例验证失败") + item = diffs["items"][0] + if item["old_key"] != "GE-old" or item["new_key"] != "xgei-new": + raise RuntimeError("缺少原始 A/B 键，样例验证失败") + evidence = item.get("evidence") or {} + raw_checks: dict[str, Any] = {} + for side in ("old", "new"): + ev = evidence.get(side) or {} + command = ev.get("command") or {} + collect = ev.get("collect") or {} + if command.get("raw_command") != "show interface brief" or not collect.get("collected_at"): + # Some evidence versions name the timestamp at batch granularity. + if command.get("raw_command") != "show interface brief" or not any(collect.get(k) for k in ("started_at", "ended_at", "batch_started_at")): + raise RuntimeError("双侧命令/采集时间证据缺失") + bid = collect["batch_id"] + cid = command.get("id") or command.get("command_id") + if not cid: + raise RuntimeError("证据没有可请求的命令 ID") + raw = biz.get_batch_command(db, bid, cid) + if "验收样例/非现场故障" not in raw["raw_text"]: + raise RuntimeError("命令原文回读失败") + raw_checks[side] = {"batch_id": bid, "command_id": cid, "raw_text": raw["raw_text"], + "collected_at": raw["collected_at"]} + # Read paths behind all eight MCP tools; HTTP/stdio is the lead's next step. + assert mig.list_projects(db) and biz.list_tasks(db) + assert mig.list_batches(db, ids["project_id"]) and biz.list_batches(db, ids["old_current_task_id"]) + assert mig.list_runs(db, cutover["id"]) + assert mig.get_monitor_context(db, project_id=ids["project_id"]) + assert mig.board(db, cutover["id"], run_id=run_id) + assert biz.get_batch(db, batch_ids["new_current"])["commands"] + tasks = db.query(BizStateTask).all() + if not all(t.status == "draft" and t.collect_manual_only and not t.collect_running for t in tasks): + raise RuntimeError("样例任务安全状态检查失败") + db.commit() + return { + "declaration": DECLARATION, "database": str(database), "database_url": url, + "source_root": str(ROOT), "created_at": now.isoformat() + "Z", **ids, + "cutover_batch_id": cutover["id"], "run_id": run_id, + "collect_batch_ids": batch_ids, "command_ids": command_ids, + "expected": {"scenario": "旧侧接口仍UP，新侧映射接口DOWN，割接验收未完成，应产生红项", + "old_key": "GE-old", "new_key": "xgei-new", "minimum_red_count": 1}, + "observed": {"red_diff_count": diffs["total"], "open_red_count": reds["open_count"], + "verdict": item["verdict"], "color": item["color"], + "accept_status": finished["batch"]["accept_status"], + "accept_summary": finished["accept_summary"], "raw_evidence": raw_checks}, + "safety": {"network_blocked": True, "task_status": "draft", "manual_only": True, + "task_items_enabled": False, "device_credentials": False, "real_device_ip": False}, + } + finally: + engine.dispose() + + +def main() -> int: + for stream in (sys.stdout, sys.stderr): + if hasattr(stream, "reconfigure"): + stream.reconfigure(encoding="utf-8", errors="replace") + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--database", required=True, help="artifacts 内新 SQLite 文件的绝对路径，文件名含 acceptance") + parser.add_argument("--metadata", help="artifacts 内新 JSON 文件的绝对路径，文件名含 acceptance") + args = parser.parse_args() + try: + database = artifact_path(args.database) + metadata = artifact_path(args.metadata, metadata=True) if args.metadata else None + database.parent.mkdir(parents=True, exist_ok=True) + # Exclusive creation prevents overwrite, including races after validation. + with database.open("xb"): + pass + result = seed(database) + text = json.dumps(result, ensure_ascii=False, indent=2) + if metadata: + metadata.parent.mkdir(parents=True, exist_ok=True) + with metadata.open("x", encoding="utf-8") as output: + output.write(text + "\n") + print(text) + return 0 + except Exception as exc: + # Known local validation messages are safe; other failures omit internals. + message = str(exc) if isinstance(exc, (ValueError, RuntimeError)) else type(exc).__name__ + print(json.dumps({"ok": False, "error": message, "declaration": DECLARATION}, ensure_ascii=False), file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skills/README.md b/skills/README.md index df5570b..45031a3 100644 --- a/skills/README.md +++ b/skills/README.md @@ -6,7 +6,7 @@ |-------|--------------|----------------------------------------| | `ops` | `ops/netx-ops/` | NMS 告警/库存/SQL + managed CLI + `findTopologyPaths` | | `topology` | `topology/netx-topology/` | 画布 / Fabric / dual_unit / 布图 | -| `biz-monitor` | `biz-monitor/netx-biz-monitor/` | 割接/业务监控只读分析（**netxops 宿主工具**直连 REST；非 netx-mcp） | +| `bizMonitor` | `biz-monitor/netx-biz-monitor/` | 割接/业务监控只读分析（独立 `netx-bizmonitor-mcp` 或 netxops 宿主工具） | | `biz-state` | `biz-state/netx-biz-state/` | 状态监控：TextFSM / parser / Profile（Cursor Agent；无 DSH 工具） | ## DSH 怎么用 @@ -14,8 +14,8 @@ 1. 运行时优先：`NETX_SKILLS_ROOT` → 旁路 `../netx/skills` → 包内 `presets/netxops/skills` 2. 发 npm 前：`powershell -File netxops/scripts/sync-skills-from-netx.ps1` 3. Settings → 能力组开关：开哪组就注册哪组的 **tools + 对应 skill**（含 **bizMonitor**） -4. 其它预设可强制挂：`dsh-netxops/tools-ops|topology|biz-monitor`（旧别名 `tools-nms` / `tools-common` → ops） +4. 其它预设可强制挂：`dsh-netxops/tools-ops|topology|biz-monitor` -MCP / Cursor：skill 根直接指本目录。MCP 包内不镜像 skills。业务监控分析工具只在 **netxops**，不要往 netx-mcp 加。 +MCP / Cursor：skill 根直接指本目录，包内不镜像 skills。不装 netxops 时，分别使用 `netx-mcp`（15 工具）、`netx-topology-mcp`（16 工具）和 `netx-bizmonitor-mcp`（8 工具）；配置见根 `mcp.all.json`。独立包均通过 HTTP 访问 netx，不直连数据库。 **Cursor Agent**：加 biz_state 监控命令时启用或点名 **netx-biz-state**（`skills/biz-state/netx-biz-state/`）。 diff --git a/skills/biz-monitor/netx-biz-monitor/SKILL.md b/skills/biz-monitor/netx-biz-monitor/SKILL.md index ebd44a9..ec06e28 100644 --- a/skills/biz-monitor/netx-biz-monitor/SKILL.md +++ b/skills/biz-monitor/netx-biz-monitor/SKILL.md @@ -1,32 +1,30 @@ --- name: netx-biz-monitor description: >- - Overnight cutover / biz_state monitor analysis via netxops host tools - (netx__listBizMonitors / listBizMonitorBatches / getBizMonitor*). List - projects/tasks then batches when no id is given; read definitions, board, - reds/diffs with evidence (device-raw A/B, show commands), and judge tool - false-positive vs real business fault. Does not create templates/tasks. + 通过独立 netx-bizmonitor-mcp 或 netxops 分析割接与业务状态。 + 无 ID 先查目录和批次，结合看板、变红项、双侧原始 A/B 键和采集命令原文判断故障。 + 只读已有项目与评价结果，不创建模板、触发采集、执行评价或关闭红票。 --- # netx-biz-monitor（割接 / 业务监控只读分析） 人配好监控与任务；AI 当晚做归因。 -工具在 **netxops**（`netx__*`），直连 netx REST，**不是** netx-mcp。 +独立 **netx-bizmonitor-mcp** 提供下表裸名；netxops 使用 `netx__` 加同名。两者均访问 netx REST，需 `biz-monitor:read`。安装和三组配置见 [MCP_BIZMONITOR.md](../../../docs/MCP_BIZMONITOR.md)。 ## 工具（能力组 `bizMonitor`） | 工具 | 用途 | |------|------| -| `netx__listBizMonitors` | **入口目录**：列出割接/监控对比项目 + 业务监控任务（**无需** project_id） | -| `netx__listBizMonitorBatches` | **批次目录**：`project_id`→割接批次；`task_id`→采集批次；`kind=runs`→evaluate run | -| `netx__getBizMonitorContext` | 任务定义：项目、监控/对比模板、归一化规则、端口映射、关联任务与命令项 | -| `netx__getBizMonitorBoard` | 批次进度、evaluate run、sheet 卡、verdict 统计 | -| `netx__listBizMonitorReds` | 红单 + **evidence** | -| `netx__getBizMonitorDiffs` | run 明细（可滤 color=red） | -| `netx__getBizCollectBatch` | 某次采集：commands / metrics | -| `netx__getBizCollectCommandRaw` | 单命令全量 show 输出 | +| `listBizMonitors` | **入口目录**：列出割接/监控对比项目 + 业务监控任务（**无需** project_id） | +| `listBizMonitorBatches` | **批次目录**：`project_id`→割接批次；`task_id`→采集批次；`batch_id,kind=runs`→evaluate run | +| `getBizMonitorContext` | 任务定义：项目、监控/对比模板、归一化规则、端口映射、关联任务与命令项 | +| `getBizMonitorBoard` | 批次进度、evaluate run、sheet 卡、verdict 统计 | +| `listBizMonitorReds` | 红单 + **evidence** | +| `getBizMonitorDiffs` | run 明细，`color=red`；`offset/limit` 分页；`q` 为关键词 | +| `getBizCollectBatch` | 采集工作簿摘要、commands 和 sheet 目录；不含全部指标行或原文 | +| `getBizCollectCommandRaw` | 以采集 batch_id + command_id 获取单命令原文，按需调用，可能很大 | -交叉验证真障时再用 ops：`netx__execManagedNe` / 拓扑路径。 +交叉验证真障时再用 ops 的 `execManagedNe` / 拓扑路径。独立 MCP 仅允许单行 `show` / `display`，不执行配置或 shell 脚本。 ## 字段约定（必守） @@ -41,9 +39,9 @@ description: >- 2. `listBizMonitorBatches(project_id)` 或 `(task_id)` — 拿到 **完整 batch_id**；需要 run 时再 `kind=runs` / `include_runs` 3. `getBizMonitorContext(project_id)` — 弄清盯什么 4. `getBizMonitorBoard(batch_id)` — 进度 / missing / skipped -5. `listBizMonitorReds` 或 `getBizMonitorDiffs(color=red)` +5. `listBizMonitorReds` 或 `getBizMonitorDiffs(run_id,color=red,limit=100,offset=0)`；按 total 翻页，不能只看第一页就宣称全部红项。 6. 看 evidence：parse 失败？current_missing？映射 miss？归一化误伤？ -7. 需要原文 → `getBizCollectCommandRaw` +7. 从差异 evidence 的 collect/command 取两侧**采集** batch_id/command_id → `getBizCollectCommandRaw`；若证据不全，用对应 task_id 查采集批次，再 getBizCollectBatch 查命令。不要把割接 batch_id 当成采集 batch_id。 8. 像真障 → `execManagedNe` 现场核对 ## Verdict 速查（skill 解释，工具不硬编码） @@ -64,3 +62,5 @@ description: >- 4. 再谈真实业务问题 配置整改与关红单由人做；本 skill **只读分析**。 + +目录或批次为空时，明确说明没有可评价的数据；不把空红票当作割接通过。run 尚未完成或存在 current_missing/采集失败时，不给通过结论。工具本身不启动 evaluate；需要人先在 Web 完成定义、采集和评价。`old_key/new_key` 是设备原始端口；例：归一化和映射将 GE-old 匹配到 xgei-new，结论仍分别展示 GE-old 与 xgei-new，不能用内部匹配键替代。 diff --git a/skills/ops/netx-ops/SKILL.md b/skills/ops/netx-ops/SKILL.md index e0a944b..6d82291 100644 --- a/skills/ops/netx-ops/SKILL.md +++ b/skills/ops/netx-ops/SKILL.md @@ -8,7 +8,7 @@ description: netx 运维：告警、网元、只读 CLI。 处理 netx **告警 / 网元清单 / 只读 CLI** 时遵循本手册。 画布布图不在本 skill（见 **netx-topology**）。本手册里的「路径」仅指 `findTopologyPaths` 查两端关联。 -**工具名**：DSH 为 `netx__` + stem（如 `netx__queryNmsAlarms`）；MCP 多为同名裸 stem。下文一律写 stem。 +**工具名**：DSH 为 `netx__` + stem（如 `netx__queryNmsAlarms`）；独立 `netx-mcp` 使用同名裸 stem，共 15 个。下文一律写 stem。模型入参只用 `nms_ne_id/nms_ne_ids`；旧 `ume_*`、getManagedNe 的 `id/managed_ne_id` 别名已删除。 **两套「一批」（勿混）：** @@ -23,7 +23,7 @@ description: netx 运维：告警、网元、只读 CLI。 ## 1. 总原则 -1. **先新鲜度，再结论**：`runNmsDiagnostics` 或 `aggregateNmsAlarms` → 读 `meta.last_seen_*`。过旧则按快照：时间窗落在 min~max 内，禁止默认「现在−30 分钟」。 +1. **先新鲜度，再结论**：`runNmsDiagnostics` → 读全库 `meta.last_seen_*`。用户问最近一小时则按用户时区查询真实时间窗；若为空且快照过旧，只能说当前数据无法判断。可另按快照 max 前一小时补充历史分析，明确日期，不冒充当前告警。带过滤的 aggregate 元数据只代表过滤后的窗口。 2. **证据优先**：有告警须有级别计数和/或 Top `host_name`；否则写「证据不足」，禁止臆造根因。 3. **对用户**：只展示 **host_name**；**严禁**出现任何 UUID/`ne_id`。无 `host_name` 的行当垃圾丢弃（不顶替 label、不提「缺失」）。工具入参仍可用 id，但不得写入对用户可见正文。 4. **短问**：能 ≤3 次工具调用就闭环的，先聚合/过滤，禁止无过滤翻全库。复杂关联（dying gasp、A<>B CLI）不受 3 次硬顶，但仍须有过滤与目标。 @@ -67,7 +67,10 @@ description: netx 运维：告警、网元、只读 CLI。 - 命令相同：`nms_ne_ids` 或 `ne_ids` + 共享 `commands` - 命令不同（厂商/角色不同）：`targets=[{nms_ne_id|ne_id, commands:[…]}, …]`，仍一次调用 - 超时：加大 `read_timeout_sec` 或减条数；禁止同参盲重试 -- 只读：`show` / `display` / `ping` / `traceroute` 及带 `?` 的探索；配置类不做 +- 独立 MCP 只允许单行 `show` / `display`，支持只读输出过滤和 `?` 探索；禁止配置、shell、重定向、命令拼接。后端 `linux_shell/unrestricted` 不能解除 MCP 门禁。 +- `async=true` 或多台自动异步返回 job_id 仅代表开始；约 5 秒后用 `getNeExecJob` 有界轮询，期间可继续独立工作。单台须 `terminal=true` 且 `result.ok=true`；批量还须核对 `result.summary.failed=0` 和每条 `result.results[].ok=true`。`status=succeeded` / `result.ok=true` 仍可能是部分成功，要分别报告成功和失败目标。连接失败/timeout 不能当作登录成功，也不要重提相同 job 或整批重跑已成功设备。 +- `getManagedNe.capability` 是后端设备能力；独立 MCP 仍只允许单行 show/display，后端 Linux/RouterOS 开放策略不会放宽此门禁。NAS/VPS/RouterOS 原生 CLI 当前不能通过这组 MCP 执行；连接测试须明确另用 netx Web/API，不能把 API 登录证据写成 MCP 原生 CLI 执行成功。 +- 同一 IP 有多个纳管记录时，结合用户指定厂商与 `vendor/device_type` 选择 ID，不能取第一条 generic 记录就执行。`connect_status` 是上次连接测试状态，exec 成功不会自动刷新；本次登录/查询结论以本次 job 的逐台结果和有效正文为准。空正文或只有命令头不算查询通过。 **✓ 同命令多台** @@ -180,7 +183,7 @@ Neighbour / PW / Tunnel 等控制面量多 **不当断纤**，除非用户问的 - **优先复用本会话已成功过的命令**（含同厂其他台验证过的）：新台/下一批先试这些，再对失败子集另探；不要一失败就换全员命令。 - 多台一批时：用 `targets` 给失败台换命令，或先一批共同命令 → 只对失败子集再一批探测；不要为了一台把成功台的结果丢掉重来。 -### `?` 规则（`show` / `display` / `ping` 等同一套） +### `?` 规则（`show` / `display`） 1. 报错中的 **`^`** 标出错位置 → **`^` 之前**已正确 → 对该前缀加 `?` 列下一级。 2. `?` 可紧接在部分单词后（`show optical?`、`display inter?`），不必强制空格再写 `?`。 diff --git a/skills/topology/netx-topology/SKILL.md b/skills/topology/netx-topology/SKILL.md index 43c6da7..e9c21c0 100644 --- a/skills/topology/netx-topology/SKILL.md +++ b/skills/topology/netx-topology/SKILL.md @@ -317,7 +317,7 @@ sinkTopologyDualUnits({ |--------|------| | `layout` | 小图 / 剩余块：`compact` / `corridor` / `rings` / `unstick` | | `layout_dual_unit` | **仅**重布已沉的那一只眼；椭圆弧带 + 空心眼 | -| `move_nodes` / `sink_nodes` | **后续下沉主路径**；`park` 块扫 | +| `move_nodes` | **后续下沉主路径**；`park` 块扫 | | `orbit_sweep` | **眼后精修主工具**：`until_limit` 或 `node_id`；`objective=crossing\|total` | | `clear_edge_hits` | 眼 sink **可**（须 `portal_ids`；交叉不升） | | `pull_far_chains` | 眼 sink **可**：朝门户中点收远走廊/孤立点 | @@ -376,3 +376,13 @@ sinkTopologyDualUnits({ 4. 机器上常残留多份旧 MCP 进程 → 精修前 catalog 若仍缺 action，**杀光再启**；勿用旧进程半截交付，勿改走 HTTP/临时 py。 拓扑页开「实时同步」可看落笔。勿用告警/CLI MCP 写拓扑。 + +## MCP 异步与失败处理 + +小图先用 `layoutTopologyView(view_id=...,action=layout,recipe=compact,mode=preview)` 看提案，再 `mode=apply,params={background:true}` 保存。`job_id/status=running/applied=false` 表示尚未完成；用同一台机器、同一任务目录的 `layoutTopologyView(action=job_status,params={job_id:...})` 轮询。成功需 `status=done` 且 `result.applied=true`，随后 `getTopologyView(detail=full)` 回读坐标。没有 HTTP layout job 接口。 + +任务状态在 MCP 的 `NETX_LAYOUT_JOB_DIR`（默认 data/runtime/layout_jobs）落盘，换机器/目录不能读同一 job。`job_cancel` 只请求协作取消，不承诺回滚。复制/移动/下沉是多步 HTTP 操作；遇到错误或 `write_completed=true` 先读两端现状，不盲重试。坐标保存失败时不继续删除源成员。 + +只读 token 不包含 `ne:write`，将看不到布局/建图工具；preview/job_status 目前也随 layout 工具受该权限限制。修改的是 netx 画布与 Fabric 元数据，不向真实设备下配置。需要演练时新建验收视图，避免覆盖既有图。 + +布局任务成功和评分模型评价分别记录。评分仅供诊断，需结合图形与业务需求；不能仅凭总分判定布局通过或失败。评分模型的校准留待后续专项，本轮不调整算法或阈值。 diff --git a/tests/fixtures/zte_vpnv6/README.md b/tests/fixtures/zte_vpnv6/README.md new file mode 100644 index 0000000..34a34f1 --- /dev/null +++ b/tests/fixtures/zte_vpnv6/README.md @@ -0,0 +1,3 @@ +# ZTE vpnv6 只读回放样本 + +这3个小样本于2026-10-15从工作区现有 `D:/project/chatgpt/test/show-zte/` 同名文件原样复制，作为 `test_bgp_peer_ipv4_and_ipv6_wrap` 的仓库内测试输入。内容为 show bgp 的summary及vrf邻居in/out只读输出，不含配置日志或口令。未在本次任务中连接设备采集。保留字节原貌；请勿把它们解释为合成数据或本次现场实测。 diff --git a/tests/fixtures/zte_vpnv6/show-bgp-vpnv6 b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6 new file mode 100644 index 0000000..29444b7 --- /dev/null +++ b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6 @@ -0,0 +1,13 @@ +MDN-BCP-CN1-ZM8SP#show bgp vpnv6 unicast summary | one-line +08:04:10 Indonesia Sat Sep 19 2026 +BGP router ID : 124.195.96.248 +Local AS : 4761 +All peers : 2 + Established : 2 +All prefixes : 0 +Neighbor Ver As MsgRcvd MsgSend Up/Down State/PfxRcd +FC00:A000:0:61:114::75 + 4 4761 14476867 2679817 5w4d 0 +FC00:A000:0:61:114::76 + 4 4761 14609910 2680356 5w4d 0 +MDN-BCP-CN1-ZM8SP# \ No newline at end of file diff --git a/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-in-vrf b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-in-vrf new file mode 100644 index 0000000..e25f1d5 --- /dev/null +++ b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-in-vrf @@ -0,0 +1,40 @@ +MDN-BCP-CN1-ZM8SP#show bgp vpnv6 unicast vrf vpn.giims neighbor in 2407::45:0:0:5:1 | one-line +09:50:02 Indonesia Sat Sep 19 2026 +Routes Learned From This Neighbor: +Status codes: * valid, < last valid, i - internal, s - stale +Origin codes: i - IGP, e - EGP, ? - incomplete +Local router ID 124.195.96.248, Local AS 4761, Local port 179 +Remote router ID 10.83.48.3, Remote AS 65254, Remote port 54486 +Total number of routes: 9 + Valid routes : 9 + Invalid routes : 0 + Network Next Hop Metric LocPrf RtPrf Path + +* 2407::45:0:0:2:780/125 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:7c4/128 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:7c5/128 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:800/123 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:844/128 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:845/128 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:880/123 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:8c4/128 + 2407::45:0:0:5:1 + 20 65254 ? +* 2407::45:0:0:2:8c5/128 + 2407::45:0:0:5:1 + 20 65254 ? +MDN-BCP-CN1-ZM8SP# \ No newline at end of file diff --git a/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-out-vrf b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-out-vrf new file mode 100644 index 0000000..9702bd3 --- /dev/null +++ b/tests/fixtures/zte_vpnv6/show-bgp-vpnv6-neighbor-router-out-vrf @@ -0,0 +1,11 @@ +MDN-BCP-CN1-ZM8SP#show bgp vpnv6 unicast vrf vpn.giims neighbor out 2407::45:0:0:5:1 | one-line +09:51:15 Indonesia Sat Sep 19 2026 +Routes Sent To This Neighbor: +Origin codes: i - IGP, e - EGP, ? - incomplete +Total number of routes: 1 +Network Next Hop From Metric LocPrf Path + +2407::45:0:0:5:0/127 + 2407::45:0:0:5:0 + 4761 ? +MDN-BCP-CN1-ZM8SP# diff --git a/tests/test_batch_workbook_api.py b/tests/test_batch_workbook_api.py index 2cb354f..a9e0c94 100644 --- a/tests/test_batch_workbook_api.py +++ b/tests/test_batch_workbook_api.py @@ -6,12 +6,56 @@ import unittest from unittest.mock import MagicMock, patch from fastapi import HTTPException +from sqlalchemy import create_engine, event, inspect +from sqlalchemy.orm import Session + +from netx_api.db import Base from netx_api.biz_state.service import get_batch, list_batch_metric_rows -from netx_api.models import BizStateBatch, BizStateBatchCommand +from netx_api.models import BizStateBatch, BizStateBatchCommand, BizStateMetricRow class BatchWorkbookApiTests(unittest.TestCase): + def _stored_summary(self, batch, commands, metric_counts): + """Exercise SQL summary projection against an isolated DB, with raw text deferred.""" + engine = create_engine("sqlite+pysqlite:///:memory:") + self.addCleanup(engine.dispose) + Base.metadata.create_all(engine) + with Session(engine, expire_on_commit=False) as db: + db.add(batch) + db.add_all(commands) + for metric_id, count in metric_counts: + db.add_all([ + BizStateMetricRow(batch_id=batch.id, metric_id=metric_id, seq=i) + for i in range(count) + ]) + db.commit() + db.expunge_all() + selected_commands = [] + column_loads = [] + + @event.listens_for(db, "loaded_as_persistent") + def check_deferred(session, row): + if isinstance(row, BizStateBatchCommand): + selected_commands.append(row) + self.assertIn("raw_text", inspect(row).unloaded) + + @event.listens_for(db, "do_orm_execute") + def track_column_loads(state): + if state.is_column_load: + column_loads.append(state.statement) + + with patch( + "netx_api.biz_state.service.batch_protect_info", + return_value={"protected": False, "reasons": []}, + ): + result = get_batch(db, batch.id) + self.assertEqual(len(selected_commands), len(commands)) + self.assertEqual(column_loads, [], "summary must never lazy-load raw CLI text") + for row in selected_commands: + self.assertIn("raw_text", inspect(row).unloaded) + return result + def test_get_batch_summary_has_sheets_not_metrics(self) -> None: batch = BizStateBatch( id="b1", @@ -31,35 +75,7 @@ class BatchWorkbookApiTests(unittest.TestCase): row_count=2, raw_text="RAW", ) - db = MagicMock() - db.get.side_effect = lambda model, pk: batch if pk == "b1" else None - - cmd_q = MagicMock() - cmd_q.filter.return_value.order_by.return_value.all.return_value = [cmd] - metric_count_q = MagicMock() - metric_count_q.filter.return_value.group_by.return_value.all.return_value = [("arp", 2)] - lldp_count_q = MagicMock() - lldp_count_q.filter.return_value.scalar.return_value = 0 - - def query(*_args, **_kwargs): - # First call in get_batch after protect: BatchCommand; then metric count; then lldp - # Distinguish by call count - n = query.n - query.n += 1 - if n == 0: - return cmd_q - if n == 1: - return metric_count_q - return lldp_count_q - - query.n = 0 - db.query.side_effect = query - - with patch( - "netx_api.biz_state.service.batch_protect_info", - return_value={"protected": False, "reasons": []}, - ): - out = get_batch(db, "b1") + out = self._stored_summary(batch, [cmd], [("arp", 2)]) self.assertIn("sheets", out) self.assertNotIn("metrics", out) @@ -93,32 +109,7 @@ class BatchWorkbookApiTests(unittest.TestCase): raw_text="a\nb\nc\n", message="", ) - db = MagicMock() - db.get.side_effect = lambda model, pk: batch if pk == "b1" else None - cmd_q = MagicMock() - cmd_q.filter.return_value.order_by.return_value.all.return_value = [cmd] - metric_count_q = MagicMock() - metric_count_q.filter.return_value.group_by.return_value.all.return_value = [("arp", 3)] - lldp_count_q = MagicMock() - lldp_count_q.filter.return_value.scalar.return_value = 0 - - def query(*_args, **_kwargs): - n = query.n - query.n += 1 - if n == 0: - return cmd_q - if n == 1: - return metric_count_q - return lldp_count_q - - query.n = 0 - db.query.side_effect = query - - with patch( - "netx_api.biz_state.service.batch_protect_info", - return_value={"protected": False, "reasons": []}, - ): - out = get_batch(db, "b1") + out = self._stored_summary(batch, [cmd], [("arp", 3)]) self.assertEqual(out["message"], "stopped") self.assertEqual(out["commands"][0]["raw_line_count"], 3) @@ -148,34 +139,7 @@ class BatchWorkbookApiTests(unittest.TestCase): declared_total=1669101, message="declared=1669101;parsed=100", ) - db = MagicMock() - db.get.side_effect = lambda model, pk: batch if pk == "b1" else None - cmd_q = MagicMock() - cmd_q.filter.return_value.order_by.return_value.all.return_value = [cmd] - metric_count_q = MagicMock() - metric_count_q.filter.return_value.group_by.return_value.all.return_value = [ - ("bgp_route", 100) - ] - lldp_count_q = MagicMock() - lldp_count_q.filter.return_value.scalar.return_value = 0 - - def query(*_args, **_kwargs): - n = query.n - query.n += 1 - if n == 0: - return cmd_q - if n == 1: - return metric_count_q - return lldp_count_q - - query.n = 0 - db.query.side_effect = query - - with patch( - "netx_api.biz_state.service.batch_protect_info", - return_value={"protected": False, "reasons": []}, - ): - out = get_batch(db, "b1") + out = self._stored_summary(batch, [cmd], [("bgp_route", 100)]) self.assertEqual(out["commands"][0]["raw_line_count"], 119303) self.assertEqual(out["commands"][0]["declared_total"], 1669101) @@ -263,34 +227,7 @@ class BatchWorkbookApiTests(unittest.TestCase): row_count=1, ), ] - db = MagicMock() - db.get.side_effect = lambda model, pk: batch if pk == "b1" else None - cmd_q = MagicMock() - cmd_q.filter.return_value.order_by.return_value.all.return_value = cmds - metric_count_q = MagicMock() - metric_count_q.filter.return_value.group_by.return_value.all.return_value = [ - ("bgp_peer", 3) - ] - lldp_count_q = MagicMock() - lldp_count_q.filter.return_value.scalar.return_value = 0 - - def query(*_args, **_kwargs): - n = query.n - query.n += 1 - if n == 0: - return cmd_q - if n == 1: - return metric_count_q - return lldp_count_q - - query.n = 0 - db.query.side_effect = query - - with patch( - "netx_api.biz_state.service.batch_protect_info", - return_value={"protected": False, "reasons": []}, - ): - out = get_batch(db, "b1") + out = self._stored_summary(batch, cmds, [("bgp_peer", 3)]) bgp = next(s for s in out["sheets"] if s["metric_id"] == "bgp_peer") self.assertEqual(bgp["title"], "BGP Status Summary") diff --git a/tests/test_managed_ne.py b/tests/test_managed_ne.py index effce9c..5513751 100644 --- a/tests/test_managed_ne.py +++ b/tests/test_managed_ne.py @@ -188,8 +188,9 @@ class ManagedNeApiTests(unittest.TestCase): self.assertEqual(listed.json()["total"], 1, listed.text) self.client.delete(f"/v1/managed-ne/{ne_id}") - def test_create_without_crypto_key(self): - settings.credential_secret_key = "" + @patch("netx_api.ne_service_common.credentials_configured", return_value=False) + def test_create_without_crypto_key(self, mock_configured): + # Empty settings now auto-provision a key; model an unavailable key explicitly. r = self.client.post( "/v1/managed-ne", json={ @@ -201,6 +202,7 @@ class ManagedNeApiTests(unittest.TestCase): }, ) self.assertEqual(r.status_code, 503, r.text) + mock_configured.assert_called_once() def test_delete_ume_sync_route_not_captured_by_ne_id(self): """DELETE /ume-sync must not match DELETE /{ne_id} with ne_id='ume-sync'.""" diff --git a/tests/test_mcp_http.py b/tests/test_mcp_http.py index e0ac008..c509072 100644 --- a/tests/test_mcp_http.py +++ b/tests/test_mcp_http.py @@ -100,23 +100,10 @@ def test_stdio_initialize_and_tools_list() -> None: proc.wait(timeout=5) -def test_legacy_netx_api_mcp_module_still_works() -> None: - proc = subprocess.Popen( - [sys.executable, "-m", "netx_api.mcp"], - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - encoding="utf-8", - errors="replace", - cwd=str(__import__("pathlib").Path(__file__).resolve().parents[1]), - ) - assert proc.stdin and proc.stdout - init_req = json.dumps({"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}) + "\n" - proc.stdin.write(init_req) - proc.stdin.flush() - init_line = proc.stdout.readline() - init_resp = json.loads(init_line) - assert init_resp["result"]["serverInfo"]["mode"] == "http" - proc.terminate() - proc.wait(timeout=5) + +def test_only_standalone_mcp_implementation_is_kept() -> None: + from pathlib import Path + + root = Path(__file__).resolve().parents[1] + assert not (root / "netx_api/mcp_server.py").exists() + assert not list((root / "netx_api/mcp").glob("*.py")) diff --git a/tests/test_ne_capability_jobs.py b/tests/test_ne_capability_jobs.py index 8279fb6..35af8e2 100644 --- a/tests/test_ne_capability_jobs.py +++ b/tests/test_ne_capability_jobs.py @@ -73,6 +73,7 @@ class NeExecJobTests(unittest.TestCase): ack = start_ne_exec_job( kind="exec", arguments={"ne_id": "abc", "commands": ["show version"]}, + owner_user_id="test-owner", runner=lambda: {"ok": True, "output": "hi"}, ) self.assertTrue(ack.get("ok")) @@ -80,7 +81,7 @@ class NeExecJobTests(unittest.TestCase): self.assertTrue(jid) done = None for _ in range(50): - polled = get_ne_exec_job(jid) + polled = get_ne_exec_job(jid, user_id="test-owner") if polled.get("terminal"): done = polled break diff --git a/tests/test_ne_exec_job_security.py b/tests/test_ne_exec_job_security.py new file mode 100644 index 0000000..fb5581c --- /dev/null +++ b/tests/test_ne_exec_job_security.py @@ -0,0 +1,241 @@ +"""Exec job isolation and path tests: temporary files and in-memory ASGI only.""" + +from __future__ import annotations + +import json +import time +from types import SimpleNamespace +from urllib.parse import quote +from unittest.mock import MagicMock, patch + +from fastapi import FastAPI +from fastapi.testclient import TestClient +import pytest + +from netx_api import managed_ne_router as routes +from netx_api import ne_exec_jobs as jobs +from netx_api.auth_deps import AuthContext, require_user +from netx_api.db import get_db + + +JOB_ID = "a" * 32 +OTHER_JOB_ID = "b" * 32 + + +@pytest.fixture +def job_dir(tmp_path, monkeypatch): + directory = tmp_path / "jobs" + directory.mkdir() + monkeypatch.setenv("NETX_NE_EXEC_JOB_DIR", str(directory)) + return directory + + +def write_record(directory, *, owner="alice", job_id=JOB_ID): + record = { + "job_id": job_id, "owner_user_id": owner, "status": "succeeded", "ne_count": 1, + "result": {"ok": True, "output": "temporary-test-output"}, + } + (directory / f"{job_id}.json").write_text(json.dumps(record), encoding="utf-8") + return record + + +@pytest.mark.parametrize("job_id", [ + "../canary", r"..\canary", r"C:\temp\canary", r"\\server\share\canary", + "/tmp/canary", "a" * 31, "a" * 33, "A" * 32, + "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", " " + JOB_ID, JOB_ID + "\n", "not-a-job", +]) +def test_invalid_ids_never_access_job_directory(job_id): + with patch.object(jobs, "_jobs_dir") as directory: + out = jobs.get_ne_exec_job(job_id, user_id="alice") + assert out == {"ok": False, "error_code": "invalid_job_id", "error": "invalid_job_id"} + with pytest.raises(ValueError, match="^invalid_job_id$"): + jobs._job_path(job_id) + directory.assert_not_called() + assert job_id not in json.dumps(out) + + +def test_canary_outside_job_directory_cannot_be_read(job_dir): + canary = job_dir.parent / "canary.json" + canary.write_text(json.dumps({"status": "succeeded", "result": {"canary": "non-secret"}}), encoding="utf-8") + for job_id in ("../canary", r"..\canary", str(canary.with_suffix(""))): + out = jobs.get_ne_exec_job(job_id, user_id="alice", is_admin=True) + assert out["error"] == "invalid_job_id" + assert "result" not in out + assert str(job_dir.parent) not in json.dumps(out) + + +def test_valid_id_symlink_cannot_escape_directory(job_dir): + canary = job_dir.parent / "canary.json" + canary.write_text(json.dumps({"job_id": JOB_ID, "owner_user_id": "alice", "status": "succeeded", "result": {"canary": "non-secret"}}), encoding="utf-8") + link = job_dir / f"{JOB_ID}.json" + try: + link.symlink_to(canary) + except (OSError, NotImplementedError): + pytest.skip("Host does not permit temporary symlink creation") + out = jobs.get_ne_exec_job(JOB_ID, user_id="alice", is_admin=True) + assert out["error"] == "invalid_job_id" + with pytest.raises(ValueError, match="^invalid_job_id$"): + jobs._write_job({"job_id": JOB_ID, "owner_user_id": "alice"}) + assert json.loads(canary.read_text(encoding="utf-8"))["result"]["canary"] == "non-secret" + + +def test_resolved_candidate_must_stay_inside_job_directory(job_dir): + """Exercise containment even on Windows hosts that cannot create symlinks.""" + from pathlib import Path + + original_resolve = Path.resolve + candidate = job_dir / f"{JOB_ID}.json" + outside = job_dir.parent / "canary.json" + + def resolve(path, *args, **kwargs): + return outside if path == candidate else original_resolve(path, *args, **kwargs) + + with patch.object(Path, "resolve", resolve): + assert jobs.get_ne_exec_job(JOB_ID, user_id="alice", is_admin=True)["error"] == "invalid_job_id" + + +def test_persisted_owner_is_required_and_other_user_is_denied(job_dir): + write_record(job_dir) + # Reader loads the on-disk record on each call; no in-memory owner fallback. + assert jobs.get_ne_exec_job(JOB_ID, user_id="alice")["result"]["output"] == "temporary-test-output" + denied = jobs.get_ne_exec_job(JOB_ID, user_id="bob") + assert denied["error"] == "job_forbidden" + assert "result" not in denied and "owner_user_id" not in denied + assert jobs.get_ne_exec_job(JOB_ID, user_id="admin", is_admin=True)["ok"] is True + record = json.loads((job_dir / f"{JOB_ID}.json").read_text(encoding="utf-8")) + record["owner_user_id"] = "bob" + (job_dir / f"{JOB_ID}.json").write_text(json.dumps(record), encoding="utf-8") + assert jobs.get_ne_exec_job(JOB_ID, user_id="alice")["error"] == "job_forbidden" + + +@pytest.mark.parametrize("owner", [None, "", " "]) +def test_ownerless_legacy_records_are_admin_only(job_dir, owner): + write_record(job_dir, owner=owner) + assert jobs.get_ne_exec_job(JOB_ID, user_id="alice")["error"] == "job_forbidden" + assert jobs.get_ne_exec_job(JOB_ID, user_id="admin", is_admin=True)["ok"] is True + assert jobs.get_ne_exec_job(JOB_ID, user_id="", is_admin=True)["error"] == "job_forbidden" + + +def test_record_cannot_impersonate_another_job_id(job_dir): + write_record(job_dir) + record = json.loads((job_dir / f"{JOB_ID}.json").read_text(encoding="utf-8")) + record["job_id"] = OTHER_JOB_ID + (job_dir / f"{JOB_ID}.json").write_text(json.dumps(record), encoding="utf-8") + assert jobs.get_ne_exec_job(JOB_ID, user_id="alice")["error"] == "job_not_found" + + +def test_counts_sum_mixed_batch_sources(): + args = {"targets": [{"ne_id": "a"}, {"ume_ne_id": "b"}], "ne_ids": ["c", "", None], "ume_ne_ids": ["d", "e"]} + assert jobs.count_exec_ne_targets(args) == 5 + with patch.object(jobs, "async_min_ne_count", return_value=4): + assert jobs.should_run_exec_async(args) is True + + +def test_combined_target_cap_is_enforced_before_scheduling(job_dir): + with patch.object(jobs.threading, "Thread") as thread: + out = jobs.start_ne_exec_job(kind="exec-batch", owner_user_id="alice", arguments={ + "ne_ids": [f"n{i}" for i in range(11)], "ume_ne_ids": [f"u{i}" for i in range(10)], + }, runner=lambda: pytest.fail("Oversized batch must not execute")) + assert out["error"] == "too_many_targets" + assert out["max_targets"] == 20 + assert list(job_dir.iterdir()) == [] + thread.assert_not_called() + + +def test_twenty_mixed_targets_are_accepted_and_persisted(job_dir): + ack = jobs.start_ne_exec_job(kind="exec-batch", owner_user_id="alice", arguments={ + "targets": [{"ne_id": f"t{i}"} for i in range(7)], + "ne_ids": [f"n{i}" for i in range(6)], "ume_ne_ids": [f"u{i}" for i in range(7)], + }, runner=lambda: {"ok": True, "summary": {"total": 20}, "results": []}) + assert ack["ok"] is True and ack["ne_count"] == 20 + deadline = time.monotonic() + 3 + while time.monotonic() < deadline: + done = jobs.get_ne_exec_job(ack["job_id"], user_id="alice") + if done.get("terminal"): + break + time.sleep(0.01) + assert done["status"] == "succeeded" + assert done["ne_count"] == 20 + assert done["result"]["summary"]["total"] == 20 + + +def test_job_owner_cannot_be_empty(job_dir): + with patch.object(jobs.threading, "Thread") as thread: + out = jobs.start_ne_exec_job(kind="exec", owner_user_id="", arguments={"ne_id": "n"}, runner=lambda: {}) + assert out["error"] == "job_owner_required" + assert list(job_dir.iterdir()) == [] + thread.assert_not_called() + + +@pytest.fixture +def api_client(job_dir, monkeypatch): + app = FastAPI() + app.include_router(routes.router) + auth = [AuthContext(user=SimpleNamespace(id="alice", username="alice", role="user"), auth_via="api_token", scopes=frozenset({"ne:exec"}))] + app.dependency_overrides[require_user] = lambda: auth[0] + app.dependency_overrides[get_db] = lambda: MagicMock() + monkeypatch.setattr(routes, "write_audit", lambda *args, **kwargs: None) + # The fake DB context and exec results must never open a database or device. + monkeypatch.setattr("netx_api.db.SessionLocal", MagicMock()) + monkeypatch.setattr(routes, "execute_managed_ne_commands", lambda *args, **kwargs: {"ok": True, "output": "mock-cli"}) + monkeypatch.setattr(routes, "execute_managed_ne_commands_batch", lambda *args, **kwargs: {"ok": True, "items": []}) + with TestClient(app) as client: + yield client, auth + + +def test_asgi_owner_other_user_and_admin_access(api_client, job_dir): + client, auth = api_client + write_record(job_dir) + path = f"/v1/managed-ne/exec-jobs/{JOB_ID}" + assert client.get(path).json()["result"]["output"] == "temporary-test-output" + auth[0].user.id = "bob" + assert client.get(path).status_code == 403 + auth[0].user.role = "admin" + assert client.get(path).status_code == 200 + auth[0].user.role = "user" + auth[0].scopes = frozenset({"ne:exec", "admin:users"}) + assert client.get(path).status_code == 200 + + +def test_asgi_scope_required_even_for_owner_and_admin(api_client, job_dir): + client, auth = api_client + write_record(job_dir) + auth[0].user.role = "admin" + auth[0].scopes = frozenset({"ne:read"}) + assert client.get(f"/v1/managed-ne/exec-jobs/{JOB_ID}").status_code == 403 + with patch.object(routes, "start_ne_exec_job") as start: + response = client.post("/v1/managed-ne/exec-jobs", json={"ne_id": "n", "commands": ["show version"]}) + assert response.status_code == 403 + start.assert_not_called() + + +def test_asgi_invalid_windows_path_returns_structured_error(api_client, job_dir): + client, _ = api_client + response = client.get("/v1/managed-ne/exec-jobs/" + quote(r"..\canary", safe="")) + assert response.status_code == 400 + assert response.json()["detail"]["error"] == "invalid_job_id" + assert str(job_dir) not in response.text + + +def test_asgi_create_persists_actor_and_mixed_count(api_client, job_dir): + client, auth = api_client + response = client.post("/v1/managed-ne/exec-jobs", json={ + "targets": [{"ne_id": "n1"}], "ne_ids": ["n2"], "ume_ne_ids": ["n3"], + "commands": ["show version"], "owner_user_id": "forged-owner", + }) + assert response.status_code == 200 + ack = response.json() + assert ack["ok"] is True and ack["ne_count"] == 3 + job_id = ack["job_id"] + deadline = time.monotonic() + 3 + while time.monotonic() < deadline: + done = client.get(f"/v1/managed-ne/exec-jobs/{job_id}").json() + if done.get("terminal"): + break + time.sleep(0.01) + assert done["status"] == "succeeded" + record = json.loads((job_dir / f"{job_id}.json").read_text(encoding="utf-8")) + assert record["owner_user_id"] == "alice" + assert done["ne_count"] == 3 + auth[0].user.id = "forged-owner" + assert client.get(f"/v1/managed-ne/exec-jobs/{job_id}").status_code == 403 diff --git a/tests/test_ne_netmiko_show.py b/tests/test_ne_netmiko_show.py index 5c57bb4..2847e57 100644 --- a/tests/test_ne_netmiko_show.py +++ b/tests/test_ne_netmiko_show.py @@ -5,6 +5,9 @@ from __future__ import annotations import unittest import re from unittest.mock import MagicMock, patch +from types import SimpleNamespace + +from netmiko.base_connection import BaseConnection from netx_api.ne_netmiko import drain_read_channel, send_show_command @@ -24,6 +27,85 @@ class DrainReadChannelTests(unittest.TestCase): class SendShowCommandTests(unittest.TestCase): + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_real_netmiko_cisco_reader_uses_full_prompt_after_drain(self, drain: MagicMock) -> None: + conn = BaseConnection(host="127.0.0.1", device_type="cisco_ios", auto_connect=False) + conn.base_prompt = "R2" + # Drain clears leftovers; reader then sees echo + body + full prompt. + chunks = iter(["show version\nCisco IOS Software\nR2 uptime is 15 hours\nR2#", "", ""]) + conn.channel = SimpleNamespace(read_channel=lambda: next(chunks, "")) + conn.write_channel = MagicMock() + conn.find_prompt = MagicMock() + out = send_show_command(conn, "show version", read_timeout=2) + self.assertIn("Cisco IOS Software", out) + self.assertIn("R2 uptime is 15 hours", out) + conn.write_channel.assert_called_once() + conn.find_prompt.assert_not_called() + drain.assert_called_once() + self.assertEqual(drain.call_args.kwargs.get("idle_reads"), 6) + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_cisco_does_not_probe_prompt_or_match_hostname_inside_output(self, drain: MagicMock) -> None: + conn = MagicMock() + conn.device_type = "cisco_ios_ssh" + conn.base_prompt = "R2" + body = "Cisco IOS Software\nR2 uptime is 15 hours\nSystem image file is flash:ios.bin" + + def delayed_ios_output(**kwargs: object) -> str: + if kwargs.get("auto_find_prompt", True): + return "" + if kwargs.get("cmd_verify", True): + return "" + pattern = str(kwargs.get("expect_string", "")) + if re.search(pattern, "R2 uptime is 15 hours"): + return "Cisco IOS Software" + return body + + conn.send_command.side_effect = delayed_ios_output + self.assertEqual(send_show_command(conn, "show version", read_timeout=60), body) + conn.send_command.assert_called_once() + conn.find_prompt.assert_not_called() + conn.send_command_timing.assert_not_called() + self.assertFalse(conn.send_command.call_args.kwargs["cmd_verify"]) + drain.assert_called_once() + self.assertEqual(drain.call_args.kwargs.get("idle_reads"), 6) + pattern = conn.send_command.call_args.kwargs["expect_string"] + for prompt in ("R2#", "R2>", "R2# "): + self.assertIsNotNone(re.search(pattern, prompt)) + for content in ("R2", "R2 uptime is 15 hours", "R2#show version", "prefix R2#"): + self.assertIsNone(re.search(pattern, content)) + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_cisco_prompt_hostname_is_literal(self, drain: MagicMock) -> None: + conn = MagicMock() + conn.device_type = "cisco_ios_telnet" + conn.base_prompt = "edge-1.lab" + conn.send_command.return_value = "interfaces" + self.assertEqual(send_show_command(conn, "show ip interface brief"), "interfaces") + pattern = conn.send_command.call_args.kwargs["expect_string"] + self.assertIsNotNone(re.search(pattern, "edge-1.lab#")) + self.assertIsNone(re.search(pattern, "edge-1Xlab#")) + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_cisco_timeout_does_not_resend(self, drain: MagicMock) -> None: + conn = MagicMock() + conn.device_type = "cisco_ios" + conn.base_prompt = "R2" + conn.send_command.side_effect = TimeoutError("late show output") + with self.assertRaises(TimeoutError): + send_show_command(conn, "show version") + conn.send_command.assert_called_once() + conn.send_command_timing.assert_not_called() + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_cisco_missing_cached_prompt_fails_before_sending(self, drain: MagicMock) -> None: + conn = MagicMock() + conn.device_type = "cisco_ios" + conn.base_prompt = "" + with self.assertRaisesRegex(ValueError, "cisco_base_prompt_missing"): + send_show_command(conn, "show version") + conn.send_command.assert_not_called() + @patch("netx_api.ne_netmiko.drain_read_channel") def test_mikrotik_does_not_probe_prompt_after_drain(self, drain: MagicMock) -> None: conn = MagicMock() @@ -88,6 +170,17 @@ class SendShowCommandTests(unittest.TestCase): conn.send_command_timing.assert_not_called() drain.assert_called_once_with(conn) + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_cisco_empty_output_raises_without_resend(self, drain: MagicMock) -> None: + conn = MagicMock() + conn.device_type = "cisco_ios_ssh" + conn.base_prompt = "R2" + conn.send_command.return_value = "\n" + with self.assertRaisesRegex(ValueError, "cisco_empty_show_output"): + send_show_command(conn, "show version", read_timeout=30) + conn.send_command.assert_called_once() + conn.send_command_timing.assert_not_called() + if __name__ == "__main__": unittest.main() diff --git a/tests/test_ne_session_interactive.py b/tests/test_ne_session_interactive.py index f31296f..fabe6c4 100644 --- a/tests/test_ne_session_interactive.py +++ b/tests/test_ne_session_interactive.py @@ -1,9 +1,13 @@ from __future__ import annotations import unittest +import re from unittest.mock import MagicMock, patch +from types import SimpleNamespace -from netx_api.ne_session_connect import _collection_driver_class, _zte_collection_driver_class +from netmiko.base_connection import BaseConnection + +from netx_api.ne_session_connect import _collection_driver_class, _zte_collection_driver_class, _cisco_ios_collection_driver_class, _base_connect_kwargs from netx_api.ne_session_factory import ( _build_netmiko_connection, _interactive_driver_class, @@ -50,6 +54,74 @@ class InteractiveNetmikoTests(unittest.TestCase): self.assertTrue(direct.call_args.kwargs.get("interactive")) +class CiscoCollectionPrepTests(unittest.TestCase): + def test_only_cisco_ssh_uses_cr_enter_on_the_actual_driver(self): + for device_type in ("cisco_ios", "cisco_ios_ssh", "cisco_ios_telnet", "huawei", "linux_ssh", "mikrotik_routeros"): + with self.subTest(device_type=device_type): + kwargs = _base_connect_kwargs(device_type=device_type, host="127.0.0.1", port=22, + username="fixture", password="fixture", enable_secret="") + if device_type in ("cisco_ios", "cisco_ios_ssh"): + self.assertEqual(kwargs["default_enter"], "\r") + driver = BaseConnection(**kwargs, auto_connect=False) + self.assertEqual(driver.normalize_cmd("show version"), "show version\r") + else: + self.assertNotIn("default_enter", kwargs) + + def session(self): + cls = _cisco_ios_collection_driver_class(_netmiko_driver_class("cisco_ios")) + sess = cls.__new__(cls) + sess.base_prompt = "R2.lab" + sess.RETURN = "\r" + sess._test_channel_read = MagicMock() # type: ignore[method-assign] + sess.set_base_prompt = MagicMock() # type: ignore[method-assign] + sess.write_channel = MagicMock() # type: ignore[method-assign] + sess.read_until_pattern = MagicMock() # type: ignore[method-assign] + sess.set_terminal_width = MagicMock() # type: ignore[method-assign] + sess.disable_paging = MagicMock() # type: ignore[method-assign] + sess.send_command_timing = MagicMock() # type: ignore[method-assign] + return sess + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_setup_only_establishes_hostname_prompt(self, drain: MagicMock): + sess = self.session() + order: list[str] = [] + drain.side_effect = lambda *_a, **_k: order.append("drain") + sess.set_base_prompt.side_effect = lambda **kw: order.append("prompt") + sess.session_preparation() + self.assertEqual(order, ["prompt", "drain"]) + sess.set_base_prompt.assert_called_once() + initial = sess._test_channel_read.call_args.kwargs["pattern"] + self.assertIsNotNone(re.search(initial, "R2.lab#")) + self.assertIsNone(re.search(initial, "R2.lab#terminal length 0")) + channel = SimpleNamespace(read_timeout_override=None, _read_buffer="", read_channel=lambda: "R2.lab#") + self.assertEqual(BaseConnection.read_until_pattern(channel, pattern=initial), "R2.lab#") + sess.set_terminal_width.assert_not_called() + sess.disable_paging.assert_not_called() + sess.write_channel.assert_not_called() + sess.send_command_timing.assert_not_called() + drain.assert_called_once() + self.assertEqual(drain.call_args.kwargs.get("idle_reads"), 8) + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_setup_timeout_stops_before_drain(self, drain: MagicMock): + sess = self.session() + sess.set_base_prompt.side_effect = TimeoutError("late prompt") + with self.assertRaises(TimeoutError): + sess.session_preparation() + drain.assert_not_called() + sess.write_channel.assert_not_called() + sess.send_command_timing.assert_not_called() + + @patch("netx_api.ne_netmiko.drain_read_channel") + def test_setup_never_sends_terminal_commands(self, drain: MagicMock): + sess = self.session() + sess.session_preparation() + sess.write_channel.assert_not_called() + sess.set_terminal_width.assert_not_called() + sess.disable_paging.assert_not_called() + drain.assert_called_once_with(sess, idle_reads=8, pause_sec=0.1) + + class ZteCollectionPrepTests(unittest.TestCase): def test_collection_driver_selects_zte_wrapper(self) -> None: base = _netmiko_driver_class("zte_zxros_ssh") diff --git a/tests/test_ume_hierarchy.py b/tests/test_ume_hierarchy.py index 704d662..5f546a5 100644 --- a/tests/test_ume_hierarchy.py +++ b/tests/test_ume_hierarchy.py @@ -10,7 +10,7 @@ from sqlalchemy.orm import sessionmaker import netx_api.models # noqa: F401 from netx_api.db import Base -from netx_api.models import TopoFabricNode, TopoFolder, TopoView, UmeInventoryNE, UmeTopoLink, UmeTopoNode +from netx_api.models import TopoFabricNode, TopoFolder, TopoView, TopoViewNode, UmeInventoryNE, UmeTopoLink, UmeTopoNode from netx_api.ume_topology_apply import apply_ume_topology_to_fabric from netx_api.ume_topology_flat_coords import recompute_flat_world_coords from netx_api.ume_topology_world import ensure_ume_world_and_sbn_folders, get_world_view @@ -217,6 +217,11 @@ class UmeHierarchyTests(unittest.TestCase): self.assertEqual(moved.x, 777.0) self.assertEqual(moved.y, 888.0) + placement = self.db.query(TopoViewNode).filter_by( + view_id=world.id, fabric_node_id=region.fabric_node_id, + ).one() + self.assertEqual((placement.x, placement.y), (777.0, 888.0)) + self.db.expire_all() g1 = get_level_view_graph(self.db, world) again = next(n for n in g1.nodes if n.fabric_node_id == region.fabric_node_id) self.assertEqual(again.x, 777.0) @@ -395,6 +400,57 @@ class UmeHierarchyTests(unittest.TestCase): self.assertEqual(moved.x, ox) self.assertEqual(moved.y, oy) + def test_flat_drag_accepts_negative_and_distant_display_coordinates(self): + from netx_api.topology_schemas import ViewNodeIn, ViewPositionsPatch + from netx_api.topology_views_graph import patch_view_positions + from netx_api.ume_topology_world import get_world_flat_view + + self._seed_tree() + apply_ume_topology_to_fabric(self.db) + ensure_ume_world_and_sbn_folders(self.db) + flat = get_world_flat_view(self.db) + initial = get_flat_view_graph(self.db, flat, lod="detail") + target = initial.nodes[0] + patch_view_positions(self.db, flat.id, ViewPositionsPatch( + positions=[ViewNodeIn(fabric_node_id=target.fabric_node_id, x=-20000, y=30000, locked=True)], + return_graph=False, + )) + placement = self.db.query(TopoViewNode).filter_by( + view_id=flat.id, fabric_node_id=target.fabric_node_id, + ).one() + self.assertEqual((placement.x, placement.y, placement.locked), (-20000, 30000, True)) + recompute_flat_world_coords(self.db) + self.db.expire_all() + graph = get_flat_view_graph(self.db, flat, lod="detail") + moved = next(n for n in graph.nodes if n.fabric_node_id == target.fabric_node_id) + self.assertEqual((moved.x, moved.y, moved.locked), (-20000, 30000, True)) + # Later dragging cannot move a locked placement without explicit unlock. + result = patch_view_positions(self.db, flat.id, ViewPositionsPatch( + positions=[ViewNodeIn(fabric_node_id=target.fabric_node_id, x=10, y=20)], + return_graph=False, + )) + self.assertEqual(result.skipped_locked, 1) + self.assertEqual((placement.x, placement.y), (-20000, 30000)) + + def test_level_position_patch_does_not_add_members_from_another_level(self): + from netx_api.topology_schemas import ViewNodeIn, ViewPositionsPatch + from netx_api.topology_views_graph import patch_view_positions + + self._seed_tree() + apply_ume_topology_to_fabric(self.db) + ensure_ume_world_and_sbn_folders(self.db) + world = get_world_view(self.db) + foreign = self.db.query(TopoFabricNode).filter_by(ume_ne_id="me-a1").one() + result = patch_view_positions(self.db, world.id, ViewPositionsPatch( + positions=[ViewNodeIn(fabric_node_id=foreign.id, x=1, y=2)], + return_graph=False, + )) + self.assertEqual(result.updated, 0) + self.assertEqual(result.skipped_missing, 1) + self.assertEqual(self.db.query(TopoViewNode).filter_by( + view_id=world.id, fabric_node_id=foreign.id, + ).count(), 0) + def test_create_folder_under_ume_world_remounts_to_drill(self): from netx_api.topology_schemas import TopologyFolderCreate from netx_api.topology_views_tree import create_folder diff --git a/tests/test_webcrt.py b/tests/test_webcrt.py index 786ff23..ea9a000 100644 --- a/tests/test_webcrt.py +++ b/tests/test_webcrt.py @@ -410,6 +410,9 @@ class WebcrtServiceTests(unittest.TestCase): "password": "p", "hop_enabled": True, "hop_vendor": "huawei", + "hop_host": "192.0.2.10", + "hop_username": "hop-user", + "hop_password": "test-only-hop-password", "ip_address": "10.0.0.3", "protocol": "ssh", }, @@ -479,6 +482,7 @@ class WebcrtServiceTests(unittest.TestCase): self.assertTrue( svc._webcrt_creds_ready( { + "ip_address": "192.0.2.20", "username": "ca-oper", "password": "", "hop_enabled": True, @@ -493,6 +497,7 @@ class WebcrtServiceTests(unittest.TestCase): self.assertFalse( svc._webcrt_creds_ready( { + "ip_address": "192.0.2.20", "username": "ca-oper", "password": "", "hop_enabled": True, @@ -507,6 +512,7 @@ class WebcrtServiceTests(unittest.TestCase): self.assertFalse( svc._webcrt_creds_ready( { + "ip_address": "192.0.2.20", "username": "u", "password": "", "protocol": "ssh", @@ -517,6 +523,7 @@ class WebcrtServiceTests(unittest.TestCase): self.assertTrue( svc._webcrt_creds_ready( { + "ip_address": "192.0.2.20", "username": "", "password": "", "protocol": "telnet", diff --git a/tests/test_webcrt_audit.py b/tests/test_webcrt_audit.py index a1a548e..649bf3c 100644 --- a/tests/test_webcrt_audit.py +++ b/tests/test_webcrt_audit.py @@ -332,8 +332,8 @@ class ResolveAuditCommandsTests(unittest.TestCase): ) self.assertEqual(out, [prompt + "show ip interface brief"]) - def test_zte_real_backspace_redraw_reconstructs_full_command(self) -> None: - """Real ZTE device redraws via backspace runs (not CSI cursor moves). + def test_zte_synthetic_backspace_redraw_reconstructs_full_command(self) -> None: + """Synthetic ZTE-style replay; no missing field capture is assumed. Up-arrow recalls ``show interface brief``, left-arrows move the cursor after ``show``, then ``ip`` is inserted. The device redraws using @@ -344,8 +344,6 @@ class ResolveAuditCommandsTests(unittest.TestCase): (standard terminal behaviour) so the recalled text survives the redraw and the full edited line is reconstructed. """ - import os - from netx_api.webcrt_channel import ( _live_input_line, _stdout_has_inplace_edit, @@ -353,12 +351,13 @@ class ResolveAuditCommandsTests(unittest.TestCase): render_pty_line, ) - fixture = os.path.join(os.path.dirname(__file__), "fixtures", - "zte_history_edit_stdout.txt") - with open(fixture, encoding="utf-8") as f: - stdout = f.read() - prompt = "AL5458-ACC-6120HS#" + # Recall a command, move left without deleting, then redraw the edited suffix. + # The final command never appears as one contiguous substring in the replay. + recalled = "show interface brief" + stdout = prompt + recalled + ("\x08" * len("interface brief")) + "\x1b[Kip interface brief" + self.assertNotIn("show ip interface brief", stdout) + expected = prompt + "show ip interface brief" # render_pty_line must reconstruct the full edited command. diff --git a/tests/test_zte_extended_parsers.py b/tests/test_zte_extended_parsers.py index 91bbb30..f0194c4 100644 --- a/tests/test_zte_extended_parsers.py +++ b/tests/test_zte_extended_parsers.py @@ -270,7 +270,7 @@ class ZteExtendedParserTests(unittest.TestCase): self.assertTrue(all(r["afi"] == "ipv4" for r in vrf), "vrf-scoped vpnv4 → ipv4") # Real wrapped vpnv6 summary + IPv6 neighbor in/out routes - sample_dir = Path(__file__).resolve().parents[2] / "test" / "show-zte" + sample_dir = Path(__file__).resolve().parent / "fixtures" / "zte_vpnv6" v6_sum = (sample_dir / "show-bgp-vpnv6").read_text(encoding="utf-8", errors="replace") v6_real = normalize_bgp_peer( raw_text=v6_sum, command="show bgp vpnv6 unicast summary | one-line"