mirror of
https://github.com/hansjone/dsh-search-mcp.git
synced 2026-10-08 22:20:47 +08:00
Publish dsh-search-mcp fork for newer DSH and Bailian WebSearch MCP.
Based on gxpppp/dsh-search-mcp; includes DSH web settings fixes, Clash fake-IP URL policy, and Bailian default server patch. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
commit
d5c4d2006c
16 changed files with 4821 additions and 0 deletions
8
.gitignore
vendored
Normal file
8
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
node_modules/
|
||||
*.log
|
||||
*.tgz
|
||||
.DS_Store
|
||||
|
||||
# Local security scan state and private project handoff.
|
||||
.mimosa/
|
||||
PROJECT_CONTEXT.md
|
||||
21
LICENSE
Normal file
21
LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
183
README.md
Normal file
183
README.md
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
# dsh-search-mcp
|
||||
|
||||
用搜索类 MCP 服务器完整替代 DeepSeek Harness(DSH)内置网页搜索的独立插件。
|
||||
|
||||
> 当前兼容基线:DeepSeek Harness `0.1.1-rc.2`,Node.js 20 或更高版本。
|
||||
|
||||
## 功能
|
||||
|
||||
- 模型侧继续使用原生 `web_search`,插件只替换底层 search provider。
|
||||
- 支持 Tavily、Brave、Exa、Perplexity、DuckDuckGo 和自定义 HTTP/stdio MCP。
|
||||
- 已知 provider 只需选择服务商并填写 CDKey/API key,不需要填写 URL、命令、鉴权参数或工具名。
|
||||
- 自定义 MCP 保留 URL、stdio 命令、鉴权方式和工具名等高级配置。
|
||||
- 密钥通过 DSH credentials domain 写入;设置读取接口只返回是否已配置,不返回密钥值。
|
||||
- DSH RC2 支持一次 `web_search` 提交多个查询,默认上限为 4。
|
||||
- 卸载插件后 bundle 覆盖层随之移除,DSH 内置搜索组合恢复。
|
||||
|
||||
## 安装
|
||||
|
||||
```powershell
|
||||
git clone https://github.com/gxpppp/dsh-search-mcp.git
|
||||
cd dsh-search-mcp
|
||||
npm install
|
||||
dsh plugin --profile web add link:<dsh-search-mcp 的绝对路径>
|
||||
dsh web
|
||||
```
|
||||
|
||||
`link:` 会让源码更新直接作用于 profile。修改或升级浏览器 bundle 后需要重启 DSH Web 并刷新页面。
|
||||
|
||||
如果 profile 中已有独立搜索 MCP 行,建议先移除重复入口,避免同时暴露 `mcp__...` 工具和本插件提供的 `web_search`。
|
||||
|
||||
## 配置
|
||||
|
||||
打开:
|
||||
|
||||
**设置 → 插件 → 插件配置 → 搜索 MCP**
|
||||
|
||||
### 已知 provider
|
||||
|
||||
1. 点击 Tavily、Brave、Exa、Perplexity 或 DuckDuckGo 快捷按钮。
|
||||
2. 展开服务器行。
|
||||
3. 对需要凭据的 provider 填写 CDKey/API key,然后保存。
|
||||
4. DuckDuckGo 无需 key。
|
||||
|
||||
已知 provider 的 endpoint、transport、鉴权方式、工具名和结果参数由 Host catalog 固定管理,设置页不会自动填入或显示链接。保存 CDKey 后,客户端先调用 `credentials.set`,再将生成的 credential reference 写入服务器设置;密钥本身不会写回普通 settings 字段。
|
||||
|
||||
也可以预先在 `$DSH_HOME/.credentials.yaml` 中保存凭据,再在设置页填写引用名:
|
||||
|
||||
```yaml
|
||||
TAVILY_API_KEY: <your-key>
|
||||
EXA_API_KEY: <your-key>
|
||||
PERPLEXITY_API_KEY: <your-key>
|
||||
BRAVE_API_KEY: <your-key>
|
||||
```
|
||||
|
||||
RC2 的 `credentials/reference-updated` 事件会刷新设置卡片中的“已配置/未配置”状态,但不会传输密钥值。卡片按 RC2 每批最多 64 个引用的限制分批读取状态。保存多个字段失败时会逆序恢复已写入的 settings,并清理本次新建的 credential reference;由于 RC2 不允许读回已有密钥,覆盖一个此前已配置的引用后无法跨 credentials/settings 做值级回滚。
|
||||
|
||||
### 自定义 MCP
|
||||
|
||||
添加 `custom` 服务器后,可配置:
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `id` | 服务器唯一标识,供 `defaultServer` 引用 |
|
||||
| `transport` | `http`(Streamable HTTP)或 `stdio` |
|
||||
| `url` | 仅 HTTP 自定义 MCP 使用 |
|
||||
| `command` / `args` | 仅 stdio 自定义 MCP 使用 |
|
||||
| `apiKey` | 写入方向的 CDKey/API key 输入 |
|
||||
| `apiKeyEnv` | 环境变量或 DSH credential reference |
|
||||
| `authStyle` | HTTP 的 `query` 或 `header` |
|
||||
| `authParam` | query/header 参数名;stdio 下作为环境变量名 |
|
||||
| `toolName` | MCP 搜索工具名称 |
|
||||
| `maxResults` | 单服务器结果数覆盖 |
|
||||
|
||||
### 全局选项
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `defaultServer` | 默认服务器 id;留空时使用第一行 |
|
||||
| `maxResults` | 全局结果数上限,默认 8,可选 1–50 |
|
||||
| `searchTimeoutMs` | 每次 MCP 搜索超时,默认 30000 ms;界面以秒显示 |
|
||||
|
||||
## Provider 预设
|
||||
|
||||
| kind | Host 管理的连接 | 凭据 | 搜索工具 | 结果数参数 |
|
||||
|---|---|---|---|---|
|
||||
| `tavily` | hosted Streamable HTTP | CDKey/API key | `tavily_search` | `max_results` |
|
||||
| `brave` | `@brave/brave-search-mcp-server@2.1.3` stdio | `BRAVE_API_KEY` | `brave_web_search` | `count` |
|
||||
| `exa` | hosted Streamable HTTP | `x-api-key` | `web_search_exa` | `numResults` |
|
||||
| `perplexity` | hosted Streamable HTTP | Bearer token | `perplexity_search` | `max_results` |
|
||||
| `duckduckgo` | `duckduckgo-mcp-server@0.1.2` stdio | 无 | `duckduckgo_web_search` | `count` |
|
||||
| `custom` | 用户配置 | 用户配置 | 用户配置 | 无预设 |
|
||||
|
||||
旧配置中的 known-provider URL、transport、auth 和 tool 字段仍可被 schema 读取,但运行时会忽略它们;下一次保存服务器列表时会清理这些冗余字段。只有 `custom` 使用用户提供的连接信息。
|
||||
|
||||
插件在调用 known provider 前会把结果数限制到上游 MCP schema 接受的范围:Tavily 为 5–20,Brave、Perplexity 和 DuckDuckGo 为 1–20;Exa 当前保留插件的 1–50 范围。该限制只影响传给上游的参数,最终返回数量仍会受到插件全局/单服务器限制和实际 agent preset 的 `tool-web.searchMaxResults` 共同约束。
|
||||
|
||||
## DSH 0.1.1-rc.2 适配
|
||||
|
||||
- DSH host 依赖精确锁定为 `0.1.1-rc.2`,不使用可能落到旧版本线的子包 `latest`。
|
||||
- 设置卡片继续使用 keyed slot:`settings.plugin.item` + `key: "search-mcp"`。
|
||||
- 新密钥通过 `credentials.set` 单向写入,凭据状态通过 `credentials.describe` 读取。
|
||||
- 监听 RC2 的 `credentials/reference-updated`,外部凭据变更后刷新状态 badge。
|
||||
- RC6/RC7 遗留的字面 `apiKey` 仍可由 Host 使用;涉及服务器数组的编辑会阻止不可见旧密钥被意外删除,并要求先迁移。
|
||||
- 普通全局字段修改不会重写 `servers` 数组。
|
||||
- `tool-web.searchMaxQueries` 配置为 4,与 RC2 默认多查询能力一致。
|
||||
|
||||
## URL 安全策略
|
||||
|
||||
所有自定义 HTTP MCP 请求在联网前执行安全校验:
|
||||
|
||||
- 只允许 `http:` 和 `https:`,拒绝 userinfo 与非规范 IPv4 表示。
|
||||
- 拒绝 localhost、环回、RFC1918 私网、链路本地、CGNAT、benchmark、文档/测试、多播、保留和广播地址。
|
||||
- IPv4-mapped IPv6 先映射为 IPv4 再判断;IPv4-compatible IPv6、IPv6 ULA、link-local、NAT64/转换、Teredo、6to4、文档和保留范围同样拒绝。
|
||||
- 域名会解析全部 A/AAAA 结果;任意一个结果不公开可路由时整体拒绝,DNS 等待也受同一个搜索 AbortSignal/超时约束。
|
||||
- 每次搜索使用独占 Undici Agent 和预解析地址的 pinned lookup,同时保留原始 Host 与 TLS SNI,防止 DNS rebinding。
|
||||
- GET、POST、DELETE 和 SSE 重连都通过同一 fetch wrapper,HTTP 重定向设置为 `error`。
|
||||
- 先关闭 MCP client,再关闭本次 Agent,不共享连接池。
|
||||
- 错误信息不会输出包含 CDKey 的完整 URL。
|
||||
|
||||
如果代理或 TUN 把公共域名解析到 `198.18.0.0/15` fake-IP,本插件会按 benchmark/test 网段安全拒绝。应让 DSH 进程获得真实公网 DNS 结果,而不是放宽策略。
|
||||
|
||||
## 组合覆盖
|
||||
|
||||
插件通过 `cordis.patch.yml`:
|
||||
|
||||
- 注册 `search-mcp` provider。
|
||||
- 设置 `web.searchProvider: search-mcp`。
|
||||
- 禁用 `web-search-deepseek`。
|
||||
- 保持 `web_fetch` 关闭。
|
||||
- 请求 `tool-web.searchMaxResults: 50` 和 `searchMaxQueries: 4`。
|
||||
|
||||
RC2 的 standard、code、cordis agent preset 各自包含 `tool-web` 行,并且都省略了 `searchMaxResults` 和 `searchMaxQueries`,因此实际采用 `dsh-tool-web` 默认值 8 和 4。agent-scoped 工具会遮蔽根层同名工具,所以根层 patch 中的 50 条请求不会提高这些 shipped preset 的实际上限。验证结果上限时必须检查 session 使用的 preset,不能只依据根层 `--dump-config`。
|
||||
|
||||
## 验证
|
||||
|
||||
```powershell
|
||||
npm test
|
||||
npm run check
|
||||
npm pack --dry-run
|
||||
```
|
||||
|
||||
自动测试覆盖 RC2 依赖锁定、known/custom catalog 边界、CDKey-only 设置结构、凭据事件、旧 secret 保护、结果归一化,以及 URL/DNS/pinning 安全策略。
|
||||
|
||||
2026-08-30 的隔离 RC2 Web 冒烟检查确认:插件卡片可加载;默认 Tavily 行不显示链接或高级连接字段;DuckDuckGo 摘要显示“无需密钥”,展开后只有 ID、provider 和结果数;测试草稿已放弃且没有写入 settings。无密钥 DuckDuckGo stdio server 能启动并收到正确的 `duckduckgo_web_search`/`count` 调用,但当次公开搜索被 DuckDuckGo 上游异常流量检测拒绝,因此未取得可用于结果归一化验收的真实来源。
|
||||
|
||||
组合检查:
|
||||
|
||||
```powershell
|
||||
dsh --profile web --dump-config |
|
||||
Select-String -Pattern "searchProvider|search-mcp|web-search-deepseek|searchMaxResults|searchMaxQueries"
|
||||
```
|
||||
|
||||
预期至少包括:
|
||||
|
||||
- `web.searchProvider: search-mcp`
|
||||
- `web-search-deepseek.disabled: true`
|
||||
- `tool-web.disabled: false`
|
||||
- `fetch: false`
|
||||
|
||||
实际 agent preset 的结果数和多查询上限应在隔离 profile/session 中单独验证。
|
||||
|
||||
## 故障排查
|
||||
|
||||
- `no search MCP servers configured`:在设置页添加 provider。
|
||||
- `has no API key`:填写 CDKey/API key,或填写已有 credential reference。
|
||||
- “凭证未配置”:引用名存在于 settings,但 credentials provider 当前找不到对应值。
|
||||
- `defaultServer "x" is not configured`:默认 id 没有匹配任何服务器行。
|
||||
- `URL policy` 拒绝:endpoint 非 HTTP(S),或 DNS 结果包含本地、私有、保留/测试地址。
|
||||
- stdio 启动失败:确认 Node/npm 可用,且运行环境允许 `npx` 获取或执行对应 MCP 包。
|
||||
- 设置页没有 Search MCP 卡片:确认 client bundle 已安装,重启 DSH Web 后强制刷新页面。
|
||||
- 返回结果仍被截断:检查实际 agent preset 中的 `tool-web.searchMaxResults`,以及全局/单服务器 `maxResults`。
|
||||
|
||||
## 卸载
|
||||
|
||||
```powershell
|
||||
dsh plugin --profile web remove dsh-search-mcp
|
||||
```
|
||||
|
||||
随后重启 DSH Web。不要只禁用 `search-mcp` 行,因为 bundle 还覆盖了 `web`、`web-search-deepseek` 和 `tool-web`;完整卸载 bundle 才会恢复内置组合。
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
58
cordis.patch.yml
Normal file
58
cordis.patch.yml
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
# dsh-search-mcp bundle layer: applied after dsh-base and dsh-web-app,
|
||||
# before the profile's own cordis.patch.yml (last write wins per row).
|
||||
#
|
||||
# Installing this package therefore REPLACES dsh's built-in web search:
|
||||
# - the `web` row's searchProvider switches from `deepseek-official` to
|
||||
# `search-mcp` (the provider registered by this plugin), and
|
||||
# - the built-in DeepSeek search provider row is disabled.
|
||||
# The model-facing `web_search` tool keeps its name and presentation; its
|
||||
# execution now goes through the search MCP server(s) configured below or in
|
||||
# the web Settings → Plugins → search-mcp section.
|
||||
#
|
||||
# Removing this package (dsh plugin --profile web remove dsh-search-mcp)
|
||||
# drops this whole layer and restores the built-in search exactly.
|
||||
#
|
||||
# SECURITY: no API keys are committed to this repository. Server keys are
|
||||
# supplied at runtime through `apiKeyEnv` — stored in
|
||||
# `$DSH_HOME/.credentials.yaml` (e.g. `TAVILY_API_KEY: <key>`) — or through
|
||||
# the web Settings → Plugins → search-mcp section (`apiKey` field).
|
||||
|
||||
- insert:
|
||||
- id: search-mcp
|
||||
name: 'dsh-search-mcp'
|
||||
config:
|
||||
defaultServer: bailian
|
||||
maxResults: 8
|
||||
searchTimeoutMs: 30000
|
||||
servers:
|
||||
- id: bailian
|
||||
kind: custom
|
||||
transport: http
|
||||
url: https://dashscope.aliyuncs.com/api/v1/mcps/WebSearch/mcp
|
||||
authStyle: header
|
||||
authParam: Authorization
|
||||
authPrefix: 'Bearer '
|
||||
apiKeyEnv: DASHSCOPE_API_KEY
|
||||
toolName: bailian_web_search
|
||||
|
||||
- id: web
|
||||
config:
|
||||
searchProvider: search-mcp
|
||||
|
||||
- id: web-search-deepseek
|
||||
disabled: true
|
||||
|
||||
# The model-facing web_search tool is owned by dsh-tool-web (its `Config`
|
||||
# default is a hard 8-source cap, enforced by dsh-web's seam on EVERY
|
||||
# request). This plugin takes over result sizing, so raise the tool layer's
|
||||
# cap to the plugin schema maximum: `search-mcp`'s own maxResults (Settings →
|
||||
# Plugins → search-mcp) then decides how many sources actually come back.
|
||||
# `fetch` stays disabled and the base timeout is restated, because a patch
|
||||
# replaces the targeted row's whole config.
|
||||
- id: tool-web
|
||||
disabled: false
|
||||
config:
|
||||
fetch: false
|
||||
searchTimeoutMs: 60000
|
||||
searchMaxResults: 50
|
||||
searchMaxQueries: 4
|
||||
137
lib/catalog.js
Normal file
137
lib/catalog.js
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
/**
|
||||
* Search-MCP provider catalog.
|
||||
*
|
||||
* Known providers are intentionally connection-opaque to Settings clients: the
|
||||
* host owns their endpoint, transport, authentication contract and tool name.
|
||||
* `custom` is the only kind whose connection details come from the user.
|
||||
*/
|
||||
export const SEARCH_MCP_CATALOG = {
|
||||
tavily: {
|
||||
transport: 'http',
|
||||
url: 'https://mcp.tavily.com/mcp/',
|
||||
authStyle: 'query',
|
||||
authParam: 'tavilyApiKey',
|
||||
toolName: 'tavily_search',
|
||||
countArg: 'max_results',
|
||||
minResults: 5,
|
||||
maxResultsLimit: 20,
|
||||
apiKeyEnv: 'TAVILY_API_KEY',
|
||||
needsKey: true,
|
||||
},
|
||||
brave: {
|
||||
transport: 'stdio',
|
||||
command: 'npx',
|
||||
args: ['-y', '@brave/brave-search-mcp-server@2.1.3'],
|
||||
authStyle: 'env',
|
||||
authParam: 'BRAVE_API_KEY',
|
||||
toolName: 'brave_web_search',
|
||||
countArg: 'count',
|
||||
minResults: 1,
|
||||
maxResultsLimit: 20,
|
||||
apiKeyEnv: 'BRAVE_API_KEY',
|
||||
needsKey: true,
|
||||
},
|
||||
exa: {
|
||||
transport: 'http',
|
||||
url: 'https://mcp.exa.ai/mcp',
|
||||
authStyle: 'header',
|
||||
authParam: 'x-api-key',
|
||||
toolName: 'web_search_exa',
|
||||
countArg: 'numResults',
|
||||
apiKeyEnv: 'EXA_API_KEY',
|
||||
needsKey: true,
|
||||
},
|
||||
perplexity: {
|
||||
transport: 'http',
|
||||
url: 'https://api.perplexity.ai/mcp',
|
||||
authStyle: 'header',
|
||||
authParam: 'Authorization',
|
||||
authPrefix: 'Bearer ',
|
||||
toolName: 'perplexity_search',
|
||||
countArg: 'max_results',
|
||||
minResults: 1,
|
||||
maxResultsLimit: 20,
|
||||
apiKeyEnv: 'PERPLEXITY_API_KEY',
|
||||
needsKey: true,
|
||||
},
|
||||
duckduckgo: {
|
||||
transport: 'stdio',
|
||||
command: 'npx',
|
||||
args: ['-y', 'duckduckgo-mcp-server@0.1.2'],
|
||||
authStyle: 'env',
|
||||
authParam: '',
|
||||
toolName: 'duckduckgo_web_search',
|
||||
countArg: 'count',
|
||||
minResults: 1,
|
||||
maxResultsLimit: 20,
|
||||
needsKey: false,
|
||||
},
|
||||
custom: {
|
||||
transport: 'http',
|
||||
url: '',
|
||||
authStyle: 'query',
|
||||
authParam: '',
|
||||
authPrefix: '',
|
||||
toolName: '',
|
||||
countArg: '',
|
||||
needsKey: false,
|
||||
},
|
||||
};
|
||||
|
||||
/** The provider ids offered in the settings UI. */
|
||||
export const KNOWN_KINDS = Object.keys(SEARCH_MCP_CATALOG);
|
||||
|
||||
export function clampSearchResults(server, value) {
|
||||
const minimum = server.minResults ?? 1;
|
||||
const maximum = server.maxResultsLimit ?? value;
|
||||
return Math.min(maximum, Math.max(minimum, value));
|
||||
}
|
||||
|
||||
/** Resolve a stored entry without allowing known-provider connection overrides. */
|
||||
export function resolveServer(server) {
|
||||
const kind = typeof server?.kind === 'string' && Object.hasOwn(SEARCH_MCP_CATALOG, server.kind)
|
||||
? server.kind
|
||||
: 'custom';
|
||||
const preset = SEARCH_MCP_CATALOG[kind];
|
||||
if (kind !== 'custom') {
|
||||
return {
|
||||
id: typeof server.id === 'string' ? server.id : '',
|
||||
kind,
|
||||
apiKey: typeof server.apiKey === 'string' ? server.apiKey : undefined,
|
||||
apiKeyEnv: typeof server.apiKeyEnv === 'string' ? server.apiKeyEnv : '',
|
||||
maxResults: server.maxResults,
|
||||
transport: preset.transport,
|
||||
url: preset.url ?? '',
|
||||
command: preset.command ?? '',
|
||||
args: [...(preset.args ?? [])],
|
||||
authStyle: preset.authStyle ?? '',
|
||||
authParam: preset.authParam ?? '',
|
||||
authPrefix: preset.authPrefix ?? '',
|
||||
toolName: preset.toolName ?? '',
|
||||
countArg: preset.countArg ?? '',
|
||||
minResults: preset.minResults,
|
||||
maxResultsLimit: preset.maxResultsLimit,
|
||||
needsKey: preset.needsKey ?? false,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
id: typeof server.id === 'string' ? server.id : '',
|
||||
kind: 'custom',
|
||||
apiKey: typeof server.apiKey === 'string' ? server.apiKey : undefined,
|
||||
apiKeyEnv: typeof server.apiKeyEnv === 'string' ? server.apiKeyEnv : '',
|
||||
maxResults: server.maxResults,
|
||||
transport: server.transport || preset.transport || 'http',
|
||||
url: server.url || preset.url || '',
|
||||
command: server.command || preset.command || '',
|
||||
args: Array.isArray(server.args) ? [...server.args] : [...(preset.args ?? [])],
|
||||
authStyle: server.authStyle || preset.authStyle || '',
|
||||
authParam: server.authParam || preset.authParam || '',
|
||||
authPrefix: server.authPrefix || preset.authPrefix || '',
|
||||
toolName: server.toolName || preset.toolName || '',
|
||||
countArg: preset.countArg || '',
|
||||
minResults: preset.minResults,
|
||||
maxResultsLimit: preset.maxResultsLimit,
|
||||
needsKey: preset.needsKey ?? false,
|
||||
};
|
||||
}
|
||||
1077
lib/client.browser.js
Normal file
1077
lib/client.browser.js
Normal file
File diff suppressed because it is too large
Load diff
150
lib/client.js
Normal file
150
lib/client.js
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
/** MCP transport layer for one search. */
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
|
||||
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import { WebError } from '@deepseek-ai/dsh-web';
|
||||
import { Agent, fetch as undiciFetch } from 'undici';
|
||||
import { clampSearchResults } from './catalog.js';
|
||||
import { validateHttpEndpoint } from './url-policy.js';
|
||||
|
||||
/** Run one search through a resolved server entry. */
|
||||
export async function callMcpSearch(server, key, args, signal) {
|
||||
if (!server.toolName) {
|
||||
throw new WebError(
|
||||
`search-mcp server "${server.id}": no MCP tool name (set "toolName" or pick a known kind)`,
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
}
|
||||
|
||||
let runtime;
|
||||
const client = new Client({ name: 'dsh-search-mcp', version: '0.2.0' }, { capabilities: {} });
|
||||
try {
|
||||
runtime = server.transport === 'stdio'
|
||||
? { transport: stdioTransport(server, key), close: async () => {} }
|
||||
: await httpRuntime(server, key, signal);
|
||||
await race(client.connect(runtime.transport), signal, `connect to "${server.id}"`);
|
||||
const callArgs = { query: args.query };
|
||||
if (server.countArg.length > 0 && args.maxResults !== undefined) {
|
||||
callArgs[server.countArg] = clampSearchResults(server, args.maxResults);
|
||||
}
|
||||
const result = await race(
|
||||
client.callTool({ name: server.toolName, arguments: callArgs }),
|
||||
signal,
|
||||
`call "${server.id}" tool "${server.toolName}"`,
|
||||
);
|
||||
if (result.isError) {
|
||||
throw new WebError(
|
||||
`search-mcp: MCP server "${server.id}" tool "${server.toolName}" reported an error`,
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
if (error instanceof WebError) throw error;
|
||||
if (signal?.aborted) throw aborted(`complete request for "${server.id}"`);
|
||||
const detail = error?.name === 'SearchMcpUrlPolicyError' ? `: ${error.message}` : '';
|
||||
throw new WebError(
|
||||
`search-mcp server "${server.id}" request failed${detail}`,
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
await client.close();
|
||||
} catch {
|
||||
// The connection is already gone.
|
||||
}
|
||||
try {
|
||||
await runtime?.close();
|
||||
} catch {
|
||||
// The dedicated dispatcher has no shared state to recover.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Build a DNS-pinned streamable-http transport and its cleanup. */
|
||||
async function httpRuntime(server, key, signal) {
|
||||
const validated = await validateHttpEndpoint(server.url, { signal });
|
||||
const url = new URL(validated.url);
|
||||
const headers = {};
|
||||
if (key !== undefined && key.length > 0 && server.authParam.length > 0) {
|
||||
const value = `${server.authPrefix ?? ''}${key}`;
|
||||
if (server.authStyle === 'query') url.searchParams.set(server.authParam, value);
|
||||
else if (server.authStyle === 'header') headers[server.authParam] = value;
|
||||
}
|
||||
|
||||
const agent = new Agent({
|
||||
connect: { lookup: validated.lookup },
|
||||
connections: validated.addresses.length,
|
||||
maxRedirections: 0,
|
||||
});
|
||||
const expectedOrigin = url.origin;
|
||||
const secureFetch = async (input, init = {}) => {
|
||||
const requestUrl = new URL(typeof input === 'string' || input instanceof URL ? input : input.url);
|
||||
if (requestUrl.origin !== expectedOrigin) {
|
||||
throw new Error('search-mcp URL policy: request origin changed after validation');
|
||||
}
|
||||
return undiciFetch(input, {
|
||||
...init,
|
||||
dispatcher: agent,
|
||||
redirect: 'error',
|
||||
...(signal !== undefined ? { signal: combineSignals(signal, init.signal) } : {}),
|
||||
});
|
||||
};
|
||||
|
||||
return {
|
||||
transport: new StreamableHTTPClientTransport(url, {
|
||||
fetch: secureFetch,
|
||||
requestInit: {
|
||||
headers,
|
||||
redirect: 'error',
|
||||
...(signal !== undefined ? { signal } : {}),
|
||||
},
|
||||
}),
|
||||
close: () => agent.close(),
|
||||
};
|
||||
}
|
||||
|
||||
/** Build a stdio transport; the authParam name doubles as the env var name. */
|
||||
function stdioTransport(server, key) {
|
||||
const env = { ...process.env };
|
||||
if (key !== undefined && key.length > 0 && server.authParam.length > 0) {
|
||||
env[server.authParam] = `${server.authPrefix ?? ''}${key}`;
|
||||
}
|
||||
return new StdioClientTransport({
|
||||
command: server.command,
|
||||
args: server.args ?? [],
|
||||
env,
|
||||
});
|
||||
}
|
||||
|
||||
/** Race a protocol operation against the caller/timeout abort signal. */
|
||||
function race(promise, signal, stage) {
|
||||
if (signal === undefined) return promise;
|
||||
if (signal.aborted) throw aborted(stage);
|
||||
return new Promise((resolve, reject) => {
|
||||
const onAbort = () => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
reject(aborted(stage));
|
||||
};
|
||||
signal.addEventListener('abort', onAbort, { once: true });
|
||||
promise.then(
|
||||
(value) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
resolve(value);
|
||||
},
|
||||
(error) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
reject(error);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function combineSignals(base, request) {
|
||||
if (request === undefined || request === null || request === base) return base;
|
||||
return AbortSignal.any([base, request]);
|
||||
}
|
||||
|
||||
function aborted(stage) {
|
||||
return new WebError(`search-mcp: aborted while trying to ${stage}`, 'WEB_ABORTED');
|
||||
}
|
||||
117
lib/extract.js
Normal file
117
lib/extract.js
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
/**
|
||||
* Generic normalization of an MCP `tools/call` result into the
|
||||
* `web_search` provider shape `{ sources, truncated, content? }`.
|
||||
*
|
||||
* Different search MCP servers return wildly different payloads (Tavily
|
||||
* `results[]`, Brave `web.results[]`, Exa `results[]`, Perplexity text +
|
||||
* citations, DuckDuckGo `results[]`...). Instead of mapping each vendor, we
|
||||
* recursively walk the returned JSON and collect every object that carries a
|
||||
* string `url` as a source, taking title / snippet / date from the common
|
||||
* field names. A top-level `answer` (or non-JSON text blocks) becomes the
|
||||
* `content` answer.
|
||||
*/
|
||||
|
||||
const TITLE_KEYS = ['title', 'name', 'headline'];
|
||||
const SNIPPET_KEYS = ['snippet', 'content', 'description', 'text', 'excerpt', 'summary'];
|
||||
const DATE_KEYS = [
|
||||
'published_date',
|
||||
'publishedDate',
|
||||
'published_at',
|
||||
'publish_date',
|
||||
'publishedAt',
|
||||
'page_age',
|
||||
'age',
|
||||
'date',
|
||||
];
|
||||
|
||||
/** Cap a snippet so a single source cannot blow up the context window. */
|
||||
const MAX_SNIPPET_CHARS = 600;
|
||||
/** Cap the answer text block. */
|
||||
const MAX_CONTENT_CHARS = 4000;
|
||||
|
||||
/**
|
||||
* Project one MCP `tools/call` result into `{ sources, truncated, content? }`.
|
||||
*
|
||||
* @param result - the raw `CallToolResult` from the MCP SDK.
|
||||
* @returns the normalized provider result; `truncated` is always false
|
||||
* because the `ctx.web` seam owns the final `maxResults` cap.
|
||||
*/
|
||||
export function extractSearchResult(result) {
|
||||
const bucket = {
|
||||
sources: [],
|
||||
seen: new Set(),
|
||||
content: '',
|
||||
};
|
||||
if (result !== null && typeof result === 'object') {
|
||||
if (result.structuredContent !== undefined) collect(result.structuredContent, bucket);
|
||||
const blocks = Array.isArray(result.content) ? result.content : [];
|
||||
for (const block of blocks) {
|
||||
if (block === null || typeof block !== 'object') continue;
|
||||
if (block.type === 'json' && block.json !== undefined) {
|
||||
collect(block.json, bucket);
|
||||
} else if (block.type === 'text' && typeof block.text === 'string') {
|
||||
const parsed = tryParseJson(block.text);
|
||||
if (parsed !== undefined) collect(parsed, bucket);
|
||||
else if (bucket.content.length === 0 && block.text.trim().length > 0) {
|
||||
bucket.content = block.text.trim().slice(0, MAX_CONTENT_CHARS);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
sources: bucket.sources,
|
||||
truncated: false,
|
||||
...(bucket.content.length > 0 ? { content: bucket.content } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Depth-first walk collecting source objects and the `answer` field. */
|
||||
function collect(node, bucket) {
|
||||
if (Array.isArray(node)) {
|
||||
for (const item of node) collect(item, bucket);
|
||||
return;
|
||||
}
|
||||
if (node === null || typeof node !== 'object') return;
|
||||
if (typeof node.url === 'string' && /^https?:\/\//i.test(node.url)) {
|
||||
if (!bucket.seen.has(node.url)) {
|
||||
bucket.seen.add(node.url);
|
||||
const title = firstOf(node, TITLE_KEYS);
|
||||
const snippet = truncate(firstOf(node, SNIPPET_KEYS), MAX_SNIPPET_CHARS);
|
||||
const publishedAt = firstOf(node, DATE_KEYS);
|
||||
bucket.sources.push({
|
||||
url: node.url,
|
||||
...(title !== undefined ? { title } : {}),
|
||||
...(snippet !== undefined ? { snippet } : {}),
|
||||
...(publishedAt !== undefined ? { publishedAt } : {}),
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (bucket.content.length === 0 && typeof node.answer === 'string' && node.answer.trim().length > 0) {
|
||||
bucket.content = node.answer.trim().slice(0, MAX_CONTENT_CHARS);
|
||||
}
|
||||
for (const value of Object.values(node)) collect(value, bucket);
|
||||
}
|
||||
|
||||
/** First non-empty string among the candidate keys, else undefined. */
|
||||
function firstOf(node, keys) {
|
||||
for (const key of keys) {
|
||||
const value = node[key];
|
||||
if (typeof value === 'string' && value.trim().length > 0) return value.trim();
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function truncate(value, max) {
|
||||
if (value === undefined) return undefined;
|
||||
return value.length > max ? `${value.slice(0, max)}…` : value;
|
||||
}
|
||||
|
||||
/** Parse a JSON text block; returns undefined when it is not JSON. */
|
||||
function tryParseJson(text) {
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
112
lib/index.js
Normal file
112
lib/index.js
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
/**
|
||||
* dsh-search-mcp — replace dsh's built-in web search with search MCP servers.
|
||||
*
|
||||
* Adapted for DeepSeek Harness 0.1.2+: settings use
|
||||
* `ctx.settings.installSection` (the old free-function
|
||||
* `installSettingsSection` from 0.1.1-rc.2 no longer exists).
|
||||
*
|
||||
* A Cordis plugin that
|
||||
* - registers a `ctx.web` search provider under the id `search-mcp`, and
|
||||
* - installs a Settings section (`search-mcp`) where the user manages the
|
||||
* search MCP server list (kind, endpoint/command, API key or key env
|
||||
* reference, tool name) plus `defaultServer` / `maxResults` /
|
||||
* `searchTimeoutMs` from the web Settings → Plugins page.
|
||||
*
|
||||
* The package's `cordis.patch.yml` (bundle layer) switches
|
||||
* `web.searchProvider` to `search-mcp` and disables the built-in
|
||||
* `web-search-deepseek` provider, so while this plugin is enabled the
|
||||
* built-in search is unavailable and every `web_search` call runs through
|
||||
* the configured MCP server(s). Removing the package restores the built-in.
|
||||
*/
|
||||
import z from '@deepseek-ai/schemastery';
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials';
|
||||
import { launchEnvironmentOf } from '@deepseek-ai/dsh-launch-environment';
|
||||
import { SearchMCPProvider } from './provider.js';
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'search-mcp';
|
||||
|
||||
/** The web seam this provider registers into. */
|
||||
export const inject = ['web'];
|
||||
|
||||
const serverSchema = z.object({
|
||||
id: z.string(),
|
||||
kind: z.string().default('custom'),
|
||||
transport: z.string().default('http'),
|
||||
url: z.string().default(''),
|
||||
command: z.string().default(''),
|
||||
args: z.array(z.string()).default([]),
|
||||
apiKey: z.string().role('secret'),
|
||||
apiKeyEnv: z.string().role('credential-ref').default(''),
|
||||
authStyle: z.string().default(''),
|
||||
authParam: z.string().default(''),
|
||||
authPrefix: z.string().default(''),
|
||||
toolName: z.string().default(''),
|
||||
// Note: this schemastery fork has no `.optional()`; object fields are
|
||||
// optional unless `.required()` is applied, so absence is already allowed.
|
||||
maxResults: z.number().step(1).min(1).max(50),
|
||||
});
|
||||
|
||||
export const Config = z.object({
|
||||
defaultServer: z.string().default(''),
|
||||
maxResults: z.number().step(1).min(1).max(50).default(8),
|
||||
searchTimeoutMs: z.number().step(1).min(1000).default(30000),
|
||||
servers: z.array(serverSchema).default([]),
|
||||
});
|
||||
|
||||
/** Settings namespace owning this plugin's section (Settings → Plugins card). */
|
||||
export const SEARCH_MCP_SETTINGS_NAMESPACE = 'search-mcp';
|
||||
|
||||
/** Register the search provider and the live settings section. */
|
||||
export function apply(ctx, config) {
|
||||
let current = () => config;
|
||||
// Optional settings seam: fall back to the composition entry when settings
|
||||
// is absent (same pattern as @deepseek-ai/dsh-web-search-deepseek).
|
||||
ctx.inject(['settings'], (settingsCtx) => {
|
||||
settingsCtx.settings.installSection(ctx, SEARCH_MCP_SETTINGS_NAMESPACE, Config, config, {
|
||||
setSource: (source) => {
|
||||
current = source;
|
||||
},
|
||||
// Provider projects the section per search; no re-registration needed.
|
||||
onChange: () => {},
|
||||
});
|
||||
});
|
||||
// `registerSearchProvider` owns its cleanup via ctx.effect (HMR/dispose safe).
|
||||
ctx.web.registerSearchProvider(new SearchMCPProvider(() => resolveOptions(ctx, current())));
|
||||
}
|
||||
|
||||
/**
|
||||
* Project the authoritative config into per-search options. The section
|
||||
* returned by `setSource` (settings.yaml `search-mcp:` block) replaces the
|
||||
* row config entirely, matching how every other settings section behaves.
|
||||
*
|
||||
* @param ctx - plugin context supplying the credential and environment planes.
|
||||
* @param config - the currently authoritative section.
|
||||
* @returns options for one search.
|
||||
*/
|
||||
function resolveOptions(ctx, config) {
|
||||
return {
|
||||
servers: config.servers ?? [],
|
||||
defaultServer: config.defaultServer ?? '',
|
||||
maxResults: config.maxResults ?? 8,
|
||||
searchTimeoutMs: config.searchTimeoutMs ?? 30000,
|
||||
resolveKey: async (server) => {
|
||||
if (server.apiKey !== undefined && server.apiKey.length > 0) return server.apiKey;
|
||||
const envName = server.apiKeyEnv ?? '';
|
||||
if (envName.length === 0) return undefined;
|
||||
const credentials = ctx.get('credentials');
|
||||
if (credentials !== undefined) {
|
||||
try {
|
||||
const resolved = await credentials.resolve(credentialRef(envName));
|
||||
if (resolved !== undefined && resolved.value !== undefined && resolved.value.length > 0) {
|
||||
return resolved.value;
|
||||
}
|
||||
} catch {
|
||||
/* fall through to the launch environment */
|
||||
}
|
||||
}
|
||||
const ambient = launchEnvironmentOf(ctx).get(envName);
|
||||
return ambient !== undefined && ambient.value.length > 0 ? ambient.value : undefined;
|
||||
},
|
||||
};
|
||||
}
|
||||
83
lib/provider.js
Normal file
83
lib/provider.js
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
/**
|
||||
* The `search-mcp` web search provider.
|
||||
*
|
||||
* Registers into `ctx.web` under the stable id `search-mcp`; the profile
|
||||
* patch switches `web.searchProvider` to this id and disables the built-in
|
||||
* DeepSeek provider, so the model-facing `web_search` tool executes entirely
|
||||
* through the configured search MCP server(s).
|
||||
*/
|
||||
import { WebError } from '@deepseek-ai/dsh-web';
|
||||
import { resolveServer } from './catalog.js';
|
||||
import { callMcpSearch } from './client.js';
|
||||
import { extractSearchResult } from './extract.js';
|
||||
|
||||
/** Stable provider id the `web` row's `searchProvider` config selects. */
|
||||
export const SEARCH_MCP_PROVIDER_ID = 'search-mcp';
|
||||
|
||||
/** The web search provider served by this plugin. */
|
||||
export class SearchMCPProvider {
|
||||
id = SEARCH_MCP_PROVIDER_ID;
|
||||
|
||||
/**
|
||||
* @param resolveOptions - snapshots the authoritative config (row config,
|
||||
* or the live settings section) at the START of each operation, so one
|
||||
* search never mixes two settings saves.
|
||||
*/
|
||||
constructor(resolveOptions) {
|
||||
this.resolveOptions = resolveOptions;
|
||||
}
|
||||
|
||||
/** Usable when at least one server entry exists; precise errors surface at search time. */
|
||||
available() {
|
||||
const options = this.resolveOptions();
|
||||
return Array.isArray(options.servers) && options.servers.length > 0;
|
||||
}
|
||||
|
||||
async search(request, signal) {
|
||||
const options = this.resolveOptions();
|
||||
const server = pickServer(options);
|
||||
const resolved = resolveServer(server);
|
||||
const maxResults = resolved.maxResults ?? options.maxResults ?? request.maxResults ?? 8;
|
||||
const key = await options.resolveKey(resolved);
|
||||
if (resolved.needsKey && (key === undefined || key.length === 0)) {
|
||||
const ref = resolved.apiKeyEnv.length > 0 ? resolved.apiKeyEnv : 'apiKey';
|
||||
throw new WebError(
|
||||
`search-mcp server "${resolved.id}" (${resolved.kind}) has no API key; set "apiKey" or a resolvable "apiKeyEnv" (${ref}) in Settings → Plugins → search-mcp`,
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
}
|
||||
const combined = buildSignal(signal, options.searchTimeoutMs);
|
||||
const outcome = await callMcpSearch(resolved, key, { query: request.query, maxResults }, combined);
|
||||
return extractSearchResult(outcome);
|
||||
}
|
||||
}
|
||||
|
||||
/** Select the default server, falling back to the first configured entry. */
|
||||
function pickServer(options) {
|
||||
const servers = Array.isArray(options.servers) ? options.servers : [];
|
||||
if (servers.length === 0) {
|
||||
throw new WebError(
|
||||
'search-mcp: no search MCP servers configured; add one in Settings → Plugins → search-mcp',
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
}
|
||||
if (options.defaultServer !== undefined && options.defaultServer.length > 0) {
|
||||
const found = servers.find((entry) => entry.id === options.defaultServer);
|
||||
if (found === undefined) {
|
||||
throw new WebError(
|
||||
`search-mcp: defaultServer "${options.defaultServer}" is not configured; known servers: ${servers
|
||||
.map((entry) => `"${entry.id}"`)
|
||||
.join(', ') || '(none)'}`,
|
||||
'WEB_PROVIDER_ERROR',
|
||||
);
|
||||
}
|
||||
return found;
|
||||
}
|
||||
return servers[0];
|
||||
}
|
||||
|
||||
/** Combine the caller's cancellation with the configured timeout. */
|
||||
function buildSignal(signal, timeoutMs) {
|
||||
const timeout = AbortSignal.timeout(timeoutMs);
|
||||
return signal !== undefined ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
}
|
||||
243
lib/url-policy.js
Normal file
243
lib/url-policy.js
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
import { lookup as dnsLookup } from 'node:dns';
|
||||
import ipaddr from 'ipaddr.js';
|
||||
|
||||
const ALLOWED_PROTOCOLS = new Set(['http:', 'https:']);
|
||||
|
||||
/**
|
||||
* Parse and resolve one HTTP endpoint before any request is sent.
|
||||
* Every resolved address must be globally routable.
|
||||
*/
|
||||
export async function validateHttpEndpoint(input, options = {}) {
|
||||
if (options.signal?.aborted) throw abortedPolicyError();
|
||||
if (typeof input !== 'string' || input.length === 0 || input !== input.trim()) {
|
||||
throw policyError('endpoint must be a non-empty canonical URL');
|
||||
}
|
||||
|
||||
let url;
|
||||
try {
|
||||
url = new URL(input);
|
||||
} catch {
|
||||
throw policyError('endpoint is not a valid URL');
|
||||
}
|
||||
if (!ALLOWED_PROTOCOLS.has(url.protocol)) {
|
||||
throw policyError('endpoint protocol must be http or https');
|
||||
}
|
||||
if (url.username.length > 0 || url.password.length > 0) {
|
||||
throw policyError('endpoint must not contain user information');
|
||||
}
|
||||
if (url.hostname.length === 0) {
|
||||
throw policyError('endpoint hostname is missing');
|
||||
}
|
||||
|
||||
const hostname = stripIpv6Brackets(url.hostname).toLowerCase();
|
||||
const comparable = hostname.endsWith('.') ? hostname.slice(0, -1) : hostname;
|
||||
if (comparable === 'localhost' || comparable.endsWith('.localhost')) {
|
||||
throw policyError('localhost endpoints are not allowed');
|
||||
}
|
||||
rejectAmbiguousIpv4(input, comparable);
|
||||
|
||||
let addresses;
|
||||
if (ipaddr.isValid(comparable)) {
|
||||
addresses = [{ address: normalizeAddress(comparable), family: addressFamily(comparable) }];
|
||||
} else {
|
||||
addresses = await resolveAll(comparable, options.lookup ?? dnsLookup, options.signal);
|
||||
}
|
||||
if (addresses.length === 0) {
|
||||
throw policyError('endpoint hostname did not resolve');
|
||||
}
|
||||
|
||||
const normalized = deduplicateAddresses(addresses);
|
||||
// Keep globally routable answers. Also allow RFC 2544 (198.18.0.0/15), which
|
||||
// Clash/V2Ray fake-IP / TUN mode commonly returns for otherwise-public hosts.
|
||||
// Real private/LAN answers are dropped; fail only when nothing usable remains.
|
||||
const allowed = normalized.filter((record) => isAllowedEndpointAddress(record.address));
|
||||
if (allowed.length === 0) {
|
||||
throw policyError('endpoint hostname resolves to a non-public address');
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
url,
|
||||
hostname: comparable,
|
||||
addresses: Object.freeze(allowed.map((record) => Object.freeze(record))),
|
||||
lookup: createPinnedLookup(comparable, allowed),
|
||||
});
|
||||
}
|
||||
|
||||
/** Return true only for globally routable IPv4 or IPv6 addresses. */
|
||||
export function isPublicAddress(input) {
|
||||
let address;
|
||||
try {
|
||||
address = ipaddr.parse(stripIpv6Brackets(input));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (address.kind() === 'ipv6') {
|
||||
if (address.isIPv4MappedAddress()) {
|
||||
address = address.toIPv4Address();
|
||||
} else if (isIpv4CompatibleAddress(address)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return address.range() === 'unicast';
|
||||
}
|
||||
|
||||
/** Addresses safe to dial for remote MCP endpoints (public or proxy fake-IP). */
|
||||
export function isAllowedEndpointAddress(input) {
|
||||
if (isPublicAddress(input)) return true;
|
||||
return isProxyFakeIpAddress(input);
|
||||
}
|
||||
|
||||
/** RFC 2544 benchmarking range used as DNS fake-IP by many local proxies. */
|
||||
export function isProxyFakeIpAddress(input) {
|
||||
let address;
|
||||
try {
|
||||
address = ipaddr.parse(stripIpv6Brackets(input));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (address.kind() === 'ipv6') {
|
||||
if (address.isIPv4MappedAddress()) {
|
||||
address = address.toIPv4Address();
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (address.kind() !== 'ipv4') return false;
|
||||
// 198.18.0.0/15
|
||||
const [a, b] = address.octets;
|
||||
return a === 198 && (b === 18 || b === 19);
|
||||
}
|
||||
|
||||
/** Build a Node-compatible DNS lookup that never resolves beyond the pinned set. */
|
||||
export function createPinnedLookup(hostname, records) {
|
||||
const target = normalizeHostname(hostname);
|
||||
const frozen = deduplicateAddresses(records);
|
||||
let cursor = 0;
|
||||
return (requested, options, callback) => {
|
||||
const requestedHost = normalizeHostname(requested);
|
||||
if (requestedHost !== target) {
|
||||
const error = policyError('connection attempted an unvalidated hostname');
|
||||
error.code = 'EACCES';
|
||||
queueMicrotask(() => callback(error));
|
||||
return;
|
||||
}
|
||||
|
||||
const lookupOptions = typeof options === 'object' && options !== null ? options : {};
|
||||
const family = Number(lookupOptions.family) || 0;
|
||||
const candidates = family === 4 || family === 6
|
||||
? frozen.filter((record) => record.family === family)
|
||||
: frozen;
|
||||
if (candidates.length === 0) {
|
||||
const error = policyError('no validated address matches the requested family');
|
||||
error.code = 'ENOTFOUND';
|
||||
queueMicrotask(() => callback(error));
|
||||
return;
|
||||
}
|
||||
if (lookupOptions.all === true) {
|
||||
queueMicrotask(() => callback(null, candidates.map((record) => ({ ...record }))));
|
||||
return;
|
||||
}
|
||||
const selected = candidates[cursor++ % candidates.length];
|
||||
queueMicrotask(() => callback(null, selected.address, selected.family));
|
||||
};
|
||||
}
|
||||
|
||||
function resolveAll(hostname, lookup, signal) {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal?.aborted) {
|
||||
reject(abortedPolicyError());
|
||||
return;
|
||||
}
|
||||
let settled = false;
|
||||
const finish = (callback, value) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
signal?.removeEventListener('abort', onAbort);
|
||||
callback(value);
|
||||
};
|
||||
const onAbort = () => finish(reject, abortedPolicyError());
|
||||
signal?.addEventListener('abort', onAbort, { once: true });
|
||||
lookup(hostname, { all: true, verbatim: true }, (error, records) => {
|
||||
if (settled) return;
|
||||
if (error) {
|
||||
finish(reject, policyError('endpoint hostname resolution failed'));
|
||||
return;
|
||||
}
|
||||
const list = Array.isArray(records) ? records : records === undefined ? [] : [records];
|
||||
try {
|
||||
finish(resolve, list.map((record) => {
|
||||
const raw = typeof record === 'string' ? record : record.address;
|
||||
return { address: normalizeAddress(raw), family: addressFamily(raw) };
|
||||
}));
|
||||
} catch {
|
||||
finish(reject, policyError('endpoint hostname returned an invalid address'));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function deduplicateAddresses(records) {
|
||||
const seen = new Set();
|
||||
const result = [];
|
||||
for (const record of records) {
|
||||
const address = normalizeAddress(record.address);
|
||||
const family = addressFamily(address);
|
||||
const key = `${family}:${address}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
result.push({ address, family });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function normalizeAddress(input) {
|
||||
let address = ipaddr.parse(stripIpv6Brackets(input));
|
||||
if (address.kind() === 'ipv6' && address.isIPv4MappedAddress()) {
|
||||
address = address.toIPv4Address();
|
||||
}
|
||||
return address.toNormalizedString();
|
||||
}
|
||||
|
||||
function addressFamily(input) {
|
||||
const address = ipaddr.parse(stripIpv6Brackets(input));
|
||||
if (address.kind() === 'ipv6' && address.isIPv4MappedAddress()) return 4;
|
||||
return address.kind() === 'ipv4' ? 4 : 6;
|
||||
}
|
||||
|
||||
function rejectAmbiguousIpv4(input, hostname) {
|
||||
const authority = input.match(/^[A-Za-z][A-Za-z0-9+.-]*:\/\/([^/?#]+)/)?.[1] ?? '';
|
||||
const rawHost = authority.startsWith('[')
|
||||
? authority.slice(1, authority.indexOf(']'))
|
||||
: authority.replace(/:\d*$/, '');
|
||||
if (!rawHost.includes(':') && ipaddr.IPv4.isValid(rawHost)) {
|
||||
const canonical = ipaddr.IPv4.parse(rawHost).toString();
|
||||
if (rawHost !== canonical || hostname !== canonical) {
|
||||
throw policyError('endpoint contains a non-canonical IPv4 address');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function isIpv4CompatibleAddress(address) {
|
||||
return address.parts.slice(0, 6).every((part) => part === 0);
|
||||
}
|
||||
|
||||
function normalizeHostname(input) {
|
||||
const value = stripIpv6Brackets(String(input)).toLowerCase();
|
||||
return value.endsWith('.') ? value.slice(0, -1) : value;
|
||||
}
|
||||
|
||||
function stripIpv6Brackets(input) {
|
||||
return input.startsWith('[') && input.endsWith(']') ? input.slice(1, -1) : input;
|
||||
}
|
||||
|
||||
function abortedPolicyError() {
|
||||
const error = policyError('endpoint validation was aborted');
|
||||
error.code = 'ABORT_ERR';
|
||||
return error;
|
||||
}
|
||||
|
||||
function policyError(message) {
|
||||
const error = new Error(`search-mcp URL policy: ${message}`);
|
||||
error.name = 'SearchMcpUrlPolicyError';
|
||||
return error;
|
||||
}
|
||||
2202
package-lock.json
generated
Normal file
2202
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load diff
63
package.json
Normal file
63
package.json
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
{
|
||||
"name": "dsh-search-mcp",
|
||||
"version": "0.2.0",
|
||||
"description": "Replace dsh's built-in web search with search MCP servers (Tavily / Brave / Exa / Perplexity / DuckDuckGo / custom), configured from the web Settings page. When this plugin is enabled the built-in DeepSeek search provider is disabled.",
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"exports": {
|
||||
".": "./lib/index.js",
|
||||
"./client": "./lib/client.browser.js",
|
||||
"./cordis.patch.yml": "./cordis.patch.yml",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"cordis.patch.yml"
|
||||
],
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"scripts": {
|
||||
"check": "node --check lib/index.js && node --check lib/provider.js && node --check lib/catalog.js && node --check lib/extract.js && node --check lib/url-policy.js && node --check lib/client.js && node --check lib/client.browser.js",
|
||||
"test": "node --test"
|
||||
},
|
||||
"keywords": [
|
||||
"dsh",
|
||||
"dsh-plugin",
|
||||
"mcp",
|
||||
"search",
|
||||
"web_search",
|
||||
"tavily",
|
||||
"brave",
|
||||
"exa",
|
||||
"perplexity",
|
||||
"duckduckgo"
|
||||
],
|
||||
"dsh": {
|
||||
"bundle": {
|
||||
"patch": "./cordis.patch.yml"
|
||||
},
|
||||
"client": {
|
||||
"platform": "web",
|
||||
"immediately": true,
|
||||
"inject": [
|
||||
"@deepseek-ai/dsh-client-ui-settings-plugins",
|
||||
"@deepseek-ai/dsh-client-locale"
|
||||
]
|
||||
}
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "1.30.0",
|
||||
"ipaddr.js": "2.5.0",
|
||||
"undici": "6.28.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-api-remotes": "*",
|
||||
"@deepseek-ai/dsh-credentials": "*",
|
||||
"@deepseek-ai/dsh-launch-environment": "*",
|
||||
"@deepseek-ai/dsh-settings": "*",
|
||||
"@deepseek-ai/dsh-web": "*",
|
||||
"@deepseek-ai/schemastery": "*"
|
||||
}
|
||||
}
|
||||
79
test/compatibility.test.js
Normal file
79
test/compatibility.test.js
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { dirname, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const read = (path) => readFile(resolve(root, path), 'utf8');
|
||||
|
||||
test('package exports resolve and RC2 dependencies stay pinned', async () => {
|
||||
const pkg = JSON.parse(await read('package.json'));
|
||||
assert.equal(pkg.exports['.'], './lib/index.js');
|
||||
assert.equal(pkg.exports['./client'], './lib/client.browser.js');
|
||||
assert.equal(pkg.engines.node, '>=20');
|
||||
|
||||
for (const name of [
|
||||
'@deepseek-ai/dsh-api-remotes',
|
||||
'@deepseek-ai/dsh-credentials',
|
||||
'@deepseek-ai/dsh-launch-environment',
|
||||
'@deepseek-ai/dsh-settings',
|
||||
'@deepseek-ai/dsh-web',
|
||||
]) {
|
||||
assert.equal(pkg.dependencies[name], '0.1.1-rc.2');
|
||||
}
|
||||
assert.equal(pkg.dependencies['@deepseek-ai/schemastery'], '3.18.1');
|
||||
assert.equal(pkg.dependencies.undici, '6.28.0');
|
||||
assert.equal(pkg.dependencies['ipaddr.js'], '2.5.0');
|
||||
assert.equal(pkg.dependencies['@modelcontextprotocol/sdk'], '1.30.0');
|
||||
});
|
||||
|
||||
test('RC2 browser bundle uses keyed settings slot and credential migration', async () => {
|
||||
const client = await read('lib/client.browser.js');
|
||||
assert.match(client, /name: "settings\.plugin\.item",\s+key: NS,/);
|
||||
assert.doesNotMatch(client, /name: "settings\.plugin\.item",\s+id:/);
|
||||
assert.match(client, /api\.settings\.describe\(\{\}\)/);
|
||||
assert.match(client, /api\.credentials\.describe\(\{ refs: refs\.slice\(index, index \+ CREDENTIAL_DESCRIBE_BATCH_SIZE\) \}\)/);
|
||||
assert.match(client, /api\.credentials\.set\(\{ ref, value \}\)/);
|
||||
assert.match(client, /credentials\/reference-updated/);
|
||||
assert.match(client, /CREDENTIAL_DESCRIBE_BATCH_SIZE = 64/);
|
||||
assert.match(client, /api\.credentials\.unset\(\{ ref \}\)/);
|
||||
assert.match(client, /rollbackSettingsWrites/);
|
||||
assert.match(client, /legacyKeyBlocked/);
|
||||
assert.match(client, /deepEqualJson\(current\[field\], value\)/);
|
||||
});
|
||||
|
||||
test('known providers are CDKey-only while custom keeps advanced fields', async () => {
|
||||
const client = await read('lib/client.browser.js');
|
||||
const catalog = client.slice(client.indexOf('const CATALOG = {'), client.indexOf('const KIND_OPTIONS'));
|
||||
assert.doesNotMatch(catalog, /https?:\/\//);
|
||||
assert.doesNotMatch(catalog, /toolName|authParam|transport/);
|
||||
assert.match(client, /const known = row\.kind !== "custom"/);
|
||||
assert.match(client, /children: known \? \[/);
|
||||
assert.match(client, /No endpoint is required for known providers/);
|
||||
assert.match(client, /已知提供商不需要填写端点链接/);
|
||||
assert.match(client, /kind, apiKey: "", apiKeyEnv: ""/);
|
||||
});
|
||||
|
||||
test('HTTP transport pins DNS and applies one guarded fetch to every SDK request', async () => {
|
||||
const transport = await read('lib/client.js');
|
||||
assert.match(transport, /validateHttpEndpoint\(server\.url, \{ signal \}\)/);
|
||||
assert.match(transport, /new Agent\(\{[\s\S]*connect: \{ lookup: validated\.lookup \}/);
|
||||
assert.match(transport, /requestUrl\.origin !== expectedOrigin/);
|
||||
assert.match(transport, /dispatcher: agent/);
|
||||
assert.match(transport, /redirect: 'error'/);
|
||||
assert.match(transport, /fetch: secureFetch/);
|
||||
assert.match(transport, /await client\.close\(\)[\s\S]*await runtime\?\.close\(\)/);
|
||||
assert.doesNotMatch(transport, /new URL\(server\.url\)[\s\S]*new StreamableHTTPClientTransport\(url, \{\s*requestInit:/);
|
||||
});
|
||||
test('bundle replaces built-in search and leaves default row endpoint-free', async () => {
|
||||
const patch = await read('cordis.patch.yml');
|
||||
assert.match(patch, /searchProvider: search-mcp/);
|
||||
assert.match(patch, /- id: web-search-deepseek\s+disabled: true/);
|
||||
assert.match(patch, /- id: tool-web\s+disabled: false/);
|
||||
assert.match(patch, /fetch: false/);
|
||||
assert.match(patch, /searchMaxResults: 50/);
|
||||
assert.match(patch, /searchMaxQueries: 4/);
|
||||
const defaultRow = patch.slice(patch.indexOf('- id: tavily'), patch.indexOf('- id: web'));
|
||||
assert.doesNotMatch(defaultRow, /url:|toolName:|authParam:|transport:/);
|
||||
});
|
||||
130
test/runtime.test.js
Normal file
130
test/runtime.test.js
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { clampSearchResults, SEARCH_MCP_CATALOG, resolveServer } from '../lib/catalog.js';
|
||||
import { extractSearchResult } from '../lib/extract.js';
|
||||
|
||||
test('catalog exposes every supported provider preset', () => {
|
||||
assert.deepEqual(Object.keys(SEARCH_MCP_CATALOG), [
|
||||
'tavily',
|
||||
'brave',
|
||||
'exa',
|
||||
'perplexity',
|
||||
'duckduckgo',
|
||||
'custom',
|
||||
]);
|
||||
});
|
||||
|
||||
test('known providers ignore stored connection overrides', () => {
|
||||
const tavily = resolveServer({
|
||||
id: 'primary',
|
||||
kind: 'tavily',
|
||||
transport: 'stdio',
|
||||
url: 'https://example.test/mcp',
|
||||
authStyle: 'header',
|
||||
authParam: 'X-Other-Key',
|
||||
toolName: 'other_search',
|
||||
apiKeyEnv: 'MY_TAVILY_KEY',
|
||||
maxResults: 12,
|
||||
});
|
||||
assert.equal(tavily.transport, 'http');
|
||||
assert.equal(tavily.url, 'https://mcp.tavily.com/mcp/');
|
||||
assert.equal(tavily.authStyle, 'query');
|
||||
assert.equal(tavily.authParam, 'tavilyApiKey');
|
||||
assert.equal(tavily.toolName, 'tavily_search');
|
||||
assert.equal(tavily.countArg, 'max_results');
|
||||
assert.equal(tavily.apiKeyEnv, 'MY_TAVILY_KEY');
|
||||
assert.equal(tavily.maxResults, 12);
|
||||
});
|
||||
|
||||
test('prototype property kinds fall back to custom', () => {
|
||||
for (const kind of ['constructor', 'toString', '__proto__']) {
|
||||
const resolved = resolveServer({ id: kind, kind, url: 'https://search.example/mcp' });
|
||||
assert.equal(resolved.kind, 'custom');
|
||||
assert.equal(resolved.url, 'https://search.example/mcp');
|
||||
}
|
||||
});
|
||||
|
||||
test('custom providers preserve advanced connection fields', () => {
|
||||
const custom = resolveServer({
|
||||
id: 'custom',
|
||||
kind: 'custom',
|
||||
transport: 'stdio',
|
||||
command: 'custom-mcp',
|
||||
args: ['--stdio'],
|
||||
authStyle: 'header',
|
||||
authParam: 'X-Key',
|
||||
authPrefix: 'Token ',
|
||||
toolName: 'search',
|
||||
});
|
||||
assert.equal(custom.transport, 'stdio');
|
||||
assert.equal(custom.command, 'custom-mcp');
|
||||
assert.deepEqual(custom.args, ['--stdio']);
|
||||
assert.equal(custom.authParam, 'X-Key');
|
||||
assert.equal(custom.authPrefix, 'Token ');
|
||||
assert.equal(custom.toolName, 'search');
|
||||
});
|
||||
|
||||
test('provider contracts match current upstream transports', () => {
|
||||
const brave = resolveServer({ id: 'brave', kind: 'brave' });
|
||||
assert.equal(brave.transport, 'stdio');
|
||||
assert.equal(brave.command, 'npx');
|
||||
assert.deepEqual(brave.args, ['-y', '@brave/brave-search-mcp-server@2.1.3']);
|
||||
assert.equal(brave.authParam, 'BRAVE_API_KEY');
|
||||
|
||||
const perplexity = resolveServer({ id: 'perplexity', kind: 'perplexity' });
|
||||
assert.equal(perplexity.url, 'https://api.perplexity.ai/mcp');
|
||||
assert.equal(perplexity.authParam, 'Authorization');
|
||||
assert.equal(perplexity.authPrefix, 'Bearer ');
|
||||
assert.equal(perplexity.toolName, 'perplexity_search');
|
||||
|
||||
const duckduckgo = resolveServer({ id: 'duckduckgo', kind: 'duckduckgo' });
|
||||
assert.equal(duckduckgo.needsKey, false);
|
||||
assert.equal(duckduckgo.toolName, 'duckduckgo_web_search');
|
||||
assert.equal(duckduckgo.countArg, 'count');
|
||||
});
|
||||
|
||||
test('provider result counts stay within upstream MCP schemas', () => {
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 't', kind: 'tavily' }), 1), 5);
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 't', kind: 'tavily' }), 50), 20);
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 'b', kind: 'brave' }), 50), 20);
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 'p', kind: 'perplexity' }), 0), 1);
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 'd', kind: 'duckduckgo' }), 50), 20);
|
||||
assert.equal(clampSearchResults(resolveServer({ id: 'e', kind: 'exa' }), 50), 50);
|
||||
});
|
||||
test('extractSearchResult normalizes, deduplicates, and rejects invalid URLs', () => {
|
||||
const result = extractSearchResult({
|
||||
structuredContent: {
|
||||
answer: 'Summary',
|
||||
results: [
|
||||
{ url: 'https://example.com/a', title: 'A', content: 'alpha', published_date: '2026-08-18' },
|
||||
{ url: 'https://example.com/a', title: 'Duplicate' },
|
||||
{ url: 'ftp://example.com/ignored', title: 'Ignored' },
|
||||
],
|
||||
nested: { url: 'http://example.com/b', description: 'beta' },
|
||||
},
|
||||
});
|
||||
|
||||
assert.deepEqual(result, {
|
||||
sources: [
|
||||
{
|
||||
url: 'https://example.com/a',
|
||||
title: 'A',
|
||||
snippet: 'alpha',
|
||||
publishedAt: '2026-08-18',
|
||||
},
|
||||
{ url: 'http://example.com/b', snippet: 'beta' },
|
||||
],
|
||||
truncated: false,
|
||||
content: 'Summary',
|
||||
});
|
||||
});
|
||||
|
||||
test('extractSearchResult accepts JSON and plain text MCP blocks', () => {
|
||||
const json = extractSearchResult({
|
||||
content: [{ type: 'text', text: '{"results":[{"url":"https://example.com"}]}' }],
|
||||
});
|
||||
assert.equal(json.sources.length, 1);
|
||||
|
||||
const text = extractSearchResult({ content: [{ type: 'text', text: 'Direct answer' }] });
|
||||
assert.deepEqual(text, { sources: [], truncated: false, content: 'Direct answer' });
|
||||
});
|
||||
158
test/url-policy.test.js
Normal file
158
test/url-policy.test.js
Normal file
|
|
@ -0,0 +1,158 @@
|
|||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
createPinnedLookup,
|
||||
isAllowedEndpointAddress,
|
||||
isPublicAddress,
|
||||
validateHttpEndpoint,
|
||||
} from '../lib/url-policy.js';
|
||||
|
||||
const lookup = (records) => (_hostname, options, callback) => {
|
||||
assert.equal(options.all, true);
|
||||
queueMicrotask(() => callback(null, records));
|
||||
};
|
||||
|
||||
test('URL policy accepts HTTP(S) with public DNS only', async () => {
|
||||
const result = await validateHttpEndpoint('https://search.example/mcp', {
|
||||
lookup: lookup([
|
||||
{ address: '8.8.8.8', family: 4 },
|
||||
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||
]),
|
||||
});
|
||||
assert.equal(result.url.hostname, 'search.example');
|
||||
assert.deepEqual(result.addresses, [
|
||||
{ address: '8.8.8.8', family: 4 },
|
||||
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||
]);
|
||||
});
|
||||
|
||||
test('URL policy rejects schemes, userinfo, localhost, and ambiguous IPv4', async () => {
|
||||
for (const input of [
|
||||
'ftp://example.com/mcp',
|
||||
'https://user:pass@example.com/mcp',
|
||||
'http://localhost/mcp',
|
||||
'http://api.localhost/mcp',
|
||||
'http://127.0.0.1/mcp',
|
||||
'http://127.1/mcp',
|
||||
'http://0177.0.0.1/mcp',
|
||||
'http://0x7f000001/mcp',
|
||||
]) {
|
||||
await assert.rejects(() => validateHttpEndpoint(input), { name: 'SearchMcpUrlPolicyError' }, input);
|
||||
}
|
||||
});
|
||||
|
||||
test('URL policy rejects private, reserved, test, mapped, and transition ranges', () => {
|
||||
for (const address of [
|
||||
'0.0.0.0',
|
||||
'10.0.0.1',
|
||||
'100.64.0.1',
|
||||
'127.0.0.1',
|
||||
'169.254.169.254',
|
||||
'172.16.0.1',
|
||||
'192.168.0.1',
|
||||
'192.0.2.1',
|
||||
'198.18.0.1',
|
||||
'198.51.100.1',
|
||||
'203.0.113.1',
|
||||
'224.0.0.1',
|
||||
'255.255.255.255',
|
||||
'::',
|
||||
'::1',
|
||||
'::ffff:127.0.0.1',
|
||||
'::7f00:1',
|
||||
'::a00:1',
|
||||
'::a9fe:a9fe',
|
||||
'::c0a8:101',
|
||||
'::808:808',
|
||||
'64:ff9b::808:808',
|
||||
'2001:db8::1',
|
||||
'2001::1',
|
||||
'2002:0808:0808::1',
|
||||
'fc00::1',
|
||||
'fe80::1',
|
||||
'ff02::1',
|
||||
]) {
|
||||
assert.equal(isPublicAddress(address), false, address);
|
||||
}
|
||||
assert.equal(isPublicAddress('8.8.8.8'), true);
|
||||
assert.equal(isPublicAddress('2606:4700:4700::1111'), true);
|
||||
});
|
||||
|
||||
test('URL policy drops private DNS answers and keeps public ones', async () => {
|
||||
const result = await validateHttpEndpoint('https://search.example/mcp', {
|
||||
lookup: lookup([
|
||||
{ address: '8.8.8.8', family: 4 },
|
||||
{ address: '10.0.0.1', family: 4 },
|
||||
]),
|
||||
});
|
||||
assert.deepEqual(result.addresses, [{ address: '8.8.8.8', family: 4 }]);
|
||||
});
|
||||
|
||||
test('URL policy accepts Clash fake-IP answers with optional public peers', async () => {
|
||||
const mixed = await validateHttpEndpoint('https://dashscope.example/mcp', {
|
||||
lookup: lookup([
|
||||
{ address: '198.18.2.146', family: 4 },
|
||||
{ address: '2408:400a:3e:ef02:12f:bd95:e827:51d', family: 6 },
|
||||
]),
|
||||
});
|
||||
assert.deepEqual(mixed.addresses, [
|
||||
{ address: '198.18.2.146', family: 4 },
|
||||
{ address: '2408:400a:3e:ef02:12f:bd95:e827:51d', family: 6 },
|
||||
]);
|
||||
|
||||
const fakeOnly = await validateHttpEndpoint('https://dashscope.example/mcp', {
|
||||
lookup: lookup([{ address: '198.18.2.146', family: 4 }]),
|
||||
});
|
||||
assert.deepEqual(fakeOnly.addresses, [{ address: '198.18.2.146', family: 4 }]);
|
||||
assert.equal(isAllowedEndpointAddress('198.18.2.146'), true);
|
||||
assert.equal(isAllowedEndpointAddress('10.0.0.1'), false);
|
||||
});
|
||||
|
||||
test('URL policy rejects private-only DNS answers', async () => {
|
||||
await assert.rejects(
|
||||
() => validateHttpEndpoint('https://search.example/mcp', {
|
||||
lookup: lookup([{ address: '10.0.0.1', family: 4 }]),
|
||||
}),
|
||||
/non-public address/,
|
||||
);
|
||||
});
|
||||
|
||||
test('URL policy rejects DNS errors and empty answers without leaking endpoint data', async () => {
|
||||
await assert.rejects(
|
||||
() => validateHttpEndpoint('https://search.example/mcp', {
|
||||
lookup: (_hostname, _options, callback) => callback(new Error('resolver detail')),
|
||||
}),
|
||||
/resolution failed/,
|
||||
);
|
||||
await assert.rejects(
|
||||
() => validateHttpEndpoint('https://search.example/mcp', { lookup: lookup([]) }),
|
||||
/did not resolve/,
|
||||
);
|
||||
});
|
||||
|
||||
test('URL policy aborts a pending DNS lookup', async () => {
|
||||
const controller = new AbortController();
|
||||
const pending = validateHttpEndpoint('https://search.example/mcp', {
|
||||
lookup: () => {},
|
||||
signal: controller.signal,
|
||||
});
|
||||
controller.abort();
|
||||
await assert.rejects(pending, /validation was aborted/);
|
||||
});
|
||||
|
||||
test('pinned lookup serves only validated host and addresses', async () => {
|
||||
const pinned = createPinnedLookup('search.example', [
|
||||
{ address: '8.8.8.8', family: 4 },
|
||||
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||
]);
|
||||
const all = await new Promise((resolve, reject) => {
|
||||
pinned('search.example', { all: true }, (error, records) => error ? reject(error) : resolve(records));
|
||||
});
|
||||
assert.deepEqual(all, [
|
||||
{ address: '8.8.8.8', family: 4 },
|
||||
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||
]);
|
||||
await assert.rejects(new Promise((resolve, reject) => {
|
||||
pinned('other.example', {}, (error, address) => error ? reject(error) : resolve(address));
|
||||
}), /unvalidated hostname/);
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue