mirror of
https://github.com/hansjone/dsh-search-mcp.git
synced 2026-10-08 23:43:15 +08:00
Publish dsh-search-mcp fork for newer DSH and Bailian WebSearch MCP.
Based on gxpppp/dsh-search-mcp; includes DSH web settings fixes, Clash fake-IP URL policy, and Bailian default server patch. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
commit
d5c4d2006c
16 changed files with 4821 additions and 0 deletions
8
.gitignore
vendored
Normal file
8
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
node_modules/
|
||||||
|
*.log
|
||||||
|
*.tgz
|
||||||
|
.DS_Store
|
||||||
|
|
||||||
|
# Local security scan state and private project handoff.
|
||||||
|
.mimosa/
|
||||||
|
PROJECT_CONTEXT.md
|
||||||
21
LICENSE
Normal file
21
LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
183
README.md
Normal file
183
README.md
Normal file
|
|
@ -0,0 +1,183 @@
|
||||||
|
# dsh-search-mcp
|
||||||
|
|
||||||
|
用搜索类 MCP 服务器完整替代 DeepSeek Harness（DSH）内置网页搜索的独立插件。
|
||||||
|
|
||||||
|
> 当前兼容基线：DeepSeek Harness `0.1.1-rc.2`，Node.js 20 或更高版本。
|
||||||
|
|
||||||
|
## 功能
|
||||||
|
|
||||||
|
- 模型侧继续使用原生 `web_search`，插件只替换底层 search provider。
|
||||||
|
- 支持 Tavily、Brave、Exa、Perplexity、DuckDuckGo 和自定义 HTTP/stdio MCP。
|
||||||
|
- 已知 provider 只需选择服务商并填写 CDKey/API key，不需要填写 URL、命令、鉴权参数或工具名。
|
||||||
|
- 自定义 MCP 保留 URL、stdio 命令、鉴权方式和工具名等高级配置。
|
||||||
|
- 密钥通过 DSH credentials domain 写入；设置读取接口只返回是否已配置，不返回密钥值。
|
||||||
|
- DSH RC2 支持一次 `web_search` 提交多个查询，默认上限为 4。
|
||||||
|
- 卸载插件后 bundle 覆盖层随之移除，DSH 内置搜索组合恢复。
|
||||||
|
|
||||||
|
## 安装
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
git clone https://github.com/gxpppp/dsh-search-mcp.git
|
||||||
|
cd dsh-search-mcp
|
||||||
|
npm install
|
||||||
|
dsh plugin --profile web add link:<dsh-search-mcp 的绝对路径>
|
||||||
|
dsh web
|
||||||
|
```
|
||||||
|
|
||||||
|
`link:` 会让源码更新直接作用于 profile。修改或升级浏览器 bundle 后需要重启 DSH Web 并刷新页面。
|
||||||
|
|
||||||
|
如果 profile 中已有独立搜索 MCP 行，建议先移除重复入口，避免同时暴露 `mcp__...` 工具和本插件提供的 `web_search`。
|
||||||
|
|
||||||
|
## 配置
|
||||||
|
|
||||||
|
打开：
|
||||||
|
|
||||||
|
**设置 → 插件 → 插件配置 → 搜索 MCP**
|
||||||
|
|
||||||
|
### 已知 provider
|
||||||
|
|
||||||
|
1. 点击 Tavily、Brave、Exa、Perplexity 或 DuckDuckGo 快捷按钮。
|
||||||
|
2. 展开服务器行。
|
||||||
|
3. 对需要凭据的 provider 填写 CDKey/API key，然后保存。
|
||||||
|
4. DuckDuckGo 无需 key。
|
||||||
|
|
||||||
|
已知 provider 的 endpoint、transport、鉴权方式、工具名和结果参数由 Host catalog 固定管理，设置页不会自动填入或显示链接。保存 CDKey 后，客户端先调用 `credentials.set`，再将生成的 credential reference 写入服务器设置；密钥本身不会写回普通 settings 字段。
|
||||||
|
|
||||||
|
也可以预先在 `$DSH_HOME/.credentials.yaml` 中保存凭据，再在设置页填写引用名：
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
TAVILY_API_KEY: <your-key>
|
||||||
|
EXA_API_KEY: <your-key>
|
||||||
|
PERPLEXITY_API_KEY: <your-key>
|
||||||
|
BRAVE_API_KEY: <your-key>
|
||||||
|
```
|
||||||
|
|
||||||
|
RC2 的 `credentials/reference-updated` 事件会刷新设置卡片中的“已配置/未配置”状态，但不会传输密钥值。卡片按 RC2 每批最多 64 个引用的限制分批读取状态。保存多个字段失败时会逆序恢复已写入的 settings，并清理本次新建的 credential reference；由于 RC2 不允许读回已有密钥，覆盖一个此前已配置的引用后无法跨 credentials/settings 做值级回滚。
|
||||||
|
|
||||||
|
### 自定义 MCP
|
||||||
|
|
||||||
|
添加 `custom` 服务器后，可配置：
|
||||||
|
|
||||||
|
| 字段 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `id` | 服务器唯一标识，供 `defaultServer` 引用 |
|
||||||
|
| `transport` | `http`（Streamable HTTP）或 `stdio` |
|
||||||
|
| `url` | 仅 HTTP 自定义 MCP 使用 |
|
||||||
|
| `command` / `args` | 仅 stdio 自定义 MCP 使用 |
|
||||||
|
| `apiKey` | 写入方向的 CDKey/API key 输入 |
|
||||||
|
| `apiKeyEnv` | 环境变量或 DSH credential reference |
|
||||||
|
| `authStyle` | HTTP 的 `query` 或 `header` |
|
||||||
|
| `authParam` | query/header 参数名；stdio 下作为环境变量名 |
|
||||||
|
| `toolName` | MCP 搜索工具名称 |
|
||||||
|
| `maxResults` | 单服务器结果数覆盖 |
|
||||||
|
|
||||||
|
### 全局选项
|
||||||
|
|
||||||
|
| 字段 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `defaultServer` | 默认服务器 id；留空时使用第一行 |
|
||||||
|
| `maxResults` | 全局结果数上限，默认 8，可选 1–50 |
|
||||||
|
| `searchTimeoutMs` | 每次 MCP 搜索超时，默认 30000 ms；界面以秒显示 |
|
||||||
|
|
||||||
|
## Provider 预设
|
||||||
|
|
||||||
|
| kind | Host 管理的连接 | 凭据 | 搜索工具 | 结果数参数 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `tavily` | hosted Streamable HTTP | CDKey/API key | `tavily_search` | `max_results` |
|
||||||
|
| `brave` | `@brave/brave-search-mcp-server@2.1.3` stdio | `BRAVE_API_KEY` | `brave_web_search` | `count` |
|
||||||
|
| `exa` | hosted Streamable HTTP | `x-api-key` | `web_search_exa` | `numResults` |
|
||||||
|
| `perplexity` | hosted Streamable HTTP | Bearer token | `perplexity_search` | `max_results` |
|
||||||
|
| `duckduckgo` | `duckduckgo-mcp-server@0.1.2` stdio | 无 | `duckduckgo_web_search` | `count` |
|
||||||
|
| `custom` | 用户配置 | 用户配置 | 用户配置 | 无预设 |
|
||||||
|
|
||||||
|
旧配置中的 known-provider URL、transport、auth 和 tool 字段仍可被 schema 读取，但运行时会忽略它们；下一次保存服务器列表时会清理这些冗余字段。只有 `custom` 使用用户提供的连接信息。
|
||||||
|
|
||||||
|
插件在调用 known provider 前会把结果数限制到上游 MCP schema 接受的范围：Tavily 为 5–20，Brave、Perplexity 和 DuckDuckGo 为 1–20；Exa 当前保留插件的 1–50 范围。该限制只影响传给上游的参数，最终返回数量仍会受到插件全局/单服务器限制和实际 agent preset 的 `tool-web.searchMaxResults` 共同约束。
|
||||||
|
|
||||||
|
## DSH 0.1.1-rc.2 适配
|
||||||
|
|
||||||
|
- DSH host 依赖精确锁定为 `0.1.1-rc.2`，不使用可能落到旧版本线的子包 `latest`。
|
||||||
|
- 设置卡片继续使用 keyed slot：`settings.plugin.item` + `key: "search-mcp"`。
|
||||||
|
- 新密钥通过 `credentials.set` 单向写入，凭据状态通过 `credentials.describe` 读取。
|
||||||
|
- 监听 RC2 的 `credentials/reference-updated`，外部凭据变更后刷新状态 badge。
|
||||||
|
- RC6/RC7 遗留的字面 `apiKey` 仍可由 Host 使用；涉及服务器数组的编辑会阻止不可见旧密钥被意外删除，并要求先迁移。
|
||||||
|
- 普通全局字段修改不会重写 `servers` 数组。
|
||||||
|
- `tool-web.searchMaxQueries` 配置为 4，与 RC2 默认多查询能力一致。
|
||||||
|
|
||||||
|
## URL 安全策略
|
||||||
|
|
||||||
|
所有自定义 HTTP MCP 请求在联网前执行安全校验：
|
||||||
|
|
||||||
|
- 只允许 `http:` 和 `https:`，拒绝 userinfo 与非规范 IPv4 表示。
|
||||||
|
- 拒绝 localhost、环回、RFC1918 私网、链路本地、CGNAT、benchmark、文档/测试、多播、保留和广播地址。
|
||||||
|
- IPv4-mapped IPv6 先映射为 IPv4 再判断；IPv4-compatible IPv6、IPv6 ULA、link-local、NAT64/转换、Teredo、6to4、文档和保留范围同样拒绝。
|
||||||
|
- 域名会解析全部 A/AAAA 结果；任意一个结果不公开可路由时整体拒绝，DNS 等待也受同一个搜索 AbortSignal/超时约束。
|
||||||
|
- 每次搜索使用独占 Undici Agent 和预解析地址的 pinned lookup，同时保留原始 Host 与 TLS SNI，防止 DNS rebinding。
|
||||||
|
- GET、POST、DELETE 和 SSE 重连都通过同一 fetch wrapper，HTTP 重定向设置为 `error`。
|
||||||
|
- 先关闭 MCP client，再关闭本次 Agent，不共享连接池。
|
||||||
|
- 错误信息不会输出包含 CDKey 的完整 URL。
|
||||||
|
|
||||||
|
如果代理或 TUN 把公共域名解析到 `198.18.0.0/15` fake-IP，本插件会按 benchmark/test 网段安全拒绝。应让 DSH 进程获得真实公网 DNS 结果，而不是放宽策略。
|
||||||
|
|
||||||
|
## 组合覆盖
|
||||||
|
|
||||||
|
插件通过 `cordis.patch.yml`：
|
||||||
|
|
||||||
|
- 注册 `search-mcp` provider。
|
||||||
|
- 设置 `web.searchProvider: search-mcp`。
|
||||||
|
- 禁用 `web-search-deepseek`。
|
||||||
|
- 保持 `web_fetch` 关闭。
|
||||||
|
- 请求 `tool-web.searchMaxResults: 50` 和 `searchMaxQueries: 4`。
|
||||||
|
|
||||||
|
RC2 的 standard、code、cordis agent preset 各自包含 `tool-web` 行，并且都省略了 `searchMaxResults` 和 `searchMaxQueries`，因此实际采用 `dsh-tool-web` 默认值 8 和 4。agent-scoped 工具会遮蔽根层同名工具，所以根层 patch 中的 50 条请求不会提高这些 shipped preset 的实际上限。验证结果上限时必须检查 session 使用的 preset，不能只依据根层 `--dump-config`。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
npm test
|
||||||
|
npm run check
|
||||||
|
npm pack --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
自动测试覆盖 RC2 依赖锁定、known/custom catalog 边界、CDKey-only 设置结构、凭据事件、旧 secret 保护、结果归一化，以及 URL/DNS/pinning 安全策略。
|
||||||
|
|
||||||
|
2026-08-30 的隔离 RC2 Web 冒烟检查确认：插件卡片可加载；默认 Tavily 行不显示链接或高级连接字段；DuckDuckGo 摘要显示“无需密钥”，展开后只有 ID、provider 和结果数；测试草稿已放弃且没有写入 settings。无密钥 DuckDuckGo stdio server 能启动并收到正确的 `duckduckgo_web_search`/`count` 调用，但当次公开搜索被 DuckDuckGo 上游异常流量检测拒绝，因此未取得可用于结果归一化验收的真实来源。
|
||||||
|
|
||||||
|
组合检查：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dsh --profile web --dump-config |
|
||||||
|
Select-String -Pattern "searchProvider|search-mcp|web-search-deepseek|searchMaxResults|searchMaxQueries"
|
||||||
|
```
|
||||||
|
|
||||||
|
预期至少包括：
|
||||||
|
|
||||||
|
- `web.searchProvider: search-mcp`
|
||||||
|
- `web-search-deepseek.disabled: true`
|
||||||
|
- `tool-web.disabled: false`
|
||||||
|
- `fetch: false`
|
||||||
|
|
||||||
|
实际 agent preset 的结果数和多查询上限应在隔离 profile/session 中单独验证。
|
||||||
|
|
||||||
|
## 故障排查
|
||||||
|
|
||||||
|
- `no search MCP servers configured`：在设置页添加 provider。
|
||||||
|
- `has no API key`：填写 CDKey/API key，或填写已有 credential reference。
|
||||||
|
- “凭证未配置”：引用名存在于 settings，但 credentials provider 当前找不到对应值。
|
||||||
|
- `defaultServer "x" is not configured`：默认 id 没有匹配任何服务器行。
|
||||||
|
- `URL policy` 拒绝：endpoint 非 HTTP(S)，或 DNS 结果包含本地、私有、保留/测试地址。
|
||||||
|
- stdio 启动失败：确认 Node/npm 可用，且运行环境允许 `npx` 获取或执行对应 MCP 包。
|
||||||
|
- 设置页没有 Search MCP 卡片：确认 client bundle 已安装，重启 DSH Web 后强制刷新页面。
|
||||||
|
- 返回结果仍被截断：检查实际 agent preset 中的 `tool-web.searchMaxResults`，以及全局/单服务器 `maxResults`。
|
||||||
|
|
||||||
|
## 卸载
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dsh plugin --profile web remove dsh-search-mcp
|
||||||
|
```
|
||||||
|
|
||||||
|
随后重启 DSH Web。不要只禁用 `search-mcp` 行，因为 bundle 还覆盖了 `web`、`web-search-deepseek` 和 `tool-web`；完整卸载 bundle 才会恢复内置组合。
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT
|
||||||
58
cordis.patch.yml
Normal file
58
cordis.patch.yml
Normal file
|
|
@ -0,0 +1,58 @@
|
||||||
|
# dsh-search-mcp bundle layer: applied after dsh-base and dsh-web-app,
|
||||||
|
# before the profile's own cordis.patch.yml (last write wins per row).
|
||||||
|
#
|
||||||
|
# Installing this package therefore REPLACES dsh's built-in web search:
|
||||||
|
# - the `web` row's searchProvider switches from `deepseek-official` to
|
||||||
|
# `search-mcp` (the provider registered by this plugin), and
|
||||||
|
# - the built-in DeepSeek search provider row is disabled.
|
||||||
|
# The model-facing `web_search` tool keeps its name and presentation; its
|
||||||
|
# execution now goes through the search MCP server(s) configured below or in
|
||||||
|
# the web Settings → Plugins → search-mcp section.
|
||||||
|
#
|
||||||
|
# Removing this package (dsh plugin --profile web remove dsh-search-mcp)
|
||||||
|
# drops this whole layer and restores the built-in search exactly.
|
||||||
|
#
|
||||||
|
# SECURITY: no API keys are committed to this repository. Server keys are
|
||||||
|
# supplied at runtime through `apiKeyEnv` — stored in
|
||||||
|
# `$DSH_HOME/.credentials.yaml` (e.g. `TAVILY_API_KEY: <key>`) — or through
|
||||||
|
# the web Settings → Plugins → search-mcp section (`apiKey` field).
|
||||||
|
|
||||||
|
- insert:
|
||||||
|
- id: search-mcp
|
||||||
|
name: 'dsh-search-mcp'
|
||||||
|
config:
|
||||||
|
defaultServer: bailian
|
||||||
|
maxResults: 8
|
||||||
|
searchTimeoutMs: 30000
|
||||||
|
servers:
|
||||||
|
- id: bailian
|
||||||
|
kind: custom
|
||||||
|
transport: http
|
||||||
|
url: https://dashscope.aliyuncs.com/api/v1/mcps/WebSearch/mcp
|
||||||
|
authStyle: header
|
||||||
|
authParam: Authorization
|
||||||
|
authPrefix: 'Bearer '
|
||||||
|
apiKeyEnv: DASHSCOPE_API_KEY
|
||||||
|
toolName: bailian_web_search
|
||||||
|
|
||||||
|
- id: web
|
||||||
|
config:
|
||||||
|
searchProvider: search-mcp
|
||||||
|
|
||||||
|
- id: web-search-deepseek
|
||||||
|
disabled: true
|
||||||
|
|
||||||
|
# The model-facing web_search tool is owned by dsh-tool-web (its `Config`
|
||||||
|
# default is a hard 8-source cap, enforced by dsh-web's seam on EVERY
|
||||||
|
# request). This plugin takes over result sizing, so raise the tool layer's
|
||||||
|
# cap to the plugin schema maximum: `search-mcp`'s own maxResults (Settings →
|
||||||
|
# Plugins → search-mcp) then decides how many sources actually come back.
|
||||||
|
# `fetch` stays disabled and the base timeout is restated, because a patch
|
||||||
|
# replaces the targeted row's whole config.
|
||||||
|
- id: tool-web
|
||||||
|
disabled: false
|
||||||
|
config:
|
||||||
|
fetch: false
|
||||||
|
searchTimeoutMs: 60000
|
||||||
|
searchMaxResults: 50
|
||||||
|
searchMaxQueries: 4
|
||||||
137
lib/catalog.js
Normal file
137
lib/catalog.js
Normal file
|
|
@ -0,0 +1,137 @@
|
||||||
|
/**
|
||||||
|
* Search-MCP provider catalog.
|
||||||
|
*
|
||||||
|
* Known providers are intentionally connection-opaque to Settings clients: the
|
||||||
|
* host owns their endpoint, transport, authentication contract and tool name.
|
||||||
|
* `custom` is the only kind whose connection details come from the user.
|
||||||
|
*/
|
||||||
|
export const SEARCH_MCP_CATALOG = {
|
||||||
|
tavily: {
|
||||||
|
transport: 'http',
|
||||||
|
url: 'https://mcp.tavily.com/mcp/',
|
||||||
|
authStyle: 'query',
|
||||||
|
authParam: 'tavilyApiKey',
|
||||||
|
toolName: 'tavily_search',
|
||||||
|
countArg: 'max_results',
|
||||||
|
minResults: 5,
|
||||||
|
maxResultsLimit: 20,
|
||||||
|
apiKeyEnv: 'TAVILY_API_KEY',
|
||||||
|
needsKey: true,
|
||||||
|
},
|
||||||
|
brave: {
|
||||||
|
transport: 'stdio',
|
||||||
|
command: 'npx',
|
||||||
|
args: ['-y', '@brave/brave-search-mcp-server@2.1.3'],
|
||||||
|
authStyle: 'env',
|
||||||
|
authParam: 'BRAVE_API_KEY',
|
||||||
|
toolName: 'brave_web_search',
|
||||||
|
countArg: 'count',
|
||||||
|
minResults: 1,
|
||||||
|
maxResultsLimit: 20,
|
||||||
|
apiKeyEnv: 'BRAVE_API_KEY',
|
||||||
|
needsKey: true,
|
||||||
|
},
|
||||||
|
exa: {
|
||||||
|
transport: 'http',
|
||||||
|
url: 'https://mcp.exa.ai/mcp',
|
||||||
|
authStyle: 'header',
|
||||||
|
authParam: 'x-api-key',
|
||||||
|
toolName: 'web_search_exa',
|
||||||
|
countArg: 'numResults',
|
||||||
|
apiKeyEnv: 'EXA_API_KEY',
|
||||||
|
needsKey: true,
|
||||||
|
},
|
||||||
|
perplexity: {
|
||||||
|
transport: 'http',
|
||||||
|
url: 'https://api.perplexity.ai/mcp',
|
||||||
|
authStyle: 'header',
|
||||||
|
authParam: 'Authorization',
|
||||||
|
authPrefix: 'Bearer ',
|
||||||
|
toolName: 'perplexity_search',
|
||||||
|
countArg: 'max_results',
|
||||||
|
minResults: 1,
|
||||||
|
maxResultsLimit: 20,
|
||||||
|
apiKeyEnv: 'PERPLEXITY_API_KEY',
|
||||||
|
needsKey: true,
|
||||||
|
},
|
||||||
|
duckduckgo: {
|
||||||
|
transport: 'stdio',
|
||||||
|
command: 'npx',
|
||||||
|
args: ['-y', 'duckduckgo-mcp-server@0.1.2'],
|
||||||
|
authStyle: 'env',
|
||||||
|
authParam: '',
|
||||||
|
toolName: 'duckduckgo_web_search',
|
||||||
|
countArg: 'count',
|
||||||
|
minResults: 1,
|
||||||
|
maxResultsLimit: 20,
|
||||||
|
needsKey: false,
|
||||||
|
},
|
||||||
|
custom: {
|
||||||
|
transport: 'http',
|
||||||
|
url: '',
|
||||||
|
authStyle: 'query',
|
||||||
|
authParam: '',
|
||||||
|
authPrefix: '',
|
||||||
|
toolName: '',
|
||||||
|
countArg: '',
|
||||||
|
needsKey: false,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
/** The provider ids offered in the settings UI. */
|
||||||
|
export const KNOWN_KINDS = Object.keys(SEARCH_MCP_CATALOG);
|
||||||
|
|
||||||
|
export function clampSearchResults(server, value) {
|
||||||
|
const minimum = server.minResults ?? 1;
|
||||||
|
const maximum = server.maxResultsLimit ?? value;
|
||||||
|
return Math.min(maximum, Math.max(minimum, value));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolve a stored entry without allowing known-provider connection overrides. */
|
||||||
|
export function resolveServer(server) {
|
||||||
|
const kind = typeof server?.kind === 'string' && Object.hasOwn(SEARCH_MCP_CATALOG, server.kind)
|
||||||
|
? server.kind
|
||||||
|
: 'custom';
|
||||||
|
const preset = SEARCH_MCP_CATALOG[kind];
|
||||||
|
if (kind !== 'custom') {
|
||||||
|
return {
|
||||||
|
id: typeof server.id === 'string' ? server.id : '',
|
||||||
|
kind,
|
||||||
|
apiKey: typeof server.apiKey === 'string' ? server.apiKey : undefined,
|
||||||
|
apiKeyEnv: typeof server.apiKeyEnv === 'string' ? server.apiKeyEnv : '',
|
||||||
|
maxResults: server.maxResults,
|
||||||
|
transport: preset.transport,
|
||||||
|
url: preset.url ?? '',
|
||||||
|
command: preset.command ?? '',
|
||||||
|
args: [...(preset.args ?? [])],
|
||||||
|
authStyle: preset.authStyle ?? '',
|
||||||
|
authParam: preset.authParam ?? '',
|
||||||
|
authPrefix: preset.authPrefix ?? '',
|
||||||
|
toolName: preset.toolName ?? '',
|
||||||
|
countArg: preset.countArg ?? '',
|
||||||
|
minResults: preset.minResults,
|
||||||
|
maxResultsLimit: preset.maxResultsLimit,
|
||||||
|
needsKey: preset.needsKey ?? false,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
id: typeof server.id === 'string' ? server.id : '',
|
||||||
|
kind: 'custom',
|
||||||
|
apiKey: typeof server.apiKey === 'string' ? server.apiKey : undefined,
|
||||||
|
apiKeyEnv: typeof server.apiKeyEnv === 'string' ? server.apiKeyEnv : '',
|
||||||
|
maxResults: server.maxResults,
|
||||||
|
transport: server.transport || preset.transport || 'http',
|
||||||
|
url: server.url || preset.url || '',
|
||||||
|
command: server.command || preset.command || '',
|
||||||
|
args: Array.isArray(server.args) ? [...server.args] : [...(preset.args ?? [])],
|
||||||
|
authStyle: server.authStyle || preset.authStyle || '',
|
||||||
|
authParam: server.authParam || preset.authParam || '',
|
||||||
|
authPrefix: server.authPrefix || preset.authPrefix || '',
|
||||||
|
toolName: server.toolName || preset.toolName || '',
|
||||||
|
countArg: preset.countArg || '',
|
||||||
|
minResults: preset.minResults,
|
||||||
|
maxResultsLimit: preset.maxResultsLimit,
|
||||||
|
needsKey: preset.needsKey ?? false,
|
||||||
|
};
|
||||||
|
}
|
||||||
1077
lib/client.browser.js
Normal file
1077
lib/client.browser.js
Normal file
File diff suppressed because it is too large
Load diff
150
lib/client.js
Normal file
150
lib/client.js
Normal file
|
|
@ -0,0 +1,150 @@
|
||||||
|
/** MCP transport layer for one search. */
|
||||||
|
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||||
|
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
|
||||||
|
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||||
|
import { WebError } from '@deepseek-ai/dsh-web';
|
||||||
|
import { Agent, fetch as undiciFetch } from 'undici';
|
||||||
|
import { clampSearchResults } from './catalog.js';
|
||||||
|
import { validateHttpEndpoint } from './url-policy.js';
|
||||||
|
|
||||||
|
/** Run one search through a resolved server entry. */
|
||||||
|
export async function callMcpSearch(server, key, args, signal) {
|
||||||
|
if (!server.toolName) {
|
||||||
|
throw new WebError(
|
||||||
|
`search-mcp server "${server.id}": no MCP tool name (set "toolName" or pick a known kind)`,
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let runtime;
|
||||||
|
const client = new Client({ name: 'dsh-search-mcp', version: '0.2.0' }, { capabilities: {} });
|
||||||
|
try {
|
||||||
|
runtime = server.transport === 'stdio'
|
||||||
|
? { transport: stdioTransport(server, key), close: async () => {} }
|
||||||
|
: await httpRuntime(server, key, signal);
|
||||||
|
await race(client.connect(runtime.transport), signal, `connect to "${server.id}"`);
|
||||||
|
const callArgs = { query: args.query };
|
||||||
|
if (server.countArg.length > 0 && args.maxResults !== undefined) {
|
||||||
|
callArgs[server.countArg] = clampSearchResults(server, args.maxResults);
|
||||||
|
}
|
||||||
|
const result = await race(
|
||||||
|
client.callTool({ name: server.toolName, arguments: callArgs }),
|
||||||
|
signal,
|
||||||
|
`call "${server.id}" tool "${server.toolName}"`,
|
||||||
|
);
|
||||||
|
if (result.isError) {
|
||||||
|
throw new WebError(
|
||||||
|
`search-mcp: MCP server "${server.id}" tool "${server.toolName}" reported an error`,
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof WebError) throw error;
|
||||||
|
if (signal?.aborted) throw aborted(`complete request for "${server.id}"`);
|
||||||
|
const detail = error?.name === 'SearchMcpUrlPolicyError' ? `: ${error.message}` : '';
|
||||||
|
throw new WebError(
|
||||||
|
`search-mcp server "${server.id}" request failed${detail}`,
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
await client.close();
|
||||||
|
} catch {
|
||||||
|
// The connection is already gone.
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await runtime?.close();
|
||||||
|
} catch {
|
||||||
|
// The dedicated dispatcher has no shared state to recover.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build a DNS-pinned streamable-http transport and its cleanup. */
|
||||||
|
async function httpRuntime(server, key, signal) {
|
||||||
|
const validated = await validateHttpEndpoint(server.url, { signal });
|
||||||
|
const url = new URL(validated.url);
|
||||||
|
const headers = {};
|
||||||
|
if (key !== undefined && key.length > 0 && server.authParam.length > 0) {
|
||||||
|
const value = `${server.authPrefix ?? ''}${key}`;
|
||||||
|
if (server.authStyle === 'query') url.searchParams.set(server.authParam, value);
|
||||||
|
else if (server.authStyle === 'header') headers[server.authParam] = value;
|
||||||
|
}
|
||||||
|
|
||||||
|
const agent = new Agent({
|
||||||
|
connect: { lookup: validated.lookup },
|
||||||
|
connections: validated.addresses.length,
|
||||||
|
maxRedirections: 0,
|
||||||
|
});
|
||||||
|
const expectedOrigin = url.origin;
|
||||||
|
const secureFetch = async (input, init = {}) => {
|
||||||
|
const requestUrl = new URL(typeof input === 'string' || input instanceof URL ? input : input.url);
|
||||||
|
if (requestUrl.origin !== expectedOrigin) {
|
||||||
|
throw new Error('search-mcp URL policy: request origin changed after validation');
|
||||||
|
}
|
||||||
|
return undiciFetch(input, {
|
||||||
|
...init,
|
||||||
|
dispatcher: agent,
|
||||||
|
redirect: 'error',
|
||||||
|
...(signal !== undefined ? { signal: combineSignals(signal, init.signal) } : {}),
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
transport: new StreamableHTTPClientTransport(url, {
|
||||||
|
fetch: secureFetch,
|
||||||
|
requestInit: {
|
||||||
|
headers,
|
||||||
|
redirect: 'error',
|
||||||
|
...(signal !== undefined ? { signal } : {}),
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
close: () => agent.close(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build a stdio transport; the authParam name doubles as the env var name. */
|
||||||
|
function stdioTransport(server, key) {
|
||||||
|
const env = { ...process.env };
|
||||||
|
if (key !== undefined && key.length > 0 && server.authParam.length > 0) {
|
||||||
|
env[server.authParam] = `${server.authPrefix ?? ''}${key}`;
|
||||||
|
}
|
||||||
|
return new StdioClientTransport({
|
||||||
|
command: server.command,
|
||||||
|
args: server.args ?? [],
|
||||||
|
env,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Race a protocol operation against the caller/timeout abort signal. */
|
||||||
|
function race(promise, signal, stage) {
|
||||||
|
if (signal === undefined) return promise;
|
||||||
|
if (signal.aborted) throw aborted(stage);
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const onAbort = () => {
|
||||||
|
signal.removeEventListener('abort', onAbort);
|
||||||
|
reject(aborted(stage));
|
||||||
|
};
|
||||||
|
signal.addEventListener('abort', onAbort, { once: true });
|
||||||
|
promise.then(
|
||||||
|
(value) => {
|
||||||
|
signal.removeEventListener('abort', onAbort);
|
||||||
|
resolve(value);
|
||||||
|
},
|
||||||
|
(error) => {
|
||||||
|
signal.removeEventListener('abort', onAbort);
|
||||||
|
reject(error);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function combineSignals(base, request) {
|
||||||
|
if (request === undefined || request === null || request === base) return base;
|
||||||
|
return AbortSignal.any([base, request]);
|
||||||
|
}
|
||||||
|
|
||||||
|
function aborted(stage) {
|
||||||
|
return new WebError(`search-mcp: aborted while trying to ${stage}`, 'WEB_ABORTED');
|
||||||
|
}
|
||||||
117
lib/extract.js
Normal file
117
lib/extract.js
Normal file
|
|
@ -0,0 +1,117 @@
|
||||||
|
/**
|
||||||
|
* Generic normalization of an MCP `tools/call` result into the
|
||||||
|
* `web_search` provider shape `{ sources, truncated, content? }`.
|
||||||
|
*
|
||||||
|
* Different search MCP servers return wildly different payloads (Tavily
|
||||||
|
* `results[]`, Brave `web.results[]`, Exa `results[]`, Perplexity text +
|
||||||
|
* citations, DuckDuckGo `results[]`...). Instead of mapping each vendor, we
|
||||||
|
* recursively walk the returned JSON and collect every object that carries a
|
||||||
|
* string `url` as a source, taking title / snippet / date from the common
|
||||||
|
* field names. A top-level `answer` (or non-JSON text blocks) becomes the
|
||||||
|
* `content` answer.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const TITLE_KEYS = ['title', 'name', 'headline'];
|
||||||
|
const SNIPPET_KEYS = ['snippet', 'content', 'description', 'text', 'excerpt', 'summary'];
|
||||||
|
const DATE_KEYS = [
|
||||||
|
'published_date',
|
||||||
|
'publishedDate',
|
||||||
|
'published_at',
|
||||||
|
'publish_date',
|
||||||
|
'publishedAt',
|
||||||
|
'page_age',
|
||||||
|
'age',
|
||||||
|
'date',
|
||||||
|
];
|
||||||
|
|
||||||
|
/** Cap a snippet so a single source cannot blow up the context window. */
|
||||||
|
const MAX_SNIPPET_CHARS = 600;
|
||||||
|
/** Cap the answer text block. */
|
||||||
|
const MAX_CONTENT_CHARS = 4000;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Project one MCP `tools/call` result into `{ sources, truncated, content? }`.
|
||||||
|
*
|
||||||
|
* @param result - the raw `CallToolResult` from the MCP SDK.
|
||||||
|
* @returns the normalized provider result; `truncated` is always false
|
||||||
|
* because the `ctx.web` seam owns the final `maxResults` cap.
|
||||||
|
*/
|
||||||
|
export function extractSearchResult(result) {
|
||||||
|
const bucket = {
|
||||||
|
sources: [],
|
||||||
|
seen: new Set(),
|
||||||
|
content: '',
|
||||||
|
};
|
||||||
|
if (result !== null && typeof result === 'object') {
|
||||||
|
if (result.structuredContent !== undefined) collect(result.structuredContent, bucket);
|
||||||
|
const blocks = Array.isArray(result.content) ? result.content : [];
|
||||||
|
for (const block of blocks) {
|
||||||
|
if (block === null || typeof block !== 'object') continue;
|
||||||
|
if (block.type === 'json' && block.json !== undefined) {
|
||||||
|
collect(block.json, bucket);
|
||||||
|
} else if (block.type === 'text' && typeof block.text === 'string') {
|
||||||
|
const parsed = tryParseJson(block.text);
|
||||||
|
if (parsed !== undefined) collect(parsed, bucket);
|
||||||
|
else if (bucket.content.length === 0 && block.text.trim().length > 0) {
|
||||||
|
bucket.content = block.text.trim().slice(0, MAX_CONTENT_CHARS);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
sources: bucket.sources,
|
||||||
|
truncated: false,
|
||||||
|
...(bucket.content.length > 0 ? { content: bucket.content } : {}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Depth-first walk collecting source objects and the `answer` field. */
|
||||||
|
function collect(node, bucket) {
|
||||||
|
if (Array.isArray(node)) {
|
||||||
|
for (const item of node) collect(item, bucket);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (node === null || typeof node !== 'object') return;
|
||||||
|
if (typeof node.url === 'string' && /^https?:\/\//i.test(node.url)) {
|
||||||
|
if (!bucket.seen.has(node.url)) {
|
||||||
|
bucket.seen.add(node.url);
|
||||||
|
const title = firstOf(node, TITLE_KEYS);
|
||||||
|
const snippet = truncate(firstOf(node, SNIPPET_KEYS), MAX_SNIPPET_CHARS);
|
||||||
|
const publishedAt = firstOf(node, DATE_KEYS);
|
||||||
|
bucket.sources.push({
|
||||||
|
url: node.url,
|
||||||
|
...(title !== undefined ? { title } : {}),
|
||||||
|
...(snippet !== undefined ? { snippet } : {}),
|
||||||
|
...(publishedAt !== undefined ? { publishedAt } : {}),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (bucket.content.length === 0 && typeof node.answer === 'string' && node.answer.trim().length > 0) {
|
||||||
|
bucket.content = node.answer.trim().slice(0, MAX_CONTENT_CHARS);
|
||||||
|
}
|
||||||
|
for (const value of Object.values(node)) collect(value, bucket);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** First non-empty string among the candidate keys, else undefined. */
|
||||||
|
function firstOf(node, keys) {
|
||||||
|
for (const key of keys) {
|
||||||
|
const value = node[key];
|
||||||
|
if (typeof value === 'string' && value.trim().length > 0) return value.trim();
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function truncate(value, max) {
|
||||||
|
if (value === undefined) return undefined;
|
||||||
|
return value.length > max ? `${value.slice(0, max)}…` : value;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Parse a JSON text block; returns undefined when it is not JSON. */
|
||||||
|
function tryParseJson(text) {
|
||||||
|
try {
|
||||||
|
return JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
}
|
||||||
112
lib/index.js
Normal file
112
lib/index.js
Normal file
|
|
@ -0,0 +1,112 @@
|
||||||
|
/**
|
||||||
|
* dsh-search-mcp — replace dsh's built-in web search with search MCP servers.
|
||||||
|
*
|
||||||
|
* Adapted for DeepSeek Harness 0.1.2+: settings use
|
||||||
|
* `ctx.settings.installSection` (the old free-function
|
||||||
|
* `installSettingsSection` from 0.1.1-rc.2 no longer exists).
|
||||||
|
*
|
||||||
|
* A Cordis plugin that
|
||||||
|
* - registers a `ctx.web` search provider under the id `search-mcp`, and
|
||||||
|
* - installs a Settings section (`search-mcp`) where the user manages the
|
||||||
|
* search MCP server list (kind, endpoint/command, API key or key env
|
||||||
|
* reference, tool name) plus `defaultServer` / `maxResults` /
|
||||||
|
* `searchTimeoutMs` from the web Settings → Plugins page.
|
||||||
|
*
|
||||||
|
* The package's `cordis.patch.yml` (bundle layer) switches
|
||||||
|
* `web.searchProvider` to `search-mcp` and disables the built-in
|
||||||
|
* `web-search-deepseek` provider, so while this plugin is enabled the
|
||||||
|
* built-in search is unavailable and every `web_search` call runs through
|
||||||
|
* the configured MCP server(s). Removing the package restores the built-in.
|
||||||
|
*/
|
||||||
|
import z from '@deepseek-ai/schemastery';
|
||||||
|
import { credentialRef } from '@deepseek-ai/dsh-credentials';
|
||||||
|
import { launchEnvironmentOf } from '@deepseek-ai/dsh-launch-environment';
|
||||||
|
import { SearchMCPProvider } from './provider.js';
|
||||||
|
|
||||||
|
/** Cordis plugin name used by loader diagnostics. */
|
||||||
|
export const name = 'search-mcp';
|
||||||
|
|
||||||
|
/** The web seam this provider registers into. */
|
||||||
|
export const inject = ['web'];
|
||||||
|
|
||||||
|
const serverSchema = z.object({
|
||||||
|
id: z.string(),
|
||||||
|
kind: z.string().default('custom'),
|
||||||
|
transport: z.string().default('http'),
|
||||||
|
url: z.string().default(''),
|
||||||
|
command: z.string().default(''),
|
||||||
|
args: z.array(z.string()).default([]),
|
||||||
|
apiKey: z.string().role('secret'),
|
||||||
|
apiKeyEnv: z.string().role('credential-ref').default(''),
|
||||||
|
authStyle: z.string().default(''),
|
||||||
|
authParam: z.string().default(''),
|
||||||
|
authPrefix: z.string().default(''),
|
||||||
|
toolName: z.string().default(''),
|
||||||
|
// Note: this schemastery fork has no `.optional()`; object fields are
|
||||||
|
// optional unless `.required()` is applied, so absence is already allowed.
|
||||||
|
maxResults: z.number().step(1).min(1).max(50),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const Config = z.object({
|
||||||
|
defaultServer: z.string().default(''),
|
||||||
|
maxResults: z.number().step(1).min(1).max(50).default(8),
|
||||||
|
searchTimeoutMs: z.number().step(1).min(1000).default(30000),
|
||||||
|
servers: z.array(serverSchema).default([]),
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Settings namespace owning this plugin's section (Settings → Plugins card). */
|
||||||
|
export const SEARCH_MCP_SETTINGS_NAMESPACE = 'search-mcp';
|
||||||
|
|
||||||
|
/** Register the search provider and the live settings section. */
|
||||||
|
export function apply(ctx, config) {
|
||||||
|
let current = () => config;
|
||||||
|
// Optional settings seam: fall back to the composition entry when settings
|
||||||
|
// is absent (same pattern as @deepseek-ai/dsh-web-search-deepseek).
|
||||||
|
ctx.inject(['settings'], (settingsCtx) => {
|
||||||
|
settingsCtx.settings.installSection(ctx, SEARCH_MCP_SETTINGS_NAMESPACE, Config, config, {
|
||||||
|
setSource: (source) => {
|
||||||
|
current = source;
|
||||||
|
},
|
||||||
|
// Provider projects the section per search; no re-registration needed.
|
||||||
|
onChange: () => {},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
// `registerSearchProvider` owns its cleanup via ctx.effect (HMR/dispose safe).
|
||||||
|
ctx.web.registerSearchProvider(new SearchMCPProvider(() => resolveOptions(ctx, current())));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Project the authoritative config into per-search options. The section
|
||||||
|
* returned by `setSource` (settings.yaml `search-mcp:` block) replaces the
|
||||||
|
* row config entirely, matching how every other settings section behaves.
|
||||||
|
*
|
||||||
|
* @param ctx - plugin context supplying the credential and environment planes.
|
||||||
|
* @param config - the currently authoritative section.
|
||||||
|
* @returns options for one search.
|
||||||
|
*/
|
||||||
|
function resolveOptions(ctx, config) {
|
||||||
|
return {
|
||||||
|
servers: config.servers ?? [],
|
||||||
|
defaultServer: config.defaultServer ?? '',
|
||||||
|
maxResults: config.maxResults ?? 8,
|
||||||
|
searchTimeoutMs: config.searchTimeoutMs ?? 30000,
|
||||||
|
resolveKey: async (server) => {
|
||||||
|
if (server.apiKey !== undefined && server.apiKey.length > 0) return server.apiKey;
|
||||||
|
const envName = server.apiKeyEnv ?? '';
|
||||||
|
if (envName.length === 0) return undefined;
|
||||||
|
const credentials = ctx.get('credentials');
|
||||||
|
if (credentials !== undefined) {
|
||||||
|
try {
|
||||||
|
const resolved = await credentials.resolve(credentialRef(envName));
|
||||||
|
if (resolved !== undefined && resolved.value !== undefined && resolved.value.length > 0) {
|
||||||
|
return resolved.value;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
/* fall through to the launch environment */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const ambient = launchEnvironmentOf(ctx).get(envName);
|
||||||
|
return ambient !== undefined && ambient.value.length > 0 ? ambient.value : undefined;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
83
lib/provider.js
Normal file
83
lib/provider.js
Normal file
|
|
@ -0,0 +1,83 @@
|
||||||
|
/**
|
||||||
|
* The `search-mcp` web search provider.
|
||||||
|
*
|
||||||
|
* Registers into `ctx.web` under the stable id `search-mcp`; the profile
|
||||||
|
* patch switches `web.searchProvider` to this id and disables the built-in
|
||||||
|
* DeepSeek provider, so the model-facing `web_search` tool executes entirely
|
||||||
|
* through the configured search MCP server(s).
|
||||||
|
*/
|
||||||
|
import { WebError } from '@deepseek-ai/dsh-web';
|
||||||
|
import { resolveServer } from './catalog.js';
|
||||||
|
import { callMcpSearch } from './client.js';
|
||||||
|
import { extractSearchResult } from './extract.js';
|
||||||
|
|
||||||
|
/** Stable provider id the `web` row's `searchProvider` config selects. */
|
||||||
|
export const SEARCH_MCP_PROVIDER_ID = 'search-mcp';
|
||||||
|
|
||||||
|
/** The web search provider served by this plugin. */
|
||||||
|
export class SearchMCPProvider {
|
||||||
|
id = SEARCH_MCP_PROVIDER_ID;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param resolveOptions - snapshots the authoritative config (row config,
|
||||||
|
* or the live settings section) at the START of each operation, so one
|
||||||
|
* search never mixes two settings saves.
|
||||||
|
*/
|
||||||
|
constructor(resolveOptions) {
|
||||||
|
this.resolveOptions = resolveOptions;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Usable when at least one server entry exists; precise errors surface at search time. */
|
||||||
|
available() {
|
||||||
|
const options = this.resolveOptions();
|
||||||
|
return Array.isArray(options.servers) && options.servers.length > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
async search(request, signal) {
|
||||||
|
const options = this.resolveOptions();
|
||||||
|
const server = pickServer(options);
|
||||||
|
const resolved = resolveServer(server);
|
||||||
|
const maxResults = resolved.maxResults ?? options.maxResults ?? request.maxResults ?? 8;
|
||||||
|
const key = await options.resolveKey(resolved);
|
||||||
|
if (resolved.needsKey && (key === undefined || key.length === 0)) {
|
||||||
|
const ref = resolved.apiKeyEnv.length > 0 ? resolved.apiKeyEnv : 'apiKey';
|
||||||
|
throw new WebError(
|
||||||
|
`search-mcp server "${resolved.id}" (${resolved.kind}) has no API key; set "apiKey" or a resolvable "apiKeyEnv" (${ref}) in Settings → Plugins → search-mcp`,
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const combined = buildSignal(signal, options.searchTimeoutMs);
|
||||||
|
const outcome = await callMcpSearch(resolved, key, { query: request.query, maxResults }, combined);
|
||||||
|
return extractSearchResult(outcome);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Select the default server, falling back to the first configured entry. */
|
||||||
|
function pickServer(options) {
|
||||||
|
const servers = Array.isArray(options.servers) ? options.servers : [];
|
||||||
|
if (servers.length === 0) {
|
||||||
|
throw new WebError(
|
||||||
|
'search-mcp: no search MCP servers configured; add one in Settings → Plugins → search-mcp',
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (options.defaultServer !== undefined && options.defaultServer.length > 0) {
|
||||||
|
const found = servers.find((entry) => entry.id === options.defaultServer);
|
||||||
|
if (found === undefined) {
|
||||||
|
throw new WebError(
|
||||||
|
`search-mcp: defaultServer "${options.defaultServer}" is not configured; known servers: ${servers
|
||||||
|
.map((entry) => `"${entry.id}"`)
|
||||||
|
.join(', ') || '(none)'}`,
|
||||||
|
'WEB_PROVIDER_ERROR',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return found;
|
||||||
|
}
|
||||||
|
return servers[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Combine the caller's cancellation with the configured timeout. */
|
||||||
|
function buildSignal(signal, timeoutMs) {
|
||||||
|
const timeout = AbortSignal.timeout(timeoutMs);
|
||||||
|
return signal !== undefined ? AbortSignal.any([signal, timeout]) : timeout;
|
||||||
|
}
|
||||||
243
lib/url-policy.js
Normal file
243
lib/url-policy.js
Normal file
|
|
@ -0,0 +1,243 @@
|
||||||
|
import { lookup as dnsLookup } from 'node:dns';
|
||||||
|
import ipaddr from 'ipaddr.js';
|
||||||
|
|
||||||
|
const ALLOWED_PROTOCOLS = new Set(['http:', 'https:']);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse and resolve one HTTP endpoint before any request is sent.
|
||||||
|
* Every resolved address must be globally routable.
|
||||||
|
*/
|
||||||
|
export async function validateHttpEndpoint(input, options = {}) {
|
||||||
|
if (options.signal?.aborted) throw abortedPolicyError();
|
||||||
|
if (typeof input !== 'string' || input.length === 0 || input !== input.trim()) {
|
||||||
|
throw policyError('endpoint must be a non-empty canonical URL');
|
||||||
|
}
|
||||||
|
|
||||||
|
let url;
|
||||||
|
try {
|
||||||
|
url = new URL(input);
|
||||||
|
} catch {
|
||||||
|
throw policyError('endpoint is not a valid URL');
|
||||||
|
}
|
||||||
|
if (!ALLOWED_PROTOCOLS.has(url.protocol)) {
|
||||||
|
throw policyError('endpoint protocol must be http or https');
|
||||||
|
}
|
||||||
|
if (url.username.length > 0 || url.password.length > 0) {
|
||||||
|
throw policyError('endpoint must not contain user information');
|
||||||
|
}
|
||||||
|
if (url.hostname.length === 0) {
|
||||||
|
throw policyError('endpoint hostname is missing');
|
||||||
|
}
|
||||||
|
|
||||||
|
const hostname = stripIpv6Brackets(url.hostname).toLowerCase();
|
||||||
|
const comparable = hostname.endsWith('.') ? hostname.slice(0, -1) : hostname;
|
||||||
|
if (comparable === 'localhost' || comparable.endsWith('.localhost')) {
|
||||||
|
throw policyError('localhost endpoints are not allowed');
|
||||||
|
}
|
||||||
|
rejectAmbiguousIpv4(input, comparable);
|
||||||
|
|
||||||
|
let addresses;
|
||||||
|
if (ipaddr.isValid(comparable)) {
|
||||||
|
addresses = [{ address: normalizeAddress(comparable), family: addressFamily(comparable) }];
|
||||||
|
} else {
|
||||||
|
addresses = await resolveAll(comparable, options.lookup ?? dnsLookup, options.signal);
|
||||||
|
}
|
||||||
|
if (addresses.length === 0) {
|
||||||
|
throw policyError('endpoint hostname did not resolve');
|
||||||
|
}
|
||||||
|
|
||||||
|
const normalized = deduplicateAddresses(addresses);
|
||||||
|
// Keep globally routable answers. Also allow RFC 2544 (198.18.0.0/15), which
|
||||||
|
// Clash/V2Ray fake-IP / TUN mode commonly returns for otherwise-public hosts.
|
||||||
|
// Real private/LAN answers are dropped; fail only when nothing usable remains.
|
||||||
|
const allowed = normalized.filter((record) => isAllowedEndpointAddress(record.address));
|
||||||
|
if (allowed.length === 0) {
|
||||||
|
throw policyError('endpoint hostname resolves to a non-public address');
|
||||||
|
}
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
url,
|
||||||
|
hostname: comparable,
|
||||||
|
addresses: Object.freeze(allowed.map((record) => Object.freeze(record))),
|
||||||
|
lookup: createPinnedLookup(comparable, allowed),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Return true only for globally routable IPv4 or IPv6 addresses. */
|
||||||
|
export function isPublicAddress(input) {
|
||||||
|
let address;
|
||||||
|
try {
|
||||||
|
address = ipaddr.parse(stripIpv6Brackets(input));
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (address.kind() === 'ipv6') {
|
||||||
|
if (address.isIPv4MappedAddress()) {
|
||||||
|
address = address.toIPv4Address();
|
||||||
|
} else if (isIpv4CompatibleAddress(address)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return address.range() === 'unicast';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Addresses safe to dial for remote MCP endpoints (public or proxy fake-IP). */
|
||||||
|
export function isAllowedEndpointAddress(input) {
|
||||||
|
if (isPublicAddress(input)) return true;
|
||||||
|
return isProxyFakeIpAddress(input);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** RFC 2544 benchmarking range used as DNS fake-IP by many local proxies. */
|
||||||
|
export function isProxyFakeIpAddress(input) {
|
||||||
|
let address;
|
||||||
|
try {
|
||||||
|
address = ipaddr.parse(stripIpv6Brackets(input));
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (address.kind() === 'ipv6') {
|
||||||
|
if (address.isIPv4MappedAddress()) {
|
||||||
|
address = address.toIPv4Address();
|
||||||
|
} else {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (address.kind() !== 'ipv4') return false;
|
||||||
|
// 198.18.0.0/15
|
||||||
|
const [a, b] = address.octets;
|
||||||
|
return a === 198 && (b === 18 || b === 19);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build a Node-compatible DNS lookup that never resolves beyond the pinned set. */
|
||||||
|
export function createPinnedLookup(hostname, records) {
|
||||||
|
const target = normalizeHostname(hostname);
|
||||||
|
const frozen = deduplicateAddresses(records);
|
||||||
|
let cursor = 0;
|
||||||
|
return (requested, options, callback) => {
|
||||||
|
const requestedHost = normalizeHostname(requested);
|
||||||
|
if (requestedHost !== target) {
|
||||||
|
const error = policyError('connection attempted an unvalidated hostname');
|
||||||
|
error.code = 'EACCES';
|
||||||
|
queueMicrotask(() => callback(error));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const lookupOptions = typeof options === 'object' && options !== null ? options : {};
|
||||||
|
const family = Number(lookupOptions.family) || 0;
|
||||||
|
const candidates = family === 4 || family === 6
|
||||||
|
? frozen.filter((record) => record.family === family)
|
||||||
|
: frozen;
|
||||||
|
if (candidates.length === 0) {
|
||||||
|
const error = policyError('no validated address matches the requested family');
|
||||||
|
error.code = 'ENOTFOUND';
|
||||||
|
queueMicrotask(() => callback(error));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (lookupOptions.all === true) {
|
||||||
|
queueMicrotask(() => callback(null, candidates.map((record) => ({ ...record }))));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const selected = candidates[cursor++ % candidates.length];
|
||||||
|
queueMicrotask(() => callback(null, selected.address, selected.family));
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveAll(hostname, lookup, signal) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
if (signal?.aborted) {
|
||||||
|
reject(abortedPolicyError());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let settled = false;
|
||||||
|
const finish = (callback, value) => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
signal?.removeEventListener('abort', onAbort);
|
||||||
|
callback(value);
|
||||||
|
};
|
||||||
|
const onAbort = () => finish(reject, abortedPolicyError());
|
||||||
|
signal?.addEventListener('abort', onAbort, { once: true });
|
||||||
|
lookup(hostname, { all: true, verbatim: true }, (error, records) => {
|
||||||
|
if (settled) return;
|
||||||
|
if (error) {
|
||||||
|
finish(reject, policyError('endpoint hostname resolution failed'));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const list = Array.isArray(records) ? records : records === undefined ? [] : [records];
|
||||||
|
try {
|
||||||
|
finish(resolve, list.map((record) => {
|
||||||
|
const raw = typeof record === 'string' ? record : record.address;
|
||||||
|
return { address: normalizeAddress(raw), family: addressFamily(raw) };
|
||||||
|
}));
|
||||||
|
} catch {
|
||||||
|
finish(reject, policyError('endpoint hostname returned an invalid address'));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function deduplicateAddresses(records) {
|
||||||
|
const seen = new Set();
|
||||||
|
const result = [];
|
||||||
|
for (const record of records) {
|
||||||
|
const address = normalizeAddress(record.address);
|
||||||
|
const family = addressFamily(address);
|
||||||
|
const key = `${family}:${address}`;
|
||||||
|
if (seen.has(key)) continue;
|
||||||
|
seen.add(key);
|
||||||
|
result.push({ address, family });
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeAddress(input) {
|
||||||
|
let address = ipaddr.parse(stripIpv6Brackets(input));
|
||||||
|
if (address.kind() === 'ipv6' && address.isIPv4MappedAddress()) {
|
||||||
|
address = address.toIPv4Address();
|
||||||
|
}
|
||||||
|
return address.toNormalizedString();
|
||||||
|
}
|
||||||
|
|
||||||
|
function addressFamily(input) {
|
||||||
|
const address = ipaddr.parse(stripIpv6Brackets(input));
|
||||||
|
if (address.kind() === 'ipv6' && address.isIPv4MappedAddress()) return 4;
|
||||||
|
return address.kind() === 'ipv4' ? 4 : 6;
|
||||||
|
}
|
||||||
|
|
||||||
|
function rejectAmbiguousIpv4(input, hostname) {
|
||||||
|
const authority = input.match(/^[A-Za-z][A-Za-z0-9+.-]*:\/\/([^/?#]+)/)?.[1] ?? '';
|
||||||
|
const rawHost = authority.startsWith('[')
|
||||||
|
? authority.slice(1, authority.indexOf(']'))
|
||||||
|
: authority.replace(/:\d*$/, '');
|
||||||
|
if (!rawHost.includes(':') && ipaddr.IPv4.isValid(rawHost)) {
|
||||||
|
const canonical = ipaddr.IPv4.parse(rawHost).toString();
|
||||||
|
if (rawHost !== canonical || hostname !== canonical) {
|
||||||
|
throw policyError('endpoint contains a non-canonical IPv4 address');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function isIpv4CompatibleAddress(address) {
|
||||||
|
return address.parts.slice(0, 6).every((part) => part === 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeHostname(input) {
|
||||||
|
const value = stripIpv6Brackets(String(input)).toLowerCase();
|
||||||
|
return value.endsWith('.') ? value.slice(0, -1) : value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function stripIpv6Brackets(input) {
|
||||||
|
return input.startsWith('[') && input.endsWith(']') ? input.slice(1, -1) : input;
|
||||||
|
}
|
||||||
|
|
||||||
|
function abortedPolicyError() {
|
||||||
|
const error = policyError('endpoint validation was aborted');
|
||||||
|
error.code = 'ABORT_ERR';
|
||||||
|
return error;
|
||||||
|
}
|
||||||
|
|
||||||
|
function policyError(message) {
|
||||||
|
const error = new Error(`search-mcp URL policy: ${message}`);
|
||||||
|
error.name = 'SearchMcpUrlPolicyError';
|
||||||
|
return error;
|
||||||
|
}
|
||||||
2202
package-lock.json
generated
Normal file
2202
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load diff
63
package.json
Normal file
63
package.json
Normal file
|
|
@ -0,0 +1,63 @@
|
||||||
|
{
|
||||||
|
"name": "dsh-search-mcp",
|
||||||
|
"version": "0.2.0",
|
||||||
|
"description": "Replace dsh's built-in web search with search MCP servers (Tavily / Brave / Exa / Perplexity / DuckDuckGo / custom), configured from the web Settings page. When this plugin is enabled the built-in DeepSeek search provider is disabled.",
|
||||||
|
"type": "module",
|
||||||
|
"main": "lib/index.js",
|
||||||
|
"exports": {
|
||||||
|
".": "./lib/index.js",
|
||||||
|
"./client": "./lib/client.browser.js",
|
||||||
|
"./cordis.patch.yml": "./cordis.patch.yml",
|
||||||
|
"./package.json": "./package.json"
|
||||||
|
},
|
||||||
|
"files": [
|
||||||
|
"lib",
|
||||||
|
"cordis.patch.yml"
|
||||||
|
],
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">=20"
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"check": "node --check lib/index.js && node --check lib/provider.js && node --check lib/catalog.js && node --check lib/extract.js && node --check lib/url-policy.js && node --check lib/client.js && node --check lib/client.browser.js",
|
||||||
|
"test": "node --test"
|
||||||
|
},
|
||||||
|
"keywords": [
|
||||||
|
"dsh",
|
||||||
|
"dsh-plugin",
|
||||||
|
"mcp",
|
||||||
|
"search",
|
||||||
|
"web_search",
|
||||||
|
"tavily",
|
||||||
|
"brave",
|
||||||
|
"exa",
|
||||||
|
"perplexity",
|
||||||
|
"duckduckgo"
|
||||||
|
],
|
||||||
|
"dsh": {
|
||||||
|
"bundle": {
|
||||||
|
"patch": "./cordis.patch.yml"
|
||||||
|
},
|
||||||
|
"client": {
|
||||||
|
"platform": "web",
|
||||||
|
"immediately": true,
|
||||||
|
"inject": [
|
||||||
|
"@deepseek-ai/dsh-client-ui-settings-plugins",
|
||||||
|
"@deepseek-ai/dsh-client-locale"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@modelcontextprotocol/sdk": "1.30.0",
|
||||||
|
"ipaddr.js": "2.5.0",
|
||||||
|
"undici": "6.28.0"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"@deepseek-ai/dsh-api-remotes": "*",
|
||||||
|
"@deepseek-ai/dsh-credentials": "*",
|
||||||
|
"@deepseek-ai/dsh-launch-environment": "*",
|
||||||
|
"@deepseek-ai/dsh-settings": "*",
|
||||||
|
"@deepseek-ai/dsh-web": "*",
|
||||||
|
"@deepseek-ai/schemastery": "*"
|
||||||
|
}
|
||||||
|
}
|
||||||
79
test/compatibility.test.js
Normal file
79
test/compatibility.test.js
Normal file
|
|
@ -0,0 +1,79 @@
|
||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import { dirname, resolve } from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||||
|
const read = (path) => readFile(resolve(root, path), 'utf8');
|
||||||
|
|
||||||
|
test('package exports resolve and RC2 dependencies stay pinned', async () => {
|
||||||
|
const pkg = JSON.parse(await read('package.json'));
|
||||||
|
assert.equal(pkg.exports['.'], './lib/index.js');
|
||||||
|
assert.equal(pkg.exports['./client'], './lib/client.browser.js');
|
||||||
|
assert.equal(pkg.engines.node, '>=20');
|
||||||
|
|
||||||
|
for (const name of [
|
||||||
|
'@deepseek-ai/dsh-api-remotes',
|
||||||
|
'@deepseek-ai/dsh-credentials',
|
||||||
|
'@deepseek-ai/dsh-launch-environment',
|
||||||
|
'@deepseek-ai/dsh-settings',
|
||||||
|
'@deepseek-ai/dsh-web',
|
||||||
|
]) {
|
||||||
|
assert.equal(pkg.dependencies[name], '0.1.1-rc.2');
|
||||||
|
}
|
||||||
|
assert.equal(pkg.dependencies['@deepseek-ai/schemastery'], '3.18.1');
|
||||||
|
assert.equal(pkg.dependencies.undici, '6.28.0');
|
||||||
|
assert.equal(pkg.dependencies['ipaddr.js'], '2.5.0');
|
||||||
|
assert.equal(pkg.dependencies['@modelcontextprotocol/sdk'], '1.30.0');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('RC2 browser bundle uses keyed settings slot and credential migration', async () => {
|
||||||
|
const client = await read('lib/client.browser.js');
|
||||||
|
assert.match(client, /name: "settings\.plugin\.item",\s+key: NS,/);
|
||||||
|
assert.doesNotMatch(client, /name: "settings\.plugin\.item",\s+id:/);
|
||||||
|
assert.match(client, /api\.settings\.describe\(\{\}\)/);
|
||||||
|
assert.match(client, /api\.credentials\.describe\(\{ refs: refs\.slice\(index, index \+ CREDENTIAL_DESCRIBE_BATCH_SIZE\) \}\)/);
|
||||||
|
assert.match(client, /api\.credentials\.set\(\{ ref, value \}\)/);
|
||||||
|
assert.match(client, /credentials\/reference-updated/);
|
||||||
|
assert.match(client, /CREDENTIAL_DESCRIBE_BATCH_SIZE = 64/);
|
||||||
|
assert.match(client, /api\.credentials\.unset\(\{ ref \}\)/);
|
||||||
|
assert.match(client, /rollbackSettingsWrites/);
|
||||||
|
assert.match(client, /legacyKeyBlocked/);
|
||||||
|
assert.match(client, /deepEqualJson\(current\[field\], value\)/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('known providers are CDKey-only while custom keeps advanced fields', async () => {
|
||||||
|
const client = await read('lib/client.browser.js');
|
||||||
|
const catalog = client.slice(client.indexOf('const CATALOG = {'), client.indexOf('const KIND_OPTIONS'));
|
||||||
|
assert.doesNotMatch(catalog, /https?:\/\//);
|
||||||
|
assert.doesNotMatch(catalog, /toolName|authParam|transport/);
|
||||||
|
assert.match(client, /const known = row\.kind !== "custom"/);
|
||||||
|
assert.match(client, /children: known \? \[/);
|
||||||
|
assert.match(client, /No endpoint is required for known providers/);
|
||||||
|
assert.match(client, /已知提供商不需要填写端点链接/);
|
||||||
|
assert.match(client, /kind, apiKey: "", apiKeyEnv: ""/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('HTTP transport pins DNS and applies one guarded fetch to every SDK request', async () => {
|
||||||
|
const transport = await read('lib/client.js');
|
||||||
|
assert.match(transport, /validateHttpEndpoint\(server\.url, \{ signal \}\)/);
|
||||||
|
assert.match(transport, /new Agent\(\{[\s\S]*connect: \{ lookup: validated\.lookup \}/);
|
||||||
|
assert.match(transport, /requestUrl\.origin !== expectedOrigin/);
|
||||||
|
assert.match(transport, /dispatcher: agent/);
|
||||||
|
assert.match(transport, /redirect: 'error'/);
|
||||||
|
assert.match(transport, /fetch: secureFetch/);
|
||||||
|
assert.match(transport, /await client\.close\(\)[\s\S]*await runtime\?\.close\(\)/);
|
||||||
|
assert.doesNotMatch(transport, /new URL\(server\.url\)[\s\S]*new StreamableHTTPClientTransport\(url, \{\s*requestInit:/);
|
||||||
|
});
|
||||||
|
test('bundle replaces built-in search and leaves default row endpoint-free', async () => {
|
||||||
|
const patch = await read('cordis.patch.yml');
|
||||||
|
assert.match(patch, /searchProvider: search-mcp/);
|
||||||
|
assert.match(patch, /- id: web-search-deepseek\s+disabled: true/);
|
||||||
|
assert.match(patch, /- id: tool-web\s+disabled: false/);
|
||||||
|
assert.match(patch, /fetch: false/);
|
||||||
|
assert.match(patch, /searchMaxResults: 50/);
|
||||||
|
assert.match(patch, /searchMaxQueries: 4/);
|
||||||
|
const defaultRow = patch.slice(patch.indexOf('- id: tavily'), patch.indexOf('- id: web'));
|
||||||
|
assert.doesNotMatch(defaultRow, /url:|toolName:|authParam:|transport:/);
|
||||||
|
});
|
||||||
130
test/runtime.test.js
Normal file
130
test/runtime.test.js
Normal file
|
|
@ -0,0 +1,130 @@
|
||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { clampSearchResults, SEARCH_MCP_CATALOG, resolveServer } from '../lib/catalog.js';
|
||||||
|
import { extractSearchResult } from '../lib/extract.js';
|
||||||
|
|
||||||
|
test('catalog exposes every supported provider preset', () => {
|
||||||
|
assert.deepEqual(Object.keys(SEARCH_MCP_CATALOG), [
|
||||||
|
'tavily',
|
||||||
|
'brave',
|
||||||
|
'exa',
|
||||||
|
'perplexity',
|
||||||
|
'duckduckgo',
|
||||||
|
'custom',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('known providers ignore stored connection overrides', () => {
|
||||||
|
const tavily = resolveServer({
|
||||||
|
id: 'primary',
|
||||||
|
kind: 'tavily',
|
||||||
|
transport: 'stdio',
|
||||||
|
url: 'https://example.test/mcp',
|
||||||
|
authStyle: 'header',
|
||||||
|
authParam: 'X-Other-Key',
|
||||||
|
toolName: 'other_search',
|
||||||
|
apiKeyEnv: 'MY_TAVILY_KEY',
|
||||||
|
maxResults: 12,
|
||||||
|
});
|
||||||
|
assert.equal(tavily.transport, 'http');
|
||||||
|
assert.equal(tavily.url, 'https://mcp.tavily.com/mcp/');
|
||||||
|
assert.equal(tavily.authStyle, 'query');
|
||||||
|
assert.equal(tavily.authParam, 'tavilyApiKey');
|
||||||
|
assert.equal(tavily.toolName, 'tavily_search');
|
||||||
|
assert.equal(tavily.countArg, 'max_results');
|
||||||
|
assert.equal(tavily.apiKeyEnv, 'MY_TAVILY_KEY');
|
||||||
|
assert.equal(tavily.maxResults, 12);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('prototype property kinds fall back to custom', () => {
|
||||||
|
for (const kind of ['constructor', 'toString', '__proto__']) {
|
||||||
|
const resolved = resolveServer({ id: kind, kind, url: 'https://search.example/mcp' });
|
||||||
|
assert.equal(resolved.kind, 'custom');
|
||||||
|
assert.equal(resolved.url, 'https://search.example/mcp');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('custom providers preserve advanced connection fields', () => {
|
||||||
|
const custom = resolveServer({
|
||||||
|
id: 'custom',
|
||||||
|
kind: 'custom',
|
||||||
|
transport: 'stdio',
|
||||||
|
command: 'custom-mcp',
|
||||||
|
args: ['--stdio'],
|
||||||
|
authStyle: 'header',
|
||||||
|
authParam: 'X-Key',
|
||||||
|
authPrefix: 'Token ',
|
||||||
|
toolName: 'search',
|
||||||
|
});
|
||||||
|
assert.equal(custom.transport, 'stdio');
|
||||||
|
assert.equal(custom.command, 'custom-mcp');
|
||||||
|
assert.deepEqual(custom.args, ['--stdio']);
|
||||||
|
assert.equal(custom.authParam, 'X-Key');
|
||||||
|
assert.equal(custom.authPrefix, 'Token ');
|
||||||
|
assert.equal(custom.toolName, 'search');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('provider contracts match current upstream transports', () => {
|
||||||
|
const brave = resolveServer({ id: 'brave', kind: 'brave' });
|
||||||
|
assert.equal(brave.transport, 'stdio');
|
||||||
|
assert.equal(brave.command, 'npx');
|
||||||
|
assert.deepEqual(brave.args, ['-y', '@brave/brave-search-mcp-server@2.1.3']);
|
||||||
|
assert.equal(brave.authParam, 'BRAVE_API_KEY');
|
||||||
|
|
||||||
|
const perplexity = resolveServer({ id: 'perplexity', kind: 'perplexity' });
|
||||||
|
assert.equal(perplexity.url, 'https://api.perplexity.ai/mcp');
|
||||||
|
assert.equal(perplexity.authParam, 'Authorization');
|
||||||
|
assert.equal(perplexity.authPrefix, 'Bearer ');
|
||||||
|
assert.equal(perplexity.toolName, 'perplexity_search');
|
||||||
|
|
||||||
|
const duckduckgo = resolveServer({ id: 'duckduckgo', kind: 'duckduckgo' });
|
||||||
|
assert.equal(duckduckgo.needsKey, false);
|
||||||
|
assert.equal(duckduckgo.toolName, 'duckduckgo_web_search');
|
||||||
|
assert.equal(duckduckgo.countArg, 'count');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('provider result counts stay within upstream MCP schemas', () => {
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 't', kind: 'tavily' }), 1), 5);
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 't', kind: 'tavily' }), 50), 20);
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 'b', kind: 'brave' }), 50), 20);
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 'p', kind: 'perplexity' }), 0), 1);
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 'd', kind: 'duckduckgo' }), 50), 20);
|
||||||
|
assert.equal(clampSearchResults(resolveServer({ id: 'e', kind: 'exa' }), 50), 50);
|
||||||
|
});
|
||||||
|
test('extractSearchResult normalizes, deduplicates, and rejects invalid URLs', () => {
|
||||||
|
const result = extractSearchResult({
|
||||||
|
structuredContent: {
|
||||||
|
answer: 'Summary',
|
||||||
|
results: [
|
||||||
|
{ url: 'https://example.com/a', title: 'A', content: 'alpha', published_date: '2026-08-18' },
|
||||||
|
{ url: 'https://example.com/a', title: 'Duplicate' },
|
||||||
|
{ url: 'ftp://example.com/ignored', title: 'Ignored' },
|
||||||
|
],
|
||||||
|
nested: { url: 'http://example.com/b', description: 'beta' },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(result, {
|
||||||
|
sources: [
|
||||||
|
{
|
||||||
|
url: 'https://example.com/a',
|
||||||
|
title: 'A',
|
||||||
|
snippet: 'alpha',
|
||||||
|
publishedAt: '2026-08-18',
|
||||||
|
},
|
||||||
|
{ url: 'http://example.com/b', snippet: 'beta' },
|
||||||
|
],
|
||||||
|
truncated: false,
|
||||||
|
content: 'Summary',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('extractSearchResult accepts JSON and plain text MCP blocks', () => {
|
||||||
|
const json = extractSearchResult({
|
||||||
|
content: [{ type: 'text', text: '{"results":[{"url":"https://example.com"}]}' }],
|
||||||
|
});
|
||||||
|
assert.equal(json.sources.length, 1);
|
||||||
|
|
||||||
|
const text = extractSearchResult({ content: [{ type: 'text', text: 'Direct answer' }] });
|
||||||
|
assert.deepEqual(text, { sources: [], truncated: false, content: 'Direct answer' });
|
||||||
|
});
|
||||||
158
test/url-policy.test.js
Normal file
158
test/url-policy.test.js
Normal file
|
|
@ -0,0 +1,158 @@
|
||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import {
|
||||||
|
createPinnedLookup,
|
||||||
|
isAllowedEndpointAddress,
|
||||||
|
isPublicAddress,
|
||||||
|
validateHttpEndpoint,
|
||||||
|
} from '../lib/url-policy.js';
|
||||||
|
|
||||||
|
const lookup = (records) => (_hostname, options, callback) => {
|
||||||
|
assert.equal(options.all, true);
|
||||||
|
queueMicrotask(() => callback(null, records));
|
||||||
|
};
|
||||||
|
|
||||||
|
test('URL policy accepts HTTP(S) with public DNS only', async () => {
|
||||||
|
const result = await validateHttpEndpoint('https://search.example/mcp', {
|
||||||
|
lookup: lookup([
|
||||||
|
{ address: '8.8.8.8', family: 4 },
|
||||||
|
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||||
|
]),
|
||||||
|
});
|
||||||
|
assert.equal(result.url.hostname, 'search.example');
|
||||||
|
assert.deepEqual(result.addresses, [
|
||||||
|
{ address: '8.8.8.8', family: 4 },
|
||||||
|
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy rejects schemes, userinfo, localhost, and ambiguous IPv4', async () => {
|
||||||
|
for (const input of [
|
||||||
|
'ftp://example.com/mcp',
|
||||||
|
'https://user:pass@example.com/mcp',
|
||||||
|
'http://localhost/mcp',
|
||||||
|
'http://api.localhost/mcp',
|
||||||
|
'http://127.0.0.1/mcp',
|
||||||
|
'http://127.1/mcp',
|
||||||
|
'http://0177.0.0.1/mcp',
|
||||||
|
'http://0x7f000001/mcp',
|
||||||
|
]) {
|
||||||
|
await assert.rejects(() => validateHttpEndpoint(input), { name: 'SearchMcpUrlPolicyError' }, input);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy rejects private, reserved, test, mapped, and transition ranges', () => {
|
||||||
|
for (const address of [
|
||||||
|
'0.0.0.0',
|
||||||
|
'10.0.0.1',
|
||||||
|
'100.64.0.1',
|
||||||
|
'127.0.0.1',
|
||||||
|
'169.254.169.254',
|
||||||
|
'172.16.0.1',
|
||||||
|
'192.168.0.1',
|
||||||
|
'192.0.2.1',
|
||||||
|
'198.18.0.1',
|
||||||
|
'198.51.100.1',
|
||||||
|
'203.0.113.1',
|
||||||
|
'224.0.0.1',
|
||||||
|
'255.255.255.255',
|
||||||
|
'::',
|
||||||
|
'::1',
|
||||||
|
'::ffff:127.0.0.1',
|
||||||
|
'::7f00:1',
|
||||||
|
'::a00:1',
|
||||||
|
'::a9fe:a9fe',
|
||||||
|
'::c0a8:101',
|
||||||
|
'::808:808',
|
||||||
|
'64:ff9b::808:808',
|
||||||
|
'2001:db8::1',
|
||||||
|
'2001::1',
|
||||||
|
'2002:0808:0808::1',
|
||||||
|
'fc00::1',
|
||||||
|
'fe80::1',
|
||||||
|
'ff02::1',
|
||||||
|
]) {
|
||||||
|
assert.equal(isPublicAddress(address), false, address);
|
||||||
|
}
|
||||||
|
assert.equal(isPublicAddress('8.8.8.8'), true);
|
||||||
|
assert.equal(isPublicAddress('2606:4700:4700::1111'), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy drops private DNS answers and keeps public ones', async () => {
|
||||||
|
const result = await validateHttpEndpoint('https://search.example/mcp', {
|
||||||
|
lookup: lookup([
|
||||||
|
{ address: '8.8.8.8', family: 4 },
|
||||||
|
{ address: '10.0.0.1', family: 4 },
|
||||||
|
]),
|
||||||
|
});
|
||||||
|
assert.deepEqual(result.addresses, [{ address: '8.8.8.8', family: 4 }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy accepts Clash fake-IP answers with optional public peers', async () => {
|
||||||
|
const mixed = await validateHttpEndpoint('https://dashscope.example/mcp', {
|
||||||
|
lookup: lookup([
|
||||||
|
{ address: '198.18.2.146', family: 4 },
|
||||||
|
{ address: '2408:400a:3e:ef02:12f:bd95:e827:51d', family: 6 },
|
||||||
|
]),
|
||||||
|
});
|
||||||
|
assert.deepEqual(mixed.addresses, [
|
||||||
|
{ address: '198.18.2.146', family: 4 },
|
||||||
|
{ address: '2408:400a:3e:ef02:12f:bd95:e827:51d', family: 6 },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const fakeOnly = await validateHttpEndpoint('https://dashscope.example/mcp', {
|
||||||
|
lookup: lookup([{ address: '198.18.2.146', family: 4 }]),
|
||||||
|
});
|
||||||
|
assert.deepEqual(fakeOnly.addresses, [{ address: '198.18.2.146', family: 4 }]);
|
||||||
|
assert.equal(isAllowedEndpointAddress('198.18.2.146'), true);
|
||||||
|
assert.equal(isAllowedEndpointAddress('10.0.0.1'), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy rejects private-only DNS answers', async () => {
|
||||||
|
await assert.rejects(
|
||||||
|
() => validateHttpEndpoint('https://search.example/mcp', {
|
||||||
|
lookup: lookup([{ address: '10.0.0.1', family: 4 }]),
|
||||||
|
}),
|
||||||
|
/non-public address/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy rejects DNS errors and empty answers without leaking endpoint data', async () => {
|
||||||
|
await assert.rejects(
|
||||||
|
() => validateHttpEndpoint('https://search.example/mcp', {
|
||||||
|
lookup: (_hostname, _options, callback) => callback(new Error('resolver detail')),
|
||||||
|
}),
|
||||||
|
/resolution failed/,
|
||||||
|
);
|
||||||
|
await assert.rejects(
|
||||||
|
() => validateHttpEndpoint('https://search.example/mcp', { lookup: lookup([]) }),
|
||||||
|
/did not resolve/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('URL policy aborts a pending DNS lookup', async () => {
|
||||||
|
const controller = new AbortController();
|
||||||
|
const pending = validateHttpEndpoint('https://search.example/mcp', {
|
||||||
|
lookup: () => {},
|
||||||
|
signal: controller.signal,
|
||||||
|
});
|
||||||
|
controller.abort();
|
||||||
|
await assert.rejects(pending, /validation was aborted/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pinned lookup serves only validated host and addresses', async () => {
|
||||||
|
const pinned = createPinnedLookup('search.example', [
|
||||||
|
{ address: '8.8.8.8', family: 4 },
|
||||||
|
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||||
|
]);
|
||||||
|
const all = await new Promise((resolve, reject) => {
|
||||||
|
pinned('search.example', { all: true }, (error, records) => error ? reject(error) : resolve(records));
|
||||||
|
});
|
||||||
|
assert.deepEqual(all, [
|
||||||
|
{ address: '8.8.8.8', family: 4 },
|
||||||
|
{ address: '2606:4700:4700:0:0:0:0:1111', family: 6 },
|
||||||
|
]);
|
||||||
|
await assert.rejects(new Promise((resolve, reject) => {
|
||||||
|
pinned('other.example', {}, (error, address) => error ? reject(error) : resolve(address));
|
||||||
|
}), /unvalidated hostname/);
|
||||||
|
});
|
||||||
Loading…
Add table
Add a link
Reference in a new issue