mirror of
https://github.com/hansjone/oclaw.git
synced 2026-10-10 10:20:45 +08:00
重构主控编排与运行时预热链路,统一工作区提示词/专家调度协议并补齐 wiki 记忆注入与写回闭环。
同时收敛启动与运维脚本默认行为(含 wiki worker)、更新 Admin 可观测性与相关测试,降低首轮时延并提高运行稳定性。 Made-with: Cursor
This commit is contained in:
parent
4a23b715a2
commit
dbbe3add6a
14438 changed files with 2693620 additions and 2546 deletions
147
openclaw/docs/tools/firecrawl.md
Normal file
147
openclaw/docs/tools/firecrawl.md
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
---
|
||||
summary: "Firecrawl search, scrape, and web_fetch fallback"
|
||||
read_when:
|
||||
- You want Firecrawl-backed web extraction
|
||||
- You need a Firecrawl API key
|
||||
- You want Firecrawl as a web_search provider
|
||||
- You want anti-bot extraction for web_fetch
|
||||
title: "Firecrawl"
|
||||
---
|
||||
|
||||
# Firecrawl
|
||||
|
||||
OpenClaw can use **Firecrawl** in three ways:
|
||||
|
||||
- as the `web_search` provider
|
||||
- as explicit plugin tools: `firecrawl_search` and `firecrawl_scrape`
|
||||
- as a fallback extractor for `web_fetch`
|
||||
|
||||
It is a hosted extraction/search service that supports bot circumvention and caching,
|
||||
which helps with JS-heavy sites or pages that block plain HTTP fetches.
|
||||
|
||||
## Get an API key
|
||||
|
||||
1. Create a Firecrawl account and generate an API key.
|
||||
2. Store it in config or set `FIRECRAWL_API_KEY` in the gateway environment.
|
||||
|
||||
## Configure Firecrawl search
|
||||
|
||||
```json5
|
||||
{
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
provider: "firecrawl",
|
||||
},
|
||||
},
|
||||
},
|
||||
plugins: {
|
||||
entries: {
|
||||
firecrawl: {
|
||||
enabled: true,
|
||||
config: {
|
||||
webSearch: {
|
||||
apiKey: "FIRECRAWL_API_KEY_HERE",
|
||||
baseUrl: "https://api.firecrawl.dev",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Choosing Firecrawl in onboarding or `openclaw configure --section web` enables the bundled Firecrawl plugin automatically.
|
||||
- `web_search` with Firecrawl supports `query` and `count`.
|
||||
- For Firecrawl-specific controls like `sources`, `categories`, or result scraping, use `firecrawl_search`.
|
||||
- `baseUrl` overrides must stay on `https://api.firecrawl.dev`.
|
||||
- `FIRECRAWL_BASE_URL` is the shared env fallback for Firecrawl search and scrape base URLs.
|
||||
|
||||
## Configure Firecrawl scrape + web_fetch fallback
|
||||
|
||||
```json5
|
||||
{
|
||||
plugins: {
|
||||
entries: {
|
||||
firecrawl: {
|
||||
enabled: true,
|
||||
config: {
|
||||
webFetch: {
|
||||
apiKey: "FIRECRAWL_API_KEY_HERE",
|
||||
baseUrl: "https://api.firecrawl.dev",
|
||||
onlyMainContent: true,
|
||||
maxAgeMs: 172800000,
|
||||
timeoutSeconds: 60,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Firecrawl fallback attempts run only when an API key is available (`plugins.entries.firecrawl.config.webFetch.apiKey` or `FIRECRAWL_API_KEY`).
|
||||
- `maxAgeMs` controls how old cached results can be (ms). Default is 2 days.
|
||||
- Legacy `tools.web.fetch.firecrawl.*` config is auto-migrated by `openclaw doctor --fix`.
|
||||
- Firecrawl scrape/base URL overrides are restricted to `https://api.firecrawl.dev`.
|
||||
|
||||
`firecrawl_scrape` reuses the same `plugins.entries.firecrawl.config.webFetch.*` settings and env vars.
|
||||
|
||||
## Firecrawl plugin tools
|
||||
|
||||
### `firecrawl_search`
|
||||
|
||||
Use this when you want Firecrawl-specific search controls instead of generic `web_search`.
|
||||
|
||||
Core parameters:
|
||||
|
||||
- `query`
|
||||
- `count`
|
||||
- `sources`
|
||||
- `categories`
|
||||
- `scrapeResults`
|
||||
- `timeoutSeconds`
|
||||
|
||||
### `firecrawl_scrape`
|
||||
|
||||
Use this for JS-heavy or bot-protected pages where plain `web_fetch` is weak.
|
||||
|
||||
Core parameters:
|
||||
|
||||
- `url`
|
||||
- `extractMode`
|
||||
- `maxChars`
|
||||
- `onlyMainContent`
|
||||
- `maxAgeMs`
|
||||
- `proxy`
|
||||
- `storeInCache`
|
||||
- `timeoutSeconds`
|
||||
|
||||
## Stealth / bot circumvention
|
||||
|
||||
Firecrawl exposes a **proxy mode** parameter for bot circumvention (`basic`, `stealth`, or `auto`).
|
||||
OpenClaw always uses `proxy: "auto"` plus `storeInCache: true` for Firecrawl requests.
|
||||
If proxy is omitted, Firecrawl defaults to `auto`. `auto` retries with stealth proxies if a basic attempt fails, which may use more credits
|
||||
than basic-only scraping.
|
||||
|
||||
## How `web_fetch` uses Firecrawl
|
||||
|
||||
`web_fetch` extraction order:
|
||||
|
||||
1. Readability (local)
|
||||
2. Firecrawl (if selected or auto-detected as the active web-fetch fallback)
|
||||
3. Basic HTML cleanup (last fallback)
|
||||
|
||||
The selection knob is `tools.web.fetch.provider`. If you omit it, OpenClaw
|
||||
auto-detects the first ready web-fetch provider from available credentials.
|
||||
Today the bundled provider is Firecrawl.
|
||||
|
||||
## Related
|
||||
|
||||
- [Web Search overview](/tools/web) -- all providers and auto-detection
|
||||
- [Web Fetch](/tools/web-fetch) -- web_fetch tool with Firecrawl fallback
|
||||
- [Tavily](/tools/tavily) -- search + extract tools
|
||||
Loading…
Add table
Add a link
Reference in a new issue