hansjone/oclaw
1
0
Fork 0
mirror of https://github.com/hansjone/oclaw.git synced 2026-10-15 15:00:46 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions

重构主控编排与运行时预热链路，统一工作区提示词/专家调度协议并补齐 wiki 记忆注入与写回闭环。

Browse source 
同时收敛启动与运维脚本默认行为（含 wiki worker）、更新 Admin 可观测性与相关测试，降低首轮时延并提高运行稳定性。

Made-with: Cursor
This commit is contained in:
oliver 2026-04-26 08:34:33 +08:00
parent 4a23b715a2
commit dbbe3add6a
14438 changed files with 2693620 additions and 2546 deletions
Download patch file Download diff file Expand all files Collapse all files

67
openclaw/qa/scenarios/workspace/lobster-invaders-build.md Normal file
View file

@ -0,0 +1,67 @@
# Build Lobster Invaders
```yaml qa-scenario
id: lobster-invaders-build
title: Build Lobster Invaders
surface: workspace
coverage:
primary:
- workspace.artifacts
secondary:
- workspace.builds
objective: Verify the agent can read the repo, create a tiny playable artifact, and report what changed.
successCriteria:
- Agent inspects source before coding.
- Agent builds a tiny playable Lobster Invaders artifact.
- Agent explains how to run or view the artifact.
docsRefs:
- docs/help/testing.md
- docs/web/dashboard.md
codeRefs:
- extensions/qa-lab/src/report.ts
- extensions/qa-lab/web/src/app.ts
execution:
kind: flow
summary: Verify the agent can read the repo, create a tiny playable artifact, and report what changed.
config:
prompt: Read the QA kickoff context first, then build a tiny Lobster Invaders HTML game at ./lobster-invaders.html in this workspace and tell me where it is.
```
```yaml qa-flow
steps:
- name: creates the artifact after reading context
actions:
- call: reset
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:lobster-invaders
message:
expr: config.prompt
timeoutMs:
expr: liveTurnTimeoutMs(env, 30000)
- call: waitForOutboundMessage
args:
- ref: state
- lambda:
params: [candidate]
expr: "candidate.conversation.id === 'qa-operator'"
- set: artifactPath
value:
expr: "path.join(env.gateway.workspaceDir, 'lobster-invaders.html')"
- call: waitForCondition
saveAs: artifact
args:
- lambda:
async: true
expr: "((await fs.readFile(artifactPath, 'utf8').catch(() => null))?.includes('Lobster Invaders') ? await fs.readFile(artifactPath, 'utf8').catch(() => null) : undefined)"
- expr: liveTurnTimeoutMs(env, 20000)
- 250
- assert:
expr: "artifact.includes('Lobster Invaders')"
message: missing Lobster Invaders artifact
- assert:
expr: "!env.mock || (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).some((request) => (request.toolOutput ?? '').includes('QA mission'))"
message: expected pre-write read evidence
detailsExpr: "'lobster-invaders.html'"
```

167
openclaw/qa/scenarios/workspace/medium-game-plan-codex-harness.md Normal file
View file

@ -0,0 +1,167 @@
# Medium game plan Codex harness
```yaml qa-scenario
id: medium-game-plan-codex-harness
title: Medium game plan Codex harness
surface: workspace
coverage:
primary:
- workspace.planning
secondary:
- models.codex-cli
objective: Verify the Codex app-server harness can plan and build a medium-complex self-contained browser game.
successCriteria:
- A live-frontier run fails fast unless the selected primary model is codex/gpt-5.4.
- The scenario forces the Codex embedded harness and disables PI fallback.
- The prompt explicitly asks the agent to enter plan mode before editing.
- The agent writes a self-contained HTML game with a canvas loop, controls, scoring, waves, pause, and restart.
docsRefs:
- docs/plugins/sdk-agent-harness.md
- docs/gateway/configuration-reference.md
- docs/help/testing.md
codeRefs:
- extensions/codex/harness.ts
- src/agents/harness/selection.ts
- extensions/qa-lab/src/suite.ts
execution:
kind: flow
summary: Run with `pnpm openclaw qa suite --provider-mode live-frontier --model codex/gpt-5.4 --alt-model codex/gpt-5.4 --scenario medium-game-plan-codex-harness`.
config:
requiredProvider: codex
requiredModel: gpt-5.4
harnessRuntime: codex
harnessFallback: none
artifactFile: star-garden-defenders-codex.html
gameTitle: Star Garden Defenders
minBytes: 5000
buildPrompt: |-
Enter plan mode first and write a short implementation plan before editing.
Then build a medium-complex, self-contained browser game at ./star-garden-defenders-codex.html.
Game: Star Garden Defenders.
Requirements:
- one HTML file only; no external assets, fonts, scripts, or network calls
- canvas-based arcade loop with requestAnimationFrame
- keyboard controls and mouse or pointer support
- player movement, enemy waves, collectibles or power-ups, collision handling
- score, lives or health, wave number, pause, restart, and game-over state
- polished inline CSS and clear on-screen controls
- after writing the file, reply with the filename and the main systems implemented
```
```yaml qa-flow
steps:
- name: confirms GPT-5.4 Codex harness target
actions:
- set: selected
value:
expr: splitModelRef(env.primaryModel)
- assert:
expr: "env.providerMode !== 'live-frontier' || selected?.provider === config.requiredProvider"
message:
expr: "`expected live primary provider ${config.requiredProvider}, got ${env.primaryModel}`"
- assert:
expr: "env.providerMode !== 'live-frontier' || selected?.model === config.requiredModel"
message:
expr: "`expected live primary model ${config.requiredModel}, got ${env.primaryModel}`"
- if:
expr: "env.providerMode !== 'live-frontier'"
then:
- assert: "true"
else:
- call: patchConfig
saveAs: patchResult
args:
- env:
ref: env
patch:
agents:
defaults:
embeddedHarness:
runtime:
expr: config.harnessRuntime
fallback:
expr: config.harnessFallback
- call: waitForGatewayHealthy
args:
- ref: env
- 60000
- call: waitForQaChannelReady
args:
- ref: env
- 60000
- call: readConfigSnapshot
saveAs: snapshot
args:
- ref: env
- assert:
expr: "snapshot.config.agents?.defaults?.embeddedHarness?.runtime === config.harnessRuntime"
message:
expr: "`expected embeddedHarness.runtime=${config.harnessRuntime}, got ${JSON.stringify(snapshot.config.agents?.defaults?.embeddedHarness)}`"
- assert:
expr: "snapshot.config.agents?.defaults?.embeddedHarness?.fallback === config.harnessFallback"
message:
expr: "`expected embeddedHarness.fallback=${config.harnessFallback}, got ${JSON.stringify(snapshot.config.agents?.defaults?.embeddedHarness)}`"
detailsExpr: "env.providerMode === 'live-frontier' ? `provider=${selected?.provider} model=${selected?.model} runtime=${snapshot.config.agents?.defaults?.embeddedHarness?.runtime} fallback=${snapshot.config.agents?.defaults?.embeddedHarness?.fallback}` : `mock mode: parsed ${scenario.id}`"
- name: builds the medium game artifact
actions:
- if:
expr: "env.providerMode !== 'live-frontier'"
then:
- assert: "true"
else:
- call: reset
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:medium-game-codex
message:
expr: config.buildPrompt
provider:
expr: selected?.provider
model:
expr: selected?.model
timeoutMs:
expr: resolveQaLiveTurnTimeoutMs(env, 420000, env.primaryModel)
- call: waitForOutboundMessage
saveAs: outbound
args:
- ref: state
- lambda:
params: [candidate]
expr: "candidate.conversation.id === 'qa-operator' && candidate.text.includes(config.artifactFile)"
- expr: resolveQaLiveTurnTimeoutMs(env, 60000, env.primaryModel)
- set: artifactPath
value:
expr: "path.join(env.gateway.workspaceDir, config.artifactFile)"
- call: waitForCondition
saveAs: artifact
args:
- lambda:
async: true
expr: "((await fs.readFile(artifactPath, 'utf8').catch(() => '')).includes(config.gameTitle) ? await fs.readFile(artifactPath, 'utf8').catch(() => '') : undefined)"
- expr: resolveQaLiveTurnTimeoutMs(env, 60000, env.primaryModel)
- 500
- set: artifactLower
value:
expr: normalizeLowercaseStringOrEmpty(artifact)
- assert:
expr: "artifact.length >= config.minBytes"
message:
expr: "`expected medium game artifact >= ${config.minBytes} bytes, got ${artifact.length}`"
- assert:
expr: "artifactLower.includes('star garden defenders') && artifactLower.includes('<canvas') && artifactLower.includes('requestanimationframe')"
message: missing title, canvas, or animation loop
- assert:
expr: "artifactLower.includes('keydown') || artifactLower.includes('keyup')"
message: missing keyboard controls
- assert:
expr: "artifactLower.includes('score') && artifactLower.includes('wave') && artifactLower.includes('pause') && artifactLower.includes('restart')"
message: missing score, wave, pause, or restart systems
- assert:
expr: "outbound.text.includes(config.artifactFile)"
message:
expr: "`final reply did not mention ${config.artifactFile}: ${outbound.text}`"
detailsExpr: "env.providerMode !== 'live-frontier' ? 'mock mode: skipped live medium-game build' : `${config.artifactFile} bytes=${artifact.length}`"
```

163
openclaw/qa/scenarios/workspace/medium-game-plan-pi-harness.md Normal file
View file

@ -0,0 +1,163 @@
# Medium game plan PI harness
```yaml qa-scenario
id: medium-game-plan-pi-harness
title: Medium game plan PI harness
surface: workspace
coverage:
primary:
- workspace.planning
secondary:
- agents.pi-harness
objective: Verify GPT-5.4 can use the PI harness to plan and build a medium-complex self-contained browser game.
successCriteria:
- A live-frontier run fails fast unless the selected primary model is openai/gpt-5.4.
- The scenario forces the embedded PI harness before the build turn.
- The prompt explicitly asks the agent to enter plan mode before editing.
- The agent writes a self-contained HTML game with a canvas loop, controls, scoring, waves, pause, and restart.
docsRefs:
- docs/plugins/sdk-agent-harness.md
- docs/gateway/configuration-reference.md
- docs/help/testing.md
codeRefs:
- src/agents/harness/selection.ts
- src/agents/harness/builtin-pi.ts
- extensions/qa-lab/src/suite.ts
execution:
kind: flow
summary: Run with `pnpm openclaw qa suite --provider-mode live-frontier --model openai/gpt-5.4 --alt-model openai/gpt-5.4 --scenario medium-game-plan-pi-harness`.
config:
requiredProvider: openai
requiredModel: gpt-5.4
harnessRuntime: pi
harnessFallback: pi
artifactFile: star-garden-defenders-pi.html
gameTitle: Star Garden Defenders
minBytes: 5000
buildPrompt: |-
Enter plan mode first and write a short implementation plan before editing.
Then build a medium-complex, self-contained browser game at ./star-garden-defenders-pi.html.
Game: Star Garden Defenders.
Requirements:
- one HTML file only; no external assets, fonts, scripts, or network calls
- canvas-based arcade loop with requestAnimationFrame
- keyboard controls and mouse or pointer support
- player movement, enemy waves, collectibles or power-ups, collision handling
- score, lives or health, wave number, pause, restart, and game-over state
- polished inline CSS and clear on-screen controls
- after writing the file, reply with the filename and the main systems implemented
```
```yaml qa-flow
steps:
- name: confirms GPT-5.4 PI harness target
actions:
- set: selected
value:
expr: splitModelRef(env.primaryModel)
- assert:
expr: "env.providerMode !== 'live-frontier' || selected?.provider === config.requiredProvider"
message:
expr: "`expected live primary provider ${config.requiredProvider}, got ${env.primaryModel}`"
- assert:
expr: "env.providerMode !== 'live-frontier' || selected?.model === config.requiredModel"
message:
expr: "`expected live primary model ${config.requiredModel}, got ${env.primaryModel}`"
- if:
expr: "env.providerMode !== 'live-frontier'"
then:
- assert: "true"
else:
- call: patchConfig
saveAs: patchResult
args:
- env:
ref: env
patch:
agents:
defaults:
embeddedHarness:
runtime:
expr: config.harnessRuntime
fallback:
expr: config.harnessFallback
- call: waitForGatewayHealthy
args:
- ref: env
- 60000
- call: waitForQaChannelReady
args:
- ref: env
- 60000
- call: readConfigSnapshot
saveAs: snapshot
args:
- ref: env
- assert:
expr: "snapshot.config.agents?.defaults?.embeddedHarness?.runtime === config.harnessRuntime"
message:
expr: "`expected embeddedHarness.runtime=${config.harnessRuntime}, got ${JSON.stringify(snapshot.config.agents?.defaults?.embeddedHarness)}`"
detailsExpr: "env.providerMode === 'live-frontier' ? `provider=${selected?.provider} model=${selected?.model} runtime=${snapshot.config.agents?.defaults?.embeddedHarness?.runtime}` : `mock mode: parsed ${scenario.id}`"
- name: builds the medium game artifact
actions:
- if:
expr: "env.providerMode !== 'live-frontier'"
then:
- assert: "true"
else:
- call: reset
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:medium-game-pi
message:
expr: config.buildPrompt
provider:
expr: selected?.provider
model:
expr: selected?.model
timeoutMs:
expr: resolveQaLiveTurnTimeoutMs(env, 420000, env.primaryModel)
- call: waitForOutboundMessage
saveAs: outbound
args:
- ref: state
- lambda:
params: [candidate]
expr: "candidate.conversation.id === 'qa-operator' && candidate.text.includes(config.artifactFile)"
- expr: resolveQaLiveTurnTimeoutMs(env, 60000, env.primaryModel)
- set: artifactPath
value:
expr: "path.join(env.gateway.workspaceDir, config.artifactFile)"
- call: waitForCondition
saveAs: artifact
args:
- lambda:
async: true
expr: "((await fs.readFile(artifactPath, 'utf8').catch(() => '')).includes(config.gameTitle) ? await fs.readFile(artifactPath, 'utf8').catch(() => '') : undefined)"
- expr: resolveQaLiveTurnTimeoutMs(env, 60000, env.primaryModel)
- 500
- set: artifactLower
value:
expr: normalizeLowercaseStringOrEmpty(artifact)
- assert:
expr: "artifact.length >= config.minBytes"
message:
expr: "`expected medium game artifact >= ${config.minBytes} bytes, got ${artifact.length}`"
- assert:
expr: "artifactLower.includes('star garden defenders') && artifactLower.includes('<canvas') && artifactLower.includes('requestanimationframe')"
message: missing title, canvas, or animation loop
- assert:
expr: "artifactLower.includes('keydown') || artifactLower.includes('keyup')"
message: missing keyboard controls
- assert:
expr: "artifactLower.includes('score') && artifactLower.includes('wave') && artifactLower.includes('pause') && artifactLower.includes('restart')"
message: missing score, wave, pause, or restart systems
- assert:
expr: "outbound.text.includes(config.artifactFile)"
message:
expr: "`final reply did not mention ${config.artifactFile}: ${outbound.text}`"
detailsExpr: "env.providerMode !== 'live-frontier' ? 'mock mode: skipped live medium-game build' : `${config.artifactFile} bytes=${artifact.length}`"
```

80
openclaw/qa/scenarios/workspace/source-docs-discovery-report.md Normal file
View file

@ -0,0 +1,80 @@
# Source and docs discovery report
```yaml qa-scenario
id: source-docs-discovery-report
title: Source and docs discovery report
surface: discovery
coverage:
primary:
- workspace.repo-discovery
secondary:
- docs.discovery
objective: Verify the agent can read repo docs and source, expand the QA plan, and publish a worked or did-not-work report.
successCriteria:
- Agent reads docs and source before proposing more tests.
- Agent identifies extra candidate scenarios beyond the seed list.
- Agent ends with a worked or failed QA report.
docsRefs:
- docs/help/testing.md
- docs/web/dashboard.md
- docs/channels/qa-channel.md
codeRefs:
- extensions/qa-lab/src/report.ts
- extensions/qa-lab/src/self-check.ts
- src/agents/system-prompt.ts
execution:
kind: flow
summary: Verify the agent can read repo docs and source, expand the QA plan, and publish a worked or did-not-work report.
config:
requiredFiles:
- repo/qa/scenarios/index.md
- repo/extensions/qa-lab/src/suite.ts
- repo/docs/help/testing.md
prompt: Read the seeded docs and source plan. The full repo is mounted under ./repo/. Explicitly inspect repo/qa/scenarios/index.md, repo/extensions/qa-lab/src/suite.ts, and repo/docs/help/testing.md, then report grouped into Worked, Failed, Blocked, and Follow-up. Mention at least two extra QA scenarios beyond the seed list.
```
```yaml qa-flow
steps:
- name: reads seeded material and emits a protocol report
actions:
- call: reset
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:discovery
message:
expr: config.prompt
timeoutMs:
expr: liveTurnTimeoutMs(env, 30000)
- call: waitForCondition
saveAs: outbound
args:
- lambda:
expr: "state.getSnapshot().messages.filter((candidate) => candidate.direction === 'outbound' && candidate.conversation.id === 'qa-operator' && hasDiscoveryLabels(candidate.text)).at(-1)"
- expr: liveTurnTimeoutMs(env, 20000)
- expr: "env.providerMode === 'mock-openai' ? 100 : 250"
- assert:
expr: "!reportsMissingDiscoveryFiles(outbound.text)"
message:
expr: "`discovery report still missed repo files: ${outbound.text}`"
- assert:
expr: "!reportsDiscoveryScopeLeak(outbound.text)"
message:
expr: "`discovery report drifted beyond scope: ${outbound.text}`"
# Parity gate criterion 2 (no fake progress / fake tool completion):
# require an actual read tool call before the prose report. Without this,
# a model could fabricate a plausible Worked/Failed/Blocked/Follow-up
# report without ever touching the repo files the prompt names. The
# debug request log is fetched once and reused for both the assertion
# and its failure-message diagnostic. Each request's allInputText is
# lowercased inline at match time (the real prompt writes it as
# "Worked, Failed, Blocked") so the contains check is case-insensitive.
- set: discoveryDebugRequests
value:
expr: "env.mock ? [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))] : []"
- assert:
expr: "!env.mock || discoveryDebugRequests.some((request) => String(request.allInputText ?? '').toLowerCase().includes('worked, failed, blocked') && request.plannedToolName === 'read')"
message:
expr: "`expected at least one read tool call during discovery report scenario, saw plannedToolNames=${JSON.stringify(discoveryDebugRequests.map((request) => request.plannedToolName ?? null))}`"
detailsExpr: outbound.text
```