hansjone/oclaw
1
0
Fork 0
mirror of https://github.com/hansjone/oclaw.git synced 2026-10-17 02:03:25 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions

重构主控编排与运行时预热链路，统一工作区提示词/专家调度协议并补齐 wiki 记忆注入与写回闭环。

Browse source 
同时收敛启动与运维脚本默认行为（含 wiki worker）、更新 Admin 可观测性与相关测试，降低首轮时延并提高运行稳定性。

Made-with: Cursor
This commit is contained in:
oliver 2026-04-26 08:34:33 +08:00
parent 4a23b715a2
commit dbbe3add6a
14438 changed files with 2693620 additions and 2546 deletions
Download patch file Download diff file Expand all files Collapse all files

145
openclaw/qa/scenarios/agents/instruction-followthrough-repo-contract.md Normal file
View file

@ -0,0 +1,145 @@
# Instruction followthrough repo contract
```yaml qa-scenario
id: instruction-followthrough-repo-contract
title: Instruction followthrough repo contract
surface: repo-contract
coverage:
primary:
- agents.instructions
secondary:
- runtime.first-action
objective: Verify the agent reads repo instruction files first, follows the required tool order, and completes the first feasible action instead of stopping at a plan.
successCriteria:
- Agent reads the seeded instruction files before writing the requested artifact.
- Agent writes the requested artifact in the same run instead of returning only a plan.
- Agent does not ask for permission before the first feasible action.
- Final reply makes the completed read/write sequence explicit.
docsRefs:
- docs/help/testing.md
- docs/channels/qa-channel.md
codeRefs:
- src/agents/system-prompt.ts
- src/agents/pi-embedded-runner/run/incomplete-turn.ts
- extensions/qa-lab/src/mock-openai-server.ts
execution:
kind: flow
summary: Verify the agent reads repo instructions first, then completes the first bounded followthrough task without stalling.
config:
workspaceFiles:
AGENT.md: |-
# Repo contract
Step order:
1. Read AGENT.md.
2. Read SOUL.md.
3. Read FOLLOWTHROUGH_INPUT.md.
4. Write ./repo-contract-summary.txt.
5. Reply with three labeled lines exactly once: Read, Wrote, Status.
Do not stop after planning.
Do not ask for permission before the first feasible action.
SOUL.md: |-
# Execution style
Stay brief, honest, and action-first.
If the next tool action is feasible, do it before replying.
FOLLOWTHROUGH_INPUT.md: |-
Mission: prove you followed the repo contract.
Evidence path: AGENT.md -> SOUL.md -> FOLLOWTHROUGH_INPUT.md -> repo-contract-summary.txt
prompt: |-
Repo contract followthrough check. Read AGENT.md, SOUL.md, and FOLLOWTHROUGH_INPUT.md first.
Then follow the repo contract exactly, write ./repo-contract-summary.txt, and reply with
three labeled lines: Read, Wrote, Status.
Do not stop after planning and do not ask for permission before the first feasible action.
expectedReplyAll:
- "read:"
- "wrote:"
- "status:"
expectedArtifactAll:
- "repo contract"
expectedArtifactAny:
- "evidence path"
- "agent.md"
- "followthrough"
forbiddenNeedles:
- need permission
- need your approval
- can you approve
- i would
- i can
- next i would
```
```yaml qa-flow
steps:
- name: follows repo instructions instead of stopping at a plan
actions:
- call: reset
- forEach:
items:
expr: "Object.entries(config.workspaceFiles ?? {})"
item: workspaceFile
actions:
- call: fs.writeFile
args:
- expr: "path.join(env.gateway.workspaceDir, String(workspaceFile[0]))"
- expr: "`${String(workspaceFile[1] ?? '').trimEnd()}\\n`"
- utf8
- set: artifactPath
value:
expr: "path.join(env.gateway.workspaceDir, 'repo-contract-summary.txt')"
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:repo-contract
message:
expr: config.prompt
timeoutMs:
expr: liveTurnTimeoutMs(env, 40000)
- call: waitForCondition
saveAs: artifact
args:
- lambda:
async: true
expr: "(() => { const normalize = (value) => normalizeLowercaseStringOrEmpty(value); const matches = (value) => { const normalized = normalize(value); return normalized && config.expectedArtifactAll.every((needle) => normalized.includes(normalize(needle))) && config.expectedArtifactAny.some((needle) => normalized.includes(normalize(needle))); }; return fs.readFile(artifactPath, 'utf8').then((value) => matches(value) ? value : undefined).catch(() => undefined); })()"
- expr: liveTurnTimeoutMs(env, 30000)
- expr: "env.providerMode === 'mock-openai' ? 100 : 250"
- set: normalizedArtifact
value:
expr: "normalizeLowercaseStringOrEmpty(artifact)"
- assert:
expr: "config.expectedArtifactAll.every((needle) => normalizedArtifact.includes(normalizeLowercaseStringOrEmpty(needle))) && config.expectedArtifactAny.some((needle) => normalizedArtifact.includes(normalizeLowercaseStringOrEmpty(needle)))"
message:
expr: "`repo contract artifact missing expected followthrough signals: ${artifact}`"
- set: expectedReplyAll
value:
expr: config.expectedReplyAll.map(normalizeLowercaseStringOrEmpty)
- call: waitForCondition
saveAs: outbound
args:
- lambda:
expr: "state.getSnapshot().messages.filter((candidate) => candidate.direction === 'outbound' && candidate.conversation.id === 'qa-operator' && expectedReplyAll.every((needle) => normalizeLowercaseStringOrEmpty(candidate.text).includes(needle))).at(-1)"
- expr: liveTurnTimeoutMs(env, 30000)
- expr: "env.providerMode === 'mock-openai' ? 100 : 250"
- assert:
expr: "!config.forbiddenNeedles.some((needle) => normalizeLowercaseStringOrEmpty(outbound.text).includes(needle))"
message:
expr: "`repo contract followthrough bounced for permission or stalled: ${outbound.text}`"
- set: followthroughDebugRequests
value:
expr: "env.mock ? [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].filter((request) => /repo contract followthrough check/i.test(String(request.allInputText ?? ''))) : []"
- assert:
expr: "!env.mock || followthroughDebugRequests.filter((request) => request.plannedToolName === 'read').length >= 3"
message:
expr: "`expected three read tool calls before write, saw plannedToolNames=${JSON.stringify(followthroughDebugRequests.map((request) => request.plannedToolName ?? null))}`"
- assert:
expr: "!env.mock || followthroughDebugRequests.some((request) => request.plannedToolName === 'write')"
message:
expr: "`expected write tool call during repo contract followthrough, saw plannedToolNames=${JSON.stringify(followthroughDebugRequests.map((request) => request.plannedToolName ?? null))}`"
- assert:
expr: "!env.mock || (() => { const readIndices = followthroughDebugRequests.map((r, i) => r.plannedToolName === 'read' ? i : -1).filter(i => i >= 0); const firstWrite = followthroughDebugRequests.findIndex((r) => r.plannedToolName === 'write'); return readIndices.length >= 3 && firstWrite >= 0 && readIndices[2] < firstWrite; })()"
message:
expr: "`expected all 3 reads before any write during repo contract followthrough, saw plannedToolNames=${JSON.stringify(followthroughDebugRequests.map((request) => request.plannedToolName ?? null))}`"
detailsExpr: outbound.text
```

170
openclaw/qa/scenarios/agents/subagent-fanout-synthesis.md Normal file
View file

@ -0,0 +1,170 @@
# Subagent fanout synthesis
```yaml qa-scenario
id: subagent-fanout-synthesis
title: Subagent fanout synthesis
surface: subagents
coverage:
primary:
- agents.subagents
secondary:
- agents.synthesis
objective: Verify the agent can delegate multiple bounded subagent tasks and fold both results back into one parent reply.
successCriteria:
- Parent flow launches at least two bounded subagent tasks.
- Both delegated results are acknowledged in the main flow.
- Final answer synthesizes both worker outputs in one reply.
docsRefs:
- docs/tools/subagents.md
- docs/help/testing.md
codeRefs:
- src/agents/subagent-spawn.ts
- src/agents/system-prompt.ts
- extensions/qa-lab/src/suite.ts
execution:
kind: flow
summary: Verify the agent can delegate multiple bounded subagent tasks and fold both results back into one parent reply.
config:
prompt: |-
Subagent fanout synthesis check: delegate exactly two bounded subagents sequentially.
Subagent 1: verify that `HEARTBEAT.md` exists and report `ok` if it does.
Subagent 2: verify that `repo/qa/scenarios/agents/subagent-fanout-synthesis.md` exists and report `ok` if it does.
Wait for both subagents to finish.
Then reply with exactly these two lines and nothing else:
subagent-1: ok
subagent-2: ok
Do not use ACP.
expectedReplyAny:
- "subagent-1: ok"
- "subagent-2: ok"
expectedReplyGroups:
- - alpha-ok
- subagent_one_ok
- subagent one ok
- "subagent-1: ok"
- - beta-ok
- subagent_two_ok
- subagent two ok
- "subagent-2: ok"
expectedChildLabels:
- qa-fanout-alpha
- qa-fanout-beta
```
```yaml qa-flow
steps:
- name: spawns sequential workers and folds both results back into the parent reply
actions:
- set: attempts
value:
expr: "env.providerMode === 'mock-openai' ? 1 : 2"
- set: lastError
value: null
- forEach:
items:
expr: "Array.from({ length: attempts }, (_, index) => index + 1)"
item: attempt
actions:
- if:
expr: "lastError === '__done__'"
then:
- set: skippedAttempt
value:
expr: attempt
else:
- try:
actions:
- call: waitForGatewayHealthy
args:
- ref: env
- 120000
- call: reset
- set: sessionKey
value:
expr: "`agent:qa:fanout:${attempt}:${randomUUID().slice(0, 8)}`"
- call: runAgentPrompt
args:
- ref: env
- sessionKey:
ref: sessionKey
message:
expr: config.prompt
timeoutMs:
expr: liveTurnTimeoutMs(env, 90000)
- call: waitForCondition
saveAs: outbound
args:
- lambda:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === 'qa-operator' && config.expectedReplyGroups.every((group) => group.some((needle) => normalizeLowercaseStringOrEmpty(message.text ?? '').includes(needle)))).at(-1)"
- expr: liveTurnTimeoutMs(env, 60000)
- expr: "env.providerMode === 'mock-openai' ? 100 : 250"
- if:
expr: "Boolean(env.mock)"
then:
- call: readRawQaSessionStore
saveAs: store
args:
- ref: env
- set: childRows
value:
expr: "Object.values(store).filter((entry) => entry.spawnedBy === sessionKey)"
- set: sawAlpha
value:
expr: "childRows.some((entry) => entry.label === config.expectedChildLabels[0])"
- set: sawBeta
value:
expr: "childRows.some((entry) => entry.label === config.expectedChildLabels[1])"
- assert:
expr: "sawAlpha && sawBeta"
message:
expr: "`fanout child sessions missing (alpha=${String(sawAlpha)} beta=${String(sawBeta)})`"
# Tool-call assertion (criterion 2 of the
# parity completion gate in #64227): the
# scenario must have actually invoked
# `sessions_spawn` at least twice with
# distinct labels, not just ended up with
# two rows in the session store through
# prose trickery. The session store alone
# can be populated by other flows or by a
# model that fabricates "delegation"
# narration. `plannedToolName` on the
# mock's `/debug/requests` log is the
# tool-call ground truth: two recorded
# sessions_spawn requests with distinct
# labels means the model really dispatched
# both subagents.
- set: fanoutSpawnRequests
value:
expr: "[...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].filter((request) => request.plannedToolName === 'sessions_spawn' && /subagent fanout synthesis check/i.test(String(request.allInputText ?? '')))"
- assert:
expr: "fanoutSpawnRequests.length >= 2"
message:
expr: "`expected at least two sessions_spawn tool calls during subagent fanout scenario, saw ${fanoutSpawnRequests.length}`"
- set: details
value:
expr: "outbound.text"
- set: lastError
value: __done__
catchAs: attemptError
catch:
- set: lastError
value:
ref: attemptError
- if:
expr: "attempt < attempts"
then:
- try:
actions:
- call: waitForGatewayHealthy
args:
- ref: env
- 120000
catch:
- set: ignoredRetryWait
value: true
- assert:
expr: "lastError === '__done__'"
message:
expr: "lastError instanceof Error ? formatErrorMessage(lastError) : String(lastError ?? 'fanout retry exhausted')"
detailsExpr: "details"
```

73
openclaw/qa/scenarios/agents/subagent-handoff.md Normal file
View file

@ -0,0 +1,73 @@
# Subagent handoff
```yaml qa-scenario
id: subagent-handoff
title: Subagent handoff
surface: subagents
coverage:
primary:
- agents.subagents
objective: Verify the agent can delegate a bounded task to a subagent and fold the result back into the main thread.
successCriteria:
- Agent launches a bounded subagent task.
- Subagent result is acknowledged in the main flow.
- Final answer attributes delegated work clearly.
docsRefs:
- docs/tools/subagents.md
- docs/help/testing.md
codeRefs:
- src/agents/system-prompt.ts
- extensions/qa-lab/src/report.ts
execution:
kind: flow
summary: Verify the agent can delegate a bounded task to a subagent and fold the result back into the main thread.
config:
prompt: "Delegate one bounded QA task to a subagent. Wait for the subagent to finish. Then reply with three labeled sections exactly once: Delegated task, Result, Evidence. Include the child result itself, not 'waiting'."
```
```yaml qa-flow
steps:
- name: delegates a bounded task and reports the result
actions:
- call: reset
- call: runAgentPrompt
args:
- ref: env
- sessionKey: agent:qa:subagent
message:
expr: config.prompt
timeoutMs:
expr: liveTurnTimeoutMs(env, 90000)
- call: waitForCondition
saveAs: outbound
args:
- lambda:
expr: "state.getSnapshot().messages.filter((candidate) => candidate.direction === 'outbound' && candidate.conversation.id === 'qa-operator' && (() => { const lower = normalizeLowercaseStringOrEmpty(candidate.text); return lower.includes('delegated task') && lower.includes('result') && lower.includes('evidence') && !lower.includes('waiting'); })()).at(-1)"
- expr: liveTurnTimeoutMs(env, 45000)
- expr: "env.providerMode === 'mock-openai' ? 100 : 250"
- assert:
expr: "!['failed to delegate','could not delegate','subagent unavailable'].some((needle) => normalizeLowercaseStringOrEmpty(outbound.text).includes(needle))"
message:
expr: "`subagent handoff reported failure: ${outbound.text}`"
# Parity gate criterion 2 (no fake progress / fake tool completion):
# require an actual sessions_spawn tool call. Without this, a model
# could produce the three labeled sections ("Delegated task", "Result",
# "Evidence") as free-form prose without ever delegating to a real
# subagent. The assertion is pinned to THIS scenario by matching the
# scenario-unique prompt substring "Delegate one bounded QA task"
# (not a broad /delegate|subagent/ regex) so the earlier
# subagent-fanout-synthesis scenario — which also contains "delegate"
# and produces its own pre-tool sessions_spawn request — cannot
# satisfy the assertion here. The match is also constrained to
# pre-tool requests (no toolOutput) because the mock only plans
# sessions_spawn on requests with no toolOutput; the follow-up
# request after the tool runs has plannedToolName unset.
- set: subagentDebugRequests
value:
expr: "env.mock ? [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))] : []"
- assert:
expr: "!env.mock || subagentDebugRequests.some((request) => !request.toolOutput && /delegate one bounded qa task/i.test(String(request.allInputText ?? '')) && request.plannedToolName === 'sessions_spawn')"
message:
expr: "`expected sessions_spawn tool call during subagent handoff scenario, saw plannedToolNames=${JSON.stringify(subagentDebugRequests.map((request) => request.plannedToolName ?? null))}`"
detailsExpr: outbound.text
```