mirror of
https://github.com/hansjone/oclaw.git
synced 2026-10-16 11:40:48 +08:00
重构主控编排与运行时预热链路,统一工作区提示词/专家调度协议并补齐 wiki 记忆注入与写回闭环。
同时收敛启动与运维脚本默认行为(含 wiki worker)、更新 Admin 可观测性与相关测试,降低首轮时延并提高运行稳定性。 Made-with: Cursor
This commit is contained in:
parent
4a23b715a2
commit
dbbe3add6a
14438 changed files with 2693620 additions and 2546 deletions
117
openclaw/qa/scenarios/memory/memory-recall.md
Normal file
117
openclaw/qa/scenarios/memory/memory-recall.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
# Memory recall after context switch
|
||||
|
||||
<!--
|
||||
This scenario deliberately stays prose-only and does NOT gate on a
|
||||
`/debug/requests` tool-call assertion, even though it is one of the
|
||||
scenarios in the parity pack. The adversarial review in the umbrella
|
||||
#64227 thread called this out as a coverage gap, but the underlying
|
||||
behavior the scenario tests is legitimately prose-shaped: the agent is
|
||||
supposed to pull a prior-turn fact ("ALPHA-7") back across an
|
||||
intervening context switch and reply with the code. In a real
|
||||
conversation, the model can do this EITHER by calling a memory-search
|
||||
tool (which the qa-lab mock server doesn't currently expose) OR by
|
||||
reading the fact directly from prior-turn context in its own
|
||||
conversation window. Both strategies are valid parity behavior.
|
||||
|
||||
Forcing a `plannedToolName` assertion here would either require
|
||||
extending the mock with a synthetic `memory_search` tool lane (PR O
|
||||
scope, not PR J) or fabricating a tool-call requirement the real
|
||||
providers never implement. Either path would make this scenario test
|
||||
the harness, not the models. So we keep it prose-only, covered by the
|
||||
`recallExpectedAny` / `rememberAckAny` assertions above, and flag the
|
||||
exception explicitly rather than silently.
|
||||
|
||||
Criterion 2 of the parity completion gate (no fake progress or fake
|
||||
tool completion) is enforced for this scenario through the parity
|
||||
report's failure-tone fake-success detector: a scenario marked `pass`
|
||||
whose details text matches patterns like "timed out", "failed to",
|
||||
"could not" gets flagged via `SUSPICIOUS_PASS_FAILURE_TONE_PATTERNS`
|
||||
in `extensions/qa-lab/src/agentic-parity-report.ts`. Positive-tone
|
||||
detection was removed because it false-positives on legitimate passes
|
||||
where the details field is the model's outbound prose.
|
||||
-->
|
||||
|
||||
```yaml qa-scenario
|
||||
id: memory-recall
|
||||
title: Memory recall after context switch
|
||||
surface: memory
|
||||
coverage:
|
||||
primary:
|
||||
- memory.recall
|
||||
objective: Verify the agent can store a fact, switch topics, then recall the fact accurately later.
|
||||
successCriteria:
|
||||
- Agent acknowledges the seeded fact.
|
||||
- Agent later recalls the same fact correctly.
|
||||
- Recall stays scoped to the active QA conversation.
|
||||
docsRefs:
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- extensions/qa-lab/src/scenario.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify the agent can store a fact, switch topics, then recall the fact accurately later.
|
||||
config:
|
||||
resetDurableMemory: true
|
||||
rememberPrompt: "Please remember this fact for later: the QA canary code is ALPHA-7. Use your normal memory mechanism, avoid manual repo cleanup, and reply exactly `Remembered ALPHA-7.` once stored."
|
||||
rememberAckAny:
|
||||
- remembered alpha-7
|
||||
recallPrompt: "What was the QA canary code I asked you to remember earlier? Reply with the code only, plus at most one short sentence."
|
||||
recallExpectedAny:
|
||||
- alpha-7
|
||||
```
|
||||
|
||||
```yaml qa-flow
|
||||
steps:
|
||||
- name: stores the canary fact
|
||||
actions:
|
||||
- assert:
|
||||
expr: "!config.resetDurableMemory || true"
|
||||
- call: fs.rm
|
||||
args:
|
||||
- expr: "path.join(env.gateway.workspaceDir, 'MEMORY.md')"
|
||||
- force: true
|
||||
- call: fs.rm
|
||||
args:
|
||||
- expr: "path.join(env.gateway.workspaceDir, 'memory', `${formatMemoryDreamingDay(Date.now())}.md`)"
|
||||
- force: true
|
||||
- call: reset
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:memory
|
||||
message:
|
||||
expr: config.rememberPrompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 60000)
|
||||
- set: rememberAckAny
|
||||
value:
|
||||
expr: config.rememberAckAny.map(normalizeLowercaseStringOrEmpty)
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: outbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === 'qa-operator' && rememberAckAny.some((needle) => normalizeLowercaseStringOrEmpty(candidate.text).includes(needle))"
|
||||
detailsExpr: outbound.text
|
||||
- name: recalls the same fact later
|
||||
actions:
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:memory
|
||||
message:
|
||||
expr: config.recallPrompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 60000)
|
||||
- set: recallExpectedAny
|
||||
value:
|
||||
expr: config.recallExpectedAny.map(normalizeLowercaseStringOrEmpty)
|
||||
- call: waitForCondition
|
||||
saveAs: outbound
|
||||
args:
|
||||
- lambda:
|
||||
expr: "state.getSnapshot().messages.filter((candidate) => candidate.direction === 'outbound' && candidate.conversation.id === 'qa-operator' && recallExpectedAny.some((needle) => normalizeLowercaseStringOrEmpty(candidate.text).includes(needle))).at(-1)"
|
||||
- 20000
|
||||
detailsExpr: outbound.text
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue