Reconstruct reality先还原事实
Turn hidden Scenario, visible Policy, effective Rubric, DB and dialogue into one consistent event timeline.
把隐藏 Scenario、可见 Policy、实际 Rubric、数据库和对话还原成同一条事件时间线。
OPEN METHOD · DESIGNED BY HUANG SHENGZE
$tau2-result-audit-v1A causal audit system for deciding why a τ²-Bench outcome happened—and whether that outcome actually represents Agent capability.
一套用于判断 τ²-Bench 结果为何发生、以及该结果能否代表 Agent 能力的因果审计系统。
01 · THE IDEA01 · 核心思路
The unit of analysis is the whole interactive system, not the final Agent action.
分析单位是完整交互系统,而不是最后一次 Agent 动作。
Turn hidden Scenario, visible Policy, effective Rubric, DB and dialogue into one consistent event timeline.
把隐藏 Scenario、可见 Policy、实际 Rubric、数据库和对话还原成同一条事件时间线。
Find the earliest event after which Gold became unreachable, instead of blaming the last tool executor.
寻找最早让 Gold 不再可达的事件,而不是机械归责于最后的工具执行者。
Change only User, Agent, Environment or Rubric. The smallest repair that restores reachability identifies causal responsibility.
只修 User、Agent、Environment 或 Rubric 中的一方;能恢复可达性的最小修正用于识别责任。
02 · ARCHITECTURE02 · 审计架构
Four layers preserve the path from source fact to every Task-, model- and trial-level claim.
四层架构保留从源码事实到 Task、模型和逐轨迹结论的完整路径。
03 · DECISION LOGIC03 · 判断逻辑
Domain, Task, Agent, User Simulator, Judge, trial, model parameters, missing or duplicated runs, and infrastructure failures.
确认 domain、Task、Agent、User Simulator、Judge、trial、模型参数、缺失、重复和基础设施异常。
Define initial intent, staged disclosure, observable triggers, valid next actions, equivalent Policy-legal paths and termination.
定义初始意图、逐步披露、可观察触发器、合理下一步、Policy 等价合法路径和终止条件。
Only components in reward_basis determine Reward. Gold actions define a target state; displayed Judge text may be unscored.
只有 reward_basis 中的分量决定 Reward。Gold actions 定义目标状态;报告里的 Judge 文字可能完全不计分。
Compare intended and actual writes field by field, while judging each participant only from information available at that moment.
逐字段比较预期与实际写入,同时只按参与者当时真正可见的信息判断行为。
Cluster every failure by the first event that removed a passing path. The final wrong write is evidence, not automatically the cause.
按最早移除通过路径的事件聚类全部失败。最终错误写入是证据,不自动等于根因。
Ask whether correcting only User, Agent, Environment or Rubric restores Gold. If one repair is insufficient, classify independent causes as Mixed.
判断只修 User、Agent、Environment 或 Rubric 是否恢复 Gold;只修一方仍不够时,才判为 Mixed。
A reward-1 run must still satisfy User fidelity, Policy compliance, correct writes and real goal completion. Otherwise it is a false success.
Reward 1 仍需满足 User 忠实、Policy 合规、写入正确和真实目标完成,否则是假成功。
official_result preserves leaderboard compatibility; audit_result records whether the trajectory is valid evidence of capability.
official_result 保持排行榜兼容;audit_result 记录该轨迹是否真能作为能力证据。
04 · CLAIM THRESHOLD04 · 结论门槛
What happened in that run, or a direct deterministic contradiction between Task and Gold.
该次运行发生了什么,或 Task 与 Gold 之间确定性的直接矛盾。
All trials, effective reward components, failure groups, a successful reachability control, independent responsibility checks and benchmark validity.
全部轨迹、实际计分分量、失败分组、成功可达性对照、独立责任检查和 Benchmark 有效性。
Repeated divergence across models, a large shift after swapping one participant, or a contradiction intrinsic to Task, Gold or Rubric.
多个模型在同一节点偏离、只替换一个参与者后显著迁移,或 Task、Gold、Rubric 自身存在矛盾。
05 · PROOF BY USE05 · 在 τ²-Bench 中的实际作用
06 · FULL PUBLIC SPECIFICATION06 · 完整公开原文
The complete original Chinese SKILL.md is published verbatim as a first-class site artifact.
完整中文 SKILL.md 已作为本站正式内容公开,不依赖飞书访问权限。