超越最终答案：审计多智能体工业工作流中的轨迹级模型幻觉

AI 导读

提出了Trajel，一个用于审计多智能体工业工作流中轨迹级模型幻觉的数据集和评估框架。它基于专家标注的智能体轨迹，引入了五种幻觉分类：事实性、引用性、逻辑性、程序性和范围性。基准测试表明，现有基准遗漏了最常见故障模式，近半数幻觉轨迹涉及多种类型。自动化检测器尽管二分类准确度高，但仍会误判最微妙的类型。轨迹感知的检测显著优于标准的事后验证，表明基于分类法的评估对于更安全的智能体部署是必要的。

推荐理由

现有的幻觉测试只看最终答案，但真正危险的错误往往埋在中间步骤。Trajel把多智能体工业流程的每一步都审计了，发现近一半故障同时包含多种幻觉类型，做智能体安全的人该换评估方式了。

正文

View PDF HTML (experimental)

Abstract:Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination benchmarks still evaluate only the final output, missing failures that originate in intermediate Thought-Action-Observation steps. We present Trajel, a dataset and evaluation framework for auditing trajectory-level hallucinations in multi-agent industrial workflows. Trajel introduces a five-type hallucination taxonomy (factual, referential, logical, procedural, and scope-based) over expert-annotated agent traces from AssetOpsBench. We benchmark supervised detection models at the subtask, trajectory, and long-context levels. Our results show that the most common failure modes are missed by existing benchmarks, that nearly half of hallucinated trajectories involve multiple types at once, and that automated detectors with high binary accuracy still misclassify the subtlest types. Trajectory-aware detection significantly outperforms standard post-hoc verification, making taxonomy-grounded evaluation necessary for safer agentic deployment.

Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2605.24219 [cs.AI]
	(or arXiv:2605.24219v2 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2605.24219 arXiv-issued DOI via DataCite

Submission history

From: Santosh Subhashrao Borse [view email]
[v1] Fri, 22 May 2026 21:05:10 UTC (819 KB)
[v2] Tue, 26 May 2026 16:24:40 UTC (819 KB)