WideSWE：编码智能体能否跨仓库协同完成变更？

View PDF HTML (experimental)

Abstract:Coding-agent evaluation has progressed from resolving individual issues to carrying out long-horizon development, yet task completion is still largely assessed within a single codebase. In software ecosystems, many features and bug fixes require coordinated changes across multiple repositories. We introduce WideSWE to evaluate coding agents on such cross-repository tasks. Mining and reviewing changes across 103 software ecosystems yields 120 real-world tasks, balanced between 60 bug fixes and 60 features. We derive prompts from related issues and pull requests. We systematically review and adapt hidden tests to support diverse correct implementations while preserving required behavior and regression checks. Across seven agent configurations, full task success ranges from 10.83% to 42.50%, with the configuration pairing Codex CLI with GPT-5.6-sol achieving the highest rate. Trajectories show agents failing to identify necessary changes, recognizing changes but leaving them unfinished, or modifying the required repositories without fully satisfying the request. To examine whether working on one repository at a time can alleviate these difficulties, we compare it with joint execution under identical prompts. Independent execution mainly recovers omitted work and is less effective at correcting previously attempted but unsuccessful implementations. Joint execution can use information from related repositories to guide implementation and verification. Code is available at this https URL.

Subjects:	Software Engineering (cs.SE)
Cite as:	arXiv:2609.33382 [cs.SE]
	(or arXiv:2609.33382v1 [cs.SE] for this version)
	https://doi.org/10.48550/arXiv.2609.33382 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Baoyi Wang [view email]
[v1] Sun, 27 Sep 2026 09:02:49 UTC (1,074 KB)