HuggingFace Daily Papers（社区热门论文）· HuggingFace Daily Papers（社区热门论文）·· 2026-04-17精选AI 评分81
Qwen3.5-Omni技术报告
AI 导读
阿里通义千问团队发布Qwen3.5-Omni，模型拥有数百亿参数和256k上下文，基于超1亿小时音视频数据训练，在215项基准测试中达SOTA水平，关键音频任务超越Gemini-3.1 Pro。采用混合注意力MoE架构，支持10小时音频与400秒720P视频理解。提出ARIA技术优化流式语音合成稳定性，支持10种语言。模型具备Audio-Visual Vibe Coding能力，可直接基于音视频指令生成代码。
推荐理由
Qwen3.5-Omni把语音、视频、文本全模态做到一个模型里，且在215项音频任务上超越Gemini 3.1 Pro，这是国内团队在全模态模型上的里程碑，做交互产品的值得关注。
正文
Abstract:In this work, we present Qwen3.5-Omni, the latest advancement in the Qwen-Omni model family. Representing a significant evolution over its predecessor, Qwen3.5-Omni scales to hundreds of billions of parameters and supports a 256k context length. By leveraging a massive dataset comprising heterogeneous text-vision pairs and over 100 million hours of audio-visual content, the model demonstrates robust omni-modality capabilities. Qwen3.5-Omni-plus achieves SOTA results across 215 audio and audio-visual understanding, reasoning, and interaction subtasks and benchmarks, surpassing Gemini-3.1 Pro in key audio tasks and matching it in comprehensive audio-visual understanding. Architecturally, Qwen3.5-Omni employs a Hybrid Attention Mixture-of-Experts (MoE) framework for both Thinker and Talker, enabling efficient long-sequence inference. The model facilitates sophisticated interaction, supporting over 10 hours of audio understanding and 400 seconds of 720P video (at 1 FPS). To address the inherent instability and unnaturalness in streaming speech synthesis, often caused by encoding efficiency discrepancies between text and speech tokenizers, we introduce ARIA. ARIA dynamically aligns text and speech units, significantly enhancing the stability and prosody of conversational speech with minimal latency impact. Furthermore, Qwen3.5-Omni expands linguistic boundaries, supporting multilingual understanding and speech generation across 10 languages with human-like emotional nuance. Finally, Qwen3.5-Omni exhibits superior audio-visual grounding capabilities, generating script-level structured captions with precise temporal synchronization and automated scene segmentation. Remarkably, we observed the emergence of a new capability in omnimodal models: directly performing coding based on audio-visual instructions, which we call Audio-Visual Vibe Coding.
| Subjects: | Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2604.15804 [cs.CL] |
| (or arXiv:2604.15804v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.15804 arXiv-issued DOI via DataCite |
Submission history
From: Jin Xu [view email]
[v1]
Fri, 17 Apr 2026 08:05:46 UTC (2,893 KB)
[v2]
Tue, 21 Apr 2026 03:35:14 UTC (2,914 KB)
来源：HuggingFace Daily Papers（社区热门论文） · arxiv.org