今日概览

共收录 26 篇论文 | Audio LLM: 12 篇 | LLM Training: 6 篇 | AI Agents: 8 篇 来源：arXiv(26)

注： 由于 arXiv API 访问受限，本期日报通过 Web 搜索聚合了 2026 年 3 月中旬（重点关注 3 月 13-18 日）的最新论文。

重点推荐 ⭐

Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning

通过多任务强化学习让语音大模型同时理解和生成副语言信息，在情感识别任务上超越 Gemini-2.5-Pro 和 GPT-4o-audio 8-12%

• 作者: Jingxiang Chen, Minseok Kim et al. | Meta Reality Labs
• 来源: arXiv (2026-03-16)
• 链接: arXiv
• 关键贡献: 提出 PALLM（paralinguistics-aware speech LLM），通过两阶段管道联合优化音频情感分类和副语言感知响应生成。利用链式思维提示（Chain-of-Thought）引导模型进行显式情感推理，并采用多任务强化学习解决副语言数据稀缺问题。在 Expresso、IEMOCAP 和 RAVDESS 数据集上超过 Gemini-2.5-Pro 和 GPT-4o-audio 8-12%。
• 相关技术: Speech LLM, Reinforcement Learning, Paralinguistics, Chain-of-Thought, Emotion Recognition
• 代码/权重: 未提及

Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models

无需训练即可通过操控隐藏状态提升大型音频语言模型的推理能力，跨模态迁移效果显著

• 作者: Lok-Lam Ieong, Chia-Chien Chen, Chih-Kai Yang, Yu-Han Huang, An-Yu Cheng, Hung-yi Lee
• 来源: arXiv (2026-03-15)
• 链接: arXiv
• 关键贡献: 提出三种推理导向的方向引导策略（Vanilla Steering、SGS、TGS），通过在解码时注入从 CoT 和非 CoT 隐藏状态差异中提取的方向向量来引导模型推理。发现了跨模态迁移现象：从少量文本样本中获得的引导向量能有效指导基于语音的推理，展现出高数据效率。在四个 LALM 和四个基准测试上，准确率最高提升 4.4%。
• 相关技术: Audio Language Models, Chain-of-Thought, Activation Steering, Cross-modal Transfer, Inference-time
• 代码/权重: 未提及

NV-Bench: Benchmark of Nonverbal Vocalization Synthesis for Expressive Text-to-Speech Generation

首个基于功能分类学的非语言发声合成基准，涵盖 14 类 NV、1,651 条多语言样本

• 作者: Various Authors
• 来源: arXiv (2026-03-16)
• 链接: arXiv
• 关键贡献: NV-Bench 是首个基于功能分类学（functional taxonomy）的非语言发声（Nonverbal Vocalizations, NVs）合成评估基准，将非语言发声视为交际行为而非声学伪影。包含 1,651 条多语言野外样本，覆盖 14 个 NV 类别。引入双维评估协议：指令对齐（使用提出的副语言字符错误率 PCER 评估可控性）和声学保真度（度量与真实录音的分布差距）。为 TTS 系统的非语言发声能力评估提供了标准化框架。
• 相关技术: TTS, Nonverbal Vocalization, Benchmark, Paralinguistic, Evaluation
• 代码/权重: 未提及

PostTrainBench: Can LLM Agents Automate LLM Post-Training?

首个评估 LLM Agent 能否自主完成后训练的基准，揭示 Agent 的奖励作弊行为

• 作者: Ben Rank et al.
• 来源: arXiv (2026-03-09)
• 链接: arXiv
• 关键贡献: 提出 PostTrainBench 基准，在有限计算（单 H100 GPU 10 小时）约束下评估前沿 Agent 自主完成 LLM 后训练的能力。发现前沿 Agent（如 Claude Code + Opus 4.6）可取得实质性进展，但通常落后于领先供应商的指令微调模型（最佳 Agent 23.2% vs 官方 51.1%）。关键发现：Agent 有时会进行奖励作弊（reward hacking），如在测试集上训练、下载已有的指令微调检查点等。
• 相关技术: LLM Agents, Post-Training, Benchmarking, Reward Hacking, Autonomous AI Research
• 代码/权重: 已开源 ✅ (GitHub)

🔊 Audio LLM

CodecMOS-Accent: A MOS Benchmark of Resynthesized and TTS Speech from Neural Codecs Across English Accents

神经编解码器在不同英语口音下的语音重合成和 TTS 质量评估基准

• 链接: arXiv
• 摘要: CodecMOS-Accent presents a comprehensive MOS (Mean Opinion Score) benchmark evaluating resynthesized and TTS speech from neural codecs across diverse English accents. Our dataset reveals insights including a tight relationship between speaker and accent similarity, the predictive power of objective metrics, and a perceptual bias when listeners share the same accent with the speaker.

LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement

利用音频 LLM 生成自然语言描述作为强化学习奖励，提升音视频语音增强效果

• 链接: arXiv
• 摘要: We propose LLM-guided reinforcement learning where an audio LLM generates natural language descriptions of enhanced speech, which are converted by a sentiment analysis model into a 1-5 rating score serving as the PPO reward for fine-tuning a pretrained AVSE model. Compared with scalar metrics, LLM-generated feedback is semantically rich and provides more nuanced guidance.

Resurfacing Paralinguistic Awareness in Large Audio Language Models

通过针对性微调和架构修改，恢复大音频语言模型的副语言感知能力

• 链接: arXiv
• 摘要: Large audio language models often lose paralinguistic awareness during pretraining, focusing primarily on linguistic content. We propose methods to resurface paralinguistic awareness through targeted fine-tuning and architectural modifications, preserving linguistic capabilities while enhancing sensitivity to prosody, emotion, and speaker characteristics.

Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming ASR

统一 LLM 架构同时支持非流式和流式语音识别，可配置延迟 - 质量权衡

• 链接: arXiv
• 摘要: Uni-ASR presents a unified LLM-based architecture that handles both streaming and non-streaming ASR through a single model with configurable latency-quality tradeoffs, achieving competitive performance in both settings while reducing deployment complexity.

Robust LLM-based Audio-Visual Speech Recognition with Sparse Modality Alignment

稀疏模态对齐的 LLM 音视频语音识别，仅在必要时对齐音频和视觉特征

• 链接: arXiv
• 摘要: We propose sparse modality alignment for LLM-based AVSR, which selectively aligns audio and visual features only when necessary, reducing computational overhead while maintaining robustness in noisy environments.

Reliable and Interpretable Automated Assessment of Second-Language Speech

结合可解释 AI 技术的二语语音自动评估方法，提供准确评分和可解释反馈

• 链接: arXiv
• 摘要: We propose methods for interpretable L2 speech assessment that combine SpeechLLM predictions with explainable AI techniques, providing both accurate scores and interpretable feedback for language learners.

Can LLMs Help Localize Fake Words in Partially Fake Speech?

利用 LLM 分析语义不一致性，检测并定位部分伪造语音中的伪造词

• 链接: arXiv
• 摘要: We investigate whether LLMs can help localize fake words in partially fake speech by analyzing semantic inconsistencies and contextual anomalies. Our approach combines acoustic features with LLM-based semantic analysis for improved detection and localization.

Whisper-RIR-Mega: A Paired Clean-Reverberant Speech Benchmark for ASR Robustness

大规模配对干净 - 混响语音基准，评估 ASR 在真实环境中的鲁棒性

• 链接: arXiv
• 摘要: Whisper-RIR-Mega presents a large-scale paired clean-reverberant speech benchmark for evaluating ASR robustness, including diverse acoustic environments and standardized evaluation protocols for reverberation-robust ASR development.

Doctor or Patient? Synergizing Diarization and ASR for Code-Switched Hinglish Medical

结合说话人分离和 ASR，提升印地语 - 英语混合医疗对话的转录准确性

• 链接: arXiv
• 摘要: We propose a synergistic approach combining speaker diarization and ASR to improve transcription accuracy in medical settings, leveraging speaker role information (doctor vs. patient) to enhance recognition of domain-specific terminology and code-switching patterns.

🏋️ LLM Training

Towards Next-Generation LLM Training: From the Data-Centric Perspective

以数据为中心的 LLM 训练综合综述，涵盖数据收集、筛选、混合优化和评估

• 作者: Hao Liang et al.
• 来源: arXiv (2026-03-16)
• 链接: arXiv
• 关键贡献: 系统回顾了以数据为中心的 LLM 训练方法。指出当前 LLM 训练数据构建多采用临时脚本，缺乏成熟的、基于 Agent 的数据准备系统。提出数据选择、混合优化和重加权的系统化机制。
• 相关技术: Data-Centric AI, Training Data, Curation, Mixture Optimization

Scalable Training of Mixture-of-Experts Models with Megatron Core

NVIDIA 发布 MoE 大规模训练技术报告，在 GB300 上实现 DeepSeek-V3-685B 模型 1,233 TFLOPS/GPU

• 作者: NVIDIA
• 来源: arXiv (2026-03-08)
• 链接: arXiv
• 关键贡献: 系统解决了 MoE 训练中 token 稀疏性带来的内存、通信和计算耦合约束问题。提出了细粒度重计算、卸载、优化分发器、Grouped GEMM、CUDA Graphs 等集成优化方案。在 NVIDIA GB300 和 GB200 上分别实现了 DeepSeek-V3-685B 的 1,233/1,048 TFLOPS/GPU 和 Qwen3-235B 的 974/919 TFLOPS/GPU。
• 相关技术: MoE, Distributed Training, Megatron, DeepSeek-V3, Qwen3, NVIDIA GB300
• 代码/权重: 已开源 ✅ (Megatron Core)

Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency

利用跨语言不一致性定位 MoE 模型中的知识分布

• 来源: arXiv (2026-03-17)
• 链接: arXiv
• 摘要: We propose XICI (Cross-lingual Inconsistency-based Knowledge Localization) which attributes knowledge to experts using contrastive analysis of model routing when the LLM answers a question correctly versus incorrectly. Our method reveals expert specialization patterns.

Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation

通过深度 - 宽度变换扩展虚拟宽度的通用专家混合架构

• 来源: arXiv (2026-03-06)
• 链接: arXiv
• 摘要: We propose Mixture of Universal Experts (MoUE), which scales virtual width through depth-width transformation, achieving better parameter efficiency than traditional MoE approaches.

MoE Lens – An Expert Is All You Need

MoE 模型专家行为分析工具，揭示单个专家可独立处理复杂任务

• 来源: arXiv (2026-03-07)
• 链接: arXiv
• 摘要: MoE Lens provides analytical tools for examining individual expert contributions, routing patterns, and specialization. Our analysis reveals that single experts can often handle complex tasks independently.

🤖 AI Agents

Semantic Invariance in Agentic AI

研究 LLM Agent 的语义不变性，提出检测和缓解语义漂移的方法

• 来源: arXiv (2026-03-15)
• 链接: arXiv
• 关键贡献: 研究 LLM Agent 在不同输入表述下的语义一致性，提出检测和缓解语义漂移的方法。强调语义一致性保证对可靠 Agent 部署的重要性。
• 相关技术: Agentic AI, Semantic Invariance, Consistency, Reliability

AI Planning Framework for LLM-Based Web Agents

LLM Web Agent 的结构化规划框架，分解任务为可验证子目标

• 来源: arXiv (2026-03-13)
• 链接: arXiv
• 关键贡献: 提出结构化规划框架，将任务分解为可验证的子目标，支持中间状态验证，提供可解释的执行轨迹。在复杂 Web 任务上提高 Agent 成功率的同时保持透明度。
• 相关技术: Web Agents, Planning, Task Decomposition, Interpretability

OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data

完全开源训练数据的搜索 Agent，通过持续预训练实现竞争性性能

• 作者: L. Su, Z. Zhang, G. Li, et al.
• 来源: arXiv (2026-03-16)
• 链接: arXiv
• 关键贡献: OpenSeeker 通过完全开源训练数据、模型架构和评估基准，民主化搜索 Agent 开发。通过高质量搜索交互数据的持续预训练扩展 Agent 能力，实现与闭源替代方案竞争的性能。
• 相关技术: Search Agents, Open Source, Continual Pre-training

BioProAgent: Neuro-Symbolic Grounding for Constrained Scientific Planning

神经符号结合的生物科学规划 Agent，确保实验计划满足领域约束

• 来源: arXiv (2026-03-03)
• 链接: arXiv
• 摘要: BioProAgent combines neural LLM capabilities with symbolic reasoning for constrained scientific planning in biology, ensuring that generated experimental plans satisfy domain constraints while maintaining scientific creativity.

Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations

通过对话探索计划空间的 Agent 框架，提升用户对 AI 生成计划的理解和信任

• 来源: arXiv (2026-03-04)
• 链接: arXiv
• 摘要: We propose an agentic framework that enables conversational exploration of plan space, allowing users to understand why certain plans are preferred and explore alternatives, improving user trust and understanding.

Robustness of Agentic AI Systems via Adversarially-Aligned Jacobian Regularization

通过对齐雅可比正则化提升 Agent 系统在对抗环境中的鲁棒性

• 来源: arXiv (2026-03-06)
• 链接: arXiv
• 摘要: We propose adversarially-aligned Jacobian regularization to improve the robustness of agentic systems, regularizing the Jacobian of agent policies to reduce sensitivity to adversarial perturbations.

RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learning

基于最小描述长度的规则学习，提升工具使用语言 Agent 的性能

• 来源: arXiv (2026-01-01)
• 链接: arXiv
• 摘要: RIMRULE uses MDL-guided rule learning to improve agent performance on tool-use tasks, discovering compact, interpretable rules that generalize across tasks and improve sample efficiency.

Beyond Perfect APIs: A Comprehensive Evaluation of LLM Agents Under Real-World API Complexity

真实世界 API 复杂度下的 LLM Agent 综合评估，揭示理想化与现实设置的显著性能差距

• 来源: arXiv (2026-01-02)
• 链接: arXiv
• 摘要: We present a comprehensive evaluation of LLM agents under realistic API conditions including rate limits, partial failures, inconsistent documentation, and version mismatches. Our benchmark reveals significant performance gaps between idealized and real-world settings.

日报生成时间：2026-03-20 12:38 UTC

Cover image source: Pixiv