arXiv cs.CL论文
超越语音字幕：基于语音奖励的风格规划用于对话式文本转语音
该论文指出自然语言风格描述在LLM与可控TTS间存在信息压缩问题，导致描述保真度不等于合成控制有效性。作者通过实证研究表明候选指令的语音-文本对齐度对下游声学相似性预测能力较弱，并提出一种基于语音奖励的风格规划方法以解决此问题。
这篇是正式发表的长论文，站内提供中文解读，全文请到原文阅读 PDF。
Abstract:Natural-language style descriptions provide an interpretable interface between large language models (LLMs) and controllable text-to-speech (TTS). However, using descriptions as pseudo-labels compresses target acoustics into text, and descriptive fidelity need not imply effective control of a particular synthesizer. We empirically show that speech-text alignment only weakly predicts downstream acoustic similarity among candidate instructions for the same utterance. We therefore propose Speech-Rewarded Style Planning (SRSP), which trains a text-based style planner through a frozen downstream TTS model. Given dialogue history and response text, the planner generates candidate instructions and is optimized with group-relative policy optimization (GRPO), using the teacher-forced likelihood of target speech tokens as the reward. On an English subset of the ISCSLP 2026 CoT-TTS corpus, SRSP achieves higher speech-style and emotion similarity to target speech and lower mel-cepstral distortion than the Base LLM and target-audio-informed captioning baselines. LLM-based expressive speech evaluation further shows gains over all baselines in contextual appropriateness and reference consistency.