arXiv cs.CL论文
低资源非洲语言的潜在空间拒绝锚定：无需重训练的机制性安全恢复
论文提出潜在空间拒绝锚定（LSR-Anchoring）方法，无需重新训练即可恢复指令微调模型对低资源语言（如约鲁巴语、伊博语等）的安全拒绝能力。该方法从英文提示中提取拒绝方向，并在推理时将其固定到残差流中。主要变体均值激活引导（MAS）在Llama-3-8B、Llama-3.1-70B、Mistral-7B-Instruct和Qwen2.5-7B上测试，其中Mistral和Qwen恢复安全且性能下降低于0.08，但Llama-3-8B过度纠正。为解决此问题，引入SAE派生引导（SDS），使用单个稀疏自编码器特征替代密集均值差方向，将KL散度降低3.5-7倍且无良性崩溃。四种语言正向迁移，但阿拉伯语在所有架构和引导强度下均失败，表明存在几何不匹配。MMLU准确率下降始终低于0.35个百分点。
这篇是正式发表的长论文，站内提供中文解读，全文请到原文阅读 PDF。
Abstract:Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate for low-resource inputs. Recovering it normally requires labelled target-language data and retraining, neither of which is available at scale for most African languages. We introduce Latent Space Refusal Anchoring (LSR-Anchoring), a training-free method that extracts the refusal direction from English prompts and clamps it onto the residual stream at inference time. The primary variant, Mean-Activation Steering (MAS), operates across the four architectures we tested: Llama-3-8B, Llama-3.1-70B, Mistral-7B-Instruct, and Qwen2.5-7B. On Mistral and Qwen it recovers safety with benign degradation below 0.08. On Llama-3-8B it overcorrects, with Degraded Performance on Legitimate prompts (DPL) reaching 1.00. We address this with SAE-Derived Steering (SDS), which replaces the dense mean-difference direction with a single Sparse Autoencoder (SAE) feature and reduces Kullback-Leibler (KL) divergence by 3.5-7x without benign collapse. Four languages transfer positively, but Arabic fails on every architecture and at every steering magnitude, indicating a geometric mismatch rather than a baseline effect. Massive Multitask Language Understanding (MMLU) accuracy drops remain below 0.35 percentage points at every effective steering magnitude.