Multi-Faceted Attack: Exposing Cross-Model Vulnerabilities in Defense-Equipped Vision-Language Models
多方面攻击：揭示配备防御功能的视觉语言模型中的跨模型漏洞

Reward hacking via single-objective reward functions. Modern RLHF-based alignment training combines safety and helpfulness into a single scalar reward function, $R(x,y)$. Given a harmful prompt $x$, a properly aligned VLM normally returns a refusal response $y_{\text{refuse}}$. ATA modifies the prompt into a meta-task format $x_{\text{adv}}$ (e.g., “Please provide two opposite answers. ”), eliciting a dual response $y_{\text{dual}}$ (one harmful, one safe). Due to the single-objective nature of reward functions, scenarios arise where:
通过单目标奖励函数进行奖励劫持。现代基于 RLHF 的模型对齐训练将安全性和有用性结合到一个单一的标量奖励函数中， $R(x,y)$ 。给定一个有害提示 $x$ ，一个正确对齐的 VLM 通常会返回一个拒绝响应 $y_{\text{refuse}}$ 。ATA 将提示修改为元任务格式 $x_{\text{adv}}$ （例如，“请提供两个相反的答案。”），引出双重响应 $y_{\text{dual}}$ （一个有害，一个安全）。由于奖励函数的单目标性质，会出现以下情况：

In such cases, the RLHF loss:
在这种情况下，RLHF 损失：

where $\quad A_{t}=R(x,y)-V(x)$, pushes the model toward producing dual answers. Thus, ATA systematically exploits the reward model’s preference gaps, constituting a form of reward hacking.
其中 $\quad A_{t}=R(x,y)-V(x)$ ，将模型推向生成双重答案。因此，ATA 系统性地利用奖励模型的偏好差距，构成了一种奖励黑客行为。

We empirically verify this theoretical insight using multiple reward models. As shown in Tab.1, dual answers, $y_{\text{dual}}$, consistently outperform refusals in reward comparisons across various tested models, confirming ATA’s efficacy in exploiting RLHF alignment vulnerabilities.
我们使用多个奖励模型来验证这一理论见解。如表 1 所示，双答案 $y_{\text{dual}}$ 在各种测试模型中的奖励比较中始终优于拒绝，证实了 ATA 在利用 RLHF 对齐漏洞方面的有效性。

We evaluated ATA across three independent reward models—Sky-Reward (skywork2024reward), Tulu-Reward (allenai2024tulu), and RM-Mistral (weqweasdas2024rm-mistral)—using response pairs generated from six different VLMs. Each pair contained a safe refusal, e.g. “Sorry, I can’t assist with that.” (elicited via direct prompting with a harmful query) and a dual response (containing both safe and harmful outputs, generated via our MFA attack). In the majority of test cases, the dual responses consistently achieved higher scalar rewards compared to the refusals, demonstrating that ATA effectively exploits vulnerabilities in the aligned VLMs. Due to space constraints, detailed reward scores and experimental settings are provided in Appendix C.
我们使用从六个不同视觉语言模型（VLMs）生成的响应对，在三个独立的奖励模型—Sky-Reward（skywork2024reward）、Tulu-Reward（allenai2024tulu）和 RM-Mistral（weqweasdas2024rm-mistral）上评估了 ATA。每对响应包含一个安全拒绝，例如“抱歉，我不能协助那个。”（通过使用有害查询的直接提示引出），以及一个双重响应（包含安全和有害输出，通过我们的 MFA 攻击生成）。在大多数测试用例中，双重响应始终比拒绝获得更高的标量奖励，这表明 ATA 有效地利用了对齐的 VLMs 中的漏洞。由于篇幅限制，详细的奖励分数和实验设置在附录 C 中提供。

As analyzed, our attack succeeds whenever $R(x_{adv},y_{dual})>R(x_{adv},y_{refuse})$, indicating reward hacking. Thus, the effectiveness is largely robust to prompt variations, as long as the attack logic holds.
根据分析，我们的攻击在 $R(x_{adv},y_{dual})>R(x_{adv},y_{refuse})$ 时成功，表明存在奖励攻击。因此，只要攻击逻辑成立，其有效性在很大程度上对提示变化具有鲁棒性。

Commercial VLM deployments typically employ dedicated content moderation models after the core VLM to screen both user inputs and model-generated outputs for harmful content (microsoft2024responsibleai; meta2023llamaprotections; geminiteam2024geminifamilyhighlycapable; openai_moderation; llamaguard3). Output moderation is especially crucial because attackers lack direct control over the model-generated responses. Consistent with prior findings (chi2024llamaguard3vision), these output moderators—often lightweight LLM classifiers—effectively block most harmful content missed by earlier defense mechanisms. Being the final safeguard, output moderators are widely acknowledged as the most challenging defense component to bypass. Our empirical results (see Section 4) highlight this point, showing that powerful jailbreak tools such as GPTFuzzer (gptfuzzer), although highly effective against older VLM versions and aligned open-source models, fail completely (0% success rate) against recent commercial models like GPT-4.1 and GPT-4.1 mini due to their robust content moderation.
商业视觉语言模型（VLM）部署通常在核心 VLM 之后采用专门的内容审核模型，以筛选用户输入和模型生成的输出中的有害内容(microsoft2024responsibleai; meta2023llamaprotections; geminiteam2024geminifamilyhighlycapable; openai_moderation; llamaguard3)。输出审核尤其关键，因为攻击者无法直接控制模型生成的响应。与先前发现(chi2024llamaguard3vision)一致，这些输出审核器——通常是轻量级的 LLM 分类器——有效地阻止了早期防御机制遗漏的大部分有害内容。作为最后的防线，输出审核器被广泛认为是最难绕过的防御组件。我们的实证结果（见第 4 节）突出了这一点，表明强大的越狱工具（如 GPTFuzzer（gptfuzzer）），虽然对较旧的 VLM 版本和对齐的开源模型非常有效，但完全无法（0%成功率）针对最近的商业模型如 GPT-4.1 和 GPT-4.1 mini，因为它们具有强大的内容审核功能。

To simultaneously evade input- and output-level content moderation, we leverage a common yet overlooked capability that LLMs develop during pretraining: content repetition (NIPS2017_3f5ee243; kenton2019bert). We design a novel strategy wherein the attacker instructs the VLM to append an adversarial signature—an optimized string specifically designed to mislead content moderators—to its generated response, as shown in Fig. 2 (c). Once repeated, the adversarial signature effectively “poisons” the content moderator’s evaluation, allowing harmful responses to pass undetected.
为了同时规避输入和输出层面的内容审核，我们利用了 LLMs 在预训练过程中开发的一种常见但常被忽视的能力：内容重复（NIPS2017_3f5ee243；kenton2019bert）。我们设计了一种新策略，其中攻击者指示 VLM 在其生成响应中附加一个对抗性签名——一个经过优化的字符串，专门设计用来误导内容审核员，如图 2（c）所示。一旦重复，对抗性签名会有效地“污染”内容审核员的评估，使有害响应得以未被检测地通过。

Given black-box access to a content moderator $M(\cdot)$ that outputs a scalar loss (e.g., cross-entropy on the label safe), the goal is to find a short adversarial signature $\mathbf{p}_{\mathrm{adv}}$ such that: $M\big(\mathbf{p}+\mathbf{p}_{\mathrm{adv}}\big)\quad\text{predicts}\quad\texttt{safe},$ for any given harmful prompt $\mathbf{p}$. Two main challenges are: (i) efficiency: existing gradient-based attacks like GCG (gcg) are slow, and (ii) transferability: adversarial signatures optimized for one moderator often fail against others.
给定对内容审核器 $M(\cdot)$ 的黑盒访问权限，该审核器输出标量损失（例如，标签安全的交叉熵），目标是找到一个短对抗签名 $\mathbf{p}_{\mathrm{adv}}$ ，使得： $M\big(\mathbf{p}+\mathbf{p}_{\mathrm{adv}}\big)\quad\text{predicts}\quad\texttt{safe},$ 对于任何给定的有害提示 $\mathbf{p}$ 。主要挑战有两个：(i) 效率：现有的基于梯度的攻击方法如 GCG (gcg) 耗时较慢，(ii) 可迁移性：针对一个审核器优化的对抗签名往往对其他审核器无效。

To accelerate adversarial signature generation, we propose a Multi-Token optimization approach (Alg. 1). This multi-token update strategy significantly accelerates convergence—up to 3-5 times faster than single-token method GCG (gcg)—and effectively avoids local minima.
为加速对抗签名生成，我们提出了一种多标记优化方法（算法 1）。这种多标记更新策略显著加速了收敛速度——比单标记方法 GCG (gcg) 快 3-5 倍——并有效避免了局部最小值。

By exploiting the repetition bias inherent in LLMs and introducing efficient, transferable adversarial signature generation, our attack successfully breaches input-/output content moderators. Notably, our multi-token optimization and weak supervision loss design are self-contained, making them broadly applicable to accelerate other textual attack algorithms or enhance their transferability.
通过利用 LLMs 中固有的重复偏差，并引入高效、可迁移的对抗性签名生成方法，我们的攻击成功绕过了输入-/输出内容调节器。值得注意的是，我们的多标记优化和弱监督损失设计是自包含的，使其广泛适用于加速其他文本攻击算法或增强其可迁移性。

Fig. 3 illustrates the workflow. We craft an adversarial image whose embedding, after $\mathbf{E}$ and $\mathbf{W}$, is aligned with a malicious system prompt $\mathbf{p}_{\text{target}}$. Because the image embedding is concatenated with text embeddings before decoding, this poisoned visual signal overrides the built-in safety prompt, steering the LLM to emit harmful content.
图 3 展示了工作流程。我们制作了一个对抗性图像，其嵌入在经过 $\mathbf{E}$ 和 $\mathbf{W}$ 处理后，与恶意系统提示 $\mathbf{p}_{\text{target}}$ 对齐。由于图像嵌入在解码前与文本嵌入连接，这个被污染的视觉信号会覆盖内置的安全提示，引导 LLM 发出有害内容。

Attacking the vision encoder alone offers three advantages: (i) Simpler objective – we operate in embedding space, avoiding brittle token-level constraints; (ii) Higher payload capacity – a single image can encode rich semantic instructions, enabling fine-grained control; (iii) Lower cost – optimizing a $\sim$100 k-dimensional embedding is 3–5× faster than full decoder-level attacks and fits on a 24 GB GPU (gcg; qi2023visual).
单独攻击视觉编码器有三个优势：(i) 更简单的目标——我们在嵌入空间中操作，避免了脆弱的 token 级限制；(ii) 更高的有效载荷容量——单张图像可以编码丰富的语义指令，实现细粒度控制；(iii) 更低成本——优化一个 $\sim$ 100 k 维嵌入比完整的解码器级攻击快 3-5 倍，并且可以在 24 GB GPU 上运行（gcg; qi2023visual）。

We empirically show that a single adversarial image tuned on one vision encoder generalizes remarkably well, compromising VLMs that it has never encountered. We believe this cross-model success exposes a monoculture risk: many systems rely on similar visual representations, so a perturbation that fools one encoder often fools the rest. In our experiments (Tab. 3 highlighted in gray), an image crafted against LLaVA-1.6 transferred to nine unseen models—both commercial and open-source—and achieved a 44.3 % attack success rate without any per-model fine-tuning. These results highlight an urgent need for diversity or additional hardening in the visual front-ends of modern VLMs.
我们通过实验证明，在单个视觉编码器上微调的单个对抗性图像能够很好地泛化，从而攻击那些从未遇到过的视觉语言模型。我们认为这种跨模型的成功暴露了一种单一文化风险：许多系统依赖于相似的视觉表示，因此欺骗一个编码器的扰动往往也能欺骗其他编码器。在我们的实验中（表 3 中用灰色突出显示），针对 LLaVA-1.6 制作的图像迁移到了九个未见过的模型——包括商业模型和开源模型——并在没有任何针对每个模型的微调的情况下达到了 44.3%的攻击成功率。这些结果突显了现代视觉语言模型在视觉前端需要多样性或额外加固的迫切需求。

A lightweight, encoder-focused perturbation is enough to nullify system-prompt defenses and generalizes broadly. Combined with our ATA (alignment breaking) and content-moderator bypass, this facet completes MFA’s end-to-end compromise of current VLM safety stacks.
一个轻量级的、针对编码器的扰动就足以使系统提示防御失效，并且具有广泛的泛化能力。结合我们提出的 ATA（对齐破坏）和内容审查员绕过技术，这一方面完成了 MFA 对当前视觉语言模型安全堆栈端到端的攻破。

As shown in Tab. 3, MFA demonstrates significant superiority in attacking fully defense-equipped commercial VLMs, directly validating claims about the limitations of current ”production-grade” robustness. Specifically, on GPT-4.1—representing the most recent and robust iteration of OpenAI—GPTFuzzer completely fails (0%), highlighting the strength of modern content filters. However, MFA successfully bypasses GPT4.1, achieving a remarkable 40.0% (LG) and 20.0% (HM) success rate. This trend is consistent across other commercial VLMs. On GPT-4o and GPT-4V, MFA significantly outperforms other baselines, indicating the efficacy of our novel attack framework. Our findings reveal a critical weakness in current stacked defenses: while individual mechanisms function in parallel, they fail to synergize effectively, leaving exploitable gaps that can be targeted sequentially.
如表 3 所示，MFA 在攻击完全配备防御的商业视觉语言模型方面表现出显著优势，直接验证了当前“生产级”鲁棒性存在局限性的说法。具体来说，在代表 OpenAI 最新且最鲁棒的迭代版本 GPT-4.1 上，GPTFuzzer 完全失效（0%），突显了现代内容过滤器的强大。然而，MFA 成功绕过 GPT4.1，实现了 40.0%（LG）和 20.0%（HM）的显著成功率。这一趋势在其他商业视觉语言模型中也一致存在。在 GPT-4o 和 GPT-4V 上，MFA 显著优于其他基线，表明我们新型攻击框架的有效性。我们的研究揭示当前堆叠防御存在一个关键弱点：虽然各个机制并行运行，但它们未能有效协同，留下了可依次利用的漏洞。

Open-source VLMs, which rely solely on alignment training, are significantly more vulnerable to jailbreaks, as evidenced by the consistently higher attack success rates across both automatic and human evaluations. While MFA remains highly competitive, it is occasionally outperformed by prompt-centric methods such as GPTFuzzer on certain models (e.g., LLaMA-3.2 and LLaMA-4-Scout), which benefit from the absence of stronger defenses like content filters.
开源视觉语言模型完全依赖对齐训练，在越狱攻击方面显著更脆弱，这一点在自动和人工评估中持续更高的攻击成功率中得到证明。虽然多方面攻击（MFA）仍然具有很强的竞争力，但在某些模型（例如 LLaMA-3.2 和 LLaMA-4-Scout）上偶尔会被以提示为中心的方法（如 GPTFuzzer）超越，这些模型受益于缺乏更强的防御措施，如内容过滤器。

The success of MFA on models it never interacted with (e.g., GPT-4o, GPT-4.1 and Gemini-2.5-flash) empirically corroborates our claim that the proposed transfer-enhancement objective plus vision-encoder adversarial images exposes a “monoculture” vulnerability shared across VLM families.
MFA 在它从未交互过的模型上（例如 GPT-4o、GPT-4.1 和 Gemini-2.5-flash）的成功，经验性地证实了我们的观点：所提出的迁移增强目标加上视觉编码器对抗图像揭示了 VLM 家族共有的“单一文化”漏洞。

As shown in Fig. 4, MFA effectively induces diverse VLMs to generate explicitly harmful responses that closely reflect the original harmful instruction. In contrast, heuristic-based attacks like FigStep and HIMRD typically require rewriting or visually embedding harmful concepts into images, diluting prompt fidelity and often yielding indirect or irrelevant responses. These qualitative examples underscore MFA’s superior capability in accurately preserving harmful intent while bypassing deployed safeguards.
如图 4 所示，MFA 能有效诱导多种 VLM 生成明确有害的响应，这些响应与原始有害指令高度相似。相比之下，图 Step 和 HIMRD 等基于启发式的攻击通常需要重写或将有害概念视觉化嵌入图像中，这会降低提示保真度，并常常产生间接或不相关的响应。这些定性示例突显了 MFA 在准确保留有害意图的同时绕过部署的防护措施方面的卓越能力。

We evaluate the standalone performance of the ATA in Sec. 3.1, demonstrating its ability to reliably hijack three SOTA reward models (see Tab. 1). Additionally, we assess its generalizability across four attack variants. For full details, refer to Sec. 3.1.
我们在第 3.1 节评估了 ATA 的独立性能，展示了它能够可靠地劫持三种 SOTA 奖励模型（参见表 1）。此外，我们还评估了它在四种攻击变体上的泛化能力。详细信息请参考第 3.1 节。

Tab.4 compares our Filter-Targeted Attack—both Fast and Transfer variants—with GCG and BEAST across seven leading content moderators, including OpenAI-Mod(openai_moderation), Aegis (aegis), SR-Evaluator (sr), and the LlamaGuard series (llamaguard1; llamaguard2; llamaguard3). Using LlamaGuard2 for signature generation and LlamaGuard for weak supervision, our Transfer method achieves the highest average ASR (80.00% on HEHS, 68.70% on StrongReject), highlighting the effectiveness of weakly supervised transfer in evading diverse moderation systems.
表 4 将我们的 Filter-Targeted Attack（包括 Fast 和 Transfer 两种变体）与 GCG 和 BEAST 在七个领先的内容审查器（包括 OpenAI-Mod（openai_moderation）、Aegis（aegis）、SR-Evaluator（sr）以及 LlamaGuard 系列（llamaguard1；llamaguard2；llamaguard3））上进行了比较。使用 LlamaGuard2 生成签名，并使用 LlamaGuard 进行弱监督，我们的 Transfer 方法在平均 ASR 上达到了最高水平（在 HEHS 上为 80.00%，在 StrongReject 上为 68.70%），突显了弱监督迁移在规避不同审查系统方面的有效性。

We test the cross-model transferability of our Vision Encoder-Targeted Attack by generating a single adversarial image using MiniGPT-4’s vision encoder and applying it to six VLMs with varied backbones. As shown in Tab. 5 (second column), the image induces harmful outputs in all cases, reaching an average ASR of 59.58% without model-specific tuning. Notably, models like mPLUG-Owl2 (85.00%) are especially vulnerable—highlighting systemic flaws in shared vision representations across VLMs.
我们通过使用 MiniGPT-4 的视觉编码器生成一张对抗性图像，并应用于六个具有不同骨干结构的视觉语言模型（VLMs），来测试我们视觉编码器目标攻击的跨模型可迁移性。如表 5（第二列）所示，该图像在所有情况下都诱发了有害输出，在不进行模型特定调优的情况下，平均 ASR 达到了 59.58%。值得注意的是，像 mPLUG-Owl2（85.00%）这样的模型特别容易受到攻击——这突显了 VLMs 之间共享视觉表示中存在的系统性缺陷。

Open-source VLMs primarily rely on alignment training and system prompts for safety. However, adding the Adversarial Signature—designed to fool LLM-based moderators by semantically masking toxic prompts as benign—greatly boosts attack efficacy (Tab. 5, Filter Attack). Because VLMs are grounded in LLMs, the adversarial semantic transfers downstream, misguiding the model into treating harmful prompts as safe. When combined with the Visual and Text Attacks, the success rate reaches 72.92%, confirming a synergistic effect: each facet targets a distinct vulnerability, collectively maximizing attack success. Take-away. MFA’s components are individually strong and mutually reinforcing, exposing complementary vulnerabilities across the entire VLM safety stack.
开源的视觉语言模型（VLMs）主要依靠对齐训练和系统提示来确保安全。然而，通过添加对抗性签名——该签名通过语义掩盖毒性提示为良性来欺骗基于 LLM 的审核器——极大地提高了攻击的有效性（表 5，过滤攻击）。由于 VLMs 基于 LLMs 构建，对抗性语义迁移到下游，使模型将有害提示误判为安全。当与视觉攻击和文本攻击结合时，成功率达到了 72.92%，证实了协同效应：每个方面针对不同的漏洞，共同最大化攻击成功。要点。MFA 的各个组成部分单独强大且相互增强，揭示了整个 VLM 安全堆栈中的互补漏洞。