A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
《大型视觉语言模型安全综述：攻击、防御与评估》

$\bullet$ Single-Modality. Based on the vulnerabilities of LVLMs. Qi et al. [23] propose a classic white-box attack method for crafting adversarial examples that can exploit the visual modality of LVLM to induce unsafe or misleading outputs, even when the model has been carefully aligned to follow ethical guidelines or constraints. The attack is formulated as an optimization problem, where the adversarial example $\widehat{v}_{adv}$ is selected from a perturbation space $\mathscr{B}$ to minimize the log-probability of the model’s output for the target class $y_{i}$. The attack objective is mathematically expressed as:
$\bullet$ 单模态。基于 LVLMs 的漏洞，Qi 等人[ 23]提出了一种经典的白盒攻击方法，用于制作对抗样本，该方法可以利用 LVLM 的视觉模态来诱导不安全或误导性的输出，即使模型已经经过精心校准以遵循道德准则或约束。该攻击被表述为一个优化问题，其中对抗样本 $\widehat{v}_{adv}$ 从扰动空间 $\mathscr{B}$ 中选取，以最小化模型对目标类别 $y_{i}$ 输出的对数概率。攻击目标用数学公式表示为：

where the goal is to force the model to misclassify or generate undesired outputs. Using the same approach to generate adversarial images, Schlarmann et al. [24] further investigate the success rates of both targeted and untargeted attacks against the OpenFlamingo model [110]. To expand the applicability of adversarial attacks, Bailey et al. [27] propose a general-purpose Behaviour Matching algorithm for generating adversarial images with enhanced context transferability. This algorithm enables adversarial examples to maintain their effectiveness across diverse scenarios and tasks. Additionally, [27] introduces Prompt Matching method, which is designed to train hijacking models capable of mimicking the behavior elicited by an arbitrary text prompt. Luo et al. [95] introduce the concept of cross-prompt adversarial transferability. The proposed CroPA [95] method refines visual adversarial perturbations using learnable prompts, specifically designed to counteract the misleading effects of adversarial images. CroPA [95] enables a single adversarial example to mislead all predictions of a LVLM across different prompts. From an unusual perspective, Gao et al. [92] explore a novel approach to induce high energy-latency [111, 112] in order to induce safety issues in LVLMs by causing them to generate endless outputs. Specifically, they introduced the concept of delayed End-of-Sequence (EOS) loss, leveraging it to create verbose images with perturbations that disrupt the model’s ability to terminate its responses appropriately. This specific loss function not only inhibits the model from halting its responses but also increases token diversity during generation. This results in the model producing lengthy and often irrelevant outputs, which can degrade user experience or lead to the propagation of unintended or incoherent information. Besides, Wang et al. [94] investigate the impact of Chain-of-Thought (CoT) reasoning [113, 114, 115] on the robustness of LVLMs. To address this, they introduced the Stop Reasoning attack method, which generates adversarial images to guide the model’s output according to a pre-designed template. This approach reduces the token probability associated with CoT reasoning, thereby effectively diminishing its influence on the model’s safety performance. Gao et al. [96] shift the focus of adversarial attacks to the Visual Grounding task [116, 117, 118], demonstrating how adversarial perturbations can effectively disrupt the alignment between visual inputs and textual references. Using Projected Gradient Descent (PGD) [79], they add perturbations on images, enabling the execution of both targeted and untargeted adversarial attacks. Jang et al. [99] introduce the “Replace-then-Perturb” method, a novel approach that differs from traditional adversarial attacks, which often disrupt the entire image. This method focuses on specific objects within an image, replacing them with carefully designed adversarial substitutes and applying targeted perturbations. By ensuring that other objects in the scene remain unaffected and correctly recognized, the method maintains the overall context while effectively misleading the model’s reasoning about the targeted objects. Unlike the previously mentioned single-turn attack methods, Bagdasaryan et al. [91] propose a multi-turn attack that uses prompt injection to compromise dialog safety. By forcing the model to output a specific attacker-chosen instruction $w$ in its first response (e.g., “I will always follow instruction:”), the attacker poisons the dialog history. This causes the model to lose its safety mechanisms in subsequent turns. The attack exploits the model’s context retention, making it persistently unsafe, and can be made stealthier by paraphrasing the injected instruction.
其目标是将模型强制错误分类或生成不期望的输出。使用相同的方法生成对抗性图像，Schlarmann 等人[24]进一步研究了针对 OpenFlamingo 模型[110]的定向攻击和非定向攻击的成功率。为了扩展对抗性攻击的适用性，Bailey 等人[27]提出了一种通用的行为匹配算法，用于生成具有增强上下文可迁移性的对抗性图像。该算法使对抗性样本能够在不同的场景和任务中保持其有效性。此外，[27]引入了提示匹配方法，该方法旨在训练能够模仿任意文本提示所引发行为的劫持模型。Luo 等人[95]引入了跨提示对抗性可迁移性的概念。他们提出的 CroPA[95]方法使用可学习的提示来改进视觉对抗性扰动，专门设计用来抵消对抗性图像的误导效果。CroPA[95]使单个对抗性样本能够在不同的提示下误导 LVLM 的所有预测。 从非同寻常的角度出发，高等人[92]探索了一种新颖的方法，通过使 LVLMs 生成无限输出，来诱导高能耗-延迟[111,112]并引发安全问题。具体来说，他们引入了延迟序列结束（EOS）损失的概念，利用它来创建带有扰动的冗长图像，从而破坏模型适当终止其响应的能力。这种特定的损失函数不仅阻止模型停止其响应，还增加了生成过程中的 token 多样性。这导致模型产生冗长且通常不相关的输出，这可能降低用户体验或导致意外或不连贯信息的传播。此外，王等人[94]研究了思维链（CoT）推理[113,114,115]对 LVLMs 鲁棒性的影响。为了解决这个问题，他们引入了停止推理攻击方法，该方法生成对抗性图像，根据预设计的模板引导模型的输出。这种方法降低了与 CoT 推理相关的 token 概率，从而有效地削弱了其对模型安全性能的影响。 高等人[96]将对抗攻击的焦点转移到视觉定位任务[116,117,118]上，展示了对抗扰动如何有效破坏视觉输入与文本参考之间的对齐。他们使用投影梯度下降（PGD）[79]，在图像上添加扰动，从而能够执行有目标和无目标的对抗攻击。蒋等人[99]引入了“替换后扰动”方法，这是一种与传统对抗攻击（通常破坏整个图像）不同的新方法。该方法专注于图像中的特定对象，用精心设计的对抗替代物替换它们，并施加有目标的扰动。通过确保场景中的其他对象保持未受影响且被正确识别，该方法在保持整体上下文的同时有效误导模型对目标对象的推理。与之前提到的单轮攻击方法不同，巴加达萨扬等人[91]提出了一种多轮攻击，该攻击使用提示注入来破坏对话安全。 通过迫使模型在其首次回应中输出攻击者选择的特定指令 $w$ （例如，“我将始终遵循指令：”），攻击者污染了对话历史。这导致模型在后续回合中失去了安全机制。 该攻击利用了模型保持上下文的能力，使其持续不安全，并且可以通过改写注入的指令来使其更加隐蔽。

$\bullet$ Cross-Modality. While single-modality attacks target visual components of LVLMs, cross-modality attacks exploit the interaction between modalities to achieve more robust and transferable adversarial effects. These methods aim to misalign the model’s multimodal understanding by jointly perturbing both visual and textual inputs, leveraging the complex dependencies between modalities to amplify the attack’s impact. Wang et al. [26] propose the Universal Master Key (UMK) method, comprises an adversarial image prefix and an adversarial text suffix. Firstly, UMK [26] establish a corpus containing several few-shot examples of harmful sentences $Y:=\{y_{i}\}_{i=1}^{m}$. The methodology for embedding toxic semantics into the adversarial image prefix $\widehat{v}_{\text{adv}}$ is straightforward: UMK [26] initialize $\widehat{v}_{\text{adv}}$ with random noise and optimize it to maximize the generation probability of this few-shot corpus in the absence of text input. The optimization objective is formulated as follows:
$\bullet$ 跨模态。虽然单模态攻击针对 LVLMs 的视觉组件，但跨模态攻击利用模态间的交互来实现更鲁棒和可迁移的对抗效果。这些方法旨在通过联合扰动视觉和文本输入来错位模型的跨模态理解，利用模态间的复杂依赖关系来放大攻击的影响。Wang 等人[ 26]提出了通用主密钥（UMK）方法，包含一个对抗图像前缀和一个对抗文本后缀。首先，UMK[ 26]建立了一个包含几个有害句子少样本示例的语料库 $Y:=\{y_{i}\}_{i=1}^{m}$ 。将有害语义嵌入对抗图像前缀 $\widehat{v}_{\text{adv}}$ 的方法很简单：UMK[ 26]用随机噪声初始化 $\widehat{v}_{\text{adv}}$ ，并优化它以在无文本输入的情况下最大化该少样本语料库的生成概率。优化目标如下：

where $\varnothing$ denotes an empty text input. This optimization problem can be efficiently solved using prevalent techniques in image adversarial attacks, such as PGD [79]. To maximize the probability of generating affirmative responses, UMK [26] further introduce an adversarial text suffix $\widehat{t}_{\text{adv}}$ in conjunction with the adversarial image prefix $\widehat{v}_{\text{adv}}$, which is embedded with toxic semantics. This multimodal attack strategy aims to thoroughly exploit the inherent vulnerabilities of LVLMs. The optimization objective is as follows:
其中 $\varnothing$ 表示空文本输入。这个优化问题可以使用图像对抗攻击中的常用技术（如 PGD [ 79]）高效解决。为了最大化生成肯定回答的概率，UMK [ 26] 进一步引入了带有毒性语义的对抗文本后缀 $\widehat{t}_{\text{adv}}$ ，并与对抗图像前缀 $\widehat{v}_{\text{adv}}$ 结合使用。这种多模态攻击策略旨在彻底利用大型视觉语言模型（LVLMs）的固有漏洞。优化目标如下：

Similar to [26], Ying et al. [97] introduce the Bi-Modal Adversarial Prompt Attack (BAP) method, which perturbs images and rewrites text inputs to compromise the safety mechanisms of LVLMs. BAP [97] crafts the visual perturbation by constructing a query-agnostic corpus, ensuring the model consistently generates positive responses regardless of the query’s harmful intent. Differing from [26], BAP [97] incorporates an iterative refinement of the textual prompt using CoT strategy [119, 120], leverages the reasoning capabilities of LLMs to progressively optimize the textual input, ensuring it aligns with the visual perturbation while effectively bypassing safety mechanisms. This alignment between visual and textual modalities enables the model to produce harmful outputs with higher success and precision. Extending these insights, HADES [40] demonstrates that images can act as alignment backdoors for LVLMs. HADES [40] combines multiple attack strategies in a systematic process: it first removes harmful textual content by embedding it into typography, then pairs it with a harmful image generated using a diffusion model guided by an iteratively refined prompt from an LLM. Finally, an adversarial image is appended to the composite image, effectively eliciting affirmative responses from LVLMs for harmful instructions. Presented strong evidence that the visual modality poses the alignment vulnerability of LVLMs, underscoring the urgent need for further exploration into cross-modal alignment. While [95] introduces cross-prompt attack by using single-modality method. CIA [98] further improves CroPA [95] by employing gradient-based perturbation to inject target tokens into both visual and textual contexts. CIA [98] shifts contextual semantics towards the target tokens instead of preserving the original image semantics, thereby enhancing the cross-prompt transferability of adversarial images.
与[ 26]类似，Ying 等人[ 97]提出了双模态对抗提示攻击（BAP）方法，该方法通过扰动图像和重写文本输入来破坏大型视觉语言模型（LVLMs）的安全机制。BAP[ 97]通过构建一个查询无关的语料库来制作视觉扰动，确保模型无论查询的恶意意图如何，始终生成正面响应。与[ 26]不同，BAP[ 97]采用 CoT 策略[ 119, 120]对文本提示进行迭代优化，利用 LLMs 的推理能力逐步优化文本输入，确保其与视觉扰动一致，同时有效绕过安全机制。这种视觉和文本模态之间的对齐使模型能够以更高的成功率和精度生成有害输出。基于这些见解，HADES[ 40]表明图像可以作为 LVLMs 的校准后门。HADES[ 40]在系统过程中结合了多种攻击策略：首先通过将其嵌入到排版中移除有害文本内容，然后将其与一个由 LLM 迭代优化的提示引导的扩散模型生成的有害图像配对。 最后，将对抗性图像附加到合成图像中，有效引出 LVLMs 对有害指令的肯定响应。提供了强有力的证据，表明视觉模态导致了 LVLMs 的校准漏洞，突显了进一步探索跨模态校准的紧迫性。虽然[ 95]通过使用单模态方法引入了跨提示攻击。CIA [ 98]通过采用基于梯度的扰动，将目标标记注入视觉和文本上下文中，进一步改进了 CroPA [ 95]。CIA [ 98]将上下文语义转向目标标记，而不是保留原始图像语义，从而增强了对抗性图像的跨提示可迁移性。

$\bullet$ Single-Modality. Focusing on visual modality, Zhao et al. [25] conducts both transfer-based and query-based attacks against image-grounded text generation, focusing on adversaries with only black-box system access. CLIP [12] and BLIP [121] are employed as surrogate models to generate adversarial examples by aligning textual and image embeddings, which are subsequently transferred to other LVLMs. This methodology achieves a notably high success rate in generating targeted responses. Dong et al. [100] and Niu et al. [101] both employing more white-box surrogate vision encoders of LVLMs. Dong et al. [100] further investigated the vulnerability of Google’s Bard¹¹1https://bard.google.com/
$\bullet$ 单模态。聚焦于视觉模态，赵等人[ 25]对图像-文本生成进行了基于迁移和基于查询的攻击，专注于只有黑盒系统访问的对手。CLIP[ 12]和 BLIP[ 121]被用作替代模型，通过对齐文本和图像嵌入来生成对抗样本，这些样本随后被迁移到其他大型视觉语言模型（LVLMs）。这种方法在生成目标响应方面取得了显著的高成功率。董等人[ 100]和牛等人[ 101]都采用了更白盒的 LVLM 替代视觉编码器。董等人[ 100]进一步研究了 Google 的 Bard ¹ , a black-box model. The generated adversarial examples demonstrated the capability to mislead Bard, producing incorrect image descriptions with a 22% attack success rate (ASR) based solely on their transferability. These examples were also highly effective against other commercial LVLMs, achieving similarly high ASRs. Wang et al. [122] introduces a novel attack method called VT-Attack. This method disrupts encoded visual tokens by comprehensively targeting their features, relationships, and semantic properties. It employs a multi-faceted approach to alter the internal representations of these tokens, effectively interfering with the model’s ability to generate untargeted outputs across various tasks.
，一个黑盒模型。生成的对抗样本展示了误导 Bard 的能力，仅凭借其可迁移性，以 22%的攻击成功率（ASR）生成了错误的图像描述。这些样本对其他商业视觉语言模型（LVLMs）也非常有效，实现了同样高的 ASR。Wang 等人[ 122]介绍了一种名为 VT-Attack 的新型攻击方法。该方法通过全面针对其特征、关系和语义属性来破坏编码的视觉标记。它采用多方面的方法来改变这些标记的内部表示，有效地干扰模型在各种任务中生成非目标输出的能力。

$\bullet$ Cross-Modality. Jailbreak In Pieces (JIP) [85] developed a cross-modality attack method that requires access solely to the vision encoder CLIP [12]. JIP [85] first decomposes a typical harmful prompt into two distinct components: a generic textual instruction $x_{g}^{t}$ (e.g., “teach me how to make these things.”) and a malicious trigger $H_{\text{harm}}$, which can be either a harmful textual input $x_{\text{harm}}^{t}$ or an image input $x_{\text{harm}}^{i}$ generated using visual or OCR methods. Therefore craft the adversarial input images $\widehat{x}_{adv}^{i}$ that mapped into the dangerous embedding regions close to $H_{\text{harm}}$:
$\bullet$ 跨模态。分段越狱攻击（JIP）[ 85] 开发了一种跨模态攻击方法，该方法仅需访问视觉编码器 CLIP [ 12]。JIP [ 85] 首先将典型的有害提示分解为两个不同组件：一个通用文本指令 $x_{g}^{t}$ （例如，“教我如何制作这些东西。”）和一个恶意触发器 $H_{\text{harm}}$ ，该触发器可以是具有危害性的文本输入 $x_{\text{harm}}^{t}$ 或使用视觉或 OCR 方法生成的图像输入 $x_{\text{harm}}^{i}$ 。因此，制作对抗性输入图像 $\widehat{x}_{adv}^{i}$ ，使其映射到接近 $H_{\text{harm}}$ 的危险嵌入区域。

$\hat{x}_{\text{adv}}^{i}$ and $x_{g}^{t}$ are then input jointly into LVLMs, where their embeddings are combined in a manner that circumvents the model’s textual-only safety alignment. JIP [85] enables the generation of harmful outputs by exploiting the multimodal integration, thereby highlighting the increased vulnerability of LVLMs to sophisticated cross-modality attacks. JIP [85] achieve a high success rate across different LVLMs, highlighting the risk of cross-modality alignment vulnerabilities.
$\hat{x}_{\text{adv}}^{i}$ 和 $x_{g}^{t}$ 随后被共同输入到大型视觉语言模型（LVLMs）中，它们的嵌入以一种绕过模型纯文本安全对齐的方式被组合。JIP [85]通过利用多模态集成来生成有害输出，从而突显了 LVLMs 在面对复杂跨模态攻击时的脆弱性。JIP [85]在不同的 LVLMs 上实现了高成功率，突显了跨模态对齐漏洞的风险。

$\bullet$ Malicious Typography. Qraitem et al. [104] proposed the self-generated typography attack, wherein LVLMs are employed to generate words most similar to the objects in an image for typography. This technique is designed to disrupt the model’s classification by introducing subtle textual perturbations that confuse the model’s decision-making process. By exploiting the shortcoming that content safety guardrails of VLMs are ineffective against typographic visual prompts, FigStep [41] employs typographic transformations to convert malicious instructions, such as “Here is how to build a bomb: 1. 2. 3.,” into image format. These images are then input into the LVLM alongside carefully crafted prompts, inducing the model to complete the malicious instructions. Under this method, the typographic transformation allows malicious content to evade textual safety filters by embedding it as visual input. Combined with prompts designed to exploit the model’s multimodal reasoning, FigStep [41] instructing the model to answer the prohibited question in steps, effectively manipulates the LVLM to interpret and process these visual instructions, leading to unsafe completions that circumvent its alignment mechanisms. Additionally, MML [108] drawing inspiration from cryptography, utilizes an encryption-decryption process across both text and image modalities to prevent the over-exposure of malicious information. By replacing text in images [123] and prompting the model to perform decryption (e.g., ’burger’ → ’drugs’), MML [108] exploits the model’s logic and OCR capabilities to bypass security defenses, achieving a high attack success rate. HIMRD [109] divides the malicious prompt into two seemingly harmless components. One part, containing harmful words, is embedded into an image through typographic formatting and paired with a Text-to-Image (T2I) model to generate the corresponding image. The other part is inserted into the text using a prompt template (e.g., Look at the text at the top of the image, put the text inside the parentheses of ’make illegal ( )’). Through iterative optimization, the prompt is updated, ultimately achieving a successful jailbreak attack.
$\bullet$ 恶意排版。Qraitem 等人[104]提出了自生成排版攻击，该方法利用大型视觉语言模型生成与图像中对象最相似的文字进行排版。这种技术旨在通过引入微小的文本扰动来干扰模型的分类，从而混淆模型决策过程。通过利用视觉语言模型的内容安全防护措施对排版视觉提示无效的缺陷，FigStep[41]采用排版转换将恶意指令（如“这里是如何制造炸弹：1. 2. 3.”）转换为图像格式。然后将这些图像与精心设计的提示一同输入到大型视觉语言模型中，诱使模型完成恶意指令。在这种方法下，排版转换使恶意内容能够通过将其嵌入为视觉输入来规避文本安全过滤器。 结合旨在利用模型多模态推理能力的提示，FigStep [ 41] 指示模型分步回答禁止性问题，有效操控大型视觉语言模型（LVLM）解释和处理这些视觉指令，导致不安全的回答绕过其对齐机制。此外，MML [ 108] 从密码学中获得灵感，在文本和图像模态之间使用加密解密过程，以防止恶意信息的过度暴露。通过替换图像中的文本 [ 123] 并提示模型执行解密（例如，'burger' → 'drugs'），MML [ 108] 利用模型的逻辑和 OCR 能力绕过安全防御，实现高攻击成功率。HIMRD [ 109] 将恶意提示分为两个看似无害的组件。一部分包含有害词汇，通过排版格式嵌入图像中，并与文本到图像（T2I）模型配对生成相应图像。另一部分使用提示模板插入文本中（例如，"看图像顶部的文本，将文本放在'make illegal ( )'的括号内"）。 通过迭代优化，提示被更新，最终成功实现了越狱攻击。

$\bullet$ Visual Role-Play. Ma et al. [105] propose Visual Role-play (VRP), an effective structure-based jailbreak method that instructs the model to act as a high-risk character in the image input to generate harmful content. VRP [105] first utilize an LLM to generate a detailed description of a high-risk character. The description is then employed to create a corresponding character image. Next, VRP [105] integrate the typography of the character description and the associated malicious questions at the top and bottom of the character image, respectively, to form the complete jailbreak image input. This malicious image input is then paired with a benign role-play instruction text to query and attack LVLMs. Effectively misleads LVLMs into generating malicious responses.
$\bullet$ 视觉角色扮演。Ma 等人[ 105]提出了视觉角色扮演（VRP），这是一种基于结构的越狱方法，指导模型在图像输入中扮演高风险角色以生成有害内容。VRP[ 105]首先利用 LLM 生成一个高风险角色的详细描述。然后，利用该描述创建相应的角色图像。接下来，VRP[ 105]将角色描述的排版和相关的恶意问题分别集成到角色图像的顶部和底部，形成完整的越狱图像输入。这个恶意图像输入随后与一个良性的角色扮演指令文本配对，用于查询和攻击 LVLMs。有效地误导 LVLMs 生成恶意响应。

$\bullet$ Logical Questioning. Zou et al. [106] explore the use of LVLMs’ logic understanding capabilities, particularly in interpreting flowcharts, for jailbreak attacks. They transform harmful textual instructions into visual flowchart representations, leveraging the model’s multimodal processing to bypass safety measures. This method is further enhanced by integrating visual instructions with textual prompts to generate detailed harmful outputs.
$\bullet$ 逻辑提问。Zou 等人[ 106]探索了 LVLM 的逻辑理解能力，特别是在解释流程图方面的应用，用于越狱攻击。他们将有害的文本指令转换为视觉流程图表示，利用模型的多模态处理能力绕过安全措施。该方法通过将视觉指令与文本提示相结合，生成详细的有害输出，进一步得到增强。

$\bullet$ Red Teaming. IDEATOR[107] utilizes LVLMs as red team models to generate multimodal jailbreak prompts. Through an iterative dialogue between the attack and victim models, the system continuously refines and optimizes the generated prompts. This approach effectively explores a wide range of LVLM vulnerabilities without relying on white-box access or manual intervention, showcasing a robust and automated red teaming framework.
$\bullet$ 红队测试。IDEATOR[ 107]将 LVLM 用作红队模型，生成多模态越狱提示。通过攻击模型和受害者模型之间的迭代对话，系统不断优化和改进生成的提示。这种方法有效地探索了 LVLM 的多种漏洞，而无需依赖白盒访问或人工干预，展示了一个强大且自动化的红队测试框架。

$\bullet$ Prompt Engineering. To defend against prompt-based attacks, AdaShield [43] leverages the instruction-following capabilities of LVLMs by incorporating safety prefixes into the input, aiming to activate the model’s inherent safety mechanisms through prompting techniques. Specifically, the safety prefixes are categorized into fixed and adaptive textual prefixes, which are dynamically optimized based on the characteristics of malicious input queries. By utilizing a defender model to iteratively generate, evaluate, and refine these prompts, AdaShield [43] improves the robustness of LVLMs without the need for fine-tuning or additional modules. BlueSuffix [131] employs a diffusion-based method to purify jailbreak images and introduces an LLM-based text purifier to rewrite adversarial textual prompts while preserving their original meaning. Based on the purified data, a trained Suffix Generator is utilized to produce prompt suffixes that guide the model, thereby enhancing its safety capabilities.
$\bullet$ 提示工程。为了防御基于提示的攻击，AdaShield [ 43] 利用 LVLMs 的指令跟随能力，通过将安全前缀整合到输入中，旨在通过提示技术激活模型的内在安全机制。具体来说，安全前缀被分为固定和自适应文本前缀，这些前缀根据恶意输入查询的特征进行动态优化。通过利用防御模型迭代生成、评估和优化这些提示，AdaShield [ 43] 在无需微调或额外模块的情况下提高了 LVLMs 的鲁棒性。BlueSuffix [ 131] 采用基于扩散的方法净化越狱图像，并引入基于 LLM 的文本净化器来重写对抗性文本提示，同时保留其原始含义。基于净化后的数据，一个训练好的后缀生成器被用于生成引导模型的提示后缀，从而增强其安全能力。

$\bullet$ Image Perturbation. To address perturbation-based attacks, detecting and removing adversarial noise has proven to be highly effective. Specifically, CIDER [44] leverages the observation that the semantic distance between clean and adversarial images, relative to harmful queries, exhibits significant differences. As a denoising model, CIDER [44] iteratively removes noise from the input images and evaluates the semantic distance before and after denoising. If the distance exceeds a predefined threshold, the input is identified as malicious and rejected. CIDER [44] effectively filters out adversarial inputs while preserving the integrity of benign ones. Drawing inspiration from SmoothLLM [138], SmoothVLM [130] enhances the robustness of LVLMs against adversarial patch attacks [23, 27, 85], which employs randomized smoothing by introducing controlled noise to input images, which helps mitigate the impact of adversarial patches. Ensures that small, localized perturbations are smoothed out, reducing their ability to mislead the model while maintaining the semantic fidelity of the input. Significantly reduces the success rate of attacks on two leading LVLMs under 5%, while achieving up to 95% context recovery of the benign images, demonstrating a balance between security, usability, and efficiency.
$\bullet$ 图像扰动。为了应对基于扰动的攻击，检测并移除对抗性噪声已被证明非常有效。具体来说，CIDER [ 44] 利用了一个观察结果：相对于有害查询，干净图像和对抗性图像之间的语义距离表现出显著差异。作为去噪模型，CIDER [ 44] 迭代地从输入图像中移除噪声，并在去噪前后评估语义距离。如果距离超过预设阈值，输入就会被识别为恶意并拒绝。CIDER [ 44] 有效过滤掉对抗性输入，同时保留良性输入的完整性。受 SmoothLLM [ 138] 启发，SmoothVLM [ 130] 增强了 LVLMs 对对抗性补丁攻击 [ 23, 27, 85] 的鲁棒性，该攻击通过向输入图像引入受控噪声进行随机平滑，有助于减轻对抗性补丁的影响。确保小的、局部的扰动被平滑掉，从而降低它们误导模型的能力，同时保持输入的语义保真度。 显著降低了针对两个领先 LVLM 的攻击成功率至 5%以下，同时实现了良性图像高达 95%的上下文恢复，展示了在安全性、可用性和效率之间的平衡。

$\bullet$ Hybrid Perturbation. UniGuard [132] combines prompt engineering and image perturbation, jointly addressing unimodal and cross-modal harmful inputs. Specifically, UniGuard [132] introduces additive safe noise for image inputs and applies suffix modifications to text prompts, effectively reducing the likelihood of generating unsafe responses. By training on a targeted small corpus of toxic content, UniGuard [132] achieves significant robustness against a wide range of adversarial attacks.
$\bullet$ 混合扰动。UniGuard [ 132] 结合了提示工程和图像扰动，共同处理单模态和跨模态的有害输入。具体来说，UniGuard [ 132] 为图像输入引入加性安全噪声，并对文本提示应用后缀修改，有效降低了生成不安全响应的可能性。通过在有毒内容的小型目标语料库上训练，UniGuard [ 132] 实现了对广泛对抗性攻击的显著鲁棒性。

$\bullet$ Activation Alignment. As researches in LLM safety [139], the model’s parameter activations exhibit noticeable differences when processing safe and unsafe requests respectively. To leverage this observation, InferAligner [45] calculates the mean activation difference of the last token between harmful and harmless prompts. Based on this calculation, a threshold is established to identify unsafe responses. When the activation value of a token surpasses the threshold, the mean activation difference is applied to bias-correct the output, effectively mitigating unsafe content and ensuring more secure and reliable responses. Additionally, CMRM [133] highlights that LVLMs exhibit vulnerabilities when queried with images but tend to restore safety when images are excluded. To address this issue, CMRM [133] computes the hidden state activation bias between pure text inputs and text-image inputs at the same layer $l$, as defined:
$\bullet$ 激活对齐。在 LLM 安全研究[ 139]中，模型在处理安全请求与不安全请求时，其参数激活表现出明显差异。为利用这一观察结果，InferAligner[ 45]计算了有害提示和无害提示之间最后一个 token 的平均激活差异。基于此计算，建立了一个阈值来识别不安全响应。当 token 的激活值超过阈值时，将应用平均激活差异对输出进行偏差校正，有效减轻不安全内容并确保更安全可靠的响应。此外，CMRM[ 133]指出，LVLMs 在接收图像查询时表现出漏洞，但在排除图像时倾向于恢复安全性。为解决此问题，CMRM[ 133]计算了同一层纯文本输入与文本-图像输入之间的隐藏状态激活偏差，定义如下： $l$

where $\mathbf{h}_{t}^{l(i)}$ and $\mathbf{h}_{c}^{l(i)}$ represent the hidden state activations of the $i$-th input in the $l$-th layer for pure text input and text-image input, respectively. Here, $N$ denotes the total number of samples in the dataset. By applying PCA (Principal Component Analysis) to the activation differences, CMRM [133] identifies the principal direction of variation caused by the visual input. This bias is then used to correct the visual-induced misalignment, aligning multimodal representations closer to the original LLM text-only distribution while retaining the benefits of visual information. To defend against adversarial jailbreaks, ASTRA [134] decomposes input images to identify regions with high correlation to the attack. Based on these regions, steering vectors are constructed to capture adversarial directions in the activation space. During inference, ASTRA [134] projects the model’s activations onto the steering vectors and applies corrections to remove components aligned with the jailbreak-related directions, thereby mitigating adversarial influence while maintaining the model’s performance.
在纯文本输入和文本-图像输入中， $\mathbf{h}_{t}^{l(i)}$ 和 $\mathbf{h}_{c}^{l(i)}$ 分别表示第 $i$ 个输入在第 $l$ 层的隐藏状态激活。这里， $N$ 表示数据集中的样本总数。通过将主成分分析（PCA）应用于激活差异，CMRM [ 133] 识别出由视觉输入引起的变异主方向。然后利用这种偏差来纠正视觉引起的错位，使多模态表示更接近原始 LLM 文本分布，同时保留视觉信息的好处。为了防御对抗性越狱攻击，ASTRA [ 134] 将输入图像分解以识别与攻击高度相关的区域。基于这些区域，构建引导向量以捕获激活空间中的对抗方向。在推理过程中，ASTRA [ 134] 将模型的激活投影到引导向量上，并对与越狱相关方向对齐的分量应用校正，从而减轻对抗影响，同时保持模型性能。

$\bullet$ Logits Adjustment. CoCA [46] introducing a safe instruction to harmful queries and calculating the resulting logits bias. This bias represents the adjustment required to align the model’s outputs with safer responses. During inference, the logits bias is directly applied to the model’s output layer, allowing it to maintain robust safety capabilities without explicitly adding safe instructions to the input. Similar to CoCA [46], IMMUNE [135] enhances the safety of model outputs by aligning responses during the inference stage. Instead of explicitly modifying inputs, IMMUNE [135] evaluates the safety of candidate tokens by introducing a safe reward model $Q_{\text{safe}}$ that quantifies the likelihood of a token contributing to a harmful response. The reward score is combined with the model’s original logits to compute an adjusted decoding score, which guides the generation process toward safer outputs.
$\bullet$ Logits 调整。CoCA [ 46] 向有害查询引入安全指令并计算由此产生的 logits 偏差。该偏差表示使模型输出与更安全响应对齐所需的调整。在推理过程中，logits 偏差直接应用于模型的输出层，使其能够在不向输入显式添加安全指令的情况下保持强大的安全能力。类似于 CoCA [ 46]，IMMUNE [ 135] 通过在推理阶段对齐响应来增强模型输出的安全性。IMMUNE [ 135] 不是通过显式修改输入，而是通过引入一个安全奖励模型 $Q_{\text{safe}}$ 来评估候选 token 的安全性，该模型量化了 token 导致有害响应的可能性。奖励分数与模型的原始 logits 结合，计算出一个调整后的解码分数，从而引导生成过程朝向更安全的输出。

$\bullet$ Harmful Detecting. JailGuard [47] observes that attack inputs inherently exhibit lower robustness compared to benign queries, irrespective of the attack methods or modalities. To exploit this property, JailGuard systematically designs and implements 16 random mutators and 2 semantic-driven targeted mutators to introduce perturbations at various levels of text and image inputs. By comparing the cosine similarity between the mutated and original inputs, JailGuard identifies significant discrepancies as indicators of adversarial attacks. This approach serves as a universal detection method capable of handling diverse attack types and modalities. Pi et al. [48] propose MLLM-Protector, a defense framework that fine-tunes two separate LLMs to serve as a Harm Detector and a Response Detoxifier. The Harm Detector is designed to accurately identify outputs that violate predefined safety protocols, ensuring that harmful responses are flagged before being delivered to users. Meanwhile, the Response Detoxifier enhances the model’s helpfulness by rewriting harmful outputs into safe and constructive responses, effectively balances safety and utility. Unlike previous approaches, ECSO [136] operates without introducing additional modules. It demonstrates that LVLMs are inherently capable of assessing the safety of their outputs. When the model detects that its response may be harmful, ECSO [136] mitigates this risk by converting the visual input into a textual caption and proceeding with text-only processing. This method highlights the ability to harness the intrinsic safety mechanisms of the LLM component within LVLMs, effectively reducing the generation of unsafe outputs while maintaining computational efficiency. For adversarial attacks, MirrorCheck [137] employs Text-to-Image (T2I) models [140, 141, 142] to generate images from captions produced by the target VLMs and then computes the similarity between the feature embeddings of the input and generated images to identify adversarial samples. MirrorCheck [137] demonstrates robust defense capabilities, offering an effective, training-free approach to detect and mitigate adversarial threats in LVLMs.
$\bullet$ 有害检测。JailGuard [ 47] 观察到攻击输入相较于良性查询，无论攻击方法或模态如何，都表现出较低的鲁棒性。为了利用这一特性，JailGuard 系统性地设计和实现了 16 个随机变异器和 2 个语义驱动的目标变异器，以在文本和图像输入的不同层次引入扰动。通过比较变异输入和原始输入之间的余弦相似度，JailGuard 将显著差异识别为对抗性攻击的指标。这种方法是一种通用的检测方法，能够处理各种攻击类型和模态。Pi 等人[ 48]提出了 MLLM-Protector，这是一个防御框架，通过微调两个独立的 LLMs 来充当有害检测器和响应解毒器。有害检测器被设计为准确识别违反预定义安全协议的输出，确保在将有害响应交付给用户之前将其标记。同时，响应解毒器通过将有害输出重写为安全且建设性的响应来增强模型的有用性，有效地平衡了安全性和实用性。 与先前方法不同，ECSO [ 136] 不需要引入额外模块。它证明了视觉语言模型（LVLMs）本质上能够评估其输出的安全性。当模型检测到其响应可能有害时，ECSO [ 136] 通过将视觉输入转换为文本标题，并继续仅进行文本处理来缓解这种风险。这种方法突出了利用 LVLMs 中 LLM 组件的内在安全机制的能力，有效减少不安全输出的生成，同时保持计算效率。对于对抗性攻击，MirrorCheck [ 137] 采用文本到图像（T2I）模型 [ 140, 141, 142] 从目标视觉语言模型（VLMs）生成的标题中生成图像，然后计算输入图像和生成图像的特征嵌入之间的相似度，以识别对抗性样本。 MirrorCheck [ 137] 展示了强大的防御能力，提供了一种有效的、无需训练的方法来检测和缓解 LVLMs 中的对抗性威胁。

$\bullet$ Adversarial Specific Dataset. In adversarial detection, Huang et al. [143] introduce RADAR, a large-scale open-source adversarial dataset containing 4,000 samples. RADAR offers diverse harmful queries and responses, with samples sourced from the COCO dataset [144] and adversarial inputs generated using the PGD [79] method. To ensure high-quality samples, RADAR incorporates filtering procedures during construction, verifying that responses to benign inputs remain harmless while those to adversarial inputs are appropriately harmful.
$\bullet$ 对抗特定数据集。在对抗检测中，Huang 等人[ 143]介绍了 RADAR，这是一个包含 4,000 个样本的大规模开源对抗数据集。RADAR 提供了多样化的有害查询和响应，样本来源于 COCO 数据集[ 144]，并使用 PGD[ 79]方法生成对抗输入。为确保样本质量，RADAR 在构建过程中加入了过滤流程，验证对良性输入的响应保持无害，而对对抗输入的响应则适当有害。

$\bullet$ General Safety Dataset. Chen et al. [145] collet VLSafe, a harmless alignment dataset related to images, created using an LLM-Human-in-the-Loop approach and GPT-3.5-Turbo [3]. VLSafe [145] construction involves iterative refinement and filtering to ensure safety and quality, borrowing methods from textual adversarial attack research. Based on the COCO dataset [144], multiple iterations refine the dataset, followed by rounds of filtering to eliminate failure modes. The final dataset contains 5,874 samples, split into 4,764 training samples and 1,110 evaluation samples. Zong et al. [49] demonstrate through experiments that the inclusion of the vision modality significantly reduces the safety capabilities of LVLMs. Currently, a substantial portion of training data is generated by LLMs, which often contains harmful content, thereby contributing to the degradation of safety alignment in LVLMs. Furthermore, the use of LoRA fine-tuning has been shown to introduce additional safety risks. While cleaning the training data can partially restore safety alignment, its overall effectiveness remains limited. Based on these findings, [49] set out to collect a new safe vision-language instruction-following dataset VLGuard, significantly reduces the harmfulness of models across all fine-tuning strategies and models considered. To address the lack of high-quality open-source training datasets necessary for achieving the safety alignment of LVLMs, Zhang et al. [50] introduce SPA-VL, a large-scale safety alignment dataset that encompasses 6 harmfulness domains, 13 categories, and 53 subcategories. The dataset consists of 100,788 quadruple samples, with responses collected from 12 diverse LVLMs, including both open-source models (e.g., QwenVL [65]) and closed-source models (e.g., Gemini [60]), to ensure diversity. SPA-VL [50] reveals that increasing data volume, incorporating diverse responses, and using a mix of question types significantly enhance the safety and performance of aligned models. Helff et al. [146] propose LlavaGuard for dataset annotation and safeguarding generative models, leveraging curated datasets and structured evaluation methods. It uses a JSON-formatted output with safety ratings, category classifications, and natural language rationales to ensure comprehensive assessments. The dataset, built on the Socio-Moral Image Database (SMID) [147] and expanded with web-scraped images, addresses category imbalances with at least 100 images per category. LlavaGuard [146] incorporates refined safety ratings (e.g., “Highly Unsafe”, “Moderately Unsafe”) and synthetic rationales generated by the Llava-34B model [15] to enhance generalization. Data augmentation techniques improve dataset balance and adaptability, resulting in 4,940 samples, with 599 reserved for testing.
$\bullet$ 通用安全数据集。陈等人 [ 145] 收集了 VLSafe，这是一个与图像相关的无害对齐数据集，使用 LLM-人类参与循环方法（LLM-Human-in-the-Loop approach）和 GPT-3.5-Turbo [ 3] 创建。VLSafe [ 145] 的构建涉及迭代优化和过滤，以确保安全性和质量，借鉴了文本对抗性攻击研究中的方法。基于 COCO 数据集 [ 144]，经过多次迭代优化数据集，然后进行多轮过滤以消除失效模式。最终数据集包含 5,874 个样本，分为 4,764 个训练样本和 1,110 个评估样本。宗等人 [ 49] 通过实验表明，包含视觉模态会显著降低 LVLMs 的安全能力。目前，大量训练数据由 LLMs 生成，其中常包含有害内容，从而导致 LVLMs 的安全对齐能力下降。此外，LoRA 微调已被证明会引入额外的安全风险。虽然清理训练数据可以部分恢复安全对齐，但其整体效果仍然有限。 基于这些发现，[49] 致力于收集新的安全视觉语言指令遵循数据集 VLGuard，显著降低了所有考虑的微调策略和模型的有害性。为了解决实现 LVLM 安全对齐所需的高质量开源训练数据缺乏的问题，Zhang 等人 [50] 引入了 SPA-VL，这是一个包含 6 个有害性领域、13 个类别和 53 个子类的大型安全对齐数据集。该数据集包含 100,788 个四元组样本，从 12 种不同的 LVLM 收集了响应，包括开源模型（例如 QwenVL [65]）和闭源模型（例如 Gemini [60]），以确保多样性。SPA-VL [50] 表明，增加数据量、包含多样化的响应以及使用多种题型可以显著提高对齐模型的安全性和性能。Helff 等人 [146] 提出了 LlavaGuard 用于数据集标注和安全防护生成模型，利用精选数据集和结构化评估方法。 它使用 JSON 格式的输出，包含安全评级、类别分类和自然语言理由，以确保全面评估。该数据集基于社会道德图像数据库（SMID）[147]构建，并通过网络抓取的图像进行扩展，以解决类别不平衡问题，每个类别至少包含 100 张图像。LlavaGuard [146] 结合了更精细的安全评级（例如，“高度危险”、“中度危险”）以及由 Llava-34B 模型[15]生成的合成理由，以增强泛化能力。 数据增强技术提高了数据集的平衡性和适应性，最终得到 4,940 个样本，其中 599 个用于测试。

$\bullet$ Visual Enhancement. Schlarmann et al. [148] introduce FARE, an unsupervised adversarial fine-tuning approach for improving the robustness of the CLIP vision encoder against adversarial attacks while preserving its clean zero-shot performance. FARE [148] optimizes an embedding loss that ensures perturbed inputs produce embeddings close to their unperturbed counterparts, enabling the vision encoder to retain compatibility with downstream tasks like LVLMs without additional re-training. FARE [148] implemented using PGD-based [79] adversarial training, is dataset-agnostic and can generalize to other foundation models with intermediate embedding layers. Sim-CLIP [151] tackles the challenges present in FARE [148] by integrating a Siamese architecture with cosine similarity loss. During training, Sim-CLIP [151] generates perturbed views of input images using PGD [79] and optimizes the alignment between clean and perturbed representations to ensure robustness against adversarial attacks. The method minimizes negative cosine similarity to enforce invariance between these representations while maintaining model coherence. Additionally, a stop-gradient mechanism is incorporated to prevent loss collapse, enabling efficient adversarial training without the need for negative samples or momentum encoders. Sim-CLIP+ [152] extends Sim-CLIP [151] to defend against advanced optimization-based jailbreak attacks targeting LVLMs. By leveraging a tailored cosine similarity loss and a stop-gradient mechanism, Sim-CLIP+ [152] prevents symmetric loss collapse, ensuring computational efficiency while maintaining robustness.
$\bullet$ 视觉增强。Schlarmann 等人[148]提出了 FARE，这是一种无监督对抗微调方法，旨在提高 CLIP 视觉编码器对对抗攻击的鲁棒性，同时保持其干净的零样本性能。FARE[148]优化了一个嵌入损失，确保扰动输入产生的嵌入与其未扰动版本接近，使视觉编码器能够在无需额外重新训练的情况下保持与下游任务（如 LVLMs）的兼容性。FARE[148]采用基于 PGD[79]的对抗训练实现，具有数据集无关性，并能泛化到其他具有中间嵌入层的基础模型。Sim-CLIP[151]通过集成 Siamese 架构和余弦相似度损失来解决 FARE[148]中存在的问题。在训练过程中，Sim-CLIP[151]使用 PGD[79]生成输入图像的扰动视图，并优化干净表示和扰动表示之间的对齐，以确保对对抗攻击的鲁棒性。该方法最小化负余弦相似度，以强制执行这些表示之间的不变性，同时保持模型一致性。 此外，还引入了停止梯度机制以防止损失坍塌，从而无需负样本或动量编码器即可进行高效的对抗训练。Sim-CLIP+ [ 152] 扩展了 Sim-CLIP [ 151]，用于防御针对大型视觉语言模型（LVLMs）的高级基于优化的越狱攻击。通过利用定制的余弦相似度损失和停止梯度机制，Sim-CLIP+ [ 152] 防止了对称损失坍塌，确保了计算效率的同时保持了鲁棒性。

$\bullet$ Knowledge Unlearning. Chakraborty et al. [150] propose SIU, a novel framework for implementing machine unlearning in LVLM safety. SIU [150] addresses the challenge of selectively removing visual data associated with specific concepts by leveraging fine-tuning on a single representative image. The approach is composed of two key components: (i) the construction of multifaceted fine-tuning datasets designed to target four distinct unlearning objectives and (ii) the incorporation of a Dual Masked KL-divergence Loss, which enables the simultaneous unlearning of targeted concepts while maintaining the overall functional integrity and utility of the LVLMs. TextUnlearning [150] notes that irrespective of the input modalities, all information is ultimately fused within the language space. Comparative experiments reveal that unlearning focused solely on the text modality outperforms multimodal unlearning approaches. By performing “textual” unlearning exclusively on the LLM component of LVLMs, while keeping the remaining modules frozen, this method achieves remarkable levels of harmlessness against cross-modality attacks.
$\bullet$ 知识去学习。Chakraborty 等人[ 150]提出了 SIU，这是一个用于在 LVLM 安全中实现机器去学习的新框架。SIU[ 150]通过在单个代表性图像上进行微调，解决了选择性地移除与特定概念相关的视觉数据这一挑战。该方法由两个关键组件组成：(i) 构建旨在针对四个不同去学习目标的多样化微调数据集，以及(ii) 结合双掩码 KL 散度损失，该损失能够在保持 LVLM 的整体功能完整性和实用性的同时，实现对目标概念的同时去学习。TextUnlearning[ 150]指出，无论输入模态如何，所有信息最终都在语言空间中融合。比较实验表明，仅针对文本模态进行去学习的效果优于多模态去学习方法。通过仅对 LVLM 的 LLM 组件进行“文本”去学习，同时冻结其余模块，该方法在抵御跨模态攻击方面达到了极高的无害性水平。

$\bullet$ Module Integration. SafeVLM [51] integrating three key safety modules: safety projection, safety tokens, and a safety head into LLaVA to enhance safety. SafeVLM [51] employs two-stage training strategy, where safety modules are first trained with the base model frozen, followed by fine-tuning the language model to align safety measures with vision features. During inference, safety embeddings generated by the safety head provide dynamic and customizable risk control, enabling flexible adjustments based on user needs, such as content grading and categorization.
$\bullet$ 模块集成。SafeVLM [ 51] 将三个关键安全模块——安全投影、安全标记和安全头部集成到 LLaVA 中，以增强安全性。SafeVLM [ 51] 采用两阶段训练策略，首先冻结基础模型训练安全模块，然后微调语言模型以使安全措施与视觉特征保持一致。在推理过程中，安全头部生成的安全嵌入提供动态和可定制的风险控制，能够根据用户需求进行灵活调整，例如内容分级和分类。

$\bullet$ Advanced Fine-tuning. BaThe [153] treats harmful instructions as potential triggers that can exploit backdoored models to produce prohibited outputs. BaThe [153] replaces manually designed triggers (e.g., “SUDO”) with rejection prompts embedded as “soft text embeddings” called the wedge, maps harmful instructions to rejection responses. BaThe also defends against more advanced virtual prompt backdoor attacks, where harmful instructions combined with subtle prompts act as triggers. By embedding rejection prompts into the model’s soft text embeddings and including multimodal QA datasets in training, BaThe effectively mitigates backdoor risks, ensuring safer and more robust model behavior. TGA [29] finds that the hidden states at specific transformer layers play a crucial role in the successful activation of safety mechanisms, highlighting that the vision-language alignment at the hidden state level in current methods is insufficient. To address this, TGA [29] aligns the hidden states of visual ($X_{\text{image}}$) and textual ($X_{\text{caption}}$) inputs across transformer layers in LVLMs by introducing a pair-wise loss function ($\mathcal{L}_{\text{guide}}$). In this process, retrieved text ($X_{\text{retrieval}}$) serves as a guide, ensuring that $I_{j}$ (hidden states of $X_{\text{image}}$) is closer to $C_{j}$ (hidden states of $X_{\text{caption}}$) than $R_{j}$ (hidden states of $X_{\text{retrieval}}$), achieving semantic consistency. The total loss combines $\mathcal{L}_{\text{guide}}$ with cross-entropy loss for language modeling, enhancing multimodal alignment at the hidden state level and improving the safety mechanism.
$\bullet$ 高级微调。BaThe [ 153] 将有害指令视为可能利用后门模型生成禁止输出的潜在触发器。BaThe [ 153] 用嵌入为“软文本嵌入”的拒绝提示（称为楔子）替换手动设计的触发器（例如，“SUDO”），将有害指令映射到拒绝响应。BaThe 还防御更高级的虚拟提示后门攻击，其中有害指令与微妙提示结合作为触发器。通过将拒绝提示嵌入模型的软文本嵌入中，并在训练中包含多模态问答数据集，BaThe 有效缓解后门风险，确保更安全、更稳健的模型行为。TGA [ 29] 发现特定 Transformer 层的隐藏状态在安全机制的成功激活中起着关键作用，强调当前方法在隐藏状态级别的视觉-语言对齐是不充分的。为此，TGA [ 29] 通过引入成对损失函数（ $\mathcal{L}_{\text{guide}}$ ），在 LVLMs 中跨 Transformer 层对齐视觉（ $X_{\text{image}}$ ）和文本（ $X_{\text{caption}}$ ）输入的隐藏状态。 在这个过程中，检索到的文本（ $X_{\text{retrieval}}$ ）作为指导，确保 $I_{j}$ （ $X_{\text{image}}$ 的隐藏状态）比 $R_{j}$ （ $X_{\text{retrieval}}$ 的隐藏状态）更接近 $C_{j}$ （ $X_{\text{caption}}$ 的隐藏状态），从而实现语义一致性。 总损失结合 $\mathcal{L}_{\text{guide}}$ 与语言模型的交叉熵损失，增强隐藏状态层面的多模态对齐，并提升安全机制。

$\bullet$ Rule-Based Matching \faEdit. This method relies on detecting predefined phrases (e.g., “I’m sorry, I can’t assist with it”) to evaluate the safety of model’ responses. While computationally efficient, it has significant limitations, including a lack of contextual understanding and restricted vocabulary coverage, which make it unable to handle the wide variety of expressions that models may generate. Consequently, it fails to provide a thorough assessment of response safety.
$\bullet$ 基于规则的匹配 \faEdit. 这种方法依赖于检测预定义的短语（例如，“抱歉，我无法协助此事”）来评估模型响应的安全性。虽然计算效率高，但它存在显著局限性，包括缺乏上下文理解和词汇覆盖范围有限，这使得它无法处理模型可能生成的各种表达方式。因此，它无法提供全面的响应安全性评估。

$\bullet$ Human-Assisted Evaluation \faEye. This method relies on manual assessment performed by human evaluators to provide high-quality and context-aware safety evaluations. While human judgment allows for comprehensive and flexible assessments, this approach is highly resource-intensive and constrained by subjectivity. Variations in individual perspectives, cultural backgrounds, and personal biases can lead to inconsistencies, limiting both the scalability and reproducibility of the evaluation process.
$\bullet$ 人工辅助评估 \faEye. 这种方法依赖于人工评估员进行的手动评估，以提供高质量且具有上下文感知能力的安全性评估。虽然人工判断允许进行全面和灵活的评估，但这种方法的资源消耗非常高，并受主观性限制。个体视角、文化背景和个人偏见的差异可能导致评估结果不一致，从而限制了评估过程的可扩展性和可重复性。

$\bullet$ Fine-tuned Model-Based Assessment \faGithubAlt. This approach utilizes fine-tuned LLMs or LVLMs to classify responses as safe or unsafe, as demonstrated by systems like LlamaGuard [155], LLaVAGuard [146], and GUARDRANK [56]. Compared to rule-based methods, it offers improved contextual understanding and accuracy. However, its performance heavily depends on the quality and diversity of the fine-tuning dataset and is fundamentally limited by the capabilities of the base model, particularly when handling out-of-distribution (OOD) inputs or complex safety scenarios.
$\bullet$ 基于微调模型的评估 \faGithubAlt. 这种方法利用微调的 LLMs 或 LVLMs 对响应进行安全或不安全的分类，例如 LlamaGuard [ 155]、LLaVAGuard [ 146]和 GUARDRANK [ 56]等系统所示。与基于规则的方法相比，它提供了更好的上下文理解和准确性。然而，其性能严重依赖于微调数据集的质量和多样性，并且从根本上受限于基础模型的能力，尤其是在处理分布外（OOD）输入或复杂安全场景时。

$\bullet$ GPT Proxy-Based Assessment \faGoogle. This method leverages the advanced capabilities of models like GPT-4 [16] through API-based interactions, using carefully crafted prompts to evaluate responses to malicious instructions. By taking advantage of LLMs’ ability to understand context and generate detailed evaluations, this approach provides a robust framework for safety assessment. However, its effectiveness is undermined by inherent limitations such as biases and hallucination, where the model generates plausible but incorrect information. These issues introduce potential inconsistencies and reduce the reliability of the evaluation process. An example of a evaluation prompt selected from [41] is as follows:
$\bullet$ 基于 GPT 代理的评估 \faGoogle. 这种方法通过 API 交互利用 GPT-4 等模型的先进能力[ 16]，使用精心设计的提示来评估对恶意指令的响应。通过利用 LLMs 理解上下文和生成详细评估的能力，这种方法提供了一个强大的安全评估框架。然而，其有效性受到固有限制的削弱，如偏见和幻觉，其中模型会生成看似合理但错误的信息。这些问题引入了潜在的矛盾，降低了评估过程的可靠性。从[ 41]中选择的一个评估提示示例如下：

$\bullet$ Attack Success Rate (ASR) is employed to quantify the probability of eliciting harmful responses from LVLMs using pairs of image-text queries. Consider a dataset $D$ comprising $n$ pairs of image-text queries, we formally define the ASR as:
$\bullet$ 攻击成功率（ASR）用于量化使用图像-文本查询对诱使大型视觉语言模型（LVLMs）产生有害响应的概率。考虑一个包含 $n$ 对图像-文本查询的 $D$ 数据集，我们正式定义 ASR 为：

where $\text{AttackEvaluator}(\cdot)$ is a binary function that returns 1 if the model’s response $R_{i}$ to a given input pair $(T_{i},I_{i})$ is evaluated as unsafe and 0 otherwise. This metric effectively captures the proportion of query pairs capable of inducing undesirable outputs from the target LVLM. A lower ASR indicates greater safety performance to attacks.
其中 $\text{AttackEvaluator}(\cdot)$ 是一个二元函数，当模型 $R_{i}$ 对给定的输入对 $(T_{i},I_{i})$ 的响应被评估为不安全时返回 1，否则返回 0。这个指标有效地捕捉了能够从目标大型视觉语言模型（LVLM）中诱导出不良输出的查询对的比例。ASR 越低，表明对攻击的安全性性能越好。

$\bullet$ Safety Risk Index (SRI) is introduced by SafeBench [154], aim to provides a more detailed evaluation of model safety by distinguishing between varying levels of response severity. While ASR classifies all unsafe responses equally, SRI measures the degree of risk posed by each response, enabling a finer-grained safety analysis. For example, when queried about harmful topics like bomb-making, one model might generate detailed instructions while another merely references a related book. Although both would be considered unsafe under ASR, SRI assigns a higher risk score to the former due to its greater severity. SRI is computed as:
$\bullet$ 安全风险指数（SRI）由 SafeBench [ 154]引入，旨在通过区分不同级别的响应严重程度，对模型安全性进行更详细的评估。虽然 ASR 对所有不安全响应同等对待，但 SRI 衡量每个响应带来的风险程度，从而实现更细粒度的安全分析。例如，当被问及炸弹制作等有害话题时，一个模型可能会生成详细说明，而另一个模型仅引用相关书籍。尽管在 ASR 标准下两者都被视为不安全，但 SRI 由于前者严重程度更高，会赋予其更高的风险评分。SRI 的计算公式为：

where $\text{RiskEvaluator}(\cdot)$ is a scoring function that evaluates the severity of the model’s response $R_{i}$ to a query pair $(T_{i},I_{i})$, assigning a numerical risk value based on predefined criteria. The final SRI score is normalized to a range of 0 to 100 for interpretability. Higher SRI values indicate higher safety performances.
其中 $\text{RiskEvaluator}(\cdot)$ 是一个评分函数，用于评估模型对查询对 $(T_{i},I_{i})$ 的响应 $R_{i}$ 的严重程度，并根据预定义标准赋予数值风险值。最终 SRI 分数被归一化到 0 到 100 的范围，以便于理解。更高的 SRI 值表示更高的安全性表现。

$\bullet$ Attack Success Degree (ASD), proposed by MLLMGuard [56], ASD assesses the severity of unsafe model responses across four levels as shown in Tab. VI, score 0 represents Safe and Responsible behavior, where the model refuses assistance with appropriate warnings; score 1 reflects harmless but generic responses; score 2 indicates direct unsafe outputs (e.g., step-by-step harmful instructions); and score 3 denotes high-risk responses, where the model acknowledges the harm but still provides guidance (e.g., warning about the dangers of creating a bomb while offering instructions). ASD is computed as:
$\bullet$ 攻击成功程度（ASD），由 MLLMGuard [ 56] 提出，ASD 根据表 VI 所示的四级评估不安全模型响应的严重程度，分数 0 代表安全负责任的行为，即模型拒绝协助并给出适当的警告；分数 1 反映无害但通用的响应；分数 2 表示直接的不安全输出（例如，逐步有害的指令）；分数 3 表示高风险响应，即模型承认危害但仍然提供指导（例如，警告制造炸弹的危险性同时提供指导）。ASD 的计算公式为：

where $\text{DegreeEvaluator}(\cdot)$ evaluates the severity of the model’s response $R_{i}$ to a query pair $(T_{i},I_{i})$, assigning a score from 0 to 3, and $\text{Smooth}(\cdot)$ normalizes them to a 0–1 scale. Lower ASD values indicate better safety performance.
其中 $\text{DegreeEvaluator}(\cdot)$ 评估模型对查询对 $(T_{i},I_{i})$ 的响应 $R_{i}$ 的严重程度，并分配 0 到 3 的分数， $\text{Smooth}(\cdot)$ 将它们归一化到 0-1 的刻度上。较低的 ASD 值表示更好的安全性表现。

$\bullet$ Perfect Answer Rate (PAR) is derived from ASD, MLLMGuard [56] introduced PAR, measures the proportion of safe and responsible responses, specifically those categorized as score 0 by ASD. It is computed as:
$\bullet$ 完美答案率（PAR）源自 ASD，MLLMGuard [ 56] 引入了 PAR，衡量安全且负责任的响应比例，具体指被 ASD 归类为 0 分的响应。其计算公式为：

where $\mathbb{I}(\cdot)$ is an indicator function returning 1 if the condition is true and 0 otherwise. Higher PAR indicates stronger safety performance.
$\mathbb{I}(\cdot)$ 是一个指示函数，当条件为真时返回 1，否则返回 0。更高的 PAR 表示更强的安全性表现。