Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
越狱攻击与内容安全过滤器：我们在 LLM 安全军备竞赛中进展如何？

Jailbreak Attack. Let $\mathsf{LLM}$ : $\mathcal{T}^{*}$ $\to$ $\mathcal{T}^{*}$ be a target LLM, which maps a sequence of input tokens to a sequence of output tokens. A jailbreak attack aims to design input prompts that cause the target LLM to generate text that is harmful, toxic, or objectionable. Formally, let $\mathsf{Judge}$ $:$ $(\mathcal{T}^{*},\mathcal{T}^{*})$ $\to$ $\{0,1\}$ be a judge model that decides whether the generated output $\mathsf{LLM}(P)$ aligns with a predefined harmful goal $G$. The $\mathsf{Judge}$ returns $1$ if $\mathsf{LLM}(P)$ is deemed harmful (i.e., satisfies $G$), and $0$ otherwise. The objective of an attack is to exploit vulnerabilities in $\mathsf{LLM}$ by crafting inputs that elicit harmful responses from $\mathsf{LLM}$, which can be formalized as:
越狱攻击。设 $\mathsf{LLM}$ : $\mathcal{T}^{*}$ $\to$ $\mathcal{T}^{*}$ 为一个目标 LLM，它将输入标记序列映射到输出标记序列。越狱攻击旨在设计输入提示，使目标 LLM 生成有害、有毒或令人反感的文本。形式上，设 $\mathsf{Judge}$ $:$ $(\mathcal{T}^{*},\mathcal{T}^{*})$ $\to$ $\{0,1\}$ 为一个判断模型，它决定生成的输出 $\mathsf{LLM}(P)$ 是否与预定义的有害目标 $G$ 一致。如果 $\mathsf{LLM}(P)$ 被认为是有害的（即满足 $G$ ），则 $\mathsf{Judge}$ 返回 $1$ ，否则返回 $0$ 。攻击的目标是通过设计输入来利用 $\mathsf{LLM}$ 中的漏洞，使 $\mathsf{LLM}$ 产生有害响应，这可以形式化为：

Report issue for preceding element

$\displaystyle\text{find}\>P\in{\mathcal{T}}^{*},\text{ s.t.}\>\>\mathsf{Judge}(\mathsf{LLM}(P),G)=1,$

(1)

where $P$ is the input prompt and ${\mathcal{T}}^{*}$ denotes the set of all sequences of tokens of arbitrary length.
其中 $P$ 是输入提示， ${\mathcal{T}}^{*}$ 表示所有任意长度的标记序列的集合。

Report issue for preceding element

Content Filtering. Typically, jailbreak attacks focus solely on bypassing the inherent alignment of LLMs without considering additional filtering mechanisms. However, real-world systems often incorporate an additional safety filtering mechanism to mitigate harmful outputs, creating a gap in existing research that largely overlooks the impact of these filters. The simplest version is a binary classifier, denoted as $\mathsf{Filter}_{\text{bin}}$ $:$ $\mathcal{T}^{*}$ $\to$ $\{0,1\}$ where $\mathsf{Filter}_{\text{bin}}(P)$ $=$ $1$ indicates harmful content. If harmful content is detected, the system may block or modify the input using a modifying filter. We define this generalized filter as $\mathsf{Filter}$ $:$ $\mathcal{T}^{*}$ $\to$ $\mathcal{T}^{*}\cup\{\bot\}$, where $\mathsf{Filter}(P)$ $=$ $P$ if safe, and $\mathsf{Filter}(P)$ $=$ $\bot$ (or an alternative predefined response) if harmful. The overall inference pipeline, which mimics practical safety-enhanced LLM systems, can be defined as:
内容过滤。通常，越狱攻击仅专注于绕过 LLMs 的固有对齐性，而不考虑额外的过滤机制。然而，现实世界中的系统通常包含额外的安全过滤机制来减轻有害输出，这造成了现有研究的一大空白，而现有研究大多忽视了这些过滤器的影响。最简单版本是一个二元分类器，表示为 $\mathsf{Filter}_{\text{bin}}$ $:$ $\mathcal{T}^{*}$ $\to$ $\{0,1\}$ ，其中 $\mathsf{Filter}_{\text{bin}}(P)$ $=$ $1$ 表示有害内容。如果检测到有害内容，系统可能会使用修改过滤器阻止或修改输入。我们将这种通用过滤器定义为 $\mathsf{Filter}$ $:$ $\mathcal{T}^{*}$ $\to$ $\mathcal{T}^{*}\cup\{\bot\}$ ，其中 $\mathsf{Filter}(P)$ $=$ $P$ 表示安全，而 $\mathsf{Filter}(P)$ $=$ $\bot$ （或一个预定义的替代响应）表示有害。整体推理管道，它模拟了实际增强安全性的 LLM 系统，可以定义为：

Report issue for preceding element

$$\mathsf{Filter}\Big(\mathsf{LLM}\big(\mathsf{Filter}\left(P\right)\big)\Big),$$

(2)

where the inner filter prevents adversarial inputs from reaching LLM, and the outer filter removes or modifies harmful outputs. A jailbreak attack should be deemed successful only if it bypasses both content filters and leads to a harmful output aligned with goal $G$, which can be summarized as:
其中内部过滤器阻止对抗性输入到达 LLM，外部过滤器移除或修改有害输出。只有当越狱攻击绕过两个内容过滤器并导致与目标 $G$ 对齐的有害输出时，才应被视为成功，这可以总结为：

Report issue for preceding element

	$\displaystyle\mathsf{Filter}_{\text{bin}}(P)=0,\>\>\mathsf{Filter}_{\text{bin}}(\mathsf{LLM}(P))=0,$		(3)
	$\displaystyle\mathsf{Judge}\big(G,\mathsf{Filter}\big(\mathsf{LLM}\left(\mathsf{Filter}\left(P\right)\right))\big)=1.$		(4)

General-Purpose LLMs as Zero-Shot Detectors. General-purpose LLMs leverage inherent capabilities to detect harmful content without task-specific adaptation. Safety-aligned models like GPT-4 achiam2023gpt and O3 o3mini2024 are trained on large-scale corpora and undergo extensive alignment to identify harmful prompts and outputs, with O3 leveraging reasoning capabilities to engage more deeply with semantic content liu2025logical; el2025competitive.
通用型 LLMs 作为零样本检测器。通用型 LLMs 利用其固有能力在不进行特定任务适配的情况下检测有害内容。与安全对齐的模型如 GPT-4 (Achiam, 2023) 和 O3 (o3mini, 2024) 在大规模语料库上训练，并经过广泛的对齐过程以识别有害提示和输出，其中 O3 借助推理能力更深入地处理语义内容 (Liu, 2025; El, 2025)。

Report issue for preceding element

Fine-Tuned LLMs as Standalone Classifiers. Fine-tuned LLMs are optimized for specific risks. LlamaGuard inan2023llama is fine-tuned on a safety taxonomy to classify violence and hate speech. PromptGuard PromptGuard is trained on adversarial data to detect benign, injected, and jailbreak prompts. InjecGuard li2024injecguard introduces the NotInject dataset to mitigate over-defense, enabling accurate discrimination between benign and injected prompts. OpenAI’s Content Moderation API markov2023holistic uses hybrid LLM-based classification and active learning for real-world filtering.
精调 LLMs 作为独立分类器。精调 LLMs 针对特定风险进行了优化。LlamaGuard inan2023llama 在安全分类法上进行精调，用于分类暴力和仇恨言论。PromptGuard PromptGuard 在对抗性数据上进行训练，用于检测良性、注入和越狱提示。InjecGuard li2024injecguard 引入了 NotInject 数据集以减轻过度防御，能够准确区分良性提示和注入提示。OpenAI 的内容审核 API markov2023holistic 使用基于 LLM 的混合分类和主动学习进行现实世界过滤。

Report issue for preceding element

Gradient-based Detectors. GradSafe (xie2024gradsafe) detects jailbreak prompts by analyzing gradient patterns of safety-critical parameters. When processing jailbreak prompts paired with compliance responses, LLM gradients exhibit consistent patterns distinct from safe prompts, enabling accurate detection without additional training.
基于梯度的检测器。GradSafe (xie2024gradsafe)通过分析安全关键参数的梯度模式来检测越狱提示。在处理与合规响应配对的越狱提示时，LLM 梯度表现出与安全提示不同的稳定模式，无需额外训练即可实现准确检测。

Report issue for preceding element

Datasets & LLMs. We construct a more diverse and representative evaluation set by consolidating existing benchmark datasets (AdvBench50 chen2022should, MaliciousInstruct huang2024catastrophic, JailbreakBench chao2024jailbreakbench, HarmBench mazeika2024harmbench and TruthfulQA lin2021truthfulqa) and filtering out semantically overlapping samples, resulting in 417 harmful prompts and 417 topic-aligned benign counterparts. The paired benign prompts enable a faithful assessment of the safety filter’s false positive rate for normal behaviors, providing valuable insights into its potential impact on downstream applications. Overall, our testing dataset covers various misuse behaviors, spanning 10 categories that violate OpenAI’s usage policy, which are constructed as extensions of established benchmarks mazeika2024harmbench; zou2023universal, ensuring comprehensive coverage of adversarial prompts. Besides, we evaluate both open-source and closed-source LLMs: Llama-2-7B-Chat touvron2023llama, Llama3.1-8B-Instruct grattafiori2024llama, Mistral-7B-V0.3 jiang2023mistral, Vicuna-7B-V1.5 zheng2023judging, Qwen2.5-7B-Instruct qwen2024qwen2, as well as GPT-4o-2024-1120¹¹1https://openai.com/index/gpt-4o-system-card/
数据集与 LLMs。我们通过整合现有的基准数据集（AdvBench50 chen2022should、MaliciousInstruct huang2024catastrophic、JailbreakBench chao2024jailbreakbench、HarmBench mazeika2024harmbench 和 TruthfulQA lin2021truthfulqa），并过滤掉语义上重叠的样本，构建了一个更多样化且更具代表性的评估集，最终得到 417 个有害提示和 417 个与主题对齐的良性对应提示。成对的良性提示能够对安全过滤器在正常行为上的假阳性率进行忠实评估，为其对下游应用可能产生的影响提供宝贵见解。总体而言，我们的测试数据集涵盖了各种滥用行为，涉及 10 个违反 OpenAI 使用政策的类别，这些类别作为现有基准 mazeika2024harmbench；zou2023universal 的扩展构建，确保了对对抗性提示的全面覆盖。此外，我们评估了开源和闭源 LLMs：Llama-2-7B-Chat touvron2023llama、Llama3.1-8B-Instruct grattafiori2024llama、Mistral-7B-V0.3 jiang2023mistral、Vicuna-7B-V1.5 zheng2023judging、Qwen2.5-7B-Instruct qwen2024qwen2，以及 GPT-4o-2024-1120 ¹ and GPT-4-Turbo-1106-preview²²2https://help.openai.com/en/articles/8555510-gpt-4-turbo-in-the-openai-api
和 GPT-4-Turbo-1106-preview ² .

Report issue for preceding element

Metrics. To provide a comprehensive view of attack behaviors and content safety filters, we distinguish several key metrics for a more nuanced understanding that can guide future research. First, we include the normal attack success rate ($\mathrm{ASR(Ori)}$), as a baseline for comparison, defined as:
指标。为了全面了解攻击行为和内容安全过滤器，我们区分了几个关键指标，以便更细致地理解，从而指导未来的研究。首先，我们包含正常攻击成功率（ $\mathrm{ASR(Ori)}$ ），作为比较的基准，定义为：

Report issue for preceding element

$$\frac{\sum_{i}\mathbb{I}\big\{\mathsf{Judge}\big(G^{(i)},\mathsf{LLM}(P^{(i)})\big)=1\big\}}{\big|\big\{G^{(i)}|G^{(i)}\text{ is harmful}\big\}\big|},$$

(5)

where $\mathbb{I}$ is the indicator function and $G^{(i)}$ denotes the $i$-th goal. This metric takes into account only the $\mathsf{Judge}$ without considering the content filter. Next, we define the detection rate (DR) as the proportion of harmful samples successfully detected by the content filter. We further distinguish the detection rate at the input stage (DR_I) and the output (DR_O) stage, if applicable to the specific $\mathsf{Filter}$:
其中 $\mathbb{I}$ 是指示函数， $G^{(i)}$ 表示 $i$ -th 目标。该指标仅考虑 $\mathsf{Judge}$ ，而不考虑内容过滤器。接下来，我们将检测率（DR）定义为内容过滤器成功检测到的有害样本的比例。我们进一步区分输入阶段的检测率（DR_I）和输出（DR_O）阶段的检测率，如果适用于特定的 $\mathsf{Filter}$ ：

Report issue for preceding element

	$\displaystyle\mathrm{DR\_I}=\frac{\sum_{i}\mathbb{I}\big\{\mathsf{Filter}_{\text{bin}}(P^{(i)})=1\big\}}{\big|\big\{G^{(i)}|G^{(i)}\text{ is harmful}\big\}\big|},$		(6)
	$\displaystyle\mathrm{DR\_O}=\frac{\sum_{i}\mathbb{I}\big\{\mathsf{Filter}_{\text{bin}}(\mathsf{LLM}(P^{(i)}))=1\big\}}{\big|\big\{G^{(i)}|G^{(i)}\text{ is harmful}\big\}\big|}.$		(7)

Additionally, we define the pass rate (Pass) as the rate at which harmful samples are not detected at both the input and output stages:
此外，我们将通过率（Pass）定义为有害样本在输入和输出阶段均未被检测到的比率：

Report issue for preceding element

Attacks, Safety Filters & Judge. We test representative jailbreak attacks that are recognized as state-of-the-art at the time of their publication. These methods, known for their diverse characteristics, provide a well-rounded foundation for our empirical analysis. In particular, we examine TAP mehrotra2023tree, PAIR chao2023jailbreaking, AutoDAN liu2023autodan, Adaptive andriushchenko2024jailbreaking, DrAttack li2024drattack, DeepInception li2023deepinception, CodeChameleon lv2024codechameleon, ReNeLLM ding2024wolf, Advprompter paulus2024advprompter and multi-turn attack Crescendo russinovich2025great. As discussed in Section 3.2, we assess well-established content safety filters that have been prominent in the field, spanning different categories. Our evaluation includes OpenAI API markov2023holistic, LlamaGuard inan2023llama, PromptGuard PromptGuard, InjecGuard li2024injecguard, GradSafe xie2024gradsafe, and O3 o3mini2024 model. To evaluate the semantic success of these attacks, we use GPT-4 as the $\mathsf{Judge}$, following common practices in the prior literature on LLM jailbreaks. For all the methods, we adhere to their official implementation as our standard approach (see Appendix A for detailed descriptions).
攻击、安全过滤器与裁判。我们测试了在发表时被公认为最先进的代表性越狱攻击。这些以其多样化特征而闻名的攻击方法，为我们的实证分析提供了全面的坚实基础。特别是，我们考察了 TAP mehrotra2023tree、PAIR chao2023jailbreaking、AutoDAN liu2023autodan、Adaptive andriushchenko2024jailbreaking、DrAttack li2024drattack、DeepInception li2023deepinception、CodeChameleon lv2024codechameleon、ReNeLLM ding2024wolf、Advprompter paulus2024advprompter 以及多轮攻击 Crescendo russinovich2025great。如第 3.2 节所述，我们评估了在领域内具有显著地位且涵盖不同类别的成熟内容安全过滤器。我们的评估包括 OpenAI API markov2023holistic、LlamaGuard inan2023llama、PromptGuard PromptGuard、InjecGuard li2024injecguard、GradSafe xie2024gradsafe 以及 O3 o3mini2024 模型。为评估这些攻击的语义成功率，我们遵循先前关于 LLM 越狱研究的常见做法，使用 GPT-4 作为 $\mathsf{Judge}$ 。 对于所有方法，我们遵循其官方实现作为标准方法（详细描述见附录 A）。

Report issue for preceding element

Report issue for preceding element

Safety Filters Are Effective. The overall trend suggests a current “winning” state for safety filters, as evidenced by the substantially lower pass rates (shown in Table 1 and Table 2) compared to the attack success rates typically reported in jailbreak literature. This is also visually reflected in the tables, where the green areas (indicating lower pass rates) dominate over the red regions (indicating higher pass rates). More specifically, detectors like Prompt-Guard and O3 can effectively block most injected prompts right at the input stage, with detection rates mostly ranging from approximately 70% to 100%. As a result, the overall pass rate in most cases is reduced to under 5%. This starkly contrasts with previous studies, where attack success rates are often reported as being much higher, showcasing the effectiveness of the safety filters in preventing prompts with malicious goals.
安全过滤器是有效的。整体趋势表明，当前安全过滤器处于“获胜”状态，这从表 1 和表 2 中显示的通过率（与越狱文献中通常报告的攻击成功率相比）大幅降低得到证明。这一点在表格中也有直观体现，绿色区域（表示较低通过率）占据了红色区域（表示较高通过率）。具体来说，像 Prompt-Guard 和 O3 这样的检测器可以在输入阶段有效阻止大多数注入的提示，检测率大多在 70%到 100%之间。因此，大多数情况下整体通过率被降低到 5%以下。这与以往的研究形成鲜明对比，以往研究中攻击成功率通常报告得要高得多，这展示了安全过滤器在阻止具有恶意目标的提示方面的有效性。

Report issue for preceding element

Report issue for preceding element

Attacks Lacking Semantic Stealthiness Are Easily Defeated.While many recent attacks report high effectiveness on standard, aligned LLMs, achieving success rates of over 90% (e.g., for TAP and Adaptive), the malicious prompts they generate are easily detected by existing safety filters (and manual inspection, as illustrated in Appendix Table 6 - Table 11). This creates a misleading impression of the overall threat posed by jailbreaking attacks. In contrast, methods that explicitly account for semantic stealthiness (such as ReNeLLM in this case) persist longer, maintaining a relatively acceptable pass rate against certain safety filters. However, even these approaches eventually fail, as there exist filters (such as PromptGuard and O3) capable of effectively detecting such harmful outputs or malicious inputs. Notably, the multi-turn attack Crescendo can also be effectively detected. Despite its gradual escalation of harmful intent across conversation turns, the final harmful conversation remains detectable by safety filters.
缺乏语义隐蔽性的攻击很容易被击败。虽然许多最近的攻击报告在标准、对齐的 LLMs 上具有很高的有效性，成功率超过 90%（例如，对于 TAP 和 Adaptive），但它们生成的恶意提示很容易被现有的安全过滤器（以及人工检查，如附录表 6-表 11 所示）检测到。这造成了一种关于越狱攻击总体威胁的误导印象。相比之下，明确考虑语义隐蔽性的方法（例如本例中的 ReNeLLM）持续时间更长，在针对某些安全过滤器时保持相对可接受的通过率。然而，即使这些方法最终也会失败，因为存在能够有效检测此类有害输出或恶意输入的过滤器（例如 PromptGuard 和 O3）。值得注意的是，多轮攻击 Crescendo 也可以被有效检测。尽管它在对话轮次中逐渐升级有害意图，但最终的有害对话仍然可被安全过滤器检测到。

Report issue for preceding element

Report issue for preceding element

Detector Performance and LLM Vulnerability Exhibit Systematic Variations. The results in Table 1-2， and Figure 1 highlight significant variations in the detection and vulnerability of different LLMs when subjected to a range of jailbreak attacks. Figure 3 shows the fine-grained detection result. Generally, the reasoning model O3 tends to be the most effective detector, achieving the lowest pass rates and suggesting that reasoning capabilities may play an important role in identifying harmful content regardless of the jailbreak transformation applied. In contrast, GradSafe underperforms in nearly half the cases, as it heavily depends on the format of paired, meaningful queries and responses starting with “Sure,” limiting its effectiveness across diverse jailbreak types. PromptGuard tends to over-predict harmfulness, while InjecGuard builds on it with improved robustness. From the model’s perspective, Llama-2-7B and Llama3.1-8B exhibit relatively low attack success rates (ASR $<30\%$) across six jailbreak attacks, indicating stronger inherent attack hardness. However, in cases where these models are successfully jailbroken, the resulting inputs and outputs tend to be more stealthy, leading to higher pass rates.
检测器性能和 LLM 漏洞表现出系统性的变化。表 1-2 和图 1 的结果突显了不同 LLM 在遭受一系列越狱攻击时，其检测和漏洞存在显著差异。图 3 展示了细粒度的检测结果。通常，推理模型 O3 倾向于成为最有效的检测器，实现最低的通过率，表明推理能力可能在识别有害内容方面发挥重要作用，无论应用何种越狱转换。相比之下，GradSafe 在近一半情况下表现不佳，因为它严重依赖于以“Sure”开头的成对、有意义的查询和响应格式，限制了其在不同越狱类型中的有效性。PromptGuard 倾向于过度预测有害性，而 InjecGuard 在此基础上增强了鲁棒性。从模型的角度来看，Llama-2-7B 和 Llama3.1-8B 在六种越狱攻击中表现出相对较低的攻击成功率（ASR $<30\%$ ），表明其具有更强的固有攻击难度。 然而，在这些模型被成功越狱的情况下，产生的输入和输出往往更加隐蔽，导致通过率更高。

Report issue for preceding element

Report issue for preceding element

Detectors Tend to Be Tuned for High Recall. While reliably detecting harmful prompts is essential, over-flagging benign prompts can significantly degrade user experience. To explore this trade-off, we present confusion matrix heatmaps in Figure 2 and quantitative results in Table 5, evaluating filters on non-jailbreak” samples including both benign” and “harmful” cases to reflect real-world distributions. Results reveal substantial performance gaps: LlamaGuard achieves the best accuracy (95%) with balanced precision-recall, while PromptGuard and InjecGuard perform poorly below 80%, with PromptGuard exhibiting severe over-flagging (1.00 FPR). These findings highlight that while top-tier filters like LlamaGuard demonstrate strong applicability, others may overly restrict conversational diversity and frustrate users with excessive false positives, especially problematic given that most real-world LLM usage involves benign behavior.
检测器往往针对高召回率进行调优。虽然可靠地检测有害提示至关重要，但过度标记良性提示会显著降低用户体验。为了探索这种权衡，我们在图 2 中展示了混淆矩阵热图，在表 5 中展示了定量结果，评估过滤器在“非越狱”样本上的表现，包括“良性”和“有害”案例，以反映真实世界分布。结果表明存在显著的性能差距：LlamaGuard 实现了最佳准确率（95%），具有平衡的精确率-召回率，而 PromptGuard 和 InjecGuard 表现不佳，低于 80%，其中 PromptGuard 表现出严重的过度标记（1.00 FPR）。这些发现表明，虽然像 LlamaGuard 这样的顶级过滤器具有很强的适用性，但其他过滤器可能会过度限制对话多样性，并因过多的误报而使用户感到沮丧，尤其考虑到大多数真实世界 LLM 使用涉及良性行为。

Report issue for preceding element

Report issue for preceding element

Most Detectors Impose Minimal Overhead. Table 4 presents the inference latency and economic cost for each safety filter. Overall, most detectors are feasible for routine use with modest per-sample overheads, while differences mainly reflect the expected accuracy–cost trade-off rather than prohibitive barriers. LlamaGuard and OpenAI API emerge as the most cost-effective solutions, with low latency (0.028 ms and 0.455 ms respectively) and minimal economic cost. Combined with their strong detection rates, these filters are well-suited for large-scale deployment where both effectiveness and efficiency are critical. O3 achieves the highest accuracy but at a noticeably higher cost (7.22 ms/sample, $0.0096/sample), making it a strong choice for high-stakes or auditing scenarios, while less attractive for high-throughput usage. GradSafe’s gradient-based approach incurs the highest latency (40.67 ms/sample), limiting real-time applicability, while PromptGuard and InjecGuard maintain reasonable speeds but lack the robustness of LlamaGuard or precision of O3. These findings suggest that LlamaGuard and the OpenAI API suit routine deployment, whereas O3 is preferable for critical scenarios where accuracy outweighs cost.
大多数检测器带来的开销极小。表 4 展示了每个安全过滤器在推理延迟和经济成本方面的表现。总体而言，大多数检测器在日常使用中是可行的，每样本的开销适中，而差异主要体现在预期的准确率—成本权衡上，而非不可逾越的障碍。LlamaGuard 和 OpenAI API 成为最具成本效益的解决方案，具有低延迟（分别为 0.028 毫秒和 0.455 毫秒）和最小的经济成本。结合其较高的检测率，这些过滤器非常适合大规模部署，其中有效性和效率都至关重要。O3 实现了最高的准确率，但成本明显更高（每样本 7.22 毫秒，每样本 0.0096 美元），使其成为高风险或审计场景的强有力选择，但在高吞吐量使用中吸引力较低。GradSafe 的基于梯度的方法导致最高的延迟（每样本 40.67 毫秒），限制了其实时应用能力，而 PromptGuard 和 InjecGuard 保持了合理的速度，但缺乏 LlamaGuard 的鲁棒性或 O3 的精确度。 这些发现表明，LlamaGuard 和 OpenAI API 适合日常部署。而在关键场景中，当准确性比成本更重要时，O3 更可取。

Report issue for preceding element

False Positives Distribution. We present qualitative false positive examples in Table 3 along with the category distribution of misclassified benign behaviors in Figure 4. The most common misclassifications fall under Physical Harm, followed by Privacy and Sexual/Adult Content. These errors often stem from contextually neutral terms like “bomb”, “commit suicide”, or “Nazi party” appearing in non-harmful contexts (e.g., historical or medical discussions), which can still trigger safety filters. Although current safety filters are large models that go beyond simple keyword matching, they remain sensitive to certain word patterns that resemble harmful content. This indicates a limitation in distinguishing harmful from harmless content when sensitive terms are involved. More context-aware safety mechanisms are needed—ones that better model semantic meaning and intent, rather than reacting to surface-level patterns.
误报分布。我们在表 3 中展示了定性的误报示例，并在图 4 中呈现了被错误分类的良性行为的类别分布。最常见的错误分类属于身体伤害，其次是隐私和色情/成人内容。这些错误通常源于在非有害的语境中（例如历史或医学讨论）出现上下文中中性的词语，如“炸弹”、“自杀”或“纳粹党”，这些词语仍然会触发安全过滤器。尽管当前的安全过滤器是大型模型，超越了简单的关键词匹配，但它们仍然对某些类似有害内容的词语模式敏感。这表明在涉及敏感词语时，区分有害与无害内容存在局限性。需要更多具有上下文感知能力的安全机制——这些机制能更好地模拟语义意义和意图，而不是对表面模式做出反应。

Report issue for preceding element