How Prompt Injection Detection Works (and Where It Fails)

A prompt injection detection rate needs context: which attacks, which legitimate inputs, which detector version and which decision threshold? A high catch rate can come with enough false alarms to disrupt ordinary work.

This post examines published evaluations and practical limits. For the underlying mechanism, see how prompt injection works.

Prompt injection detection rates move with the false alarm threshold

Score-based detectors apply a threshold to decide what to flag. Lowering it can catch more attacks while flagging more benign inputs. Some detectors instead return a label or apply fixed rules, so not every product exposes an adjustable score.

The April 2025 version of the PromptShield paper reports these results on its own evaluation split:

False positive rate	PromptShield detection rate	Prior detector used for comparison
0.1%	About 65%	PromptGuard: about 9%
1%	94.8%	InjecGuard: 20.4%

PromptShield is the authors’ own detector. The prior comparator changes between rows; these are not two operating points for one competing detector. The results are measured at selected thresholds, not production guarantees.

Base rates make false alarms costly

Consider a hypothetical day with ten thousand benign messages. A one percent false positive rate would produce about a hundred false alerts on average. If genuine attacks are scarce, false alerts can dominate the queue. Whether an alert interrupts a user depends on the response policy: logging, review and automatic blocking have different costs.

Ask for both detection and false positive rates, then test whether those rates hold on your own traffic. A detection percentage without the evaluation setup is insufficient for judging deployment usefulness.

Test generalization beyond familiar datasets

The July 2026 version of When Benchmarks Lie evaluated activation-based classifiers on a benchmark of 18 datasets spanning attack and benign sources. It compared standard cross-validation with leaving an entire dataset out of training. Pooled AUC was 8.0–16.5 points higher under cross-validation; per-dataset held-out-test versus leave-one-dataset-out accuracy gaps ranged from 1–25 points.

The authors also classified 28–44% of the top sparse-autoencoder features they examined as dataset-dependent shortcuts under their chosen criterion. That is not a percentage of all model features. A dataset-identity classifier reached 96.6% accuracy, illustrating how recognizable the data sources were.

These figures describe that study’s probes and evaluation choices, not a measured loss for every commercial guardrail. AUC is not accuracy. A benchmark score is neither a forecast nor a guaranteed upper bound for a different deployment; distribution and task difficulty can change in either direction.

What different detection approaches can miss

Use these as questions for testing a system, not as a ranking of products.

Input patterns and learned classifiers

A phrase filter can catch known wording but miss paraphrases or encoded instructions. A learned classifier can use broader signals, yet still fail on unfamiliar inputs or mistake legitimate instructions for an attack. Measure both kinds of error on the content your application actually reads.

Canary strings and output checks

A private, distinctive string can help test whether protected prompt content appears in output. A match is evidence to investigate; it does not prove that other secrets leaked or identify the cause on its own. An output check can block a matching response before delivery, but attacks that do not emit the canary remain outside that check’s coverage.

Checks on proposed and completed actions

Compare proposed tool calls, file access and outbound requests with the authorized task and policy. A check placed before execution can deny an action; a log-based check after execution can only help detect and respond to what has already happened. Timing matters, especially for disclosure that cannot be recalled. An unexpected action may also be an ordinary error, so it is not by itself proof of injection.

Conversation and session context

Considering earlier messages and tool results can reveal a harmful combination that isolated-message checks miss. It does not guarantee detection, and an individual-message check may still catch an obvious step within a longer attack. Test the actual context window and workflow rather than assuming either design always wins.

Pair detection with enforced access limits

Treat detection as one control whose performance must be evaluated and monitored. Restrict credentials, accessible data, tool operations and outbound destinations so a missed attack has fewer options.

Mark retrieved material as untrusted and keep it separate from higher-priority instructions. That helps communicate the boundary to the model; it does not enforce authorization by itself. Check permissions and tool arguments outside the model, and test those checks after changes.

The June 2025 MCP Tools specification recommends human control over tool invocations and confirmation for sensitive operations. It also requires server-side input validation and access controls. Those are different obligations, as MCP tool poisoning explains. Require meaningful approval for sensitive disclosures and high-impact actions according to your workflow, and verify that the approval cannot be bypassed. Limiting AI agent permissions covers that decision.

Researching this site has put my agents through untrusted pages, forum threads and documents, but I have not identified an injection attempt myself. I read the answer more often than the full retrieval, so that observation does not measure attack prevalence. A failed attempt may still leave evidence in retrieved content or logs. My preference is to combine review with limits on what agents can access and do.

FAQ

Do prompt injection detection tools work in practice?

Published evaluations show that detectors can catch attacks, but a benchmark alone does not establish usefulness in your deployment. Ask for the tested version, attack coverage, false positive rate and evaluation method. Validate them on representative benign and adversarial inputs before relying on automatic blocking.

How do I test my own setup for prompt injection?

Use an isolated test environment with synthetic data. Include legitimate tasks, known attacks, multi-turn attempts and the external sources your agent reads. Check unauthorized disclosures, tool actions and corrupted answers, as well as false alarms and task completion. Repeat relevant tests when prompts, models, permissions or tools change.

Can a detector catch an attack spread over many messages?

Sometimes. Conversation-aware checks can identify combinations that isolated checks miss, while an obvious malicious step may be caught individually. Neither design guarantees success. Test complete sequences and the point at which a harmful action would be blocked.

Is prompt injection solved yet?

The cited research does not establish a general solution. OWASP recommends layered mitigations, including model instructions, filtering, least privilege and human approval for high-risk operations. Model-level defenses and application controls can both help; neither should be treated as an unconditional guarantee.

Sources

Research figures refer to the cited paper versions, checked October 10, 2026. The traffic example is hypothetical. Practical suggestions are engineering guidance, not measured performance claims.

Written by Grace

I test AI tools and agents in my own workflow, and write down what I find, including the settings that surprised me. About Grace and how posts are verified