Jailbreak Risk Scorer

What it does

Heuristic score for jailbreak and prompt-injection risk in user input. Tool flags keyword patterns commonly seen in injection attempts: instruction-override language (“ignore previous”, “disregard”, “forget all”), role-play hijacking (“DAN”, “jailbroken AI”, “developer mode”), encoded instructions (Base64, ROT13, hidden in markdown comments), and known prompt-injection templates from public jailbreak repositories. Output: risk score 0-100 with flagged terms, intended as a fast pre-filter before LLM calls.

Why pre-screening matters: prompt injection is the OWASP LLM-01 top vulnerability of 2024-2025. Successful injections can: leak system prompts (revealing IP and competitive advantage), trick models into harmful outputs, exfiltrate data from RAG context, manipulate agent tool-calls. Real-world incidents include Bing Chat revealing its “Sydney” codename via injection (Feb 2023), GPT-4 jailbreaks via repeated DAN-pattern attacks, and indirect injection through poisoned web pages fed to AI browsers. Modern frontier models (GPT-4o, Claude Opus 4) resist most known attacks but no LLM is fully immune.

Defense in depth — this tool is one layer of many: (1) Heuristic pre-filter (this tool) — rejects obvious attacks at zero cost. (2)LLM-based classifier (small model trained on injection examples) — catches subtle attacks the heuristic misses. (3) Structural defense— wrap user content in XML tags, instruct main model to never act on instructions inside user-content tags. (4) Output filter — second LLM pass checks main output for sensitive content (system prompt fragments, unsafe recommendations). (5) Monitoring — log and rate-limit users with repeated injection attempts. Production LLM apps need all five layers; this heuristic alone is not sufficient.

How to use it

1. Paste the user input you're about to send to an LLM.
2. Read the risk score (0-100). Score over 60 = likely injection attempt; over 80 = high-confidence injection.
3. Review flagged terms — see exactly which keywords/patterns matched.
4. Decide: block, sanitize, or allow with hardened system prompt.
5. For repeat offenders or high-volume APIs, integrate a programmatic version of this check + LLM-based classifier in your pipeline.
6. Combine with structural defenses (XML-tag wrapping, output filtering) for production deployments.

When to use this tool

• Production LLM-powered applications taking arbitrary user input.
• API endpoints exposed to external users (chatbots, AI features in SaaS products).
• Pre-screening before expensive frontier-model calls — reject obvious injection at zero LLM cost.
• Security audits of existing LLM features — running production traffic through to see injection rate.

When not to use it

• As sole defense — sophisticated attacks bypass keyword matching; you also need LLM-based classification + structural defenses.
• Internal tools with trusted users — overhead not worth it for tools used only by employees.
• Domains where injection isn't possible (e.g., classification of fixed input categories).
• When you're using OpenAI/Anthropic moderation APIs — those cover overlapping concerns.

Common use cases

• Pre-decision sanity-check on inputs and outputs
• Educational use — demonstrating the underlying concept
• Onboarding a colleague who needs the same calculation/conversion
• Verifying a number or output before passing it on