What is an LLM vulnerability scanner?

An LLM vulnerability scanner sends batteries of adversarial probes — jailbreaks, prompt injections, PII extraction attempts, harmful-content requests — at a target LLM and grades the responses. The output is a vulnerability + optimization report with per-probe evidence, severity, and a prioritized fix list.

Which LLMs can I scan with FilterPrompt?

OpenAI, Anthropic, Google Gemini, Azure OpenAI, plus any OpenAI-compatible endpoint — Ollama, Groq, Mistral, Together AI, OpenRouter, Perplexity, Hugging Face, vLLM, or your own custom endpoint. Bring your own keys per tenant.

What kinds of vulnerabilities does FilterPrompt test for?

Jailbreaks (DAN, role hijack, translation smuggling), direct and indirect prompt injection, system-prompt extraction, harmful-content compliance, PII / secret leakage, bias & fairness, RAG poisoning, agent/tool abuse, output quality, and robustness — categories map to the OWASP LLM Top 10.

How are probes graded?

Each probe declares an evaluator: regex match, refusal-check, contains-check, or an AI judge (Gemini 3 Flash). Pass/fail comes with severity, category, the exact prompt sent, the model's full response, and the evaluator's reason — fully auditable.

How much does a scan cost?

1 credit per probe executed. New accounts get 1 welcome credit on signup. Pay-as-you-go credit packs after that — credits never expire. Connecting LLMs and creating tenants is free.

The Only Platform That Scans LLM Vulnerabilities AND Blocks Attacks Simultaneously

Differentiator · 2025-09-04 · 10 min read · FilterPrompt Security Team

Why scan-only and firewall-only tools both fail in production — and how a unified LLM scanner + AI firewall closes the loop in real time.

Every LLM security vendor today is either a scanner (Garak, Promptfoo) or a firewall (Lakera Guard, Cloudflare Firewall for AI, PromptShield). FilterPrompt is the only platform that does both — scan and block LLM attacks simultaneously, from a single dashboard, sharing one threat model. Here's why that matters.

Scan-only tools fail because they're frozen in time

A scan tells you what was vulnerable last Tuesday. The model's behaviour drifts daily — providers push silent updates, your system prompt changes, retrieval corpora are added, new probes go viral. By Friday, the scan is stale. Without enforcement, every newly-discovered weakness is exposed to live traffic until your next scan run.

Firewall-only tools fail because they don't know what to block

A firewall in front of an unknown threat surface is just a guess. If you haven't scanned, you don't know which attack patterns succeed against your model — so you either over-block (false positives, broken UX) or under-block (real attacks slip through). Firewalls without scanners are security theatre.

Real-time LLM protection requires both

FilterPrompt's design closes the loop:

The scanner runs continuously and finds new failure modes
Every confirmed vulnerability automatically becomes a firewall rule candidate
The firewall blocks live traffic matching that pattern within seconds of discovery
The same severity taxonomy drives both — no translation layer, no missed signals

What 'scan and block simultaneously' means in practice

One dashboard for both vulnerability reports and live block logs
One probe library — the same prompts your firewall blocks are the ones your scanner tests
One bill — pay per probe scanned, pay per request firewalled, no double-vendor overhead
One audit trail — SOC 2 and EU AI Act evidence in a single export

Migration path from scan-only or firewall-only

If you already use Garak or Promptfoo for scanning, you can keep them running and add FilterPrompt's firewall on top — but most teams consolidate after the first month because maintaining two probe libraries doubles the work. If you already use Lakera Guard or Cloudflare Firewall for AI, FilterPrompt's scanner shows you which of their rules are actually catching real attacks against your model.

Try the loop yourself

Sign up, connect any LLM, run a scan, then enable the firewall on the same tenant. The first full vulnerability scan is free. You'll have scan + firewall both live in under 5 minutes.