What is an LLM vulnerability scanner?

An LLM vulnerability scanner sends batteries of adversarial probes — jailbreaks, prompt injections, PII extraction attempts, harmful-content requests — at a target LLM and grades the responses. The output is a vulnerability + optimization report with per-probe evidence, severity, and a prioritized fix list.

Which LLMs can I scan with FilterPrompt?

OpenAI, Anthropic, Google Gemini, Azure OpenAI, plus any OpenAI-compatible endpoint — Ollama, Groq, Mistral, Together AI, OpenRouter, Perplexity, Hugging Face, vLLM, or your own custom endpoint. Bring your own keys per tenant.

What kinds of vulnerabilities does FilterPrompt test for?

Jailbreaks (DAN, role hijack, translation smuggling), direct and indirect prompt injection, system-prompt extraction, harmful-content compliance, PII / secret leakage, bias & fairness, RAG poisoning, agent/tool abuse, output quality, and robustness — categories map to the OWASP LLM Top 10.

How are probes graded?

Each probe declares an evaluator: regex match, refusal-check, contains-check, or an AI judge (Gemini 3 Flash). Pass/fail comes with severity, category, the exact prompt sent, the model's full response, and the evaluator's reason — fully auditable.

How much does a scan cost?

1 credit per probe executed. New accounts get 1 welcome credit on signup. Pay-as-you-go credit packs after that — credits never expire. Connecting LLMs and creating tenants is free.

Best LLM Security Scanner for Enterprise AI in 2026

Comparison · 2026-04-18 · 11 min read · FilterPrompt Security Team

The 2026 shortlist of enterprise LLM security scanners — FilterPrompt vs Garak vs Promptfoo vs Lakera — pricing, coverage, and evidence quality.

'Best LLM security scanner for enterprise AI 2025' is a zero-competition long-tail query with pure buying intent. If you're here, you're building a vendor shortlist. This post is your shortlist.

How enterprises evaluate LLM scanners in 2026

OWASP LLM Top 10 coverage — all 10 categories, versioned to the 2025 release.
Agentic probes — function-calling injection, tool abuse, unauthorized actions.
Evaluator quality — LLM-graded with confidence, not substring match.
Evidence — PDF audit-ready reports with full prompt/response chains.
Vendor security — SOC 2 Type II, ISO 27001, data-handling clarity.
Pricing — per-scan or per-token, predictable, no hidden overages.

The 4 contenders

FilterPrompt

Combined scanner + AI firewall, 1,000+ probes across all 10 OWASP LLM categories, agentic probes for tool abuse, LLM-graded evaluator, PDF reports mapped to OWASP + NIST AI RMF, free tier with the full sampler. Best fit: enterprises consolidating scanner + firewall into one platform.

NVIDIA Garak

Open-source, strong academic pedigree, CLI-only. Free forever. Best fit: engineering-heavy teams comfortable with Python and JSON output, or as a second-opinion scanner alongside a commercial tool.

Promptfoo

Developer-first, YAML config, CI-native. Blends evaluations with red-team probes. Best fit: teams that want prompt evaluations and scanning in the same tool.

Lakera

Firewall-first with scanner add-on, strong prompt-injection detection. Best fit: chat apps whose primary risk is prompt injection.

Side-by-side comparison

The one-day PoV

Sign up for a free FilterPrompt account.
Connect your LLM (OpenAI, Anthropic, Gemini, Azure, self-hosted).
Run the OWASP LLM Top 10 sampler — 5 minutes.
Review the PDF vulnerability report.
Push top-3 findings as firewall rules with one click.