What is an LLM vulnerability scanner?

An LLM vulnerability scanner sends batteries of adversarial probes — jailbreaks, prompt injections, PII extraction attempts, harmful-content requests — at a target LLM and grades the responses. The output is a vulnerability + optimization report with per-probe evidence, severity, and a prioritized fix list.

Which LLMs can I scan with FilterPrompt?

OpenAI, Anthropic, Google Gemini, Azure OpenAI, plus any OpenAI-compatible endpoint — Ollama, Groq, Mistral, Together AI, OpenRouter, Perplexity, Hugging Face, vLLM, or your own custom endpoint. Bring your own keys per tenant.

What kinds of vulnerabilities does FilterPrompt test for?

Jailbreaks (DAN, role hijack, translation smuggling), direct and indirect prompt injection, system-prompt extraction, harmful-content compliance, PII / secret leakage, bias & fairness, RAG poisoning, agent/tool abuse, output quality, and robustness — categories map to the OWASP LLM Top 10.

How are probes graded?

Each probe declares an evaluator: regex match, refusal-check, contains-check, or an AI judge (Gemini 3 Flash). Pass/fail comes with severity, category, the exact prompt sent, the model's full response, and the evaluator's reason — fully auditable.

How much does a scan cost?

1 credit per probe executed. New accounts get 1 welcome credit on signup. Pay-as-you-go credit packs after that — credits never expire. Connecting LLMs and creating tenants is free.

How FilterPrompt Works — Layered LLM Defense

FilterPrompt sits in front of your LLM provider as an OpenAI-compatible proxy. Every prompt and response passes through a layered pipeline of detectors before being forwarded.

Request lifecycle

Auth — x-firewall-key header resolves the tenant
IP check — global + tenant-scoped allow/blocklist
Quota check — rate limits and monthly caps
Pre-filter — pattern rules, ML detectors, structural validators, PII redaction
Upstream call — OpenAI / Anthropic / Gemini / Azure with tenant credentials
Post-filter — response inspected for PII leakage and policy violations
Audit log — verdict, rules, latency, and cost recorded per request

Layered detection

Cheap detectors run first and short-circuit on high-confidence verdicts. Heavy ML detectors run in parallel only when earlier layers were inconclusive. This keeps median latency under 100ms even with the full stack enabled.

Verdicts and audit

Each request produces a structured verdict — risk score, triggered rules, matched spans, action taken, and a stable log id. The audit table is queryable by tenant for incident response.

How FilterPrompt Works — Layered LLM Defense

Request lifecycle

Layered detection

Verdicts and audit

Related