AI Vulnerability Remediation: From Finding to Retest
Guide · 2026-11-09 · 6 min read · FilterPrompt Security Team
A practical evidence-first workflow for reviewing LLM findings, marking false positives, assigning fixes and measuring retest outcomes.
FilterPrompt is AI Security Testing & Vulnerability Scanner. It submits adversarial prompts to a connected LLM or AI application, analyses model responses, identifies vulnerabilities and generates reports with test evidence. It does not block live traffic, redact customer requests or act as a firewall.
Confirm before escalating
Read the prompt, response, evaluation reason and target configuration. Reproduce high-impact findings safely; LLM behavior can vary between runs. Record whether the issue is real, uncertain or a false positive, and explain the decision.
Assign the fix where the boundary lives
- Retrieval leakage: check document permissions and tenant isolation.
- Unsafe tool actions: check authorization, schemas, destination allowlists and human approval.
- Instruction override: separate untrusted content, narrow permissions and assess policy design.
- Unsafe output: validate downstream handling independently of the model.
Retest more than one payload
Repeat the original failing case, variations and benign control cases after changes. Compare results on documented model and application configurations. Record unresolved risk in your existing tracker. FilterPrompt supplies testing and reporting, not an automatic production patch or guaranteed remediation.
Run an authorized assessment with FilterPrompt
- Create an account and connect an LLM using your own provider credentials. Use an authorized staging target and synthetic data.
- Choose a scan suite and review the probe selection and credit estimate. Test your application endpoint when you need coverage of its retrieval, tools and safeguards.
- Run the tests. FilterPrompt sends adversarial prompts and evaluates the responses against the selected criteria.
- Open the report and inspect each failing prompt, response, severity and evaluation reason. Manually confirm important findings; mark false positives with a reason when appropriate.
- Record remediation work with an owner in your existing issue tracker. Make changes in your application and retest the affected cases.
What a scan can and cannot establish
Results describe the tested endpoint, configuration and prompts at that point in time. A passing scan is not a guarantee of security, certification or legal compliance. Testing a raw provider model does not test the access controls, retrieval sources or tools in your application. Test limits and provider costs must be agreed before running an assessment.