How to read a security test report
A test report is a to-do list ordered by risk. Read the summary, fix the critical and high findings first, and give each finding to your AI tool or developer with its reproduction steps.
Start with the executive summary, which says what was tested, when, and how bad the worst findings are. Then work down the findings by severity: critical and high first, then medium. Each finding should have a description, the steps to reproduce it, the impact and a recommended fix. Give the steps and the fix to your AI tool or developer, then ask the tester to confirm the fix. Findings you decide not to fix should be accepted in writing, not ignored.
What each section means
- Scope
- What was tested and what was not. A finding cannot exist outside it, so check it covers what your customer cares about.
- Executive summary
- The overall picture for non-technical readers: the worst issues and whether the app is fit to launch.
- Severity
- Critical, high, medium, low, informational. Often based on CVSS plus the tester's judgement of real impact in your app.
- Reproduction steps
- The exact requests or clicks that show the problem. The most useful part for fixing.
- Recommendation
- What to change. Good reports name the file, endpoint or policy.
- Retest status
- Whether the fix was verified. The column a buyer or auditor reads first.
What to fix first
| Severity | Typical example in an AI-built app | Fix within |
|---|---|---|
| Critical | Any user can read every customer's data | Before launch, or immediately if live |
| High | A user can change another user's records | Days |
| Medium | No rate limit on login | Weeks |
| Low | Verbose error messages | Next release |
| Informational | Missing security header | When convenient |
Handing findings to your AI tool
- Paste one finding at a time: the description, the reproduction steps and the recommendation.
- Ask for the fix and for a test that proves the fix, such as user A failing to read user B's record.
- Reproduce the original steps yourself. If they still work, the fix did not close it.
- Send the list back to the tester for retest.
Why the AI's own confirmation is not enough is on can AI fix its own security bugs. For the report's use with customers, see using the report for SOC 2.
Need a test that produces a report like this
Tell us what you built and who will read the report.
Get matchedCommon questions
Can I share the report with customers?
Usually not the full one, because it describes how to attack you. Share the executive summary or a summary letter confirming scope, dates and retest.
What if I disagree with a severity?
Ask the tester to explain the impact in your app. Severity should reflect real consequences, and good testers will adjust with a reason.