Black box test or code review?

Last reviewed 2026-09-30Written by Jacob Masse, TrazTech Inc.

A black box test attacks the running app without seeing the code, the way an outsider would. A code review reads the source for missing checks and unsafe patterns. Black box proves what is exploitable; code review finds issues faster and points to the exact line. For AI-built apps, sharing the code usually lowers the cost for the same coverage, and most small engagements combine both.

Compared

Which to choose

If a buyer wants a pentest report, you need the running-app test. If you want the most findings per dollar, share the code as well. PCI DSS 6.2.3 requires custom code review by someone other than its author before release.

Reviewing AI-generated code specifically

Reviewers of AI-built apps look for patterns the tools produce repeatedly: data access functions that take a user ID as a parameter rather than reading it from the session; queries without an organization filter; secrets read from environment variables that the framework ships to the browser; error handlers that return stack traces; generated admin routes; and duplicated logic where one copy has a check and another does not. The last is common, because tools regenerate similar code in several places rather than reusing a helper.

A review also checks the database rules directly, since many AI-built apps enforce access there rather than in code. Then the black box test confirms which of the code issues are reachable from outside, which keeps the report focused on what matters.

Getting it checked

TrazTech offers vibe-coding QA and security review, listed from $2,000 CAD. Get at least one other quote on the same scope; the questions to ask a testing firm help compare them.

Related questions

• What do I need to give the testers?
• Is an automated scanner enough?
• Reviewing AI-written code from Cursor

Common questions