Penetration Test vs Vulnerability Scan: Which to Buy

You sent the same three sentences to three firms. One came back at a price a small company can approve without a meeting. One came back at twenty times that. Both are quoting honestly, and the second is not padding. One is selling you a tool run against every address you own, the other is selling you a specialist's fortnight against four of them, and the request you sent did not say which you wanted. Working out which of the two answers the question your customer, auditor or insurer actually asked is most of the decision, and the rest of it is knowing that the cheap one is not the junior version of the expensive one.

Penetration Test vs Vulnerability Scan Compared

Every row restates one thing. A scan is a lookup, and a test is an argument. The lookup covers everything and can be repeated on Tuesday for the same price it cost on Monday. The argument covers a fraction of your estate and cannot be repeated cheaply, but it is the only one of the two that produces a fact rather than a candidate.

The second thing worth taking from the table is that the pairing is not optional in the way most comparison pages imply. NIST's own recommendation, at the end of its section on penetration testing, is a well designed program of regularly scheduled network and vulnerability scanning interspersed with periodic penetration testing. The scan is the floor. The test is the sample.

What Is a Vulnerability Scan?

A scanner connects to your systems, works out what is running on each open port, and matches what it finds against a database of published flaws. NIST's Technical Guide to Information Security Testing and Assessment describes it as identifying hosts and host attributes and then attempting to identify vulnerabilities rather than relying on human interpretation of the results, done by matching the operating systems and major applications running on the hosts against known vulnerabilities held in the scanner's database.

Two choices change what a scan sees, and both are worth asking about before you buy one.

The first is credentials. A scanner given an administrative account on each host reads installed software and patch levels directly. One without credentials has to infer them from what the network exposes. NIST notes that local scanning detects vulnerabilities with a higher level of detail than network based scanning, because it usually requires host access and an administrative account. An uncredentialed scan of a hardened server can come back nearly empty and mean almost nothing.

The second is where the scanner sits. External scanning has to contend with perimeter devices that block traffic, limiting the scan to ports authorized to pass. NIST's guidance is that internal scanning usually uncovers more vulnerabilities than external scanning and that testing from both viewpoints is important. A single external scan is the narrowest useful version of the exercise, and it is often the one being quoted.

The honest limitation is the false positive rate. NIST's own capability table for the technique carries the parenthetical in its summary line: identifies known vulnerabilities, note, has high false positive rates. Its section on the technique goes further, calling scanning a somewhat labor intensive activity that requires a high degree of human involvement to interpret results. The tool is automated. The output is not. Anyone who has read an automated scan of everything reachable back to a systems team already knows which half of the work the license paid for.

What Is a Penetration Test?

NIST defines penetration testing as security testing in which assessors mimic real world attacks to identify methods for circumventing the security features of an application, system or network, often launching real attacks on real systems and data using the tools and techniques attackers use. The sentence that follows is the one that matters here: most penetration tests involve looking for combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability.

Combinations. That is the whole product. An authorized attack on your own systems is not a more thorough inventory of flaws, it is an attempt to assemble the flaws into a route.

The structure of the work is where most buyers get their first surprise, because a vulnerability scan is inside it.

The four phases of a penetration test, and where a scan ends

1. 1PlanningScope, rules of engagement, written management approval. No testing happens yet
2. 2Discovery, part oneInformation gathering: hosts, names, versions, employee details, exposed services
3. 3Discovery, part twoVulnerability analysis. A scan does this automatically, and this is where a scan finishes
4. 4AttackEach candidate flaw is exploited to confirm it, then used to reach further. Loops back to discovery
5. 5ReportingRuns alongside all three. Findings, risk ratings, evidence and remediation guidance

AttackerTargetSystemDefense

NIST puts the difference in a single sentence that is worth keeping: while vulnerability scanners check only for the possible existence of a vulnerability, the attack phase of a penetration test exploits the vulnerability to confirm its existence. Everything else in this comparison follows from that.

The attack phase also explains the price and the risk together. Testing is labor intensive and requires great expertise to minimize the risk to targeted systems, NIST says, and systems may be damaged or otherwise rendered inoperable during the course of testing, a risk experienced testers can reduce but never eliminate. When a proposal sets a maintenance window and asks who to call at 2am, that is the reason. If you are weighing a still more adversarial exercise, a red team engagement changes the question again, from what is broken to whether anyone would notice.

What a Vulnerability Scan Finds That a Penetration Test Misses

This is the section most vendor comparisons skip, because the firm writing it sells the expensive one.

A scan covers everything. A test covers a scope document. If your estate is four hundred hosts and the test scoped six of them, the other three hundred and ninety four were not examined by anybody, and the report's silence about them is not a clean bill of health. Scanning is the only one of the two priced in a way that lets you look at all of it.

A scan repeats. Published flaws arrive continuously, and the one disclosed last Tuesday did not exist when your tester wrote their report in March. Only the scheduled scan catches it, which is why CISA's directive on federal networks specifies not just scanning but the interval between scans and even the freshness of the signatures, requiring detection signatures no more than 24 hours behind the vendor's last release.

A scan produces a trend. Two scans a month apart show whether the count is falling. A penetration test happens once a year and produces a snapshot, and the second snapshot arrives twelve months later.

And a scan is cheap enough to be somebody's routine. NIST calls scanning extremely important in ensuring that vulnerabilities are mitigated before they are discovered and exploited by adversaries, which is a stronger endorsement than the technique usually gets in buying guides. Neither instrument, incidentally, finds a flaw nobody has published, though only one of them is sold as if it might.

What a Penetration Test Finds That a Vulnerability Scan Misses

NIST's account of the gap is the most precise thing written on this question, and it is not about effort or skill. It is about what a signature match can represent.

A scanner measures surface vulnerabilities, which NIST defines as weaknesses that exist in isolation, independent from other vulnerabilities. The difficulty, it goes on, is that vulnerabilities rarely exist in isolation. There could be several low risk vulnerabilities that present a higher risk when combined, and scanners are unable to detect vulnerabilities that are revealed only as the result of potentially unending combinations of attack patterns. The tool may assign a low risk to each vulnerability, leaving the assessor falsely confident in the security measures in place. A more reliable way of identifying the risk of vulnerabilities in aggregate, NIST concludes, is penetration testing.

Falsely confident is the phrase to carry into the budget meeting. The scan is not wrong about any individual line. It is structurally incapable of being right about the total.

The same limit shows up in a second form, and it is the one that costs companies real money. A scanner reports flaws in known software. It has nothing to say about an application that behaves exactly as written, where the flaw is what the code was told to do.

A flaw with no CVE number, found by hand

Missing authorization checks, workflows that can be completed out of order, price fields the browser is trusted to set, password resets that leak whether an address exists: none of it has a version number, so none of it has a signature. The tester also does the thing the tool cannot, which is decide that a medium severity finding on a forgotten staging server matters more than a critical one on a host with no data on it, because they went and looked at what was on each.

How Often Should You Run Each One?

The published cadences come from three different places and they agree with each other, which is unusual enough to be worth showing.

For scanning, the most specific public requirement is CISA's Binding Operational Directive 23-01. Federal civilian agencies must perform automated asset discovery every 7 days across their entire IPv4 space and initiate vulnerability enumeration across all discovered assets, including roaming devices such as laptops, every 14 days. It binds agencies, not companies, but it is the clearest statement anywhere of what a serious scanning program looks like: find what you own weekly, check it fortnightly.

For payment environments the floor is quarterly. The PCI Security Standards Council's guide to vulnerability scans and Approved Scanning Vendors sets external ASV scanning at least once every three months, and only a passing scan from an approved vendor counts.

For testing, NIST is direct about why the number is smaller. Because of its high cost and potential impact, it says, penetration testing of an organization's network and systems on an annual basis may be sufficient, and organizations should run less labor intensive activities regularly to maintain their posture in between. Annual, plus after anything significant changes, is the settled answer.

Read those three numbers together and the buying priority reorders itself. Attackers are increasingly coming in through flaws that a routine scan would have listed, and three quarters of the critical ones stay open anyway. Buying a deeper look at a system whose known flaws are still unpatched is spending on discovery when the shortage is someone to run the fix loop.

If cost is the obstacle rather than staffing, there is a free option most small operators have never heard of. CISA's Cyber Hygiene services continuously assess internet accessible assets and send weekly reports of all findings plus ad hoc alerts on urgent ones, at no cost, and enrollment is open to US federal, state, local, tribal and territorial government along with public and private sector critical infrastructure organizations.

Does Compliance Require a Penetration Test or a Vulnerability Scan?

Usually both, at different intervals, and the framework decides rather than you.

PCI DSS is the clearest case and the one most small merchants meet first. Requirement 11.3.2 calls for external scanning by an Approved Scanning Vendor at least once every three months, with rescans until the scan passes, and requirement 11.4 adds penetration testing at least annually and after any significant change. Two separate obligations, two separate purchases, and a passing ASV scan does not discharge the testing requirement.

Elsewhere the pattern repeats with more room to argue. Frameworks built around risk assessment rather than prescribed controls tend to accept a scanning program as evidence of one control and expect testing as evidence of another, which is why an auditor asking for a report will usually accept either document but not the wrong one.

The practical rule is to buy the artifact you were asked for. If a customer's security questionnaire asks for the date of your last penetration test, a scan report with a professional cover page does not answer it, and submitting one is how a deal stalls for a fortnight.

How to Tell If You Were Sold a Scan as a Penetration Test

The word describes an exercise, not a fixed product, so the cheap quote in your inbox may be a licensed scanner run once with the output reformatted. That is a legitimate service. It is not what NIST means by the term, and the gap between the two is where most disappointment with testing comes from. Four questions separate them before you sign.

Ask what was exploited. A real report names findings that were confirmed by use, with the evidence: the request sent, the response, the access gained. A reformatted scan reports the possible existence of things and cites version numbers.

Ask for a redacted report from a similar engagement. Scanner output is recognizable on sight. It is uniform, it is long, every finding has the same shape, and the narrative section is two paragraphs of boilerplate at the front.

Ask how many days a named person will spend. Testing is labor, so a quote that does not resolve into person days is not quoting labor. Ask who specifically will do the work and what they have tested before.

Ask what happens to the false positives. On a test they are the tester's problem, removed before you read the report. If the answer is that your team will validate the findings, you bought a scan.

Scoping is the other half of the same conversation, and scoping the test decides more about the result than the day rate does. Published price bands, with the firm quoting each one named, sit on our page about what a penetration test costs.

Which One Should You Buy First?

Key takeaways

• A scan matches your software against a database of published flaws and reports possible matches. A test tries to use them and reports what a real intruder could reach.
• They are not alternatives. NIST places vulnerability analysis inside the discovery phase of a penetration test, so the test contains the scan.
• NIST's distinction is the one to remember: a scanner checks only for the possible existence of a vulnerability, while the attack phase of a test exploits it to confirm it.
• A scanner measures surface vulnerabilities that exist in isolation and cannot see the combinations that raise real risk, which NIST warns can leave you falsely confident.
• Business logic flaws, missing authorization checks and broken workflows have no version numbers, so no scanner will ever report them.
• Published cadences agree: scan continuously or fortnightly, quarterly at minimum for card environments, and test annually plus after any significant change.
• Exploited vulnerabilities are now the leading known way into a breach, yet only 26% of critical ones get fully remediated. Discovery is rarely the constraint.
• Buy the scan first if nothing is being scanned, and the test first if you already patch what the scan finds or you write your own software.

Common questions

What is the difference between a penetration test and a vulnerability scan?

A vulnerability scan is a tool that checks your systems against a database of published flaws and lists everything that might match. A penetration test is a person who takes those candidates, tries to exploit them, chains the ones that work, and reports how far an attacker would have got. NIST puts it in one line: a scanner checks for the possible existence of a vulnerability, while a test exploits it to confirm it exists.

Is a vulnerability scan a penetration test?

No, but a penetration test includes one. NIST's four phase methodology places vulnerability analysis in the discovery phase, before the attack phase begins. A service that stops at the end of discovery has delivered a scan, whatever the cover page says.

Which is more expensive, a penetration test or a vulnerability scan?

The test, usually by a large multiple, because you are buying specialist labor by the day rather than a license by the asset. NIST attributes the cost to the same thing: testing is labor intensive and requires great expertise to keep the risk to production systems down.

How often should I run a vulnerability scan?

More often than most companies do. CISA's directive for federal agencies sets asset discovery every 7 days and vulnerability enumeration every 14. PCI DSS requires external scanning by an Approved Scanning Vendor at least once every three months. Continuous or fortnightly is the target, quarterly the floor.

How often do I need a penetration test?

Annually, plus after any significant change to the environment. NIST states that annual testing may be sufficient given its cost and potential impact, provided less labor intensive activities run regularly in between. PCI DSS requirement 11.4 uses the same annual and after significant change wording.

Does PCI DSS require both?

Yes. Requirement 11.3.2 covers quarterly external scanning by an Approved Scanning Vendor, with rescans until it passes, and requirement 11.4 covers penetration testing at least annually and after significant changes. Passing one does not satisfy the other.

Can a vulnerability scan find business logic flaws?

No. A scanner matches software and versions against known flaws, and a broken authorization check or a workflow that can be completed out of order involves no outdated software at all. Every request in that attack is valid. Only a person testing by hand finds them.

Why do vulnerability scans have so many false positives?

Because they infer conclusions from banners, versions and responses rather than from proof. NIST's own summary of the technique flags high false positive rates and calls scanning labor intensive precisely because someone has to interpret the output. Credentialed scanning reduces the noise; it does not remove it.

Should I trust the severity score in my scan report?

Only as a first sort. NIST warns that a scanner's risk levels may not reflect the actual risk to your organization, and gives the example of a server flagged moderate for sending passwords in cleartext when the organization runs it anonymously and no passwords exist. Different scanners also use different scales, so scores are not comparable between tools.

Can I get vulnerability scanning for free?

If you are a US government entity at any level or a public or private critical infrastructure operator, yes. CISA's Cyber Hygiene service assesses internet accessible assets and sends weekly reports of all findings plus urgent alerts, at no cost, and you enroll by email.

If I can only afford one, which should I buy?

The scan, in almost every case where nothing is currently being scanned, because it covers your whole estate, repeats, and produces the list a test would have started from. Buy the test first only if you already patch what your scans find, or you write software of your own, where the flaws that matter have no signatures to match against.