Red Team vs Penetration Test: What Differs and Which to Buy

Ask three firms to test your security and you get three proposals that share almost no vocabulary. One is a two-week examination of a named web application. One wants to phone your staff, walk into your building and stay quiet for a quarter. Neither the prices nor the questions are in the same range, and both proposals are honest. They are answers to two different requests hiding inside one sentence, and this decision is working out which request you made.

Red Team vs Penetration Test Compared

Every row restates one difference. A penetration test measures your systems. A red team engagement measures your organization, by attacking your systems. In the second case the systems are the instrument, not the subject.

That is why the last two rows matter more than the first nine. The finding "nobody noticed" is worth paying for only if you had reason to expect somebody would. A company with no alerting and no rota already knows the answer, and can spend the same money on the thing whose absence produced it.

What Is a Penetration Test?

NIST gives the definition of record. Its glossary entry for penetration testing, taken from SP 800-115, calls it security testing in which evaluators mimic real world attacks in an attempt to identify ways to circumvent the security features of an application, system or network. The operative word is identify. The output is a set of ways in, each demonstrated rather than asserted, which is what separates an authorized attack on your own systems from a report generated by a scanner.

The scope document is the product as much as the report is. The General Services Administration's procedural guide for conducting penetration test exercises names fourteen distinct types, from network and web application testing through wireless, physical, cloud, serverless and social engineering testing, and requires a rules of engagement template signed by key personnel before testing starts so the limitations and liabilities are understood on both sides. Fourteen types means fourteen possible answers to "test our security", and quotes diverge because firms are quietly choosing different ones.

Within that frame the tester's incentive is breadth. Paid to examine an application for two weeks, a good tester wants every exploitable path in it, ranked, with enough detail that a developer can close each one. Being noticed costs nothing, and loud scanning is faster than quiet scanning, which is why scoping a penetration test is the part of the purchase that decides what you get.

What Is a Red Team Engagement?

The formal definition is narrower than the marketing. The NIST glossary carries the CNSSI 4009-2022 wording: a red team is a group of people authorized and organized to emulate a potential adversary's attack or exploitation capabilities against an enterprise's security posture. GSA's guide quotes the NIST SP 800-53 Revision 5 version of the exercise itself, an exercise reflecting real world conditions conducted as a simulated adversarial attempt to compromise organizational missions or business processes, giving a comprehensive assessment of the security capabilities of an organization and its systems.

Emulate an adversary, compromise a mission. Neither definition mentions vulnerabilities, and that omission is deliberate.

CISA, which runs these against critical infrastructure organizations itself, publishes the clearest description of the mechanics. Its red team assessment fact sheet splits the work in two: threat simulation, which emulates advanced persistent threat tactics and intent to access, navigate and persist in the customer's environment while demonstrating business level risk, and measurable events, a series of actions fired once the team is entrenched that are specifically intended to provoke a security response. The assessments, CISA says, are crafted to test the people, processes and technologies defending a network, and that defense is scored with observable, response-driven metrics.

Measurable events are the part almost no vendor page explains, and they are why a red team engagement is a measurement rather than an expensive break-in. Reaching domain admin quietly proves an attacker could. Deliberately making noise afterward, and writing down what happened next, proves whether it would have mattered.

What a Penetration Test Finds That a Red Team Misses

The firms selling red teaming are not the ones to ask about this, so here is a US government publication with no product to sell. NIST SP 800-115, the Technical Guide to Information Security Testing and Assessment, sets out the trade in section 2.4.2, and it is unusually direct.

The purpose of covert testing, it says, is to examine the damage an adversary can cause, and it does not focus on identifying vulnerabilities. It does not test every security control, identify each vulnerability, or assess all systems within an organization. It normally exploits the most rudimentary weaknesses available to gain access, because the cheapest route in is the one an attacker would take, and it usually stops at a defined boundary, such as a certain level of access, which prevents real harm while still showing the harm was possible.

Read that as a buyer and the loss is obvious. A red team that walks in through a help desk call on day three never touches your payment application, and the fifty findings sitting in it stay there. If a customer questionnaire, an insurer or a regulator asked what is wrong with a specific system, a red team answers a different question at several times the price.

NIST names two more things overt testing does that a covert exercise cannot. It lets IT staff give guidance that limits the testing's impact, and it doubles as training, with staff watching the methods assessors use against the controls they built. Both disappear the moment the exercise becomes a secret.

Neither exercise substitutes for continuous work either. An automated scan run weekly catches the newly published flaw in the days before either engagement would have reached it, at a fraction of the price of both.

What a Red Team Finds That a Penetration Test Misses

In 2022 CISA ran a red team assessment over a three-month period at a large critical infrastructure organization with multiple geographically separated sites, and published what happened. The team got in through spearphishing emails to two workstations at separate sites, moved laterally to a misconfigured server, compromised the domain controller and obtained the krbtgt account hash. Multi-factor authentication stopped it reaching one sensitive business system. Throughout the first phase, the advisory records, the team received no deconflictions or confirmation that the organization had caught their activity.

Then came phase two: 13 measurable events, designed to provoke a response from the people, processes and technology defending the network. The advisory prints them next to the reaction each one produced.

CISA's summary of the engagement is one sentence and it is the argument for the category: despite having a mature cyber posture, the organization did not detect the red team's activity throughout the assessment, including when the team attempted to trigger a security response. No penetration test produces that finding, because a penetration test is announced, and an announced attacker cannot measure surprise.

A second advisory shows the same instrument reading differently. CISA ran two concurrent assessments at a government services organization and a water utility. In both, the red team achieved full domain compromise and reached sensitive business systems and cloud resources. Organization A failed to detect or contain any of it. Organization B caught the initial compromise, quarantined the affected systems and forced the team into an assume breach model, where trusted agents handed over access so the exercise could continue. Both were compromised. Only one was beaten, and no vulnerability report would have told either which it was.

Why Organization A missed it is worth the advisory on its own. Its security operations center did receive medium and low severity alerts from endpoint detection and response tooling on the activity and did not act on them, because thousands of false positives from normal business operations, many carrying a higher severity, buried them. CISA's conclusion is the sentence to take into the budget meeting: detection tools are only as effective as the people, processes and procedures supporting them.

Who Knows the Test Is Happening

NIST SP 800-115 draws the line as overt against covert rather than pen test against red team, and the distinction is cleaner that way. Overt testing is performed with the knowledge and consent of the organization's IT staff. Covert testing takes an adversarial approach, without the staff's knowledge but with the full knowledge and permission of upper management. That second clause is not a formality. Unauthorized intrusion is a crime, and the only thing separating the exercise from the offense is a signed authorization from someone entitled to give it.

What keeps a covert exercise from turning into a real incident is the trusted agent. NIST describes organizations designating a trusted third party so the target does not initiate response measures without first verifying that an attack is genuinely underway, mediating between the assessors, management, the IT staff and the security staff. In practice that is two or three named people who hold the phone number, confirm that the traffic your analyst just escalated is the exercise, and stop the company calling law enforcement, pulling production or firing the employee who clicked.

They also make the fallback possible. When Organization B detected the intrusion immediately, its trusted agents provided access to a host replicating what the red team would have had, so the engagement could carry on measuring everything past the front door. Worth knowing before you sign: a red team caught on day two is a success, not a refund, and the contract should say what happens next.

None of it works without an incident response plan that names who decides. Run against an organization with no escalation path, the exercise produces a rediscovery rather than a finding.

How Long Each One Takes and What It Costs

CISA ran both assessments above over roughly three months. That is a federal team with no commercial clock on it, so read it as the shape of the work rather than a price, but the shape is the point: stealth is slow. NIST puts the cost mechanism in one line, saying covert testing is often time consuming and costly due to its stealth requirements, because a team operating in a stealth environment has to slow its scans and other actions to stay under the radar of the target's security staff.

A penetration test is scoped in days or a few weeks because nothing is being hidden. The tester moves at full speed, and the cost tracks the number of systems and the depth of the examination rather than the calendar.

NIST's summary of the trade is the fairest sentence written on the subject anywhere: overt testing is less expensive, carries less risk than covert testing, and is more frequently used, while covert testing provides a better indication of the everyday security of the target organization because system administrators will not have heightened awareness.

Firms publish far more price data for the first exercise than the second, which is why what a penetration test costs can be answered with named quotes while a red team budget cannot. Expect a multiple rather than an increment, because you are buying weeks of a specialist deliberately going slowly. If you run critical infrastructure there is a third route: CISA states its assessments are available to both public and private organizations at no cost, with availability limited and a real queue.

What You Get at the End of Each One

A penetration test report is a list. Each entry names the weakness, the system, the evidence it was exploited, a severity rating and the fix. It is built to be handed to engineers and worked through, and its value is that it can be set against last year's list to show whether anything got better.

A red team report is a narrative. It reconstructs the route from the outside to the objective and sets against it what your defenders saw, when they saw it and what they did. The finding is not "this server is unpatched", it is "we were inside for eleven days, three alerts fired, none was escalated, and here is the one that should have been".

The ends are where the money goes. A third CISA advisory, on an assessment at another critical infrastructure organization, records that the red team's spearphishing payloads ran on target machines without producing access, so the team kept looking and got in through a web shell left behind by a previous vulnerability disclosure program. A prior security exercise was the front door. The same advisory found the organization had relied too heavily on host-based endpoint detection without sufficient network layer protections, and that its perimeter network was not adequately firewalled from its internal network.

Does Compliance Require a Red Team or a Penetration Test?

Almost always the test, which settles the question for a lot of companies before preference gets a vote. PCI DSS requirement 11.4 calls for internal and external penetration testing at least annually and after any significant change, and no US framework in common commercial use makes the same demand of a red team exercise.

The federal picture says it differently. GSA's procedural guide, the document governing testing across that agency, covers penetration testing in detail and notes that GSA was still developing a separate guide for conducting red team exercises. Even a mature testing regime has the penetration testing procedure written down and the red team procedure in draft, not the other way around.

What Is Purple Teaming?

The middle option has a name and it is underused. GSA's guide defines a purple team exercise as a methodology in which red, the attacking side, and blue, the defending side, test cyber capabilities together to build security knowledge and inform improvements through continuous feedback and knowledge transfer.

In practice the attacker and your defenders sit on the same call. The attacker runs a technique, everyone watches whether it produced a log, an alert or nothing, the detection gets fixed, and the technique is run again. It is not stealthy and it does not measure surprise, so it answers a smaller question than a red team does.

It also costs a fraction of one, produces improvements the same week rather than in a report six weeks later, and needs no mature detection program to be worth running, because building that program is what it does. For most companies that read a red team proposal and flinched at the number, purple is the exercise being described in the meeting.

Which One Should You Buy?