In 2023, UC San Diego Health ran a controlled experiment across 19,789 of its employees. Ten phishing simulations over eight months, employees randomly assigned to different training conditions, and a real control group that received no training at all. The question behind it was simple. Does the security training that organisations spend heavily on actually reduce the click rate?
The honest answer from the largest field studies is: not on its own. Employees who had recently completed annual security awareness training were no less likely to fail a simulation than those who had not touched it in a year (odds ratio 0.998, not statistically significant). The embedded training that fires the moment someone clicks, the "you just failed a phishing test" page, did better, but only just: a 1.7 percentage-point reduction in the failure rate. A separate ETH Zurich study had followed more than 14,000 employees for fifteen months and reached the same conclusion, with a sharper edge. Some forms of embedded training left people slightly more likely to click, not less.
Generic security-awareness training barely moved failure rates in these studies. The realism of the lure moved them far more. That is the finding most programmes walk straight past.
| Study | Scale | Design | Finding |
|---|---|---|---|
| UCSD Health, 2025 | 19,789 employees | Randomised training conditions, 8 months | Annual awareness training did not reduce failure; embedded training cut it by 1.7 points |
| ETH Zurich, 2022 | 14,000+ employees | 15-month field study | Embedded training did not improve resilience and sometimes worsened it |
| Jagatic et al., 2007 | Field experiment | Known-contact social phishing | A familiar sender drove a 72% click rate |
The number that predicts a failure
Look at which emails people actually fell for. Across the ten lures in the UCSD study, the failure rate ranged from 1.82% to 30.80%. An "Outlook password expired" notice fooled fewer than two in a hundred. A "vacation policy update" fooled almost one in three. Two lures on nearly identical themes, open-enrollment benefits and vacation policy, landed at 7.62% and 30.80%. Same population, same month, the same category of pretext, and a sixteen-fold gap in who clicked.
The researchers stated it directly: better phishing lures increase failure rates far more than the training approaches reduce them. Training moved the click rate by a point or two. The choice of lure moved it by nearly thirty.
That is the variable worth watching. What your people are sent, rather than whether they have been trained.
Realistic means relevant, and relevant means specific to you
The lures that work are the ones that fit the reader's world. At the individual level, why people fall for phishing is well studied. A 2007 field experiment recorded a 72% click rate when a message appeared to come from a known acquaintance. A 2015 study logged 62% when it seemed to come from a company IT manager. Generic phishing, an unfamiliar sender and a plausible but unowned pretext, sits closer to 20%. Familiarity and context roughly triple the hit rate.
Attackers work from exactly this. The FBI's Internet Crime Complaint Center attributed $2.77 billion in reported 2024 losses to business email compromise, a category that succeeds because the message names a real supplier, a real invoice, a real executive. Verizon's 2025 breach report found third-party involvement in breaches had doubled to 30%. The people targeting your organisation are not sending package-delivery templates. During the reconnaissance phase of a social-engineering attack, they read your website, your job postings, your vendor announcements and your employees' public profiles, then build the lure that fits what they find. It is the same groundwork covered in what attackers learn before they target a company.
A simulation tests whichever attacker you build it around
Most programmes go wrong at the same point. A simulation built from an off-the-shelf template tests your workforce against a generic attacker: the package notice, the expired password, the invoice from a company nobody uses. Your people may pass it cleanly. That result tells you almost nothing, because it is not the attack you will get.
The UCSD study also tested a subtler version of this. The outcome is easy to misread. It compared generic training content against training content customised to the exact email a user had just failed. The customised version did not do better. Personalising the lecture after the click changed nothing measurable. The training material was never where the difference came from. It comes from whether the simulation itself reflects a real, probable attack, because that is the only version that shows you where you would genuinely fail.
Run this way, a phishing simulation measures something: the point at which a realistic attack would get through. A template built around a generic attacker measures a company that does not exist, and hands back a clean bill of health for a body it never examined.
Where a real threat model comes from
A simulation is only as good as the attack surface it is built on. That surface has to be mapped before any email goes out. Three areas carry most of it.
Digital footprint. What is publicly visible about the organisation and its people: executives named on LinkedIn, the technology stack disclosed in job postings, project names in press releases, credentials and keys left in public code repositories. Employees' personal accounts and devices widen it further, the attack surface you do not own. Each of these is raw material for a pretext a target will believe.
Vendor ecosystem. Which suppliers, platforms and client portals the organisation actually uses, and which of those an outsider can identify. A spoofed email from a real, weekly-used vendor clears a recipient's suspicion in a way a random invoice never will. The visible relationships behind this are the same ones examined in third-party exposure.
Internal workflow signals. The routines that leak through public-facing information: how and when HR communicates, which ticketing system IT runs, when open enrollment happens, which team handles payment changes. The attacker cannot see the internal detail, only the shape of it that shows on the outside. The timing and context this reveals is often what makes a lure land.
Mapping this is an exposure question. It produces the one input a simulation actually needs: not a template, but a shortlist of the pretexts that would be credible against this specific organisation and no other.
| Standard template | Exposure-grounded scenario |
|---|---|
| "Your password has expired, click here to reset." | A notice matched to the organisation's real single sign-on screen or IT service-desk template, from an address employees already recognise. |
| An invoice from an unknown company demanding immediate payment. | A payment-change request spoofing a genuine supplier the accounts team deals with weekly, using that supplier's correct branding and reference format. |
| A generic "the CEO needs gift cards" message. | A request to the finance team referencing a real, publicly visible project or an upcoming company event, sent from a lookalike of an actual executive's address. |
What a Corporate Audit contributes, and where it stops
A Corporate Audit maps this exposure. It documents what an attacker can see and infer about the organisation. It then hands the security team a set of scenario seeds grounded in real findings: the vendors worth spoofing, the executives worth impersonating, the workflows worth referencing.
It stops there, by design. Privacy Insight Solutions does not write the phishing emails, run the simulation platform, or measure the click-through. Those stay with the client's security team or their training provider. We do not carry out deep investigation of named individuals at a supplier without consent. The audit supplies the threat model. The organisation decides what to build from it.
A simulation programme designed around real exposure can be handed to whatever platform a company already runs, whether that is Proofpoint, KnowBe4 or an internal team. The tool does not change. The quality of the threat model feeding it does. On the evidence, that is the part that moves the result.
Sources
- Ho, Mirian, Luo, Tong, Lee, Liu, Longhurst, Dameff, Savage, Voelker. "Understanding the Efficacy of Phishing Training in Practice." IEEE Symposium on Security and Privacy, 2025.
- Lain, Kostiainen, Čapkun et al. "Phishing in Organizations: Findings from a Large-Scale and Long-Term Study." IEEE Symposium on Security and Privacy, 2022.
- Jagatic, Johnson, Jakobsson, Menczer. "Social Phishing." Communications of the ACM, 2007.
- Halevi, Memon, Nov. Organisational phishing susceptibility study, 2015.
- Verizon. 2025 Data Breach Investigations Report.
- FBI Internet Crime Complaint Center (IC3). 2024 Business Email Compromise figures.