The annual pen test is a relic of the past
AI has let software teams move at a pace that would have been unimaginable a few years ago. But as delivery adapts to match that pace, security testing risks falling dangerously far behind.
Are we over-reacting?
It doesn't feel that way once you look at what's actually changed. The annual pen test was built for a different era of software development. Codebases changed slowly. Releases were infrequent. A test scoped and delivered once a year could reasonably cover the product, because the product hadn't moved much since the last one. That way of doing things no longer exists.
AI-assisted development has compressed release cycles dramatically, and AI-native teams push it even further: teams are shipping more code, more product, more often. DORA's 2025 State of AI-assisted Software Development report found AI adoption is lifting delivery throughput, but incidents per pull request are up 242.7% year on year – the same acceleration that ships more features also opens more ways for something to break. Core systems that once took months to refactor can be restructured in days, authentication flows get rewritten overnight, and new services or third-party integrations get bolted on, sometimes all in the same sprint.
Every one of those changes opens a potential security gap, and every gap between tests is time a vulnerability can sit undiscovered. A pen test scoped against last quarter's codebase is testing something that's already moved on: the findings can be entirely accurate and still land too late to matter. Cadence is the real issue here, not testing spend – even with an unlimited budget, running more tests more often just produces more snapshots, and each one goes stale the moment the next release ships. Agentic Penetration Testing works on a different principle: a programme that never stops looking. The findings can be entirely accurate and still land too late to matter.
How Agentic Penetration Testing works
This model pairs AI agents that carry out testing tasks with expert human testers who direct those agents and validate what they find. Unlike a traditional pen test, which is a point-in-time exercise, this one moves at the same speed as the teams shipping the product. We call this Agentic Penetration Testing, sometimes known more broadly as penetration testing as a service, or PTaaS.
It works in two layers. The foundation is expert human testing: manual deep-dives that hunt for business logic flaws, authentication design weaknesses and chained exploits, the kind of issues no agent can find on its own.
Agents extend that coverage in between. Working to a scope a tester has set, they handle reconnaissance, scanning and known attack patterns on a regular cadence, weekly, monthly or triggered by a deployment, flagging new exposures and known vulnerabilities as they appear so nothing sits unseen for months at a time.
Agents surface findings, but turning a finding into a verdict still takes a person: false positives are common, and severity ratings are often wrong without business context attached. So under this model, every agent-surfaced finding is validated by a pen tester before it reaches the customer, on top of the tester's own manual work. Agents handle breadth and frequency; testers handle depth, context and the judgement calls a machine can't make. The result is a testing programme that moves at the same speed as the product, all year round. Every agent-surfaced finding is validated by a pen tester before it reaches the customer. Agents handle breadth and frequency; testers handle depth, context and the judgement calls a machine can't make.
When is the right time to implement Agentic Penetration Testing?
The honest answer: as soon as your codebase changes faster than your test cycle. That's most teams. Frequent releases, AI tooling threaded through the SDLC or a regulated sector under constant scrutiny: any one of those is enough to leave a snapshot-based model with too much uncovered for too long.
There's a capacity argument too. You cannot hire your way to continuous coverage; there simply aren't enough experienced pen testers in the market to do it by headcount. ISC2's 2025 Cybersecurity Workforce Study names penetration testing and offensive security as the role band where the senior-talent shortage is most acute, and where the cost of a misfire is highest. A model that pairs agents with expertise scales to match delivery cadence in a way that adding people to a roster never will.
Organisations that have made the shift tend to cite the same three gains: findings surface faster, internal teams spend less time triaging noise and remediation progress becomes something you can track continuously. Cobalt's State of Pentesting 2025 report found continuous programmes get organisations to remediation 50% faster and cut time-to-report by 2.6x against traditional, point-in-time testing.
What this means for engineering and security leaders
Most organisations already know testing has to move at the same speed as their development. The harder question is how fast they can get there.
None of this replaces secure engineering practice. Shifting left is still on the team writing the code. But even the most disciplined teams need a safety net that matches how fast they ship.
AI-accelerated development isn't going to slow down. If anything, it accelerates further as adoption widens, models improve and agents take on more of the work themselves, testing included. The products engineering teams are responsible for will keep changing at pace, and a programme built for a slower world will keep falling further behind it.
The annual pen test made sense for its time. For most development-led organisations in 2026, that time has passed.
