In this thought-provoking episode of Simply Offensive, host Philip Wylie sits down with Jacob Krell, a penetration tester and researcher at Suzu Labs. Jacob recently authored a white paper titled The Death of the CTF, exploring how the rapid rise of agentic AI is fundamentally altering the landscape of Capture The Flag (CTF) competitions and the broader security industry.
From 17% decreases in solve times to the emergence of "Hacking as a Service," this conversation is a wake-up call for anyone in the field.
The Data: A 17% Year-Over-Year Crunch
Jacob’s research into over 500 Hack The Box machines revealed a startling trend: a roughly 17% year-over-year decrease in "Root First Blood" times [02:16].
While player skills are naturally improving, Jacob argues that AI is the real driver behind this acceleration. He describes the current state of CTFs as reaching a "reckoning point" where the traditional human-only infrastructure can no longer keep up with the scale of AI.
AI as a Force Multiplier
Jacob doesn't view AI as a replacement for the hacker, but as a massive "force multiplier."
- Cognitive Offloading: Instead of manually researching CVEs or SSH version numbers, Jacob uses AI to handle the reconnaissance and research.
This allows him to focus his "human" brainpower on higher-level strategy and complex problem-solving.
- Agentic Hacking: Jacob has moved beyond simple scripts to using agents that can execute entire workflows—from port scanning with Nmap to full privilege escalation enumeration.
- System Engineering: The role of the pentester is shifting from being a manual "operator" to a "system engineer" who pilots and directs these AI agents.
The End of the Human Benchmark?
For decades, CTFs have been the gold standard for measuring human technical competence. Jacob believes that era is ending.
- The "Calculator" Analogy: Just as calculators didn't stop people from learning math but changed how we do it, AI is changing the benchmark for security skills.
- Anti-Cheat vs. AI-Buttons: Jacob suggests platforms may need to take a cue from online chess by implementing anti-AI detection or, alternatively, providing an "I solved this with AI" button to track metrics accurately.
- Nation-State Benchmarking: In the future, global CTF platforms could serve as battlegrounds to benchmark the AI capabilities of different nations (e.g., how America's Claude performs against China's DeepSeek).
Real-World Risks: Hacking as a Service
The implications of Jacob's research extend far beyond the lab.
- Exploding Threat Profiles: With AI agents, a malicious actor can set up a system to scan and exploit vulnerable internet-facing services in minutes.
- Commercialized Malice: Jacob predicts a shift from "Ransomware as a Service" to "Hacking as a Service," where attackers sell pre-configured AI agents and "cursor rules" to automate complex breaches.
- The Death of Attribution: If an agent spawns another agent to carry out an attack, determining legal responsibility and attribution becomes a nightmare for policymakers.
Jacob’s Advice for the Next Generation
Despite the "death" of the traditional CTF, Jacob remains technology-forward.
- Learn the Fundamentals: You still need to understand how to do things manually for when the "calculator" goes wrong.
- Personalized Tutoring: Use AI as an on-staff tutor to explain complex topics like multi-vector calculus or binary exploitation.
- Focus on Systems: When solving a problem, don't just ask "how do I fix this?" Ask "how can I solve this so I never have to deal with it again?".
Resources Mentioned:
- Jacob Krell’s White Paper: Look for it on the Suzu Labs website.
- Hack The Box Academy: Jacob’s top recommendation for hands-on keyboard learning.
- Project Artemis (Stanford) & DARPA AI Cyber Challenge: Real-world examples of agentic hacking frameworks.
Watch the full episode: AI Killed the CTF Star with Jacob Krell