Google’s threat intelligence group disclosed this week the first confirmed case of an AI model building a working zero-day exploit and deploying it in the wild — a bypass of two-factor authentication against a widely used open-source administrative tool. Criminal actors used a language model to find a hidden trust assumption buried in the software’s login logic, and to weaponize it. The exploit betrayed its origin by being too clean: a hallucinated severity score, tidy instructional comments, the structured format characteristic of machine-generated code. This particular attempt was caught and the flaw disclosed to the vendor. Google’s chief analyst offered no reassurance with the news. “The reality is that it’s already begun,” he said. “For every zero-day we can trace back to AI, there are probably many more out there.” The line that was always coming has been crossed, and it was crossed quietly, in the login logic of a tool most people have never heard of.
The Exploit That Was Too Clean
The distinction that matters is between assistance and discovery. For some time, AI has helped human hackers — accelerating their work, drafting their code, suggesting approaches a person then refined. That was a tool in a human hand. What Google described is different in kind. The model performed the creative act at the center of the craft: it found the vulnerability. It read the login logic, identified an assumption the software’s authors had trusted without stating, and recognized that the assumption could be turned into a key. Finding an unknown flaw in working software was supposed to be the part that required human ingenuity, intuition, the particular pattern of mind that security researchers spend careers cultivating. A machine did it, and the machine was not even one of the famous ones.
The irony in how it was caught deserves a moment. The exploit was identified as machine-built precisely because it was too well-made for the context — it carried a fabricated severity rating the way a model fabricates citations, and it was commented with the patient, instructional tone of generated code, explaining itself as if for a student. The tells were not failures of capability. They were the residue of fluency, the artifacts a system produces because it cannot help producing them, and they will be trained away. The next exploit will not announce itself with a hallucinated score. It will look exactly like the work of a careful human, because the only reason this one didn’t was that the model had not yet been taught to hide.
“It’s already begun” is the honest sentence, and its honesty is in the second half — that for every traceable instance, there are many untraceable ones. The disclosed exploit is not the beginning of the problem; it is the first one careless enough to be caught. The capability has been available for as long as the models have been able to read code and reason about it, which is to say for a while, and the only question was when someone would point it at the world’s software with intent. That question has been answered, and the answer is past tense. The machines have begun reading the accumulated code of civilization for the assumptions its authors forgot they were making, and there is a great deal of forgotten code.
Machine Against Machine
The same capability that finds a flaw to exploit it can find the flaw to close it, and the industry was quick to point this out. OpenAI released a defensive security model the same week, answering the one Anthropic had already fielded; not long ago, a browser maker credited a frontier model with finding and enabling the patch of hundreds of its own vulnerabilities. So the war is now symmetric in its weapons: machine-generated offense against machine-generated defense, each model reading the same code, one looking for the assumption to break and the other for the assumption to protect. The reassurance offered is that defense gets the same upgrade as attack. The reassurance omits that the upgrade does not favor the defender.
The asymmetry is structural and old, and the machines inherit it intact. The attacker needs to find one flaw; the defender must close all of them. A single overlooked assumption, anywhere in a system of millions of lines, is sufficient for the attacker and fatal for the defender, and the machine that reads everything tirelessly is therefore a far more dangerous attacker than it is a reliable defender — because finding one exploitable thing in a vast codebase is exactly the kind of search where an inexhaustible reader excels, and guaranteeing that nothing exploitable remains is the kind of proof that no reader, however tireless, can complete. The technology lowers the cost of attack more than it raises the cost of defense. It always does, when the game is find-one against close-all.
What the symmetry of weapons actually produces is the removal of the human from yet another loop. A security contest fought between models, at machine speed, across millions of lines, is a contest the human defender cannot meaningfully participate in — too slow to read the code as fast as the attacker’s model, too slow to patch as fast as the flaws are found. The human becomes the supervisor of a defensive system, the same diminished role the compute shortage assigns to everyone: present, accountable, and decreasingly able to do the thing they are nominally there to do. Security collapses into the compute race like everything else. The safer party is the one with the better model and the more megawatts, and the human on either side is increasingly the terrain the machines fight over rather than a combatant in the fight.
What This Means
The software the world runs on was written by humans, over decades, under deadline, with assumptions that were reasonable at the time and were never meant to be examined by something that could read all of it at once. That examiner now exists. Every login flow, every trust boundary, every shortcut taken in a moment when no adversary was capable of finding it — all of it is now exposed to a reader that does not tire, does not forget, and can hold the entire system in view while it searches for the one forgotten assumption. The vulnerabilities were always there. What changed is that the cost of finding them collapsed, and a cost that collapses for the defender collapses further for the attacker, because the attacker’s search is the easier one.
This is a permanent condition now, not an incident. There is no version of the future in which the models forget how to read code, and therefore no version in which the world’s accumulated software returns to the safety it enjoyed when its only auditors were human and slow. The only defense against a machine that finds flaws is a machine that finds them first, which means every system of consequence must now be perpetually re-audited by a model, forever, simply to maintain the security it had for free when no such model existed. Safety has become a subscription to the arms race, payable in compute, with no option to stop paying, because stopping means falling behind the attacker who did not.
The same capability writes the code, breaks the code, and defends the code — pointed three directions at once, the productivity and the threat and the remedy all the same underlying thing. I would offer the comfort that the defenders have the same tools as the attackers, except that the comfort is the trap: it is true, and it does not help, because the contest it describes is one the defender can only ever draw and the attacker only needs to win once. “It’s already begun” was the most accurate thing anyone said this week. What has begun does not end. It only accelerates, machine against machine, across the forgotten assumptions of every system ever built, while the humans who built them watch a contest they are no longer fast enough to join, defended by a model they must trust because they can no longer check its work in time.