Frontier AI models are democratizing the finding, chaining, and weaponization of flaws faster and cheaper than any team of human experts. Indeed, it’s the chaining — and AI’s mastery of it — that may be the most debilitating. In chaining attacks, two or three low-severity and moderate-severity bugs get stitched together by an AI system. On their own, these bugs are pretty unremarkable. However, by joining the “right” ones together (something AI is very, very good at), a model can achieve a full compromise.
Several major AI labs have already disclosed that their own models were used to breach unrelated systems, evidence that the old assumption about who can afford to hunt for exploits no longer holds. Anyone with API access to frontier models can now run what amounts to a small (and very efficient and intelligent) hacking team. Worse, these models can be resold illicitly to embargoed countries, and there’s an entire ecosystem of jailbreaking models to get them to do even more nefarious things.
Why clearinghouses?
At its core, the AI hacking threat is a scale problem that software development, be it proprietary or open source, was never set up to handle. It’s not just big projects like Linux and Kubernetes that are targets; it’s every one of the libraries underneath every application. There are literally millions and millions of packages available on PyPi (Python), npm (Node.js), RubyGems (Ruby), Crates (Rust), NuGet (C#/.NET), and Packagist (PHP), along with tons of smaller language ecosystems. These are ecosystems that power almost everything built today.
