Could AI Address the Cybercriminal Skills Gap?
Boy, I hope not, but we need to be prepared for cybercrime's next chess move.
NOTE 1: for reasons explained in my previous essay, I’ll replace the common use of the term ‘ransomware’ with ‘extortion’ in this essay.
NOTE 2: Also, Allan Alford was kind enough to interview me on this topic on the excellent Cyber Ranch Podcast.
I will be blunt because I don’t know how much time we have. I think it’s inevitable that innovations like LLM AI and AutoGPT will lead to the capability for ransomware crews to mostly if not fully, automate their attacks.
This concerns me for three reasons:
Most organizations cannot successfully defend against a targeted attack by an experienced extortion team, which is unlikely to change soon.
The court transcripts from the Joe Sullivan case are one of many signs that the number of highly vulnerable organizations is far higher than the number breached yearly.
And I suspect the reason we don’t have 10x or 100x the number of breaches we have today is simply because there aren’t enough attackers to take advantage of all the opportunities. It’s an odd coincidence that attackers also may have a talent gap.
I’m worried that the availability of generative AI, combined with attack automation in the form of malware and other offensive tooling, could fill this gap. I hope I’m wrong about this, so let’s dive into my reasoning and explore ways I could be wrong.
The Role of AutoGPT
A short primer on AutoGPT is necessary. AutoGPT represents a small and simple but important innovation compared to the much larger innovation that ChatGPT and LLM AIs generally represent. AutoGPT is an open-source Python project that adds two important capabilities to an LLM AI:
the ability to accomplish tasks and access the Internet directly (like OpenAI’s ChatGPT plugins that are still in private beta)
the ability to assign a ‘master’ task with a broad goal, accomplished by breaking down the goal into small tasks and launching additional GPT instances as necessary. Even if these tasks fail, the master task can continue trying different methods indefinitely until the desired result is reached.
We’ll likely see many more open-source AutoGPT alternatives if they don’t exist already, and many projects have built on and expanded AutoGPT. Even if it cannot fully automate a large-scale extortion attack today, it represents the seed of doing so.
In short, AutoGPT is the key innovation that could potentially allow cybercriminals to scale beyond any staff and talent limitations.
Automating a Typical Extortion Attack
There must not be any bottlenecks in the extortion process or massive scaling of these attacks won’t be possible. Let’s explore every major stage of an extortion attack aimed at a large enterprise or organization and consider whether it can be fully automated. It’s important to note that many of these steps are already highly automated, so cybercriminals won’t be starting from scratch.
Initial Access
This is currently the area of most abundance for cybercriminals. Throughout the history of cybercrime, criminals have had more opportunities than they’ve been able to take advantage of.
A tiny percentage of compromised organizations and systems are exploited and monetized. There is evidence for this. One example is the Joe Sullivan court transcripts mentioned in the intro. Investigators found that the pair of attackers blackmailing Uber had procured a list of over 100,000 working GitHub logins. Unable to take advantage of them all, they correlated the logins with website popularity ratings, which led to Uber becoming the attackers’ top target.
Currently, LLM AIs aren’t necessary for initial access. Sadly, there’s no bottleneck for finding easily hackable businesses.
Privilege Escalation
Once attackers gain access to a system, they typically collect as many legitimate credentials as possible and often create new accounts as necessary. LLM AIs know how to perform most, if not all, penetration testing-related tasks, so this won’t be a challenge. Existing malware can already automate most of this process.
It’s worth pointing out that the average extortion campaign looks a lot like a typical penetration test and that most organizations still aren’t capable of stopping or even detecting a typical penetration test.
Lateral Movement and Data Exfiltration
This area is also fully automated by malware today. Whether using exploits or captured credentials, laterally moving between systems on a common network isn’t a challenge for cybercriminals today. Ditto for copying off data to use as an extortion lever to make the targets pay a ransom.
Sending the Ransom Note and Negotiating
This is the primary step that has still been done by humans. Some testing shows that LLM AIs like GPT-4 can easily handle basic negotiations. It’s a very narrow task that only requires a few parameters to be defined.
Getting Paid
This is the only area still serving as a bottleneck for cybercriminals. Law enforcement has teamed up internationally and with private organizations to make it increasingly difficult to use cryptocurrency as a form of extortion payment. This practical issue must be addressed before tasking something like AutoGPT with going out and doing business extortion at scale.
BEC scams don’t suffer from this challenge. However, they are currently far more profitable than extortion and are often simpler to pull off. Much of a BEC scam can also be automated, and the money often goes from the target’s bank account to the attacker’s. There’s no need for cryptocurrency since the idea is to make it look like a legitimate payment to a vendor, or something similar.
Offensive Use of LLM AIs
Couldn’t OpenAI, Google, or Microsoft just detect the use of their products by malicious actors and block them?
Sure, but open-source LLM AIs are quickly catching up to commercial offerings. Some people think open source will quickly pass commercial offerings, as open source work isn’t bound by the same legal, ethical, or financial constraints that apply to a tech company. Cybercriminals built their own tools to ensure their malware won’t get caught by anti-virus decades ago. Today, they have dedicated researchers, engineers, and funding for projects like this.
Defensive Shortcomings
Why are Organizations still so vulnerable? Isn’t security spending at a record high?
Security products don’t defend organizations - the people and processes built around these products do. That’s where most organizations miss the mark. They acquire security products and they hire staff, but the work necessary to make products and staff effective together often doesn’t happen.
This isn’t all on organizations, though. It’s on security vendors as well. There are products on the market that flat out don’t work or don’t produce any significant value - at least not for defense against attacks. The use of security budget is not a useful indicator of risk reduction.
Chaotic Actors
Something that hasn’t been considered yet in this essay is a tool like AutoGPT in the hands of more chaotic actors, and actors with motivations other than money. What if something like Shamoon was used against a dozen affiliated organizations simultaneously? What if Anonymous, Lulzsec, and more recently, Lapsus$, gave AutoGPT some crazy goals and simply unleashed it?
Add to this the concern of alignment, which governs what a generative AI should and shouldn’t do. This is the only source of ethics or morals from a system that is effectively chaotic-neutral by default. Actors could choose to ignore alignment, or worse, intentionally align an LLM AI to be ruthless. Any moral qualms or humanity that malicious actors might have had would no longer be a limitation when AutoGPT is handed the reigns.
There’s some small comfort here in that someone has already done this with ChaosGPT, which hasn’t accomplished much in its’ goal to extinguish humanity. However, we’re effectively at 0.01 Alpha with this technology - it will get much more sophisticated and capable faster than we’ve ever seen in the software world. Especially now that GPT-4 and similar LLMs are already writing a lot of software these days.
Conclusion
In the mid-2010s, ransomware, extortion, and BEC scams caught us all unprepared. Almost a decade later, we still struggle to address these attacks, despite tens of billions poured into attempts to solve these problems. If attackers succeed in scaling these attacks up using AI, the opportunity is there for them to take. If malicious actors can successfully automate the equivalent of a typical penetration test, much damage can be done. Vulnerable organizations that have escaped notice so far or were able to ‘run faster than the bear’ could be out of time.
To avoid overdramatizing things, it’s important to contextualize what “out of time” means. It’s incredibly rare for breaches to be fatal to companies. It’s also not in the best interest of financially-focused actors to hurt their targets more than their insurance policies can afford. Still, extortion and cyber attacks can be disruptive and expensive, even if they don’t kill off businesses.
Hopefully, technologies like AutoGPT are further from this future than they might currently seem. Now is the time to start considering attackers’ next chess move and how we will defend and respond individually and as an industry.