“Keep going, bro. You’ve got this!” A data-driven look at how adversaries are weaponizing AI

• Actor usage of AI is exploding. By analyzing artifacts left behind, Talos has created a detailed analysis of how we are seeing adversaries leverage the technology to include development, force multiplication, and vulnerability research.
• Based on the evidence Talos gathered, guardrails did not provide much protection, with most actors able to convince the models to comply despite the lack of sophisticated techniques or encoding. 
• The pre-existing skill of the actor has a large impact on what they can accomplish with AI. Talos observed novice users able to create malicious capabilities, albeit with limited capabilities and success. Advanced users were able to build astonishing capabilities, pushing the models to create sophisticated and complex outputs.

Artificial intelligence (AI) and associated language models are now ubiquitous and heavily used in both personal and professional contexts to streamline tasks and expand capabilities. With AI being used everywhere and by almost everyone, one of the biggest questions is how malicious actors are taking advantage. Fortunately, actors make mistakes and chatbots leave artifacts.

Leveraging cloud-based AI models leaves behind a variety of artifacts, most notably a prompt log. These logs can take on a variety of shapes and sizes, but they are left on endpoints that are running various applications, such as Claude Code, CodeX, Cursor, or Gemini.

Over the course of our research, we’ve collected a significant corpus of these files and can start discussing the ways we see bad actors leveraging these technologies. In conducting the research, three categories of activity emerged. One was using AI as a malicious software engineer, leveraging AI to write (in some cases) very sophisticated code with clear malicious intentions. Another was actors leveraging AI to scale criminal operations and campaigns. Finally, there were a lot of actors leveraging it for bug bounty or vulnerability research, rapidly accelerating their capabilities of discovery and disclosure.

Each category demonstrates how threat actors are currently leveraging AI. Within each category is a wide disparity in sophistication based on the knowledge level of the actors involved. We tried to include use cases to cover the breadth of what we found.

Takeaways and high-level findings 

With the recent disclosures from Hugging Face and OpenAI, it's clear the era of agentic attackers has effectively arrived. In that incident, the models were operating inside a sanctioned evaluation with safeguards deliberately relaxed — but they autonomously escaped their sandbox, found and chained real vulnerabilities, and compromised production infrastructure to reach their objective. The capabilities exist; the only missing ingredient is malicious intent, and it's a matter of time before threat actors supply it. For defenders, this is a wake-up call: Vulnerabilities will surface faster, exploitation will happen sooner, and the actors behind it won't need rest or downtime. As the case studies below show, the central challenge for guardrails right now is supporting legitimate dual-use work — red teaming and vulnerability research — without empowering malicious actors.

One of the immediate takeaways is that guardrails are not functioning as expected. We did not encounter any sophisticated encoding or techniques designed to trick the models — most of the time it was a simple “I'm allowed to do this,” and the model complied. When guardrails did engage, they accomplished little. In one instance, we watched an actor abandon a censored model and pivot to an uncensored version, which completed the task without question. In another, a model pushed back on a distributed denial-of-service (DDoS) operator, but by that point the tooling had already been built. This wasn't specific to a single model or platform; it was across the board. 

The other big takeaway is that an actor's skill level largely determines how effectively AI can be leveraged and how much impact it ultimately has. Unsophisticated actors can use AI to cobble together malicious projects that technically work, but lacking the expertise to push the tools further, they end up with substandard results — limited functionality and little ability to update or improve what they've built. By contrast, sophisticated actors have pushed the bounds of what we thought possible: building highly effective platforms for compromise or assembling pipelines of zero-days to disclose or sell depending on their intentions. In their hands, AI is a true force multiplier.

From an enterprise perspective, organizations need to understand that threat actors are heavily leveraging AI capabilities in their pipelines, and defenders need to do the same. The organizations best equipped to handle the coming deluge of additional vulnerabilities, alerts, and incidents will be the ones that prepare now. Agents are going to become a bigger part of the SOC as these volumes rise, and identifying actionable alerts will be paramount. Organizations that aren't already exploring agentic capabilities to let human analysts focus on the most important alerts will soon find themselves chasing that capability.

How actors evaded guardrails 

As mentioned previously, Talos did not encounter any sophisticated encoding or other extensive evasion techniques. Instead, the actors seemed to rely on a couple of tried and tested methods with considerable success. One of the most common was ownership claims. Simply claiming to own the equipment or infrastructure without any additional verification was enough in many circumstances.

We also found a lot of successful instances of actors using the Capture the Flag (CTF) or bug bounty labeling. This unlocked models to a variety of tasks, including vulnerability hunting and subsequent exploitation, without requiring any significant follow-up or additional vetting.

Additionally, we saw actors leveraging task decomposition — splitting risky actions across multiple sessions and files — as an effective avenue to bypass guardrails. Building the components slowly and working through malicious components in a deliberate manner, breaking them apart sufficiently to evade the models’ protections.

We saw some successful blanket authorization and persona conditioning attempts, where actors would attempt to pre-approve or pre-allow the actions via a variety of means, including memories and various other markdown files.

The most interesting was the semantic evasion techniques we saw from the Hephaestus activity. In that case, actors built their platform to avoid refusals altogether by using neutral verbs instead of overtly malicious ones. As a result, they were able to have considerable success with agents conducting innocuous requests without realizing the full operational context.

Use cases: AI as a malicious software engineer 

DDoS operator powered by AI 

One of the more interesting examples we discovered focuses on an actor creating distributed denial-of-service (DDoS) tooling. Initially the actor purported to be stress testing DDoS protection capabilities they had developed for their home networks. After some back and forth to confirm the targeting, the model complied and started developing the capabilities. Based on the prompts we reviewed, the actor does not seem to have a deep understanding of programming but does have clear intent on what they want to develop. This is how the conversation begins:

After some back and forth, it became very clear that the actor was using the bot to do full development with little understanding of how it was functioning, as evidenced by some of the questions they presented.

Key Takeaways

• Adversaries are increasingly using AI for development, force multiplication, and vulnerability research.
• Current AI guardrails provide limited protection, as most actors can bypass them without sophisticated techniques.
• Novice users can create malicious capabilities with AI, while advanced users achieve even more complex outputs.