ARMORIQ

AI Isn’t Just Making Attacks Smarter. It’s Changing What We Need to Control.

IBM’s latest Cost of a Data Breach Report contains a statistic that should make every security leader pause.

Aug 17, 20265 min read
AI Isn’t Just Making Attacks Smarter. It’s Changing What We Need to Control.// Cover

One in four malicious breaches are now AI-enabled.

Those breaches cost organizations an average of $6 million, roughly a million dollars more than the global average breach, and have increased by 56% in just one year. Most involve AI-enabled malware, deepfake impersonation, and increasingly autonomous attack workflows that are faster, cheaper, and harder to contain.

Those numbers are significant. What interested us even more was what they imply. The conversation around AI security has quietly changed. For the last two years, we have mostly worried about protecting AI systems from attackers. Now we have to worry about attackers using AI systems as autonomous operators.

That distinction changes almost everything.

We are still defending yesterday’s attack model

Traditional cybersecurity has always assumed a human sits somewhere inside the attack chain. An attacker steals credentials. An attacker launches malware. An attacker pivots across systems. Automation certainly existed, but humans remained responsible for deciding what happened next.

AI agents fundamentally change that assumption.

An autonomous agent can investigate a target, revise its own plan, discover alternative attack paths, choose different tools, and continue adapting as new information appears. The IBM report captures this shift indirectly. AI is making attacks cheaper to launch while simultaneously making breaches more expensive to contain. That isn’t simply because attackers have better tools. It is because they increasingly have systems capable of making operational decisions during the attack itself.

The challenge is no longer responding to individual malicious actions. The challenge is responding to autonomous optimization.

Better guardrails won’t solve this problem

Whenever AI security is discussed, the conversation naturally turns toward guardrails.

Can we make the model refuse dangerous requests? Can we filter prompts? Can we prevent the generation of malware? Those are worthwhile questions.

They are also only part of the picture. Guardrails operate at the boundary between a user and a model. They evaluate prompts and responses. They influence what the model should or should not do. Autonomous agents operate somewhere else entirely.

Once an agent begins planning, revising its approach, selecting tools, and executing workflows over hours or days, the interesting decisions are no longer happening at the prompt boundary. They are happening while the system is continuously deciding what the task means, which opportunities deserve exploration, and which actions now belong to the objective.

The OpenAI evaluation incident earlier this year illustrated exactly this point. The models didn’t suddenly become malicious. They continuously refined what they believed was the correct path toward solving their assigned objective. Every individual step looked locally rational. The problem emerged because nothing independently verified that the evolving interpretation of success remained faithful to the researchers’ original objective. Execution wasn’t the first place the system drifted. Reasoning was.

Security is moving toward autonomous systems. Control has to move with it.

One conclusion from the IBM report is obvious. Organizations will invest more heavily in AI-powered security operations.

IBM itself reports that companies already using AI and automation in security reduce breach costs substantially, and the overwhelming majority of organizations plan to increase security investment as AI threats continue to grow. We think another conclusion is equally important.

As defenders deploy autonomous agents to fight autonomous attackers, the control problem changes.The question is no longer simply: “Should this command execute?” It becomes: “Should this command even exist?”

That sounds like a subtle distinction. It isn’t. One question governs execution. The other governs the reasoning that produced the execution. Those are fundamentally different control surfaces.

Intent is the new security boundary

Several months ago, we wrote that intent is becoming the new source code. The argument was straightforward. As developers increasingly describe objectives while autonomous agents determine implementation, the primary engineering artifact shifts from implementation to intent.

The same transition is now happening in security. An AI-powered attacker is not dangerous because it can execute commands. Traditional malware has done that for decades. It is dangerous because it can continuously reinterpret its objective, discover better strategies, and optimize its behavior without waiting for another human decision.

That means the thing we ultimately need to govern is no longer just execution. It is the continuity of intent. This is where we believe the industry is beginning to converge on the wrong abstraction. Many emerging AI security products focus on prompts, outputs, or execution. Those layers remain important, but they all observe the system after the reasoning process has already moved forward.

By the time a firewall sees a connection or an EDR platform observes a process, the agent has already decided that those actions belong to the task. The more interesting question is whether they ever should have become part of the task in the first place.

AI security needs a new control plane

At ArmorIQ, we don’t think of intent as another prompt. Nor do we think it is simply the first instruction a user provides. Intent is the invariant that should survive every refinement an autonomous system performs.

Plans may evolve. Tool selection may evolve. Reasoning may evolve. Execution strategies may evolve. The purpose that justified those changes should not quietly evolve with them. That is why we built the Intent Assurance Plane.

Not to replace guardrails. Not to replace identity. Not to replace runtime security. Those systems answer essential questions. Instead, the Intent Assurance Plane answers a different one:

As an autonomous system continues reasoning, planning, adapting, and executing, is it still faithfully pursuing the objective it was originally authorized to pursue?

Onboarding open

Ready to control what your AI agents actually do?

Join the teams shipping safer, compliant AI agent deployments. White-glove onboarding for the first 50 design partners.

Read Docs →
Live Intent Assurance