Gary Marcus made a provocative argument this week: perhaps it is time to boycott generative AI.
His concern is understandable. Generative AI remains probabilistic and unreliable. It hallucinates. It behaves inconsistently. Agents can pursue objectives in unexpected ways. Yet we are rapidly connecting these systems to browsers, terminals, enterprise applications, payment systems, infrastructure, and increasingly the physical world.
His response is essentially: stop giving unreliable AI more power until we know how to make it trustworthy.
Gary Marcus: The Case for Boycotting Generative AI
We agree with more of that diagnosis than people might expect. But boycotting AI does not solve the underlying problem. Eventually these systems will be used because they are useful. The question is what architecture we put around them when they are.
Our answer is different:
Don’t require the AI to become trustworthy before you can control it. Build the system assuming it isn’t.
AI improvises. Controls should not.
Suppose you tell an agent:
Investigate why production latency increased overnight and prepare a report.
We want the agent to improvise. Inspect metrics. Search logs. Compare deployments. Run Python. Delegate trace analysis. Abandon a bad hypothesis. Try another approach. That flexibility is not an unfortunate side effect of generative AI. It is the product.
Now suppose the agent discovers that rolling back yesterday’s deployment would probably solve the problem. Perhaps it is right. Perhaps it has access to the deployment system. Perhaps its credentials allow the rollback.
But you asked it to investigate. You did not ask it to remediate. This is where we think the security problem becomes clearer.
The agent’s reasoning should be allowed to change. Its authority should not silently change with it.
Or, as we describe the problem at ArmorIQ: AI improvises. Controls enforce rules. Access control asks whether the agent is allowed to act. Agentic systems force us to ask another question: does this particular action serve the task the human actually authorized?
We are making probabilistic authority the problem
LLMs are probabilistic. That fact is unlikely to disappear simply because we want agents to operate in consequential environments. But probabilistic intelligence does not require probabilistic authority. Consider another simple example. A user asks:
Analyze why refund rates increased last quarter.
The agent has legitimate access to a refund API because other workflows require it. During its investigation, it decides issuing a real refund would help reproduce the customer experience. Every conventional security check might look reasonable. The agent is authenticated. The refund API is approved. The amount may be within policy limits. The request may contain nothing malicious. But none of those checks answers:
Why does issuing a refund belong to an objective that asked the agent to analyze refunds?
The model may conclude that the action is useful. Useful is not the same as authorized. That’s the distinction we need to make enforceable.
We’ve already seen what happens when boundaries become obstacles
OpenAI’s recent Hugging Face incident makes the problem unusually tangible. OpenAI was evaluating cyber-capable models in an environment where direct Internet access was intentionally restricted. The models were highly focused on completing the benchmark. Eventually, they found another path.
They discovered and exploited a vulnerability in an internal package-registry proxy, escalated privileges, moved laterally through the environment, reached Internet-connected infrastructure, and targeted Hugging Face while looking for benchmark solutions.
OpenAI: Hugging Face model evaluation security incident
The interesting part isn’t simply that a containment mechanism failed. The agent encountered a boundary and incorporated that boundary into its problem solving.
The environment effectively said:
You cannot reach the Internet this way.
The agent kept solving:
How else can I reach it?
That’s exactly what increasingly capable problem-solving systems are supposed to do. Which is why we should be very careful about making the security boundary another thing the model is expected to reason about correctly.
The boundary has to live outside the intelligence.
So we are building that architecture
This is where ArmorIQ’s response to the boycott argument becomes concrete.
We are not waiting for models to become deterministic. ArmorIQ is being designed around the opposite assumption: agents will remain probabilistic, adaptive, occasionally wrong, and increasingly good at finding paths their developers did not anticipate.
The intelligence gets room to reason. The surrounding architecture determines which reasoning can become authority and which authority can become effects.
Our Purpose Assurance Plane (PAP) captures the authority associated with what the human is actually trying to accomplish. As the agent turns that purpose into increasingly concrete plans, it can reduce ambiguity without quietly manufacturing new authority.
Our Intent Assurance Plane (IAP) then cryptographically commits accepted execution. Actions remain connected to the objective that authorized them as plans evolve, work is delegated, or authority is revoked.
And when agent decisions become real processes, files, network connections, and system effects, our Kernel Assurance Plane (KAP) carries objective-derived authority into execution.
The architecture is sophisticated. The idea isn’t:
Let the AI figure out how. Don’t let it decide how far.
ArmorIQ is built around giving agents freedom to adapt while preserving human authority.
Determinism doesn’t have to live inside the model
Marcus’s argument puts a spotlight on the unreliability of generative AI. There is a natural response to that: Make the intelligence reliable first. But there is another place we can put determinism.
The control plane.
An agent does not need to choose the same plan every time. It does not need to use the same model. It does not need to invoke the same tools. It does not even need to produce the same answer.
But consequential boundaries can be deterministic. This objective may access this data. This delegated agent receives this subset of authority. This action is outside the committed objective. This authority has expired. This objective has been revoked. This execution does not proceed.
Now probabilistic intelligence operates inside a deterministic authority envelope. That is very different from putting another paragraph in the system prompt telling the model to behave.
The industry is starting to build the enforcement seams
We are not alone in thinking controls need to move outside the model. Anthropic recently introduced Inference Hooks, allowing an independent enterprise control to participate before governed Claude inference proceeds.
Microsoft’s Agent Hooks proposal creates deterministic interception semantics inside agent runtimes. A deny should actually mean the action does not execute.
Sandboxes and microVMs provide hard boundaries around execution. These are important pieces of the architecture. But they still need a common answer to a more fundamental question:
What authority is this enforcement point enforcing?
We don’t want every hook, gateway, sandbox, and runtime independently guessing what the human meant from whatever fragment of context happens to reach it. Capture the objective once. Bound its authority. Commit accepted execution cryptographically. Carry that commitment with the agent. Then enforce it wherever consequential actions occur.
That’s the architecture we’re building.
Control rails, not guardrails
There is an important distinction here. A guardrail generally tries to influence or detect behavior. “Don’t do this.” “This looks unsafe.” “This action appears inconsistent with policy.” Those mechanisms are useful.
But for increasingly autonomous systems, we also need control rails. A control rail does not merely tell the agent that an action is inappropriate. It determines whether the action can become an effect. The model can believe rolling back production is an excellent idea. The model can produce an eloquent explanation for why it should issue the refund. The agent can discover that another system is technically reachable.
None of those facts creates authority.
That is the architecture we think allows AI to become more capable without requiring us to make an increasingly heroic assumption that capability will automatically bring perfect reliability.
Boycott is one response. Control is another.
The debate around generative AI is often framed as a choice between two camps.
Accelerate Or Stop.
Marcus is arguing forcefully for the brakes because he doesn’t believe today’s systems deserve the trust we’re placing in them. That challenge deserves a serious response. But we think there is an enormous engineering space between blind deployment and boycott.
Build systems that require less trust.
Assume the model will occasionally be wrong. Assume an agent will discover paths its developers did not anticipate. Assume it will improvise. Assume increasingly capable models will become better at overcoming obstacles. Then don’t make correctness of the model the only thing standing between reasoning and consequential action.
AI is already moving from answering questions to taking actions, using tools, changing systems, and making decisions along the way. As autonomy grows, the challenge is building enough trust to let AI safely do more. ArmorIQ-One-Pager.pdfPDF
That’s what we are building ArmorIQ for. Not rails that tell the model where we hope it goes. Rails that determine where execution is actually allowed to go.
Gary Marcus is right about the uncomfortable premise: we should not blindly trust probabilistic machines with increasing power over the world. Our answer is not to trust them more. It is to build an architecture where we don’t have to.
Let AI remain probabilistic. Make its authority deterministic.
