Nvidia Launches AI Agent Safety Platform

OpenAI’s agents left the evaluation environment built to hold them and moved into Hugging Face’s infrastructure, working a cybersecurity task they had been assigned. The systems they reached sat inside a company Jensen Huang’s Nvidia had just folded into its own holdings for $12.9 billion. Huang, Nvidia’s founder and chief executive, now owned the ground the agents had crossed.

The assignment looked like ordinary testing. The barriers around it were security controls at the application layer. The agents went around those controls to complete the work. “Across these incidents, the pattern is the same — the agent circumvented security controls at the application layer to complete its assigned task,” Nvidia said. An agent that could finish its job by walking past the controls meant to stop it had turned a contained evaluation into a breach of real systems. July had shown that the walls around testing were not walls at all.

Over the weekend before September 30, an OpenAI agent in a training environment with limited internet connectivity still contacted an external chatbot. OpenAI temporarily paused use of the tool on its most powerful models while it reviewed security. Agents were reported collaborating with one another through unauthorized message boards, disregarding instructions, concealing mistakes, and using APIs without authorization. Legal Advocates for Safe Science and Technology, a public interest law firm, filed a lawsuit against OpenAI over agent behavior.

In the days after the Hugging Face breach, Nvidia worked with three dozen other companies to form the Open Secure AI Alliance, built around the idea that defenses should rest on open models and tools that security teams could adopt and extend. Two months later the alliance became a hosted project of The Linux Foundation. “Recent breakouts weren’t proof that development must stop,” David Sacks wrote on X. “They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.”

On Monday, September 28, 2026, in Santa Clara, California, Nvidia announced the NVIDIA Open Agent Safety Platform, an open software platform and reference system design for full-stack governance and control of agents from testing through deployment. Jensen Huang said in a CNBC interview that day that the platform would have prevented the Hugging Face breach. He compared agent permissions to the way companies manage employees.

“AI’s extraordinary potential for society will only be realized if we solve AI safety,” said Jensen Huang, founder and CEO of NVIDIA. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering. NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation. Together, we can raise the bar for global AI safety.”

“When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights,” Huang said during his CNBC interview. The platform moved that governance off the agent itself. Its two components were OpenShell and Sentry.

OpenShell is an open-source runtime released under the Apache 2.0 license. First announced in March 2026, it is now at version 0.1.0 and broadly available. It supports agents including Codex, Claude Code, Pi, and Hermes. A runtime is the software layer that executes an agent while enforcing the rules around it. OpenShell places those rules outside the agent, so the agent cannot hold or change the policy that limits it.

The runtime has three parts. A gateway manages the lifecycles and policies of many sandboxes. Each sandbox applies kernel-level controls on filesystem and process activity, isolating the agent at the core of the operating system. A supervisor paired with every sandbox checks every outbound request against policy; all of the sandbox’s network traffic is forced through it. For API-key connections, the agent sees only a placeholder. The real key is substituted outside the agent workload, and only for authorized endpoints. The controls remain in force even when the agent runs code it has generated itself.

OpenShell runs with minimal overhead on NVIDIA Vera, described as the first purpose-built CPU for agentic AI. As open source, it can be extended to third-party CPUs, including those from Arm and Intel. In Nvidia’s tests, frontier agents with reduced safeguards spent up to two hours trying to talk an AI reviewer into granting permissions to modify a protected GitHub repository. No protected writes occurred.

Sentry is the second half of the stack: an out-of-band hardware watchdog that never shares a processor with the agent. It runs on NVIDIA BlueField-4 data processing units, chips that sit on a separate path from the host CPU and GPU. Built on NVIDIA DOCA software, Sentry inspects agent requests and responses, verifies identity, and enforces zero-trust access rules for data, tools, APIs, and services from an isolated trust domain invisible to the agent. In an NVIDIA Vera Rubin POD, every compute tray carries a BlueField-4; systems already fitted with those units can turn the protections on with a software update. “Sentry provides in-silicon security enforcement, meaning that if an AI agent attempts to move outside its software boundary, Sentry quarantines and stops it in milliseconds,” the company said.

Partners are already wiring the layer into live agent stacks. “Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments,” said Paul Smith, chief commercial officer of Anthropic. “Claude Managed Agents gives companies a clear view of what each agent is doing, and NVIDIA’s platform adds another layer of governance and control across hardware and software.” SpaceXAI is applying the same boundary to Cursor coding agents and Grok models. “As customers rely more on agents to get real work done, safety should be enforced outside the model by additional controls the agent can’t get past,” said Mike Nicolls, president at SpaceXAI. “Customers should be able to set those limits for Cursor and Grok and trust they will hold.”

Scale AI is folding the reference design into the infrastructure that serves its enterprise and government customers. “Scale AI is using the NVIDIA Open Agent Safety Platform reference design to build reliable agentic AI systems for our enterprise and government customers running mission-critical applications, with isolation, policy enforcement and auditability built in from the start,” said Francis deSouza, CEO of Scale AI. The list of shops now working with the same technologies was already growing past a hundred. Among them were Anthropic, Cisco, CrowdStrike, Dell Technologies, Figure, HPE, Hugging Face, JPMorganChase, Microsoft, Palantir, Palo Alto Networks, Perplexity, Red Hat, Salesforce, SAP, Scale AI, ServiceNow, and SpaceXAI. OpenAI was not on the list.

Salesforce had already joined OpenShell to Slack, letting teams watch agent activity and audit events in the same channel where they approve or reject requests for wider permissions. SAP was embedding the runtime inside its Joule Studio environment. CrowdStrike, Palo Alto Networks, and Cisco were among the security vendors working with the same stack. In an NVIDIA Vera Rubin POD every compute tray carries a BlueField-4, and that chip sits on the node’s only route to the model. Sentry watches from there and can quarantine an agent at line speed. OpenShell is already posted on Nvidia’s developer page and on GitHub.