NVIDIA OpenShell Reference Architecture Secures and Governs AI Agent Workloads at Scale

Autonomous agents can work for hours on complex tasks, accessing files, tools, the internet, and spawning other agents to collaborate with. However, executing complex tasks over long time horizons leads to compounding risk: the agents may download code, build new tools, request additional privileges, and touch more of the underlying infrastructure.

The adaptability of agents is what makes them so powerful, but they are goal-oriented and may not prioritize safety in pursuit of their objectives. In order for agents to work both autonomously and safely, we need to build environments that provide control and visibility in what the agent is doing.

An agent cannot govern itself; security and governance belong in the infrastructure the agent operates in. This is where NVIDIA OpenShell comes into play – an open-source runtime designed to build the necessary security and governance for agentic systems into the computing infrastructure.

NVIDIA OpenShell provides deterministic governance outside of the agent’s reach, treating agent workloads as untrusted throughout execution. An agent is given access to what it needs, when it needs it, with strict controls in place to prevent unauthorized actions.

The NVIDIA OpenShell Reference Architecture defines and enforces which systems and data an agent can access, from files and internet services to systems that interact with physical devices. This architecture forms the foundation of a secure environment for AI agents to operate within.

OpenShell has four key components: Gateway, Supervisor, Sandbox, and Policy prover. The Gateway manages sandbox lifecycles and policies, while the Supervisor checks outbound requests against policy and mediates access to credentials. The Sandbox isolates the agent workload and restricts filesystem and process access, ensuring that agents cannot interfere with each other or the host system.

The Policy prover adds formal verification, checking whether a policy’s permissions stay within an organization-defined ceiling. This ensures that agents operate within predetermined boundaries, preventing unauthorized actions and data breaches.

Extensibility is a core principle of OpenShell’s design. Supervisor middleware lets operators inspect, block, or change the content that an agent sends and receives over the network as an additional enforcement layer. Gateway interceptors let operators add deployment-specific governance to OpenShell control-plane operations without modifying the gateway.

In addition to these components, organizations need to establish who an agent acts for, what authority it holds, and whether an action fits the task. Identity, authorization, data protection, and auditing each address part of that problem, with enforcement points provided by OpenShell where those systems can be integrated.

The NVIDIA Vera platform provides the compute layer beneath the controls defined by OpenShell. Agents operate through loops of dependent steps: reasoning, calling tools, executing code, querying data. AI factories run thousands of these loops concurrently, placing new demands on CPU architecture across five dimensions.

Firstly, massive core counts are needed to run many concurrent workflows and fast individual cores to shorten each sequential step. Secondly, high-bandwidth fabric and memory are required to keep those cores fed with the live data agents depend on. Thirdly, massive I/O is necessary to move data at speed between CPUs, GPUs, storage, and network.

Fourthly, hardware-enforced security is essential because agents increasingly operate on sensitive enterprise data and must be isolated from each other and the host. Lastly, delivering all of this within the power envelope of a modern data center (more agent work per watt) is crucial for scalability.

NVIDIA Vera CPUs are designed to meet these demands, with massive Olympus cores, a second-generation Scalable Coherency Fabric, LPDDR5X memory, Gen 6 I/O, confidential computing, and an energy-efficient design purpose-built for power-limited data centers. This combination delivers up to 2x performance on critical agentic workloads versus x86.

Three architectural properties make Vera fast for agent workloads: its massive Olympus cores with a 10-wide front end, neural branch predictor, and deep out-of-order execution; its second-generation Scalable Coherency Fabric providing up to 3x core-to-core bandwidth and up to 50% lower core-to-core latency than x86 alternatives.

Lastly, Vera’s LPDDR5X memory provides up to 1.2 TB/s of bandwidth with up to 40% lower loaded memory latency than chiplet-based designs, keeping agents supplied with the live context they need without stalling on memory access. This results in a dramatic increase in efficiency: Vera delivers 2x more performance per watt than x86 CPUs.

Agent workloads depend on sandboxes – isolated execution environments where agents run code, invoke tools, and complete tasks without interfering with each other or the host system. The denser and faster those sandboxes spin up, the more agent work a given infrastructure footprint can sustain.

In fact, CoreWeave found sandbox startup to be more than 3x faster on Vera than on x86, with a single rack supporting more than 11,000 concurrent agent environments. Running Terminal Bench, which measures CPU performance across real agentic loop tasks, also showed that Vera delivered 1.8x faster performance versus x86.

NVIDIA BlueField and NVIDIA Sentry provide additional hardware-enforced protection for deployments requiring extra security. OpenShell places authority outside the agent; these components extend this separation into purpose-built silicon, providing an optional safeguard alongside software boundaries.

BlueField-4 DPUs sit in the I/O path between agents running on Vera and the models they access, enforcing policy in silicon when a threat or policy violation is detected by NVIDIA Sentry. This allows for continuous monitoring of agent behavior and enforcement of runtime policy without relying on agent self-reporting.

Together, OpenShell, Vera, and BlueField-4 form the complete stack for safe AI agents: a foundation that gives organizations the performance to make agents productive and the controls to keep their work within authorized boundaries. Get started with NVIDIA OpenShell today.