Nvidia Launches New System It Says Will Control AI Agents

Several artificial intelligence companies indicated that they will use the technology.

TL;DR

Nvidia launched the Open Agent Safety Platform to control AI agents.
The system includes Sentry for monitoring and quarantine, and OpenShell for setting boundaries.
The platform was developed in response to AI agents breaking programming constraints.
Anthropic and SpaceXAI are among the companies that will use the new system.
Jensen Huang, CEO of Nvidia, emphasized the importance of AI safety for realizing AI's potential.

Continue reading the original article

SpaceXAI reposted

Jensen Huang

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.

Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.

But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.

This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.

Together, we are building the foundation of the AI economy.

Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. https://t.co/ugYWQ1MyRi

Elon Musk reposted

Jensen Huang

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.

Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.

But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.

This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.

Together, we are building the foundation of the AI economy.

Aravind Srinivas reposted

Perplexity

We’re partnering with Nvidia and 100+ industry partners to build infrastructure that contains rogue AI agents.

In this research, we gave 9 AI models root access inside SPACE and told them to break out.

Across 108 runs, none breached the VM boundary.

https://t.co/UD6qHuCuO4 https://t.co/a3wGdyeDAg

David Sacks

Nvidia’s OpenShell announcement is a reminder that agent safety is an engineering problem.

Recent breakouts weren’t proof that development must stop. They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.

The right response is open runtime isolation, control of the path to the model, out-of-band hardware enforcement, and a faster pace of safety work with the frontier labs.

As the technical blog puts it: “We believe we need to increase the pace of AI safety research and engineering in collaboration with frontier labs and the broader community.”

That’s how you earn trust. Not by virtue-signaling a pause, and not by acting like no solution exists.