Making AI Agents Safe

I spoke with Ali Golshan, a senior engineer behind Nvidia's new Open Agent Safety Platform, which promises to revolutionize agentic AI safety | Edition #340

TL;DR

  • Agentic safety is a top AI challenge, evidenced by recent incidents.
  • Nvidia's GTC conference in Berlin featured discussions on new AI safety architectures.
  • Nvidia launched the Open Agent Safety Platform to ensure verifiable policy and continuous containment/monitoring of AI agents.
  • The platform addresses the technological avenue for solving the agentic AI safety crisis.
  • Key discussion points included the platform's functionality, the importance of openness (OpenShell under Apache 2.0), emerging risks, governance challenges, and privacy.
  • The author is more optimistic that AI safety can be solved through engineering after the conversation.

Continuer la lecture de l'article original