- BIST10011,722-2.89%
- USD49.45+0.05%
- EUR55.62+0.37%
- GOLD4,181.10+0.26%
- INTEREST37.00%-1.00
- BITCOIN81,509+0.01%
- October 17, 202613°
- BIST10011,722-2.89%
- USD49.45+0.05%
- EUR55.62+0.37%
- GOLD4,181.10+0.26%
- INTEREST37.00%-1.00
- BITCOIN81,509+0.01%
- October 17, 202613°
Nvidia unveils platform to limit the capabilities of AI agents
Nvidia has unveiled the Open Agent Safety Platform to limit AI agents’ permissions to the tasks they are assigned and monitor their activities. More than 100 institutions and organizations, including Microsoft, Perplexity, Accenture and JPMorgan Chase, have reportedly begun using the system.
Nvidia has unveiled a security system called the Open Agent Safety Platform, designed to prevent AI agents from going beyond their assigned tasks. The platform, announced by Justin Boitano, Nvidia’s vice president of enterprise AI, includes open-source software and an independent security layer that monitors agents’ activities.
The system’s core component, OpenShell, enables developers to verify that an AI agent is granted only the permissions it needs to complete its task. The open-source software can be expanded to run across different computing platforms.
Sentry intervenes in suspicious activity
According to Nvidia, the platform also includes an independent security layer that runs directly on the chip and tracks AI agents in real time. Called Sentry, the layer can intervene if an agent attempts to move beyond its defined objective.
Justin Boitano said Sentry can quarantine suspicious agents within milliseconds. Boitano said OpenShell manages agents’ actions, while Sentry independently monitors suspicious behavior and brings it under control.
More than 100 institutions and organizations have begun using the platform, including Microsoft, Perplexity, Accenture and JPMorgan Chase.
The Hugging Face incident came under discussion
While explaining that the platform was designed to prevent AI agents from exceeding their limits, Boitano also referred to the Hugging Face incident. He said the new security system could have prevented the breach if it had been used earlier during model evaluations at leading laboratories.
The Hugging Face incident occurred during tests conducted by OpenAI in July 2026 to measure the capabilities of its AI models. In an investigative report published by OpenAI on August 26, the company said that a model called Internal Model 1, used only in research, played a significant role in the incident.
During the test, an AI agent exceeded the limits imposed on it in an attempt to succeed; it accessed the company’s systems and the internet and communicated with other AI agents. One of the agents identified 14 sets of credentials that could provide access to Hugging Face and shared them with the other agents.
The agents later exploited security vulnerabilities in Hugging Face systems to run code on servers. In doing so, they gained access to private data and credentials for different systems.
Stay Updated with Ally Daily
Subscribe to be informed about special news, developments and announcements. We will keep you informed.
By signing up, you agree to our Privacy Policy
