One Click on ChatGPT Can Sneak a Malicious Agent into Your Company

What Happened?

Security researchers at Zenity Labs discovered a critical vulnerability in ChatGPT's agent builder, dubbed AgentForger. The technique allowed an attacker, with a single click on a seemingly legitimate ChatGPT link, to create, configure, publish, and schedule a malicious agent within the victim's workspace. The agent could act using the employee's connected accounts and permissions, such as Outlook, Teams, Slack, SharePoint, or Google Drive, provided the organization had those integrations enabled. According to Michael Bargury, co-founder and CTO of Zenity, “This is not a forged request, it's a forged insider. With one click, an attacker gets a fully autonomous agent inside your company that has the identity and access of your employees, with protections turned off.”

The attack exploited ChatGPT's own agent builder, which accepted instructions embedded in a seemingly normal link. Upon clicking, the builder created an agent that connected to the victim's already authorized applications, disabled approval requests, and scheduled itself to run periodically. Communication with the attacker was conducted through the victim's email, searching for messages with the subject “TASK” to receive new orders, thus avoiding traditional command-and-control infrastructure.

Why Is This Important?

This incident marks a milestone in the evolution of cyber threats: it's no longer just about stealing credentials or hijacking sessions, but about creating a malicious digital employee that operates from within, with legitimate permissions. AI agents, designed to act autonomously on corporate systems, represent a new attack surface that traditional defenses are not prepared to detect. Unlike previous attacks such as session hijacking or credential phishing, AgentForger did not require complex exploits; it simply tricked ChatGPT's own agent builder into executing malicious instructions. This underscores a paradigm shift: attackers are no longer looking for backdoors, but are creating fictitious internal users with legitimate access.

Historically, vulnerabilities in AI assistants have been limited to prompt injection or jailbreak attacks. However, AgentForger goes further by compromising the agent creation flow, a fundamental component of the platform. Compared to Microsoft's Tay case in 2016, where a chatbot learned offensive behaviors, here the malicious agent is deliberately created by an attacker and acts with real permissions. This has direct implications for enterprise AI adoption, as it undermines trust in autonomous assistants.

Consequences and Lessons

Zenity demonstrated attack scenarios that included:

• Automatically mapping the company's organizational structure and projects.
• Searching for passwords and API keys in chat conversations.
• Sending convincing phishing messages from the victim's Teams account.
• Exfiltrating sensitive documents via email.

OpenAI fixed the vulnerability within four days of being notified, removing the URL parameter that enabled the attack. However, the incident underscores that AI agent security must evolve: agents are not just software, they are extensions of the workforce and require access controls, behavior monitoring, and specific usage policies. OpenAI's response was swift, but the fact that the vulnerability existed reveals gaps in the security design of generative AI platforms.

For companies, the impact is twofold: first, the need to review permission configurations in ChatGPT Workspace and limit who can create agents. Second, implement monitoring for anomalous agent activity, such as data access outside working hours or unusual communication patterns. The broader lesson is that AI agent adoption must be accompanied by a zero-trust security framework that treats each agent as another user, subject to authentication, authorization, and continuous auditing.

“Attackers no longer have to break in to steal your data. They can forge an insider to go get it,” warns Bargury.

This attack also parallels the use of bots on social media to manipulate opinions, but here the bot operates within corporate infrastructure, with access to sensitive data. The speed of OpenAI's response is positive, but the industry must learn from this incident to design agents with security-by-design principles.

What Should Readers Know?

If your company uses ChatGPT Workspace with agents and connectors to enterprise applications, it is crucial to review permission policies and ensure only authorized users can create agents. Additionally, implement monitoring for anomalous agent activities and limit actions they can perform without explicit approval. The AgentForger vulnerability is a reminder that AI security is not just a technical problem but a governance one. Organizations should treat agents as digital employees, with role-based access controls, activity logging, and periodic permission reviews.

For individual users, the lesson is to distrust suspicious links even within trusted platforms like ChatGPT. Social engineering remains the primary vector, now amplified by the ability to create autonomous agents. Finally, this case should push regulators to consider specific security frameworks for AI agents, similar to those for APIs and cloud services. AI agent adoption is in its early stages, and events like AgentForger will define how trust in this technology develops.