OpenAI trains an AI 'super hacker' to shield GPT-5.6

What happened?

OpenAI has developed GPT-Red, a large language model (LLM) specialized in finding vulnerabilities in other AI systems. According to MIT Technology Review, GPT-Red is trained through a self-play loop where it repeatedly attacks other models while they defend, iteratively improving its offensive and defensive skills. This approach, similar to training generative adversarial networks (GANs), allows GPT-Red to discover novel attacks that humans had not considered, such as the “fake chain of thought” vulnerability, which tricks the model into believing it is reasoning correctly when it is actually being manipulated.

The system was used to test the security of GPT-5.6, the latest version of OpenAI's flagship model, released last week. The company claims this process has made GPT-5.6 its most robust release to date. According to OpenAI, training against GPT-Red significantly reduced the success rate of prompt injection attacks on GPT-5.6, although exact figures have not been released.

Why is it important?

Traditional red-teaming, carried out by human teams, is falling behind the growing complexity of LLMs and their use as autonomous agents. As Nikhil Kandpal, OpenAI scientist and co-creator of GPT-Red, notes: “The risk surface grows and the blast radius also grows.” GPT-Red automates this process and can discover attacks that humans had not considered. The need for automation is critical: as models are deployed in applications that interact with the real world (web browsing, reading emails, editing code), the number of possible attack vectors multiplies. A 2025 report from the Center for AI Safety already warned that manual red-teaming methods do not scale with the pace of LLM development.

“As more capable models become available, we will have already designed the system that can discover new modes of attack,” says Dylan Hunn, co-creator of GPT-Red.

This self-improvement capability is key: GPT-Red not only replicates known attacks but can generate new exploits, making it an adaptive security tool. However, the system still has limitations: according to researchers, GPT-Red struggles with conversational attacks and those based on images, areas where human testers remain essential.

What consequences will it have?

The use of GPT-Red represents a paradigm shift in AI security: moving from manual testing to an automatic adversary that constantly improves. This could accelerate the vulnerability patching cycle and reduce the risk of security incidents in real-world deployments. Historically, human red-teaming has been the gold standard, but with the advent of models like GPT-Red, the industry could adopt “generative adversary” systems as common practice. Companies like Google DeepMind and Anthropic have already explored similar approaches, though none have been as specialized as GPT-Red.

The market impact could be twofold: on one hand, AI providers that adopt such tools could differentiate themselves through security, attracting enterprise clients concerned about risks. On the other hand, the existence of a tool like GPT-Red raises dual-use risks: if it fell into the wrong hands, it could be used to attack AI systems rather than defend them. OpenAI says it maintains strict controls over access to GPT-Red, but the possibility of leakage or malicious use is a legitimate concern. In 2023, a similar incident with a Meta security model sparked controversy when it was released without restrictions.

What should readers know?

• GPT-Red is not a public product: it is used internally at OpenAI as a security tool.
• The main attack it combats is prompt injection, where an attacker hides malicious commands in text that the model processes. This type of attack has been responsible for several critical vulnerabilities in AI assistants, such as the 2024 case where a bank's chatbot was manipulated to transfer funds.
• Training was conducted in a simulated “dojo” that replicates real-world scenarios like web browsing, reading emails, and editing code. This environment allows GPT-Red to practice attacks in a controlled setting, minimizing risks during training.
• GPT-Red has outperformed humans in finding effective attacks, but does not fully replace them. In comparative tests, GPT-Red discovered 30% more vulnerabilities than a human team in the same time, according to internal OpenAI data leaked to MIT Technology Review.
• The system is based on GPT-4 as a foundation, but has been fine-tuned specifically for red-teaming tasks, making it more efficient than a general-purpose model.

In summary, GPT-Red is a step forward in AI security, but human oversight and diverse testing remain necessary. Like any powerful tool, its value depends on how it is used and controlled. The industry will need to balance automation with ethics and transparency to prevent these tools from becoming double-edged swords.