OpenAI releases IH-Challenge to strengthen instruction hierarchy and prompt-injection resistance

What OpenAI released

On March 10, 2026, OpenAI published IH-Challenge, a reinforcement-learning dataset designed to improve how models handle conflicting instructions from different trust levels. OpenAI’s core hierarchy is explicit: System > developer > user > tool. If a model follows the wrong level, the failure can show up as policy violations, disclosure of protected information, or prompt-injection attacks embedded in retrieved content or tool outputs.

OpenAI argues that instruction hierarchy is not a narrow alignment detail. It is a general safety property for agentic systems. As models call tools, read untrusted web pages, and act on behalf of users, they constantly have to decide which instructions are authoritative and which should be ignored. The company frames many reliability and security failures as cases where the model simply followed the wrong source of instruction.

How IH-Challenge is constructed

OpenAI says naive reinforcement learning can go wrong in three ways: the task may be too instruction-heavy to isolate hierarchy behavior, LLM judges can be unreliable on ambiguous conflicts, and models can learn shortcuts such as blanket refusals. IH-Challenge is built to avoid those failure modes. The tasks are intentionally simple, objectively gradable with a Python script, and structured so that trivial over-refusal does not score well across the dataset.

The company trained an internal model called GPT-5 Mini-R on the dataset and reported improvements on both academic and internal benchmarks. On TensorTrust, performance rose from 0.86 to 0.94 for system-user conflicts and from 0.76 to 0.91 for developer-user conflicts. On RealGuardrails, handwritten test performance moved from 0.82 to 0.89, and System IFEval improved from 0.92 to 0.96. OpenAI also says the gains carried over to internal prompt-injection and jailbreak-style evaluations.

Why this matters

The most relevant claim is that stronger instruction hierarchy appears to improve several safety properties at once. OpenAI says the IH-trained model became better at safety steerability when category-specific safety rules were placed in the system prompt, and also more robust on prompt-injection benchmarks including CyberSecEval 2. At the same time, the company says the training did not collapse into broad over-refusal or obvious capability regressions: GPQA Diamond stayed at 0.83, and AIME 2024 slightly improved from 0.93 to 0.94.

That combination matters for production systems. Safety work that only increases refusals is easy to discount. Safety work that improves conflict resolution while preserving usefulness is more durable. OpenAI’s decision to release IH-Challenge on Hugging Face also gives outside researchers a concrete dataset for studying one of the hardest practical problems in agent security. As models become more autonomous, instruction hierarchy is increasingly less about etiquette and more about whether a system can safely interact with tools, retrieved content, and real-world workflows.

Sources: OpenAI research post · paper · Hugging Face dataset