Jalapeño's AI Inference Chip Sets New Standards for Speed and Efficiency

OpenAI has announced the results of its first custom inference chip, Jalapeño. The company claims that this new architecture delivers industry-leading speed and efficiency in artificial intelligence (AI) inference tasks. According to OpenAI, Jalapeño can serve more AI work per unit of power while also returning responses more quickly than existing hardware systems.

The performance gains offered by Jalapeño are significant. In tests conducted on three public models - GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T - the chip delivered a better combination of throughput, power efficiency, and latency compared to leading commercially available AI systems. Specifically, it achieved 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems.

The benefits of Jalapeño’s performance are not limited to just one or two specific models. The chip was designed from the start with a focus on serving modern and future language models, particularly interactive agents. As such, it excels in both prefill and decode phases, which have different bottlenecks. By minimizing data movement and communication delays, Jalapeño ensures that model state can be explicitly placed and kept local while activating the right combination of compute, memory, and networking for each inference phase.

Jalapeño’s development was also facilitated by AI itself. The team used OpenAI models to explore implementations, shorten design, measurement, and verification loops, and continuously iterate on model workloads. This allowed them to move from initial design to tapeout in just nine months. Additionally, AI helped optimize the chip’s arithmetic circuits, enabling more compute performance to be fit into the chip on schedule.

The architecture of Jalapeño is designed as a clear, predictable programming target for both humans and AI. Engineers can describe work through local tensors, explicit communication, and predictable synchronization, making it easier for AI to tackle parallel programming tasks. This flexibility was demonstrated when the team used Codex with GPT-Astra to bring three open-weight models not part of Jalapeño’s original production plan to high performance within two months.

The results from these tests are impressive. On Kimi K2.5 1T, for example, Jalapeño delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. In internal testing, the advantage of Jalapeño widened further on frontier OpenAI models, suggesting that its architecture becomes more valuable as workloads grow larger and more demanding.

Jalapeño’s impact extends beyond just improving performance in AI inference tasks. By producing more useful work from the same power and hardware, it can help serve more demand and lower the cost of delivering a successful result. This is particularly important for OpenAI, which aims to ensure that artificial general intelligence benefits all of humanity.

The deployment of Jalapeño within OpenAI’s compute infrastructure is expected to begin by the end of this year. It marks the first generation of a multigenerational roadmap, with Gen 2 already in development and Gen 3 taking shape. Each subsequent generation will build on what has been learned and further advance both efficiency and speed.

The growing demand for AI requires more compute from every available source. To meet this need, OpenAI plans to continue deploying accelerators from NVIDIA and other partners for both training and inference workloads. The company’s mission remains unchanged: to ensure that artificial general intelligence benefits all of humanity.

Higher peak mixed TPS/kW ≈1.9×

85,448 vs. 44,960 mixed / kW

Lower end-to-end latency ≈1.7×

1.03 s vs. 1.80 s

Lower min TBT ≈2.7×

0.69 vs. 1.87 ms (1,459 vs. 535 tok/s/user)

More throughput at previous TBT ≈53.7×

22,935 vs. 427 mixed / kW (at 535.28 tok/s/user)

Higher peak mixed TPS/kW ≈1.7×

19,641 vs. 11,781 mixed / kW

Lower end-to-end latency ≈3.6×

1.65 s vs. 5.99 s

Lower min TBT ≈4.1×

1.43 vs. 5.90 ms (700 vs. 169 tok/s/user)

More throughput at previous TBT ≈104.3×

12,258 vs. 118 mixed / kW (at 169.41 tok/s/user)

Higher peak mixed TPS/kW ≈1.5×

18,195 vs. 11,862 mixed / kW

Lower end-to-end latency ≈3.4×

1.56 s vs. 5.31 s

Lower min TBT ≈3.8×

1.44 vs. 5.48 ms (694 vs. 182 tok/s/user)

More throughput at previous TBT ≈56.1×

6,744 vs. 120 mixed / kW (at 182.46 tok/s/user)