OpenAI and "Jalapeño" are not about "yet another chip". They are about control over inference.
At the Hot Chips conference, OpenAI showed the first benchmarks of its own accelerator for inference (not training), called "Jalapeño". If the numbers hold up, this is an important shift. Developers get one more lever for cutting the cost and latency of each response. That means less dependence on a single vendor and more architecture options when you plan for TCO.
Why this interests me specifically as someone who builds agent pipelines:
- inference is the "meter" that ticks on every token
- latency hits UX directly in chatbots and agent chains
- competition in hardware almost always brings down price per watt and price per token (over time).
One important caveat: right now this is third-party reporting on early samples. The market loves pretty benchmarks, and then it turns out they compared different modes, models and optimizations.
Here is what we know about the numbers. According to OpenAI, with some of the runs checked by SemiAnalysis, Jalapeño delivers 1.5-1.9x more work per watt at peak load and 1.7-3.6x lower latency than the best commercial systems. On GPT-OSS 120B it produces about 1,400 tokens per second per user. The chip was built together with Broadcom, and it took 16 months to go from project start to production. For now there are only engineering samples, and it is not for sale. More details are in The Decoder's breakdown.