OpenAI unveiled its own inference chip, Jalapeño, and says it delivers up to 1.9 × the throughput per kilowatt of Nvidia’s GB200 and GB300 accelerators. The team turned a concept into a working silicon die in nine months—far faster than the usual two-to-three-year cycle for custom AI processors.

Why the sprint matters

OpenAI now runs models on hardware from Cerebras, AWS Trainium, AMD, and others instead of relying solely on Nvidia GPUs. As AI shifts from training-heavy to inference-heavy workloads, power use and cost per query become decisive. A chip that halves the energy needed for a trillion-parameter model could cut operating expenses for services handling billions of requests each day.

How AI wrote the chip

Jalapeño is an ASIC—a chip built for one purpose: running large language models. Unlike a general-purpose GPU, an ASIC strips away unnecessary logic and tailors data paths to LLM inference patterns. OpenAI’s engineers wrote the design in XLS, a hardware description language that lets machine-learning models generate code. The company says AI-generated logic makes up more than half of the chip’s core, shrinking the design pipeline from years to months.

Performance claims vs reality

OpenAI lists three headline numbers for Jalapeño:

  • Throughput per kilowatt: 1.5 – 1.9 × higher than Nvidia’s GB200/GB300.
  • Latency: 1.7 – 3.6 × lower than the same Nvidia models.
  • Power draw on a 1-trillion-parameter model: 700 W versus 1,400 W on the GB300.

If those figures hold up, a data center could run the same model at half the electricity cost while delivering faster responses. The claims focus on inference; OpenAI does not address training performance, aligning with its current emphasis on serving chat-type applications.

What this means for Nvidia

Nvidia’s upcoming Blackwell and Vera Rubin chips target the high-end accelerator market. Nvidia’s advantage lies in a mature software stack, broad developer adoption, and flexibility across AI tasks. Jalapeño, by contrast, is a narrow-purpose device that wins only when the workload matches its specialization.

The pressure on Nvidia has two angles. First, customers who can afford a custom ASIC may switch for the promised savings, chipping away at Nvidia’s inference share. Second, the demonstration that AI can help design hardware could compress competitors’ development cycles, forcing Nvidia to speed up its roadmap.

Counter-points and open questions

  • Software ecosystem: Nvidia’s CUDA libraries, profiling tools, and community support still give it a decisive edge for most developers. Integrating Jalapeño will require new toolchains and possibly code rewrites.
  • Real-world validation: The numbers come from OpenAI’s internal tests. Independent benchmarks, long-term reliability data, and scaling behavior under mixed-tenant loads remain unverified.
  • Economics of scale: An ASIC’s cost advantage grows with volume. OpenAI’s internal use may justify the investment, but external customers could face higher per-chip prices unless production scales.

Looking ahead

For now, Jalapeño proves that a nine-month sprint can produce a chip that, on paper, outperforms the incumbent GPU in the most cost-sensitive part of the AI stack. The real test will be whether that advantage survives the grind of real-world workloads.