OpenAI's AI-Designed Jalapeño Chip: Kernel Performance Jumps From 0.31% to 88.94% of Peak in 40 Hours

OpenAI has revealed how its own language models helped design the company's first custom AI accelerator, and the numbers are hard to ignore. The Jalapeño inference chip moved from first RTL to tapeout in just nine months, and after the first silicon returned from the foundry, OpenAI's models lifted a key attention kernel from 0.31% to 88.94% of the chip's theoretical performance ceiling in roughly 40 hours. It is one of the clearest real-world examples yet of AI accelerating the creation of the hardware that AI itself runs on.

Table of Contents

• What OpenAI Actually Announced
• Inside Jalapeño: The Hardware
• How LLMs Designed the Chip
• The 40-Hour Kernel Jump
• Benchmarks Against Nvidia: Reading the Fine Print
• Why It Matters for Nvidia and the AI Industry
• Limits and Open Questions
• Frequently Asked Questions

What OpenAI Actually Announced

Jalapeño is OpenAI's debut inference accelerator, built with chip partner Broadcom and systems partner Celestica. It was first announced in June, and OpenAI laid out the full details in late August, including a presentation at Hot Chips 2026 by hardware vice president Richard Ho and engineer Chris Leary. This week the story returned to the top of developer discussions after IEEE Spectrum published a detailed look at how the chip came together.

The headline is not only the silicon. It is the process. According to IEEE Spectrum, Jalapeño went from first architecture concept to first silicon in under 20 months, with only nine months separating the first register-transfer-level (RTL) code from tapeout, the moment a finished design goes to manufacturing. Ho told the publication that the team behind it averaged fewer than 100 people during the project and still sits at roughly 100 today as work continues on second- and third-generation designs.

OpenAI's own first-results write-up credits AI directly. It says AI helped the team explore implementations, shorten the design, measurement and verification loops, and even optimize the chip's arithmetic circuits so more compute would fit on schedule.

Inside Jalapeño: The Hardware

Headline specifications

Per IEEE Spectrum's reporting, Jalapeño delivers up to 13.4 petaflops of 4-bit compute and connects to 232 gigabytes of HBM4 memory at 15.4 terabytes per second. The compute die sits alongside six HBM4 stacks and an I/O chiplet, and the platform is designed for deployment in pods of 2,048 chips.

A predictable target for humans and machines

Tom's Hardware's Hot Chips coverage explains that Jalapeño is a spatial architecture. Each core has fast access to its own local slice of memory, cores talk over a dedicated high-speed network, and software must explicitly decide where tensors live and how they move. The chip can be programmed through a low-level environment in the open-source Triton ecosystem. A separate, deliberately modest general-purpose network-on-chip handles rarer traffic such as remote memory access, so performance-critical data stays on the fast fabric.

That design is intentional. OpenAI says it structured scheduling, data movement and synchronization to be clear and predictable, so both engineers and AI agents can reason about how the hardware will behave. The idea is that a machine-readable architecture gives AI a tractable way to attack parallel programming, historically one of the hardest problems in computing.

How LLMs Designed the Chip

Why XLS became the playground

Much of the front-end work was built around XLS, an open-source high-level synthesis toolchain originally developed at Google. Designers write in languages such as DSLX, a Rust-inspired domain-specific language, or C++, and XLS converts that into Verilog. Tom's Hardware reports that more than half of the core was written using XLS, with OpenAI's models searching for improvements to power, performance and area.

Leary told IEEE Spectrum that the AI was much better at software-looking tasks, and XLS resembles software. It also helped that Leary was intimately familiar with XLS, having started it during his time at Google. Andrew Kahng, a professor at UC San Diego, called the choice sensible because it is more natural for an LLM and allows fast iteration.

Which models did the work

The project began with help from models like o3 and, by the end, used precursors to GPT-6 Astra, which reached the public on 3 September 2026. Leary said the newer model can work directly in Verilog and is close to operating proprietary design tools by itself. Ho also confirmed the team had internal LLMs fine-tuned for chip design that are not publicly available, though he declined to name them. OpenAI's stated goal is to carry the lessons into its commercial models.

Front end versus back end

The bulk of the AI-assisted work targeted the front end: turning a concept into RTL and verifying it. Broadcom handled physical design from the gates onward, using its own internal workflow and OpenAI's public commercial models rather than the internal ones. OpenAI's physical-design engineers still guided floor planning and routing, and at Hot Chips the team reported that AI-guided optimization cut the area of the matrix multiplication units by 10% compared with an optimized human baseline.

Outside experts see room for more. Verkor.io co-founder Ravi Krishna argued that OpenAI's back-end approach already looks conservative because models improved noticeably from April 2026 onward, and his co-founder Suresh Krishna said there is no reason an agentic loop could not accelerate that stage too.

The 40-Hour Kernel Jump

What the numbers mean

The most striking result arrived after the first chips came back from the foundry in May. The team pointed its internal models at writing software for benchmarks such as SemiAnalysis's InferenceX. On DeepSeek's multi-head latent attention kernel benchmark, performance rose from 0.31% of the theoretical ceiling, a limit set by the chip's compute and memory bandwidth, to 88.94% in roughly 40 hours.

Ho says the result is repeatable, which matters for schedules. If AI can turn a brand-new chip into a well-tuned target in days, the gap between receiving first silicon and ramping production shrinks. In his words, all future schedule assumptions will be built around having that capability.

Hand-tuned, but not by hand

SemiAnalysis, which published a detailed analysis of the chip, describes Jalapeño kernels as being written almost like assembly, with each kernel individually tuned, some stretching to around 3,000 lines, and protected by correctness checks and a custom sanitizer. The point is that the labor of that tuning is increasingly done by coding models such as Codex rather than by scarce human specialists.

OpenAI also says the same automated workflow later optimized three additional open-weight models that were not in the chip's original roadmap, according to HyperAI's summary of the disclosure. That is the practical payoff: evolving model architectures no longer require months of manual software re-engineering.

Benchmarks Against Nvidia: Reading the Fine Print

OpenAI cites benchmarks showing up to 3.6 times lower end-to-end latency than Nvidia's GB300 systems, while using less power. Analysis from LLM Rumors of the InferenceX runs says OpenAI reported 1.5 to 1.9 times higher peak throughput per kilowatt of package power and 1.7 to 3.6 times lower latency across three models: GPT-OSS 120B, DeepSeek R1 and Kimi K2.5.

Those are impressive figures, but they come with caveats that careful readers should keep in view:

• They are vendor-reported. The tests used a fixed sequence shape of roughly 8K input tokens and 1K output tokens, so they are deployment signals rather than a universal leaderboard.
• The comparison setup matters. Commentators note that Nvidia deployments can use techniques such as speculative decoding that were not part of the disclosed runs, so a fair head-to-head needs matched conditions.
• Real fleets will tell the truth. IEEE Spectrum stresses that it remains to be seen whether these gains hold once Jalapeño enters widespread service across OpenAI's inference fleet.

Why It Matters for Nvidia and the AI Industry

For years, Nvidia's strongest defense has not been raw silicon but CUDA, the software ecosystem that makes its chips usable on day one. Jalapeño hints at how that moat could erode. If frontier coding models can rapidly write and tune kernels for a new architecture, the cost of leaving a general-purpose platform falls, at least for a lab that runs a small number of very high-volume models. SemiAnalysis framed it as a signal that the industry's obsession with universal compilers and programming models could be challenged by capable AI.

The dynamic is also almost circular. Models running on today's GPUs helped design the chip that may reduce dependence on those GPUs. OpenAI president Greg Brockman told CNBC that the speed-up from its models surprised the team.

There is a strategic message for smaller players as well. If a team of around 100 people, with a strong partner, can deliver a competitive accelerator on this timeline, the barrier to custom silicon looks lower than it did even a year ago. Kahng described the 20-month schedule as likely the best of its class today, though Verkor co-founder David Chin cautioned that Broadcom's involvement was essential and that a team starting from scratch could not replicate it.

Finally, there is an economic angle. Inference is where the day-to-day cost of AI accumulates, and lower latency and power per token can translate into faster agents and cheaper access as demand grows, which is exactly how OpenAI frames the benefit for customers.

Limits and Open Questions

AI did not replace the engineers

Ho and Leary were explicit that chip design cannot be fully automated today. Ho said the claim is not that anyone can build a state-of-the-art frontier accelerator using Codex alone, but that a small, focused team using AI well can reach quality results quickly. Engineers remain the final arbiters of the work.

New models still need new kernels

OpenAI itself notes that supporting each new model family still requires new kernels and model-specific optimizations. AI makes that workshop faster; it does not remove the factory. Foundry capacity is another constraint, since LLM-assisted design lowers the engineering bar but does not create manufacturing slots.

Training is a different game

Coverage at launch, including TechCrunch's, pointed out that Jalapeño is not expected to replace Nvidia for training the largest models. It targets inference, the workload where a purpose-built chip has the clearest advantage.

What comes next

The second generation is the real test. Ho says the team is introducing AI in many more places, including verification and physical design, and Leary noted new tools for automatically analyzing waveforms to speed up debugging. Ho added that Astra and its successors should be very good at chip design. HyperAI reports that OpenAI plans large-scale deployment of Jalapeño before the end of the year.

Frequently Asked Questions