Jalapeño's Perf-Per-Watt Number Divides by the Datasheet, Not the Meter

OpenAI's first custom inference chip posted 1.5x to 1.9x more AI work per watt on a public benchmark that records chip power telemetry. The published ratio uses rated TDP instead. Here is how to read that, and how to measure your own.

InferenceX, the benchmark OpenAI used to test its first custom inference chip, uploads chip power telemetry as an artifact of every run. It reports tokens per megawatt. It reports joules per token. Every point on its dashboard links back to the GitHub Actions job that produced it, and the raw artifacts sit there for ninety days.

OpenAI ran Jalapeño on that benchmark and then normalized the efficiency comparison using each accelerator's published chip power rating.

The very next sentence in that post says Jalapeño is rated at 700 watts and that its measured sustained power stayed at or below 550 watts on the workloads tested. The company disclosed the real number for its own chip, in the same paragraph, and then did not put it in the denominator.

That is worth sitting with, because the reflex reading is wrong.

The direction of the error is not what you expect

Take the strongest result. On GPT-OSS 120B, the appendix reports 85,448 mixed tokens per second per kilowatt for Jalapeño against 44,960 for the comparison GB200, roughly 1.9x. The labels underneath name the denominators: package TDP, Jalapeño 700 W, GB200 1,200 W.

Swap in the measured ceiling for the one system whose measured draw we know. At 550 watts instead of 700, Jalapeño's tokens-per-kilowatt rises by about 27 percent, and the 1.9x becomes roughly 2.4x. Same for the other two. DeepSeek R1 MXFP4 against a GB300: 19,641 versus 11,781, about 1.7x on rated power, closer to 2.1x on measured. Kimi K2.5 at 18,195 versus 11,862, about 1.5x rated, roughly 1.9x measured.

The denominator choice makes OpenAI's headline conservative on the side we can check. It is not a thumb on the scale in the obvious direction.

What it does instead is make the ratio impossible to re-derive. Blackwell parts do not sit pinned at package TDP through a benchmark run either, and OpenAI did not publish what the GB200 and GB300 actually drew. If those systems ran proportionally under their ratings, the reported ratios hold. If they ran closer to rating than Jalapeño did, the reported ratios understate. Nobody outside the run can tell, and the harness collected the data that would settle it.

What "package TDP" leaves out

Neither denominator in this comparison is a datacenter number. Package rating covers the accelerator. It does not cover cooling, power delivery losses, the host CPU, or the switch fabric.

That last one matters more here than usual. OpenAI's own architecture section says the network is integral to Jalapeño's design and that its large domain lets an entire workload stay inside one connected system. If the network is doing structural work in the performance story, the network belongs in the power story. A rack-level joules-per-token figure is the one that maps to an electricity bill, and it is not in this post from either side.

There is a smaller ambiguity in the labels too. "Package TDP: GB200 1,200 W" is a number without a stated boundary, and what counts as the package on a Grace Blackwell part is a real question. The post does not answer it.

The numbers that survive all of this

The appendix has figures the denominator debate cannot touch, and they are much larger than the headline.

At the GB200's previous best time between tokens on GPT-OSS 120B, roughly 535 tokens per second per user, Jalapeño delivered 22,935 mixed tokens per kilowatt against 427. That is about 53.7x. On DeepSeek R1, matched at 169 tokens per second per user, 12,258 versus 118, about 104x. On Kimi K2.5, matched at 182, about 56x.

Shift the denominator by 27 percent and 104x is still above 80x. These gaps are not power-accounting artifacts.

They describe something structural. The comparison system can reach that interactivity level, and when it does, its efficiency collapses, because holding one user's tokens flowing that fast means shrinking the batch and giving up the parallelism that made the chip efficient in the first place. That throughput-versus-interactivity tradeoff is the central axis of InferenceX's methodology, and it is the thing Jalapeño is architected to refuse. OpenAI's stated design goal was a chip whose primary job is serving interactive agents, with model state and KV cache placed to minimize movement between prefill and decode.

For anyone building agents, the matched-latency column is the column. Agent latency compounds across steps, so a system that only looks good at peak batch is a system that looks bad for your actual workload. OpenAI led with the softest number in the post and buried the strongest ones under a collapsible heading.

One more thing about the run itself. Jalapeño is not on the InferenceX dashboard's accelerator list, which currently covers NVIDIA H100 through GB300 and AMD MI300X through MI355X. So this is not a dashboard-audited result you can click through to a workflow log. It is OpenAI running an open, reproducible harness in-house and publishing the output. The harness is auditable. This particular run is not, yet.

Put this into practice

Pull the comparison side yourself. InferenceX is open source at SemiAnalysisAI/InferenceX. Every benchmark recipe is a shell script committed to /benchmarks, weekly snapshots of the full result database ship as public GitHub releases, and any chart on the dashboard exports raw CSV. GB200 and GB300 results on gpt-oss-120b and Kimi K2.5 are already there. An hour with that data tells you what the comparison system looked like before OpenAI framed it.

Measure joules per token on your own hardware instead of dividing by TDP. Sample power for the duration of a run and integrate it:

nvidia-smi --query-gpu=power.draw --format=csv,noheader,nounits -l 1 > power.log

Then divide total joules by output tokens. This takes an afternoon and it tends to change what people believe about their own quantization and batching choices, because rated power and drawn power diverge most exactly where you are making those tradeoffs.

Benchmark at your latency target, not at peak. If your agent needs 500 tokens per second per user to feel responsive, peak-batch efficiency tells you close to nothing about what you will pay. Pick the interactivity level your product actually needs and compare there.

Ask what is in the denominator every time. Rated or measured. Measured at what boundary, chip or node or rack. Measured at what interactivity. Those three questions dispose of most efficiency marketing in under a minute, and they work on every vendor, including the ones you like.

Where this is soft

I have not run Jalapeño, and neither has anyone outside OpenAI. Everything above is read off OpenAI's published post and InferenceX's public methodology, and the arithmetic is mine.

TDP normalization is a defensible convention, not a trick. When you are comparing a chip that nobody can buy against systems that ship, the package rating is the one figure both sides definitely have, and using it consistently is a real argument. It also happens to be the choice that makes the result unreproducible by anyone else, and the benchmark was already collecting the alternative.

The comparison configuration matters and the post gives labels rather than full recipes. Serving framework, precision, parallelism, and driver version all move these numbers, and InferenceX exists precisely because those details decide outcomes.

I have also skipped over a claim that deserves its own scrutiny later. OpenAI says AI-generated implementations of selected GPT-OSS attention and mixture-of-experts blocks ran 1.5 to 1.8 times faster than the human expert versions, and that Codex with GPT-Astra brought three unplanned open-weight models to high performance in two months. OpenAI flags, in its own text, that those speedups apply to selected blocks rather than full models. Credit for the self-limitation. It is still a claim about internal work with no external check.

And none of this is a purchase decision. Jalapeño deploys inside OpenAI's own infrastructure starting at the end of the year. You cannot buy one.

What to take from it

The chip is probably very good. The matched-latency numbers are hard to explain any other way, and a design that holds efficiency at high interactivity is exactly what agentic serving has been short of.

The habit worth taking is smaller than the chip. When a vendor publishes an efficiency ratio, find the denominator before you read the numerator. If the denominator is a datasheet value and the vendor also published a measured value somewhere else in the post, you have learned something about the shape of the claim, whichever direction it leans.

Next time, check the appendix first.

Sources: OpenAI, "Jalapeño's first results show industry-leading speed and efficiency in AI inference," August 25, 2026; InferenceX by SemiAnalysis, About and methodology; SemiAnalysisAI/InferenceX on GitHub.