Writing an InferenceX blog post

InferenceX blog posts are short, evidence-dense technical writeups that anchor a single headline number (e.g. "1.41x cheaper", "3.13x throughput", "7.7x peak throughput in 25 days") to (a) a Pareto chart from the live dashboard, (b) verified per-concurrency tables, and (c) the upstream PR that drove the change.

Posts live at packages/app/content/blog/{slug}.mdx. Images live at packages/app/public/images/{slug}/. The slug matches the MDX filename without .mdx.

Step 0: Get the source of truth from the user

Before writing anything, ask the user for whichever of these they have:

1. A chart image from the InferenceX dashboard (this is the visual that will ship in the post)
2. A CSV export from the dashboard with the underlying rows (this is the authoritative numeric source)
3. An "instant link" / preset URL to the chart on inferencex.semianalysis.com/inference?... (this becomes both the DashboardCTA href and the live-chart link)
4. A /gpu-specs radar image for the SKUs in the comparison (the user clicks "Radar" on the dashboard's GPU specs page, toggles only the SKUs in the post, and screenshots). Used in the "On-Paper Specs" section as the second figure. If the post is a cross-vendor or cross-generation comparison, this is high-value and you should ask for it; if it's a same-SKU version-bump comparison (e.g. SGLang v0.5.5 → v0.5.6 on the same B200 pool), skip the radar.
5. The upstream PR that caused the change (SGLang / vLLM / TRT-LLM)
6. The InferenceX recipe PR that wired it into the benchmark loop
7. Tweet / X post text if there's a marketing framing they want the lede to echo

If the user gives you only the chart, ask for the CSV — the chart is for the figure, the CSV is for the tables. If they give only the CSV, that becomes source of truth even if a chart later appears.

Step 1: Verify the numbers

When chart, CSV, and the InferenceX data dump disagree, use this priority:

1. CSV the user pasted in chat — they exported it from the dashboard, it matches whatever they saw
2. Chart image the user shared — read points off the curves visually; use it only when no CSV is available
3. InferenceX data dump (via the inferencex-data skill) — useful for sanity-check and for fields not in the CSV (TPOT, run IDs, image tags), but the dump can lag the chart by a week or more

Common gotchas:

• Workload mismatch: chart headers can mislead. Verify ISL/OSL from the data itself — 1k/1k and 8k/1k give wildly different tok/s/GPU and $/M tokens numbers. The blog title, lede, tables, and chart caption must all use the same ISL/OSL.
• Latest run only: filter to the highest run_attempt per github_run_id, then take the latest date per (config_id, conc, isl, osl). See the inferencex-data skill for the exact filter.
• Model spec verification: never invent parameter counts. Always WebSearch the model's released specs (total params, active params, expert count, attention type) before writing the architecture paragraph. Cite sources. GLM-5 is not GLM-4.5 — the numbers changed.
• TCO values: pull from the SemiAnalysis AI Cloud TCO Model. Current values (verify if older than a quarter):
  • H100 $1.30, H200 $1.41, B200 $1.95, B300 $2.34, GB200 $2.21, GB300 $2.652
  • MI300X $1.12, MI325X $1.28, MI355X $1.48
• Cost per million tokens formula: $/M tok = TCO_$/GPU/hr * 1e6 / (3600 * tput_per_gpu). Equivalently in Python: cost = tco / (3600 * tput / 1e6). Throughput is per-GPU, so GPU count cancels out for aggregated configs.
• Bandwidth units — keep uni-di vs bi-di and GB/s vs Gbit/s consistent. This is the single most common factor-of-two or factor-of-eight error in scale-up vs scale-out comparisons, and Cursor Bugbot will catch it. NVLink 5 per-GPU is 900 GB/s uni-directional (1.8 TB/s bi-directional). ConnectX-7 InfiniBand / RoCEv2 Ethernet per-GPU is 400 Gbit/s = 50 GB/s uni-directional (100 GB/s bi-di). The NVLink-to-IB/RoCE ratio is 18x in either direction, not 36x. A previous post (gb200-nvl72-kimi-k2-5-vllm-wide-ep-3x-vs-b200.mdx) shipped with 36x because it compared NVLink bi-di against IB uni-di — flag this if you encounter it in older posts. House rule for new posts: always state "uni-directional" or "uni-di" explicitly, and convert Gbit/s to GB/s in the same sentence so readers can audit the math.

Iso-interactivity interpolation — match the chart, not your shell script

The dashboard chart uses a monotone cubic Hermite spline (Steffen 1990, identical to d3.curveMonotoneX) on the upper-left Pareto frontier of (interactivity, throughput). Linear interpolation in a one-off Python REPL will not match what readers see in the chart and will get flagged in review.

Always use the bundled helper:

python3 .claude/skills/write-inferencex-blog/iso_interactivity.py
# stdin:  {"points": [{"interactivity": .., "throughput": .., "cost_per_M": ..}, ...],
#          "target_iv": 18.0, "metric_key": "cost_per_M"}
# stdout: {"value": 0.22}  // or null when target is outside frontier range

Or import it as a module from a small wrapper script if you're computing many rows at once.

Rules to follow because the helper enforces them — but you need to interpret them correctly when writing the table:

• No extrapolation. When the target interactivity falls outside the frontier's [min x, max x], the helper returns null. Render those cells as _unreachable_ (and the ratio column as _∞_ if comparing two dates/configs). Do not invent a value. This is the whole reason the chart code returns null — the recipe physically can't reach that operating point.
• Frontier is always built on (interactivity, throughput). Even when interpolating cost or TPOT or energy, the frontier itself is the upper-left envelope on throughput-vs-interactivity. Other metrics are derived values at frontier knots. This matches interpolateForGPU in the chart code: one frontier, many metrics interpolated against it.
• Multiple recipes (TP=4, TP=8, etc.) for the same hardware go into one points list together. The Pareto operation collapses them into a single combined frontier, exactly as the chart does when both recipes are toggled on.
• The Y values are clamped to the frontier's min/max to prevent cubic-spline overshoot above/below the data. Don't be surprised when the interpolated value sits at a knot value rather than between two knots — that's the spline saying "any value here would overshoot the data."

The canonical source of truth is packages/app/src/components/calculator/interpolation.ts (functions paretoFrontUpperLeft, monotoneSlopes, hermiteInterpolate) and packages/app/src/components/inference/hooks/useInterpolatedTrendData.ts (function interpolateMetricAtInteractivity). If you ever need to change the algorithm, change all three files — the TS pair plus the Python helper — in the same PR. The repository's AGENTS.md codifies this as a hard rule.

How the Pareto frontier behaves between the knots

The frontier is the set of measured (interactivity, throughput) points that are not dominated by any other point — a point is dominated if some other point in the dataset has both higher interactivity AND higher throughput. Geometrically, you sort the points by interactivity ascending, walk from left to right, and keep popping the previous point off the stack as long as the new point's throughput is greater or equal. What survives is the upper-left envelope: a staircase of points where as interactivity decreases (moving left), throughput increases (moving up), monotonically. Everything "inside" that envelope was a worse operating point on both axes simultaneously and is discarded — it could never be chosen in production.

The chart then draws a smooth curve through these surviving knots only. The curve is a piecewise cubic — between each adjacent pair of frontier knots (xᵢ, yᵢ) and (xᵢ₊₁, yᵢ₊₁), the chart draws a Hermite cubic specified by the two endpoint values and two tangent slopes mᵢ, mᵢ₊₁ at each end. The tangents are computed by Steffen's 1990 monotone construction (identical to d3's curveMonotoneX), which has one critical property: the cubic between two knots never overshoots the throughput values at those knots. If two adjacent knots have throughputs 3,000 and 4,000, the curve between them stays inside [3,000, 4,000] — no spurious bumps above 4,000 or dips below 3,000, even if the slopes from neighboring segments would push it that way. This is why simple cubic splines aren't used: they wiggle, and the chart would imply throughput values that the silicon never actually produced.

So when you interpolate at target_iv = 18, the helper does this: (1) finds the bracket [xᵢ, xᵢ₊₁] containing 18, (2) evaluates the Hermite cubic h₀₀·yᵢ + h₁₀·hh·mᵢ + h₀₁·yᵢ₊₁ + h₁₁·hh·mᵢ₊₁ at t = (18 − xᵢ) / (xᵢ₊₁ − xᵢ), and (3) clamps the result to the metric's [min, max] across the entire frontier as a final safety net against any residual overshoot. If 18 is to the left of the smallest frontier x or to the right of the largest, the helper returns null — there is no extrapolation, because the chart code itself draws no curve outside the data range.

What this means for the blog tables: the interpolated values you publish track the shape of the rendered curve between knots, not a straight line. At iso-interactivity points that happen to sit very close to a knot, the published number will land very close to that knot's measured value. In the middle of a wide segment, the spline can sit noticeably above or below the linear-interpolation guess — sometimes by 10% or more on steep parts of the curve. That difference is what readers see in the chart, so it's what the table must show.

Step 2: Slug and image directory

Slug naming follows the pattern of existing posts (see packages/app/content/blog/*.mdx):

• {hardware}-{model}-{framework}-{key-claim-or-number} — e.g. mi355x-kimi-k2-5-vllm-aiter-7x-speedup
• {framework}-{version}-{hardware}-{model}-{key-claim} — e.g. sglang-0-5-6-b200-deepseek-r1-fp4-up-to-1-8x
• Include a comparator when relevant — e.g. gb200-nvl72-kimi-k2-5-vllm-wide-ep-3x-vs-b200

Then mkdir -p packages/app/public/images/{slug}/ and ask the user to drop benchmark-light.png and benchmark-dark.png there (or do it yourself if they shared the files).

Image filename convention. Every image needs both -light.png and -dark.png variants (drop the same file in for both if the user only has one — placeholder is fine). Filenames should describe what's in the image, not its position in the post:

• benchmark-{light,dark}.png — the headline Pareto / throughput / cost chart
• {architecture}-rack-{light,dark}.png — rack diagrams (e.g. gb200-nvl72-rack-light.png)
• {topology}-topology-{light,dark}.png — NVLink / scale-up topology diagrams
• {kernel-or-feature}-timeline-{light,dark}.png — profiler timelines

Never use numeric names (figure1, figure2) — they break when figures get reordered and they tell the next reader nothing.

Step 3: Frontmatter

---
title: 'Punchy headline with the number — under ~75 chars'
subtitle: 'One-sentence explanation of the mechanism, the workload, and the comparator — ~150-200 chars'
date: '2026-MM-DD'
publishDate: '2026-MM-DD'
tags:
  - benchmark
  - gpu
  - inference
  - {model-slug, e.g. glm5, kimi, deepseek}
  - {vendor, e.g. amd, nvidia}
  - {hardware-slug, e.g. mi355x, b200, gb200}
  - {framework, e.g. sglang, vllm, trtllm}
  - {os/runtime, e.g. rocm} (optional)
---

Use date == publishDate == ISO YYYY-MM-DD. The same date appears in the lede ("measured on InferenceX on 2026-MM-DD").

Title and subtitle prefer model name + headline ratio + the interactivity point at which it peaks. Not a laundry list of framework + precision + workload + parallelism — those belong in the body. Compare:

• Good: 'GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user'
• Avoid: 'GB200 NVL72 vs B200 on DeepSeek R1 FP4 Dynamo TRT Disagg: Up to 4.4x Throughput per GPU in the Middle of the Curve'

Title gives the SKUs, the model, the headline number, and the interactivity anchor. Subtitle gives the one-sentence mechanism (e.g. "NVLink scale-up vs RoCEv2 EP cap") plus the workload (precision + ISL/OSL). Frameworks, MTP, and recipe details get a line in the body, not the title.

Step 4: Body structure

Sections, in order:

Lede (1-2 short paragraphs, no heading)

Lead with the headline number and the workload, both in the first sentence:

"14 weeks after GLM-5's release, AMD MI355X SGLang FP8 undercuts NVIDIA B200 SGLang FP8 on cost per million tokens across the entire single-node Pareto frontier on the 8k/1k workload. The peak gap is 1.41x at 18 tok/s/user with MTP ($0.30/M on B200 vs $0.22/M on MI355X — a 40% reduction)..."

Bold the peak ratio in the lede. Second paragraph: name the upstream PRs that made it happen, then end with a short framing line ("Speed is the moat.", "Software is the moat.", etc.) if it fits — don't force it.

<DashboardCTA> immediately after the lede

<DashboardCTA href="{the instant link / preset URL}">
  Click to see the full InferenceX dashboard →
</DashboardCTA>

Use the preset URL the user provided so clicking lands on the exact comparison view, not the bare dashboard. Format: https://inferencex.semianalysis.com/inference?g_model=...&i_prec=...&g_rundate=...&g_runid=...&i_active={hw1}_{fw1}%2C{hw2}_{fw2}&i_metric=y_costh&i_linelabel=1.

<Figure> hero image immediately after the top DashboardCTA

The chart image is the hero of the post — it goes right after the top <DashboardCTA>, before the model / architecture paragraph, so readers see the curves before they read the prose. Do not bury the figure halfway down without one at the top.

<Figure
  srcLight="/images/{slug}/benchmark-light.png"
  srcDark="/images/{slug}/benchmark-dark.png"
  alt="Plain-English description of the chart including model, precision, ISL/OSL, both compared SKUs/frameworks, and any toggles (MTP/non-MTP)"
  caption="Short caption. Note any non-obvious labeling convention used on the chart (e.g. 'Labels denote GPU count per config.')."
/>

Use the same <Figure> block twice: once here as the hero (so the chart anchors the post visually before the reader hits the technical prose), and once more directly below the iso-interactivity table further down (so the chart is right next to the data that derives from it, instead of forcing readers to scroll back up). Both <Figure> blocks are identical — same srcLight/srcDark/alt/caption. The repetition is intentional and matches how readers consume the post.

Architectural diagrams (rack layouts, topology diagrams) go between the architectural prose and the DashboardCTA, not buried lower down. If the post discusses a rack-scale system, scale-up domain, NVLink island, prefill/decode pool topology, or anything where a picture is worth a paragraph, drop it in immediately after the prose that motivates it. Example from gb200-nvl72-vs-b200-disagg-deepseek-r1-fp4-dynamo-trt.mdx: the lede mentions "all 72 GPUs over NVLink 5" → next paragraph explains the 8-GPU NVLink island vs 72-GPU rack → next thing the reader sees is the GB200 NVL72 rack diagram showing the 18 compute trays / 9 NVSwitch5 trays. The visual grounds the technical claim before the data tables appear. Use the same <Figure> block format as the hero chart, with srcLight/srcDark even if dark is a placeholder copy of light.

Model / architecture paragraph

One paragraph naming the model, vendor, release date (use it to compute "N weeks after release" if it sharpens the cadence framing), total/active parameters, expert count + top-K routing, attention mechanism (MLA, NSA/DSA, GQA, etc.), and context window. Always WebSearch to verify these numbers — don't carry over from a prior generation. Cite a source URL inline if the number is non-obvious.

Then a follow-on paragraph that ties the architecture details to why this PR matters on this hardware — e.g. "MI355X's FP8 KV path landed in mid-April, and the resulting decode throughput moved enough that MI355X's lower per-GPU TCO ($1.48/GPU/hr vs B200 at $1.95/GPU/hr per the SemiAnalysis AI Cloud TCO Model) now compounds into a real cost-per-token advantage instead of being swamped by software gaps."

## On-Paper Specs (cross-SKU comparisons only)

Skip this section for same-SKU version-bump posts (e.g. SGLang v0.5.5 → v0.5.6 on the same B200 pool) — the hardware hasn't changed and the reader doesn't need it. For any cross-vendor, cross-generation, or scale-up-domain comparison (B200 vs H200, MI355X vs B200, GB200 NVL72 vs B200 HGX, etc.), include this section between the model/architecture paragraph and ## What Shipped to Make This Happen. It anchors the reader on raw silicon ratios before they hit the recipe details and the measured perf/$ numbers, so the body's "the measured lift is HBM-bound, here's why" framing has somewhere to land.

Structure (~3 elements, in order):

1. One short intro paragraph — name the two SKUs and their generation, then explain that the radar normalizes each axis to the cross-vendor maximum in /gpu-specs, so the visible polygons compress against axes where a different SKU (typically GB200/GB300 NVL72 for scale-up-domain axes, GB300 NVL72 for FP4) sets the ceiling.
2. <Figure> for the radar. Save the user's radar screenshot to packages/app/public/images/{slug}/specs-radar-light.png (and -dark.png — copy the same file in if they only have one). The caption should call out (a) which SKU sets the ceiling on the most-visually-compressed axes and the absolute max value (e.g. "FP4 max is GB300 NVL72 at 15 PFLOP/s/GPU, so B200's 9 PFLOP/s reads ~60%"), (b) that the older-gen SKU reads 0% on the FP4 axis when it has no FP4 tensor cores.
3. Absolute specs table. Pull values directly from packages/app/src/lib/gpu-specs.ts — never paraphrase from memory or a vendor datasheet, because the spec file is the source of truth the dashboard renders. Include both per-GPU and scale-up-domain rows so the reader can audit the implications paragraph.

Standard row set (10 rows, in this order; drop rows that are identical and uninteresting for a same-vendor same-generation comparison):

Spec	SKU A	SKU B	B / A
HBM capacity	memory value	memory value	ratio
HBM bandwidth	memoryBandwidth	memoryBandwidth	ratio
Dense FP4 (TFLOP/s)	fp4 (or — if null)	fp4	ratio (or —)
Dense FP8 (TFLOP/s)	fp8	fp8	ratio
Dense BF16 (TFLOP/s)	bf16	bf16	ratio
Scale-up BW per GPU (uni-di)	scaleUpBandwidth (scaleUpTech)	same	ratio
Scale-up world size	scaleUpWorldSize	same	ratio
Scale-up domain HBM capacity	memory × scaleUpWorldSize	same	ratio
Scale-up domain HBM BW (aggregate)	memoryBandwidth × scaleUpWorldSize	same	ratio
TCO (SemiAnalysis AI Cloud Model)	$X/GPU/hr	$Y/GPU/hr	ratio

Render the FP4 row as — (em-dash, not "N/A") in both the value and ratio columns when the older SKU lacks FP4 tensor cores — this matches the chart's "0% on FP4" rendering and avoids the misleading appearance of an infinite ratio.

4. One "implications" paragraph that turns the raw ratios into a perf/$ bracket. Standard form: "with the same precision and the same recipe, SKU B's perf/$ ceiling vs SKU A is bounded by (FP8 ratio) / (TCO ratio) on a fully compute-bound workload and by (HBM BW ratio) / (TCO ratio) on a fully memory-bandwidth-bound workload (with NVLink BW giving a middle bound at (NVLink BW ratio) / (TCO ratio))." Then state which bracket the post's measured lift lands in and why — this is the bridge to the next section. If the precision step is the headline (FP8 → FP4 on the new SKU), close with "X is the lever that breaks the GEMM ceiling: A has zero FP4 tensor cores, B has Y PFLOP/s, and the resulting precision step compounds N×–M× on top."

## What Shipped to Make This Happen

The technical breakdown. For each PR:

• Link with [{org}/{repo} PR #{n}]({url})
• Name the author (link their GitHub handle)
• Merge date
• One sentence on what the kernel/feature does
• A bullet or two on the per-hardware fusion strategy (this is what makes the post interesting to engineers — fusion shapes differ per generation, e.g. MI355 vs MI300)
• Activation flags (e.g. --kv-cache-dtype fp8_e4m3 --nsa-prefill-backend tilelang)
• Quantitative claims from the PR description (throughput delta, accuracy delta)

If there are multiple PRs (upstream framework + InferenceX recipe), list them in causal order — upstream first, recipe second.

## The Numbers

Intro paragraph: state the workload (ISL/OSL), the disaggregation status (single-node aggregated, or disagg + N prefill / M decode), the measurement date, the cost formula, and the TCO values used.

Then one labeled table per (hardware, framework, spec_method, TP, GPU-count) combination, in this order:

1. Reference recipe being compared against (typically NVIDIA)
2. The headline-winning recipe (typically the AMD config that anchors the cost win)
3. Any additional recipes that fill out the Pareto (e.g. the same vendor's TP=8 arm if TP=4 was the anchor)
4. Non-MTP variants of the above if the post covers both

Table columns: Conc | tok/s/GPU | tok/s/user | TPOT (ms) | $/M tokens. Right-align numerics by using markdown column alignment. Show all measured concurrencies, not just the headline one. Numbers come straight from the CSV; round throughput to 1 decimal, TPOT to 2 decimals, cost to 2 decimals.

## Iso-Interactivity Cost Comparison

This is where the headline ratio gets made explicit. One short intro sentence ("Throughput per GPU at matched interactivity, interpolated along each SKU's Pareto frontier."), one sentence on the _unreachable_ convention if it appears in the table, then the table(s) — MTP and non-MTP if both are in scope, otherwise one.