LLM Context Benchmarks
Report  Q3 2026 Model  Qwen 3.6 27B Engines  03
Inference benchmark report · 0.5k → 128k tokens

Qwen 3.6
27B Dense, three engines.

A 27-billion-parameter dense model pushed across three inference stacks and two classes of hardware — Apple M5 Max with speculative decoding, Apple M5 Max with 4-bit quantization, and an NVIDIA DGX Spark with NVFP4. Every run is cold: no prefix cache, fresh prompt, 128 generated tokens.

Context range
0.5k – 128k
Engines tested
03
Peak generation
82.8tok/s
Peak prefill
1,077tok/s
§ 01 The contenders

Same model. Different maths under the hood.

Each engine runs Qwen 3.6 27B Dense through a distinct serving strategy — speculative drafting, low-bit quantization, and a GPU-tuned server. The hardware, the quant, and the runtime all move the numbers.

Engine 01 · Apple Silicon

MTPLX

mtplx-qwen36-27b-optimized-speed-mtp
MTP · depth 3turbo profile
Apple M5 Max · 128 GB18 CPU · 40 GPU cores · unified memorySpeculative decoding (draft + verify)
Engine 02 · Apple Silicon

oMLX

Qwen3.6-27B-oQ4-mtp
oQ4 · 4-bitMTP
Apple M5 Max · 128 GB18 CPU · 40 GPU cores · unified memoryOptimized 4-bit quantization on MLX
Engine 03 · NVIDIA

vLLM

nvidia/Qwen3.6-27B-NVFP4
NVFP4 · 4-bit floatcontinuous batch
NVIDIA DGX Spark · 122 GB20 cores · aarch64 · data-center classvLLM server, paged attention
§ 02 At a glance

Four numbers that frame the story.

Averages are computed over the 1k–128k context range so all three engines share the same baseline.

Fastest generation
55.3tok/s
MTPLX · avg
Peaks at 82.8 tok/s on 0.5k context — ~2× the next engine.
Fastest prefill
901.8tok/s
vLLM · avg
Processes prompts at up to 1,077 tok/s on 4k context.
Lowest TTFT @ 128k
202s
vLLM
Reaches first token ~1.9× faster than the Apple engines at 128k.
Memory at 128k
44.4GB
MTPLX peak
Starts at 15.6 GB and stays well inside the 128 GB budget.
§ 03 Key findings

What the data actually says.

01

A laptop with speculative decoding out-generates the data-center GPU.

MTPLX leads generation throughput at every context size — from 70.8 tok/s at 1k down to 26.5 tok/s at 128k. Against vLLM that is a 1.1–2.5× lead, and it holds even at 128k where memory pressure is highest. Multi-token-prediction (MTP) is doing the work: it accepts roughly 8 of every 10 drafted tokens.

02

The DGX Spark is the prefill king — and it widens its lead with scale.

vLLM posts the highest prompt-processing speed at every context, peaking at 1,077 tok/s on 4k. More importantly it retains throughput as context grows: at 128k it still prefill-rates 638 tok/s, where the Apple engines fall to ~340–370. That is why its time-to-first-token at 128k is roughly half.

03

Quantization alone is not enough — oMLX is steady but mid-pack.

The oQ4 build on the same M5 Max is the most consistent Apple engine: generation holds ~40 tok/s flat from 0.5k to 8k before fading. It beats MTPLX on raw prefill between 16k and 128k, but without speculative decoding it can't match MTPLX on generation, and it can't match vLLM on prefill.

04

Both Apple engines live comfortably inside unified memory.

MTPLX scales from 15.6 GB at 0.5k to 44.4 GB at 128k — a 128k-context, 27B model serving on a laptop with 80+ GB to spare. Time-per-output-token climbs from 12 ms to 38 ms over the same range, a clean fingerprint of KV-cache pressure.

§ 04 Throughput curves

Speed, plotted against context length.

Context grows logarithmically from 0.5k to 128k tokens. Each line is one engine; the colored glow marks the leader. Hover any point for exact values.

§ 05 Throughput at scale

How well do they batch?

Concurrent requests (batch 1 → 16) on oMLX and vLLM. Both are servers built to multiplex — throughput per stream rises as the batch fills.

§ 06 Spotlight · speculative decoding

Why MTPLX generates so fast.

Multi-token-prediction drafts 3 candidate tokens per step and verifies them in one forward pass. High acceptance means most drafted tokens are kept — turning each step into several real tokens of output.

§ 07 The full ledger

Every context, every engine.

Switch metrics. The best value in each row is marked — highest for throughput, lowest for time-to-first-token.

Values rounded. Averages (§ 02) span the 1k–128k range. MTPLX & oMLX also ran 0.5k; vLLM started at 1k. "—" = not measured for that engine.
§ 08 Method

How this was measured.

Test setup

  • Prompt source: a fixed literary corpus tokenized to {size}k.txt files (0.5k–128k).
  • Generation budget: 128 tokens per run, task: "summarize the above text".
  • Every run is cold / no-cache — the prefix cache is explicitly bypassed.
  • Context sizes tested: 0.5, 1, 2, 4, 8, 16, 32, 64, 128k (vLLM from 1k).
  • Batch sweep: 1, 2, 4, 8, 16 concurrent requests (oMLX, vLLM).

What the metrics mean

  • Prompt TPS — prefill speed: input tokens processed per second.
  • Generation tok/s — decode speed: output tokens produced per second.
  • TTFT — time-to-first-token: latency from request to first output.
  • TPOT — time-per-output-token: steady-state decode latency (MTPLX).
  • MTP acceptance — share of drafted tokens verified and kept.