SGLang vs vLLM: Which LLM Inference Engine Is Faster?

Which One Is Actually Faster?

It depends on how much of your traffic repeats itself.

The two engines optimise different bottlenecks. SGLang's RadixAttention keeps a radix tree of KV-cache prefixes and reuses any prefix two requests happen to share, with no configuration. That is a large win in multi-turn chat, agent loops and anything with a long fixed system prompt — and close to no win at all when every prompt is unique. vLLM's PagedAttention attacks a different problem: fragmentation and over-reservation in the KV cache. It pays off on every workload, but it pays off most where memory, not prefix reuse, is what is capping your batch size.

So the question "which is faster" only has an answer once you have described the traffic. The table below is the fastest way to place yourself.

---

Which Engine Should I Pick For My Workload?

This is the table to read if you read nothing else.

If your answer is "some of each", that is normal, and it is the reason both projects are alive. Split by route before you split by engine.

---

What Are SGLang and vLLM?

SGLang and vLLM are the two leading open-source inference servers for running large language models in production. Both expose an OpenAI-compatible HTTP API, both do continuous batching, and both are far faster than calling model.generate() in a loop. They differ in what they cache and how they schedule.

SGLang

SGLang is a serving framework from LMSYS, the group behind Chatbot Arena and Vicuna. The project states that it runs on over 400,000 GPUs worldwide and lists xAI (Grok) and Microsoft Azure (DeepSeek R1 on AMD) among its production users — those are the project's own figures, published on the SGLang repository, not independently audited.

Its core idea is RadixAttention: a radix tree over KV-cache blocks that automatically finds and reuses prefixes shared between requests.

Current release at the time of writing: v0.5.18, published 22 August 2026, which moved the stack to PyTorch 2.13 and removed the torchao integration.

vLLM

vLLM started in the Sky Computing Lab at UC Berkeley and is now a broad community project with unusually wide hardware and model coverage — see the hardware table below.

Its core idea is PagedAttention: treat the KV cache like OS virtual memory, in fixed-size pages allocated on demand. If you want that same PagedAttention core with a much wider set of samplers, the Aphrodite Engine, a vLLM fork, is the drop-in variant to look at.

Current release at the time of writing: v0.27.1, published 11 August 2026.

Both projects release frequently — the two versions above landed eleven days apart. Any benchmark you read, including the ones cited here, is a snapshot of one specific version pair and should be read that way.

---

What Do the Published Benchmarks Actually Claim?

Almost every headline number in this space is a first-party claim. That does not make it wrong, but it does mean the baseline matters more than the multiplier. Here is each widely-quoted figure with its actual source and scope.

Read down that column and the pattern is clear: there is no single, current, independent head-to-head between the two engines. The comparisons that exist are either first-party, or third-party one-offs on one model and one box. Which is why the next section matters more than this one.

---

What Do Practitioners Actually Report?

If you came here from a forum thread looking for the unfiltered version, this is the section you wanted. The most useful public evidence is not a vendor leaderboard — it is the issue trackers, where engineers post the run that surprised them, with the flags attached. Every report below is real and linked, is a snapshot against one specific version pair on one workload, and is now closed.

Three things fall out of that table, and they are the actual takeaways of this page:

1. The winner flips by model family, not just by workload. Issue #9180 is the cleanest evidence: one person, one stack, opposite results for the text and vision variants of the same model.
2. Prefix reuse is the mechanism, so a workload without shared prefixes neuters SGLang's main advantage. Notice that #7471 used a shared-prefix dataset and still went the other way — kernel quality for a specific architecture can outweigh the caching win entirely.
3. On small models the serving layer can become the bottleneck before the GPU does. #21061 is a 0.5B model on an L4, where per-request overhead dominates. Do not extrapolate a 70B result from it, and do not extrapolate it to a 70B either.

If you are scaling past a single node, the routing layer becomes as important as the engine — see our notes on multi-GPU and multi-node inference.

---

RadixAttention vs PagedAttention: What Is the Real Difference?

RadixAttention (SGLang)

RadixAttention uses a radix tree (trie) over cached KV blocks:

How RadixAttention works
├── Automatic prefix discovery across requests
├── Radix tree stores common prefixes once
├── LRU eviction for cache management
├── Cache-aware scheduling maximises reuse
└── No manual configuration required

• Automatic: discovers shared prefixes without you declaring them
• Dynamic: adapts as conversation patterns change
• Best for: multi-turn chat, agents, iterative reasoning, long fixed system prompts

When several requests share a system prompt or a conversation history, that overlap is stored once. Later requests hitting the same prefix skip recomputation.

PagedAttention (vLLM)

PagedAttention treats the KV cache like OS virtual memory:

How PagedAttention works
├── KV cache split into fixed-size "pages"
├── Pages allocated on demand, no upfront guessing
├── Copy-on-write for shared sequences
└── Waste confined to the last block of a sequence

• Memory-efficient: the vLLM launch post puts the residual waste "under 4%", against "60% – 80%" for the systems it was compared to in 2023
• Predictable: fixed page sizes make capacity planning tractable
• Best for: high-concurrency batch work, memory-constrained deployments

Instead of reserving a contiguous block sized for the longest sequence you might see, PagedAttention allocates small pages as the sequence grows.

They Are Not Mutually Exclusive

This is the detail most comparisons skip. vLLM also ships automatic prefix caching (enable_prefix_caching), and SGLang also manages its KV cache in blocks. The difference is emphasis: SGLang's scheduler is built around prefix reuse, while in vLLM it is one component among many. The practical consequence is that a lot of published "X is much slower" results are really prefix caching on versus prefix caching off. Check that flag on both servers before you believe any gap you measure — including the ones in the tables above.

---

Installation: What Do I Actually Run?

What follows is only enough to get each server answering requests. For the full vLLM path — flags, GPU-memory tuning, client setup and the things that break on first run — use the dedicated vLLM setup and installation guide.

SGLang

Broad requirements: a recent CUDA 12 toolchain, an NVIDIA GPU at SM75 or above (T4, RTX 20xx, A10, A100, L4, L40S, H100 and newer), and enough disk for the model plus compiled kernels. Exact Python, CUDA and driver minimums move release to release — take them from the official SGLang installation docs, not from an article.

pip install --upgrade pip
pip install uv
uv pip install "sglang[all]"

Docker, which is what you want in production:

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<your-token>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
    --model-path meta-llama/Llama-3.1-8B-Instruct \
    --host 0.0.0.0 --port 30000

vLLM

Broad requirements: Linux, CUDA 12 recommended, and an NVIDIA GPU at compute capability 7.0 or above (V100, T4, A100, L4, H100 and newer) — or one of the non-NVIDIA backends in the table further down. Current version minimums are in the official vLLM installation docs.

pip install vllm

docker run --gpus all --ipc=host \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    vllm/vllm-openai:latest \
    --model meta-llama/Llama-3.1-8B-Instruct

Pin the version in both cases. Given how often these projects release, "latest" is not a reproducible benchmark configuration.

Benchmarking Them Honestly on Your Own Traffic

The only comparison that will settle this for you is one run on your own request distribution. Both projects ship a benchmark harness that speaks the OpenAI API, so point the same script at both servers:

# SGLang on :30000, vLLM on :8000 — same dataset, same concurrency
python -m sglang.bench_serving \
    --backend sglang --host 127.0.0.1 --port 30000 \
    --dataset-name sharegpt --num-prompts 1000

python -m sglang.bench_serving \
    --backend vllm --host 127.0.0.1 --port 8000 \
    --dataset-name sharegpt --num-prompts 1000

Two rules that decide whether the result means anything: enable prefix caching on both servers or neither, and use a dataset with your real prefix-sharing rate. Benchmarking SGLang on unique one-shot prompts hides its whole advantage; benchmarking it on a dataset where every request shares a 2,000-token preamble manufactures one.

---

How Do the Features Compare?

Speculative Decoding

A small draft model proposes several tokens, the large model validates them in one pass. SGLang implements EAGLE and EAGLE3; vLLM implements EAGLE, Medusa and n-gram. The gain you get is governed by the draft model's acceptance rate on your prompts, which is why neither project's headline speedup transfers reliably — measure it on your traffic before you plan capacity around it.

---

Which Models Does Each Support?

SGLang

DeepSeek: SGLang carries MLA-optimised kernels, and the v0.4 data-parallel attention work was benchmarked on DeepSeek-Coder-V2. It is the usual first choice for running DeepSeek R1 locally.

vLLM

vLLM's Transformers backend is the practical differentiator: if a model exists in HuggingFace Transformers, vLLM can usually serve it on day one, at some performance cost versus a hand-written kernel.

---

What Hardware Does Each Run On?

Hardware breadth is vLLM's clearest structural advantage, and for some teams it settles the question before performance is even discussed. Neither engine targets consumer devices, though — for Apple Silicon, Android or the browser, MLC-LLM's cross-platform inference compiles models to Metal, Vulkan and WebGPU instead. If you are on a single NVIDIA card and chasing raw latency rather than concurrency, TensorRT-LLM is the third option worth pricing. Before any of that, check your model actually fits: see VRAM requirements by model size.

---

How Do I Call Them?

Both expose OpenAI-compatible endpoints, so switching engines is a base-URL change.

SGLang:

python -m sglang.launch_server \
    --model-path meta-llama/Llama-3.1-8B-Instruct \
    --port 30000 --host 0.0.0.0

vLLM:

vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000

Same client, either server:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",  # or :8000 for vLLM
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="default",  # vLLM expects the full model path here
    messages=[{"role": "user", "content": "Explain quantum computing"}],
    temperature=0.7
)
print(response.choices[0].message.content)

Because the API surface matches, running both behind a router and sending each route to whichever engine wins for it is a legitimate design, not a cop-out.

---

Troubleshooting

SGLang: out of memory

python -m sglang.launch_server \
    --model-path meta-llama/Llama-3.1-8B-Instruct \
    --context-length 8192

SGLang: throughput lower than expected on a small model

Check host CPU utilisation before you blame the GPU. SGLang #21061 documents a case on an L4 where the Python routing layer saturated a single core long before the GPU was busy.

vLLM: out of memory

vllm serve meta-llama/Llama-3.1-8B-Instruct \
    --gpu-memory-utilization 0.8

vLLM: slower than SGLang on a prefix-heavy workload

Confirm automatic prefix caching is actually enabled before concluding anything. A large share of "vLLM is slower" comparisons are prefix caching on versus prefix caching off.

---

Common Questions

Is SGLang faster than vLLM?

Not universally. It is usually faster where requests share long prefixes, because RadixAttention reuses them by default. Published third-party reports go both ways: vLLM #18136 recorded roughly 4x for SGLang on Qwen3-32B-AWQ across 4x A10G, while SGLang #7471 recorded roughly 2x for vLLM on Gemma 3 across 2x H100. Both are one workload on one box against one pair of versions.

What is the difference between RadixAttention and PagedAttention?

RadixAttention (SGLang) is about reuse: a radix tree finds prefixes that requests share and stores them once. PagedAttention (vLLM) is about waste: fixed-size pages allocated on demand stop the KV cache reserving memory it never uses. They solve different problems, and both engines now implement some form of each.

Which should I use in production?

Start from the workload table above, then benchmark both on your own request distribution with prefix caching set identically. If your traffic splits — long-lived chat sessions on one route, one-shot batch generation on another — running both behind a router is reasonable.

Can I use SGLang or vLLM with Ollama?

They are alternatives to it, not add-ons. Ollama wraps llama.cpp and optimises for a single user on one machine. SGLang and vLLM optimise for many concurrent users on server GPUs. For personal use, compare Ollama, Jan and LM Studio instead; both SGLang and vLLM work behind Open WebUI if you want a chat front end.

Do I need an H100?

No. Both run on any NVIDIA GPU at compute capability 7.0/SM75 or above — a T4, an L4 or an RTX 20-series card will serve small models. What the GPU decides is which model fits and how much concurrency you get, not whether the software runs.

Why do the benchmark numbers I find online disagree so much?

Because the variables that matter are rarely reported: version pair, prefix-sharing rate in the dataset, whether prefix caching was on for both, quantization, tensor-parallel degree, and concurrency. Change any one and the winner can flip — which is what SGLang #9180 shows happening between the text and vision variants of a single model.

---

Key Takeaways

1. There is no current, independent, published head-to-head between the two. Treat every headline multiplier as a first-party claim or a one-off report.
2. SGLang's advantage is prefix reuse. It scales with how much your requests repeat each other, and approaches zero when they don't.
3. vLLM's advantage is breadth plus memory discipline — more hardware, more model architectures, and PagedAttention's KV-cache accounting.
4. The winner flips by model family, not only by workload — see the vision-versus-text result in SGLang #9180.
5. Both expose OpenAI-compatible APIs, so switching, or running both behind a router, is cheap.
6. Benchmark on your own traffic, with prefix caching configured identically on both sides, or the result is noise.

---

Next Steps

1. Install and tune vLLM if you have already decided on breadth
2. Run DeepSeek R1 locally — the workload where SGLang's kernels matter most
3. Scale across multiple GPUs and nodes once one card is not enough
4. Check VRAM requirements before you pick either
5. Compare local AI tools if this is for personal use rather than a server