SGLang Logo
GitHub

High-Performance Serving Framework for LLMs and Multimodal Models

SGLang powers fast, scalable inference for large language and multimodal models.

SGLang Data Processing Architecture
Google
NVIDIA
Intel
Modal
Microsoft AI
Cognition
LinkedIn
Sierra
Reflection
MiniMax
RadixArk
CoreWeave
Cerebras
Periodic Labs
Arcee AI
Krea
DeepInfra
Fleet
Boson
Trajectory
LanceDB
HUD
PaleBlueDot AI
Yotta Labs
Ornith
Applied Compute
SF Compute
Netpreme
Inco AI
Eigent
Osmosis
Hark
SemiAnalysis
Runpod
ai&
Liquid AI
Crusoe
Harmonic
Cloudflare
Palo Alto Networks
Decagon
humans&
IBM
Nebius
Samsung
Snowflake
AWS
AMD
Meta
SGLang Summit 2026, presented by LMSYS

Join us in San Francisco

Open foundations for the next era of AI.

November 12–13, 2026
Fort Mason, San Francisco

Join the Summit

Production-Grade Inference

Built for large-scale deployments, delivering reliable, low-latency, high-throughput serving from a single accelerator to distributed clusters.

Model & Hardware Flexibility

Supports large language, vision-language, and diffusion models for text, multimodal understanding, and image and video generation across diverse hardware platforms.

Advanced Optimizations

Incorporates disaggregated prefill/decode, speculative decoding, parallelisms, a zero-overhead scheduler, and optimized GPU kernels.

Get Started in Seconds

Select your preferences and run the deployment command. SGLang is designed to be easy to install and deploy.

1

Install via pip or docker

The easiest ways to get started.

2

Launch the server

Start the server with a single command pointing to your model.

3

Query the API

Use standard OpenAI-compatible endpoints to interact with your model.

Build
Platform
Package
CUDA Version
Run this Command:
uv pip install sglang sglang-kernel \
  --extra-index-url https://sgl-project.github.io/whl/cu129/ \
  --extra-index-url https://download.pytorch.org/whl/cu129 \
  --index-strategy unsafe-best-match

Trusted in Production

Hear from the teams serving their models with SGLang.

NVIDIA

Agentic AI demands faster, more efficient inference and training, enabled by deep co-design across software and hardware. Through our longstanding collaboration with the SGLang team, SGLang and Miles deliver optimized performance across NVIDIA GPU generations, including the latest Vera Rubin.

Ujval Kapasi
Vice President of AI & HPC Frameworks and Libraries, NVIDIA
Cloudflare

Cloudflare’s strength is doing more with less, and in an era where GPU compute is scarce and expensive, SGLang helps us push to get every FLOP out of our GPUs. The team has been a wonderful partner to the open-source community, and we couldn’t ask for better partners in the world of inference.

Michelle Chen
Group PM – AI Platform, Cloudflare
Palo Alto Networks

At Palo Alto Networks, we're exploring SGLang to serve Small Language Models (SLMs) for real-time malicious traffic detection. SGLang's efficient inference lets us meet the low-latency, high-throughput demands of security workloads. We’re excited to be part of the SGLang community as we push this work forward.

Iris Dai
Senior Manager, Software Engineering, Palo Alto Networks
Cerebras

SGLang allows us to build a disaggregated inference stack that pairs flexible prefill with Cerebras’ high-speed decode engine. Together with RadixArk, we’ve demonstrated promising performance gains, and we’re excited to bring this integration into production and continue working on new features and optimizations.

Qi Jin
EVP of Software, Cerebras
CoreWeave

Some of the most important work in inference happens in the open-source community. Bringing SGLang to CoreWeave Inference in preview, gives customers more ways to build with open models and tools, and more control over how they optimize their inference deployments.

Urvashi Chowdhary
VP of Product Management for AI Services, CoreWeave
Nebius

SGLang is pushing open-model serving forward through rapid day-zero support and deep optimization across models and hardware. Our work with the SGLang community shows what becomes possible when a high-performance inference engine meets infrastructure designed to take frontier models from release to production.

Ryan Hanrui Wang
SVP, Applied AI, Nebius
Cognition

SGLang powers RL rollouts and the production inference behind SWE-2, helping us deliver near-frontier coding performance at a lower cost per task. Its scheduling, memory, and speculative-decoding optimizations enable us to use our GPUs efficiently across both training and serving, so we can continue pushing the cost–performance frontier for our users.

Alex Lin
Researcher, Cognition
Hark

At Hark, SGLang is a key part of the inference infrastructure behind our multimodal data curation pipelines. Together with Ray Serve LLM and KubeRay, it lets researchers focus on building better training data.

Kai-Hsun Chen
Infrastructure Lead, Hark
Osmosis AI

At Osmosis, we use SGLang as the rollout engine in our reinforcement learning stack. Its features like native weight-update APIs and LoRA support help us keep inference tightly integrated with training, so we can iterate quickly on models learning to solve difficult tasks at the frontier.

Andy Lyu
CTO and Co-founder, Osmosis AI

Broad Model & Hardware Support

A single engine that runs across various models and hardware.

Supported Hardware

NVIDIA GPUs
AMD GPUs
Google TPU
CPU Servers
Ascend NPUs
XPU
See all supported hardware

Supported Models

DeepSeek
Qwen
GPT-OSS
Llama
Mistral
GLM
See all supported models

Release Notes

v0.5.23Latest

1007 PRs·257 contributors·58 new contributors

New models

Muse Spark OpenBeam-1.0MiniMax-M3.1-FlashMiniCPM-V 4.7KevCosmos3-Nano-SimFastH3 OmniFLUX.3 Image KleinHY-NomasMOSS-Image

Highlights

  • DeepSeek-V4.1 on GB300 gets up to 1.5x more prefill throughput under load and about 28% lower first-token latency on 32K to 64K prompts.

    #44403#45059#45107

  • SGLang low-latency mode: the opt-in NVIDIA megakernel and AMD monokernel plugins run DeepSeek-V4.1-Flash decode as fused persistent kernels, for up to 44% faster single-user decode on GB300 and 75% on MI350X.

    megakernelmonokernel#44945

  • MoE models such as DeepSeek-V4.1-Flash, GLM-5.3-Flash and gpt-oss reach the first token 30% to 58% sooner on 8K prompts on H200.

    #43363
+4 more highlights
See all releases on GitHub

Join the Community

From first-time users to teams debugging complex deployments, the community is open to everyone.

GitHub
Report bugs, request features, and contribute code.
Slack
Chat with the community and get real-time help.
Discussions
Share ideas and discuss the future of SGLang.

For enterprise adoption, deployment, technical consulting, sponsorship, or partnerships, contact sglang@lmsys.org.

⌂