🚀 Installation

Current stable release: v0.1.3 on PyPI.

Two install paths. Docker is recommended — UCX, flash-attn, sglang, and CUDA are prebuilt.

Intel GPU (XPU)? This page targets NVIDIA CUDA. For Intel Arc GPUs, see Installation — Intel XPU, which uses pyproject_xpu.toml + the PyTorch XPU wheel index instead of the CUDA-only pins below.

🐳 Option A: Docker (recommended)

1. Pull the image

docker pull hongccc/sglang-omni:dev

Only the dev tag is published today. It moves with main — pin by digest for reproducible runs:

docker pull lmsysorg/sglang-omni@sha256:<digest>

2. Run the container

docker run -it \
    --shm-size 32g \
    --gpus all \
    --ipc host \
    --network host \
    --privileged \
    hongccc/sglang-omni:dev \
    /bin/zsh

3. Install sglang-omni inside the container

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

uv pip install --prerelease=allow "sglang-omni==0.1.3"

🛠️ Option B: Manual install

Build prerequisites first:

• UCX 1.20.x with CUDA + verbs — upstream, or reuse flags in docker/Dockerfile.

• flash-attn-4 >=4.0.0b18, matching torch==2.11.0 and SGLang 0.5.16’s nvidia-cutlass-dsl 4.6.0 pin.

Then:

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

uv pip install --prerelease=allow "sglang-omni==0.1.3"

Latest on the index without a pin: uv pip install --prerelease=allow sglang-omni.

Install from source

For development or unreleased changes:

git clone git@github.com:sgl-project/sglang-omni.git
cd sglang-omni

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

uv pip install --prerelease=allow -v -e .   # drop -e for a non-editable install