Installation — CUDA#

Install SGLang-Omni on Linux with an NVIDIA GPU. Docker is recommended. The current stable release is v0.2.0 on PyPI.

Other platforms#

Using a different device? Choose the installation guide for your hardware: Apple Silicon, Ascend NPU, Intel XPU, Intel CPU, and Moore Threads MUSA.

Install SGLang-Omni#

Choose Docker for a prebuilt CUDA environment, pip / uv to install into your own environment, or From source to work on the code.

Docker (recommended)#

Prerequisites. Use a CUDA-compatible NVIDIA driver, Docker, and the NVIDIA Container Toolkit. The image includes UCX, flash-attn, SGLang, and CUDA.

Pull the image.

docker pull hongccc/sglang-omni:dev

Note

The :dev tag moves with main. For reproducible runs, use the pulled image’s repository digest in place of hongccc/sglang-omni:dev in both the pull and run commands:

docker image inspect --format '{{index .RepoDigests 0}}' hongccc/sglang-omni:dev

Start a container.

docker run -it \
    --shm-size 32g \
    --gpus all \
    --ipc host \
    --network host \
    --privileged \
    hongccc/sglang-omni:dev \
    /bin/zsh

Install SGLang-Omni inside the container.

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

uv pip install --prerelease=allow "sglang-omni==0.2.0"

pip / uv#

Prerequisites. Use a CUDA environment with glibc 2.34 or newer and install these build dependencies first:

UCX 1.20.x with CUDA + verbs — upstream, or reuse flags in docker/Dockerfile.
flash-attn-4 >=4.0.0b18, matching torch==2.14.1 and SGLang 0.5.23’s nvidia-cutlass-dsl 4.8.0 pin.

Prepare the environment.

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

Install SGLang-Omni.

uv pip install --prerelease=allow "sglang-omni==0.2.0"

To install the latest version on the index, run uv pip install --prerelease=allow sglang-omni.

From source#

Use this path for development or unreleased changes. Start with the pip / uv prerequisites.

Get the source.

git clone git@github.com:sgl-project/sglang-omni.git
cd sglang-omni

Install in editable mode.

pip install --upgrade pip
pip install uv

uv venv .venv -p 3.12
source .venv/bin/activate

uv pip install --prerelease=allow -v -e .   # drop -e for a non-editable install

Next steps#

Start the Qwen3-ASR server, send a request, and check the transcript. For model-specific configuration and other request formats, see the Qwen3-ASR Cookbook.

Troubleshooting#

See Troubleshooting for installation errors. Compare uv pip check output against the expected CUDA dependency messages.