🚀 Installation — Intel CPU#

Installs sglang-omni for CPU-only inference. The default installation targets CUDA, so this path uses the separate pyproject_cpu.toml and the PyTorch CPU wheel index.

--no-build-isolation is required for the editable sglang-omni install.

Why a separate pyproject#

pip install -e . resolves pyproject.toml. In CUDA-oriented checkouts, that can pull CUDA-only wheels and replace a CPU torch stack. pyproject_cpu.toml pins the torch family to CPU wheels and omits accelerator-only packages.

Prerequisites#

Python >= 3.10,<3.13
uv
A CPU build of SGLang from the matching upstream release.
Standard audio runtime libraries such as ffmpeg and libsndfile.

🐳 Option A: Docker#

# Clone the SGLang-omni repository
git clone https://github.com/sgl-project/sglang-omni.git
cd sglang-omni

# Build the docker image
docker build -f docker/cpu.Dockerfile -t sglang-omni:cpu .

# Initiate a docker container
docker run -it --shm-size 32g --ipc host --network host sglang-omni:cpu

The image installs upstream SGLang with its CPU pyproject, then installs sglang-omni with pyproject_cpu.toml. It sets SGLANG_USE_CPU_ENGINE=1 for the runtime.

🛠️ Option B: Manual install#

Create and activate an environment first:

git clone https://github.com/sgl-project/sglang-omni.git
cd sglang-omni
OMNI_DIR="$(pwd)"

uv venv .venv -p 3.12
source .venv/bin/activate
uv pip install --upgrade pip "packaging>=24.2" "setuptools>=77.0.0" wheel

Install the matching CPU SGLang build:

git clone https://github.com/sgl-project/sglang ../sglang
cd ../sglang
git checkout v0.5.18

cd python
cp pyproject_cpu.toml pyproject.toml
uv pip install -e . --no-build-isolation --extra-index-url https://download.pytorch.org/whl/cpu

cd sglang/kernels/aot
cp pyproject_cpu.toml pyproject.toml
uv pip install -e . --no-build-isolation --extra-index-url https://download.pytorch.org/whl/cpu

Install sglang-omni with the CPU pyproject:

cd "$OMNI_DIR"
bash scripts/cpu/install_cpu.sh

Verify#

python -c "import sglang_omni, torch; print(sglang_omni.__file__, torch.__version__)"
which sgl-omni

The torch version should resolve to a CPU build. CPU-specific unit tests live in one directory, so CI (and you) can select them without touching the accelerator suites:

SGLANG_USE_CPU_ENGINE=1 pytest tests/unit_test/cpu -v

SGLANG_USE_CPU_ENGINE=1 is required at runtime. Without it the platform layer reports device_type == "cpu" while is_cpu() stays False, so code that branches on the platform silently takes the accelerator path.

Audio decoding fails with `libtorchcodec_core*.so`#

utils/audio.py decodes through torchcodec, which loads FFmpeg’s shared libraries at import. An unsupported FFmpeg version can surface as the unhelpful Could not load this library error.

torchcodec ships loaders for FFmpeg majors 4–8 only; FFmpeg 9 satisfies none of them. Pin an older major if needed. Docker users get a supported major from apt.

🚀 Installation — Intel CPU

Contents