TTS Process Topology

StageConfig.process is the source of truth for process topology. Omitting a CLI placement option preserves the topology declared by the selected model or YAML config.

For example, a config can make vocoder isolation persistent:

stages:
  - name: vocoder
    process: vocoder

Or it can keep the vocoder in a shared process:

stages:
  - name: vocoder
    process: pipeline

Temporary Placement Overrides

Two CLI options change process placement without editing the source config.

--isolate-stage STAGE puts one stage in a dedicated process:

python -m sglang_omni.cli serve \
  --model-path MODEL \
  --isolate-stage vocoder

--stage-process STAGE=PROCESS reads left to right: place STAGE in PROCESS. Repeating one process name colocates stages in it. For example, the following explicit override produces the topology already declared by the built-in Higgs-TTS config:

python -m sglang_omni.cli serve \
  --model-path bosonai/higgs-tts-3-4b \
  --stage-process preprocessing=tts_frontend \
  --stage-process audio_encoder=tts_frontend

tts_frontend : preprocessing, audio_encoder
pipeline     : tts_engine
vocoder      : vocoder

--isolate-stage X is shorthand for --stage-process X=X, so it can only ever produce a singleton. Use --stage-process for grouped topologies. Naming the same resolved stage more than once, or naming it in both options, is an error. Both options are repeatable, and literal stage names take precedence over role aliases — the stable vocoder role resolves to Ming-Omni-TTS’s model-specific audio_decode stage.

Requesting a stage the model already runs alone is an idempotent no-op: the declared topology and every declared fraction are returned unchanged.

How a Placement Request Is Validated

Moving a stage asks two independent questions, and a model answers them with two separate declarations:

• process_safe_edges() — does the handoff stay correct once it crosses a process boundary? Declared per edge, not per stage, because correctness depends on which handoff is split rather than on which stage moved. Grouping preprocessing with audio_encoder leaves their shared handoff local and only crosses audio_encoder -> tts_engine, so it is checked against that edge alone. Being a non-TP stage is not sufficient, because some stages exchange state through process-local registries a second process cannot read.

• process_edge_resources() — which GPU memory fractions to apply when an override makes that edge cross processes on a GPU already in use. A recommendation, not a capability. An edge with no recommendation is still splittable when the config already declares fractions or nothing else shares its GPU; otherwise placement validation names the stages whose fractions are missing.

An unsupported handoff is reported before a missing fraction, because declaring fractions would not make that split correct.

Applicability by Model

Model	Process-safe edges	Recommended fractions	Unsupported edges
Higgs-TTS	preprocessing -> audio_encoder, audio_encoder -> tts_engine, tts_engine -> vocoder	none needed; the config already declares 0.03 / 0.85 / 0.10	—
FishAudio S2-Pro	preprocessing -> tts_engine, tts_engine -> vocoder	tts_engine -> vocoder	—
Voxtral TTS	tts_generation -> vocoder	tts_generation -> vocoder	preprocessing -> tts_generation
Ming-Omni-TTS	tts_engine -> audio_decode	tts_engine -> audio_decode	preprocessing -> reference_encode, reference_encode -> tts_engine
MOSS-TTS Local (single-GPU)	tts_engine -> vocoder	tts_engine -> vocoder	preprocessing -> tts_engine — preprocessing publishes into a process-local PreparedRequestQueue the AR stage pops
MOSS-TTS Local (split)	none	—	all; placement declares GPU 0 while the codec runs on cuda:1, so the colocated fractions do not describe this topology
Qwen3-TTS	tts_engine -> vocoder	tts_engine -> vocoder	preprocessing -> tts_engine — prepared requests live in _PREPROCESSING_CONTEXT / _PREPARED_REQUESTS, read in-process by the AR engine builder
MOSS-TTS Delay	tts_engine -> vocoder	tts_engine -> vocoder	preprocessing -> tts_engine — same process-local PreparedRequestQueue handoff
Audar-TTS	preprocessing -> reference_encoder, reference_encoder -> tts_engine, tts_engine -> vocoder	none yet — declare fractions before splitting	—
Zonos2	preprocessing -> speaker_encode, speaker_encode -> tts_engine, tts_engine -> vocoder	none yet — declare fractions before splitting	—

Higgs-TTS already groups preprocessing and audio_encoder in a tts_frontend process and places vocoder in its own process by default. Reapplying either placement is a no-op; the options can still fully separate the frontend stages or regroup them under another process name.

Audar-TTS and Zonos2 are declared safe from stage state carried entirely in StagePayload.data, but neither has benchmark coverage yet and neither ships recommended fractions, so a split on a shared GPU fails with the missing-fraction error until the operator declares runtime.resources.total_gpu_memory_fraction for every stage on that GPU.

Resource and Performance Trade-offs

Splitting a stage out creates another OS process and usually another CUDA context. It can improve throughput by overlapping vocoder scheduling and GPU work with generation, but it also changes IPC and serialization paths, can increase idle VRAM, and may duplicate process-local caches or runtime state. Grouping stages that share a cache or a local handoff keeps that cost down, which is what --stage-process expresses and --isolate-stage cannot.

When multiple processes share one GPU, all affected GPU stages must declare compatible runtime.resources.total_gpu_memory_fraction values, and their total must fit the placement limit. Supported models apply recommended fractions to the copied config only when a placement option is present, preserving explicitly configured fractions. An explicit value below the recommendation emits a warning, and conflicting recommendations for one stage are rejected. Omitting both options therefore leaves the declared process topology and the default placement totals unchanged.

These fractions are placement-accounting declarations, not proof of an allocator-enforced runtime limit. A factory receives total_gpu_memory_fraction only when its signature accepts that argument, and an SGLang mem_fraction_static override can represent a different runtime value. Keep runtime overrides consistent with the placement declaration. Unsafe declared same-GPU topologies are rejected before startup.

Performance depends on the model, hardware, concurrency, request shape, and streaming mode. Measure the target workload before making isolation persistent in model or YAML configuration.