dots.tts

dots.tts#

dots.tts is a text-to-speech model from rednote-hilab. It outputs 48 kHz speech and clones a speaker from a short reference clip plus its transcript.

dots.tts is a continuous-latent model, not a codec model. The backbone emits no audio tokens. Each AR step gives a hidden state; a MeanFlow DiT uses it to sample one latent patch (4 frames × 128 dims); a semantic encoder turns the patch back into the next backbone input; an AudioVAE decodes latents to waveform. No codebook, no token sampler. So temperature and top_k do nothing here — use the solver knobs (num_steps, guidance_scale) instead.

Component

Spec

Backbone

Qwen2 1.5B decoder (28 L, hidden=1536, GQA 12/2)

Acoustic tail

MeanFlow DiT (18 L, hidden=1024, 16 heads) + VAE semantic encoder

Latent patch

4 frames × 128 dims (one patch ≈ 160 ms of audio)

Context length

2,048 tokens

Sample rate

48 kHz

Solver

MeanFlow + Euler, engine-wide num_steps=4 (SOAR: flow matching + CFG, num_steps=10)

Supported checkpoints#

Checkpoint

Status

dots-studio/dots.tts-mf

MeanFlow. Continuous batching, num_steps=4. examples/configs/dots_tts.yaml

dots-studio/dots.tts-soar

Flow matching. Single request at a time (max_running_requests=1) with CFG, num_steps=10. examples/configs/dots_tts_soar.yaml

dots-studio/dots.tts-base

Flow matching, same as SOAR. Serve it with examples/configs/dots_tts_soar.yaml and --model-path dots-studio/dots.tts-base

Prerequisites#

Install sglang-omni by following Installation, then download and launch the server:

hf download dots-studio/dots.tts-mf

sgl-omni serve \
  --model-path dots-studio/dots.tts-mf \
  --config examples/configs/dots_tts.yaml \
  --allowed-local-media-path docs/_static/audio \
  --port 8000

To serve SOAR instead, swap both the checkpoint and the config:

hf download dots-studio/dots.tts-soar

sgl-omni serve \
  --model-path dots-studio/dots.tts-soar \
  --config examples/configs/dots_tts_soar.yaml \
  --allowed-local-media-path docs/_static/audio \
  --port 8000

SOAR is a flow-matching checkpoint. It runs the single-request solver with classifier-free guidance, so its config pins max_running_requests: 1 and num_steps: 10; continuous batching is MeanFlow-only. Every request example below works on either checkpoint — only the model field changes.

examples/configs/dots_tts.yaml is the canonical MeanFlow deployment. It is already tuned; compiled acoustic tail and vocoder (optimize: true, on by default); continuous batching at max_running_requests=16; and the backbone decode CUDA graph. --model-path alone keeps the compiled tail and batching but leaves backbone decode eager, which is slower per request (see Performance). Use the config file.

The examples below read local clips from docs/_static/audio. To fetch reference audio over HTTP instead, allow the domains you need, e.g. --allowed-media-domain huggingface.co.

Synthesizing Speech#

dots.tts needs a reference clip and its transcript. The speaker comes entirely from the reference (x-vector plus prompt latents), so there is no zero-shot voice preset. Under the default continuous-batching deployment, a request without references is rejected.

Voice Cloning#

The reference transcript matters. It is prefixed to your input text so the model can align prompt audio with prompt text. A wrong transcript hurts cloning quality.

  1. Use curl:

curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Have a nice day and enjoy south california sunshine.",
    "references": [{
      "audio_path": "docs/_static/audio/male-voice.wav",
      "text": "Hey, Adam here. Let'\''s create something that feels real, sounds human, and connects every time."
    }],
    "seed": 42
  }' \
  --output output.wav

  1. Use Python

import requests

resp = requests.post(
    "http://localhost:8000/v1/audio/speech",
    json={
        "model": "dots-studio/dots.tts-mf",
        "input": "Have a nice day and enjoy south california sunshine.",
        "references": [{
            "audio_path": "docs/_static/audio/male-voice.wav",
            "text": "Hey, Adam here. Let's create something that feels real, sounds human, and connects every time.",
        }],
        "seed": 42,
    },
)
resp.raise_for_status()
with open("output.wav", "wb") as f:
    f.write(resp.content)

ref_audio / ref_text are accepted as a shorthand for references[0].audio_path / references[0].text.

Streaming#

Streaming lets you play audio while generation is still running, which cuts time-to-first-audio. dots.tts streams raw 48 kHz PCM: the AudioVAE decoder emits a waveform chunk every few latent patches instead of waiting for the whole utterance.

Set "stream": true and "response_format": "pcm":

curl -N -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Get the trust fund to the bank early.",
    "references": [{
      "audio_path": "docs/_static/audio/female-voice.wav",
      "text": "By repeating what students say, teachers can demonstrate that they are listening. By extending what students say."
    }],
    "stream": true,
    "response_format": "pcm",
    "seed": 42
  }' \
  --output output.pcm

The -N flag disables curl’s output buffering so chunks are written as they arrive. The response is 16-bit mono PCM at 48 kHz with no in-band JSON framing; convert it with:

ffmpeg -f s16le -ar 48000 -ac 1 -i output.pcm output.wav

Request parameters#

Top-level fields:

Parameter

Type

Default

Description

model

string

served model

Served dots.tts model identifier

input

string

(required)

Text to synthesize

references

list

(required)

Reference audio for cloning. Each item has audio_path (local path, file URL, data URL, or HTTP URL) and text (transcript). Exactly one reference is accepted

ref_audio / ref_text

string

null

Shorthand for references[0].audio_path / references[0].text

response_format

string

"wav"

Output audio format (wav, mp3, flac, opus, aac, pcm)

stream

bool

false

Enable raw PCM streaming

seed

int

null

Seed for the flow sampler. Fixes the output for a given request

language

string

null

Language tag for the prompt text; auto or auto_detect detects it from the input

instructions

string

null

Style instructions; switches the prompt template to instruction_tts

Solver knobs go under stage_params.latent_engine. They are not top-level fields: CreateSpeechRequest drops unknown top-level keys, so sending num_steps next to input changes nothing and reports no error.

Parameter

Type

Default

Description

speaker_scale

float

1.5

Scales the speaker x-vector before conditioning. Higher values push harder toward the reference timbre

guidance_scale

float

1.2

Classifier-free guidance strength for the flow sampler

eos_threshold

float

0.8

Probability above which the EOS head ends generation. Lower values cut utterances earlier

num_steps

int

4 (MF), 10 (SOAR)

Flow solver steps. MeanFlow fixes this engine-wide: another value fails the request. SOAR and base run one request at a time and honour it

ode_method

string

"euler"

Flow solver. MeanFlow accepts only euler

max_generate_length

int

500

Cap on generated latent patches (≈ 160 ms each), bounded by the engine’s max_generate_length

normalize_text

bool

false

Run the upstream text normalizer (numbers, symbols) before tokenizing

 
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Have a nice day and enjoy south california sunshine.",
    "references": [{
      "audio_path": "docs/_static/audio/male-voice.wav",
      "text": "Hey, Adam here. Let'\''s create something that feels real, sounds human, and connects every time."
    }],
    "seed": 42,
    "stage_params": {"latent_engine": {"speaker_scale": 2.0, "eos_threshold": 0.6}}
  }' \
  --output output.wav

A rejected solver value comes back as HTTP 500 with the engine’s message, for example dots.tts num_steps is fixed for continuous batching. It is a validation failure, not a server fault.

temperature, top_p, and top_k do not apply. The backbone token logits are unused; the acoustic tail is a deterministic flow solve once the seed is fixed.

Performance#

We report throughput on Seed-TTS EN. Client --max-concurrency sweep against a single dots.tts server started from examples/configs/dots_tts.yaml (max_running_requests=16, bf16, num_steps=4, backbone decode CUDA graph and graph-captured acoustic tail on). Each row is the mean of two runs, seed 42. Hardware: 1x H100.

Concurrency

Throughput (req/s)

Mean latency

RTF (per-req)

audio_s/s

WER

1

0.90

1.11 s

0.284

3.58

1.06%

2

1.48

1.35 s

0.329

6.16

1.25%

4

2.43

1.64 s

0.399

10.14

1.36%

8

3.99

2.00 s

0.486

16.65

1.31%

16

4.64

3.43 s

0.830

19.36

1.31%

32

4.43

7.15 s

1.797

18.47

1.27%

Zero failed requests in every run, and no sample above 50% WER. c=1 is a 50-sample latency probe; the other rows use the full 1,088-sample set. WER is measured on the first run of each row.

  • Concurrency — Maximum number of in-flight client requests (--max-concurrency).

  • Throughput (req/s) — Completed requests divided by total benchmark wall-clock time.

  • Mean latency — Average end-to-end time per request (send to full response received).

  • RTF (per-req) — Average ratio of processing time to generated audio duration per request. <1 is faster than real time.

  • audio_s/s — Total seconds of audio produced divided by total benchmark wall-clock time.

  • WER — Corpus word error rate of the generated speech, transcribed with Qwen/Qwen3-ASR-1.7B.

To reproduce, start the server as in Prerequisites and run the benchmark against it:

python -m benchmarks.eval.benchmark_tts_seedtts \
  --meta zhaochenyang20/seed-tts-eval-arrow \
  --model dots-studio/dots.tts-mf \
  --ref-format references \
  --base-url http://127.0.0.1:8000 --port 8000 \
  --lang en --max-concurrency 16 --warmup 8 --seed 42 \
  --generate-only --use-existing-server \
  --output-dir results/dots-seedtts-en-c16

python -m benchmarks.eval.benchmark_tts_seedtts \
  --meta zhaochenyang20/seed-tts-eval-arrow \
  --model dots-studio/dots.tts-mf \
  --ref-format references --lang en --seed 42 \
  --transcribe-only --port 8000 \
  --output-dir results/dots-seedtts-en-c16