# dots.tts

[dots.tts](https://huggingface.co/dots-studio/dots.tts-mf) is a text-to-speech model from rednote-hilab. It outputs 48 kHz speech and clones a speaker from a short reference clip plus its transcript.

dots.tts is a continuous-latent model, not a codec model. The backbone emits no audio tokens. Each AR step gives a hidden state; a MeanFlow DiT uses it to sample one latent patch (4 frames × 128 dims); a semantic encoder turns the patch back into the next backbone input; an AudioVAE decodes latents to waveform. No codebook, no token sampler. So `temperature` and `top_k` do nothing here — use the solver knobs (`num_steps`, `guidance_scale`) instead.

| Component | Spec |
|---|---|
| Backbone | Qwen2 1.5B decoder (28 L, hidden=1536, GQA 12/2) |
| Acoustic tail | MeanFlow DiT (18 L, hidden=1024, 16 heads) + VAE semantic encoder |
| Latent patch | 4 frames × 128 dims (one patch ≈ 160 ms of audio) |
| Context length | 2,048 tokens |
| Sample rate | 48 kHz |
| Solver | MeanFlow + Euler, engine-wide `num_steps=4` (SOAR: flow matching + CFG, `num_steps=10`) |

## Supported checkpoints

| Checkpoint | Status |
|---|---|
| [`dots-studio/dots.tts-mf`](https://huggingface.co/dots-studio/dots.tts-mf) | MeanFlow. Continuous batching, `num_steps=4`. `examples/configs/dots_tts.yaml` |
| [`dots-studio/dots.tts-soar`](https://huggingface.co/dots-studio/dots.tts-soar) | Flow matching. Single request at a time (`max_running_requests=1`) with CFG, `num_steps=10`. `examples/configs/dots_tts_soar.yaml` |
| [`dots-studio/dots.tts-base`](https://huggingface.co/dots-studio/dots.tts-base) | Flow matching, same as SOAR. Serve it with `examples/configs/dots_tts_soar.yaml` and `--model-path dots-studio/dots.tts-base` |

## Prerequisites

Install `sglang-omni` by following [Installation](../get_started/installation.md), then download and launch the server:

```bash
hf download dots-studio/dots.tts-mf

sgl-omni serve \
  --model-path dots-studio/dots.tts-mf \
  --config examples/configs/dots_tts.yaml \
  --allowed-local-media-path docs/_static/audio \
  --port 8000
```

To serve SOAR instead, swap both the checkpoint and the config:

```bash
hf download dots-studio/dots.tts-soar

sgl-omni serve \
  --model-path dots-studio/dots.tts-soar \
  --config examples/configs/dots_tts_soar.yaml \
  --allowed-local-media-path docs/_static/audio \
  --port 8000
```

SOAR is a flow-matching checkpoint. It runs the single-request solver with classifier-free guidance, so its config pins `max_running_requests: 1` and `num_steps: 10`; continuous batching is MeanFlow-only. Every request example below works on either checkpoint — only the `model` field changes.

`examples/configs/dots_tts.yaml` is the canonical MeanFlow deployment. It is already tuned; compiled acoustic tail and vocoder (`optimize: true`, on by default); continuous batching at `max_running_requests=16`; and the backbone decode CUDA graph. `--model-path` alone keeps the compiled tail and batching but leaves backbone decode eager, which is slower per request (see [Performance](#performance)). Use the config file.

The examples below read local clips from `docs/_static/audio`. To fetch reference audio over HTTP instead, allow the domains you need, e.g. `--allowed-media-domain huggingface.co`.

## Synthesizing Speech

dots.tts needs a reference clip and its transcript. The speaker comes entirely from the reference (x-vector plus prompt latents), so there is no zero-shot `voice` preset. Under the default continuous-batching deployment, a request without `references` is rejected.

### Voice Cloning

The reference transcript matters. It is prefixed to your input text so the model can align prompt audio with prompt text. A wrong transcript hurts cloning quality.

1. Use curl:

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Have a nice day and enjoy south california sunshine.",
    "references": [{
      "audio_path": "docs/_static/audio/male-voice.wav",
      "text": "Hey, Adam here. Let'\''s create something that feels real, sounds human, and connects every time."
    }],
    "seed": 42
  }' \
  --output output.wav
```

2. Use Python

```python
import requests

resp = requests.post(
    "http://localhost:8000/v1/audio/speech",
    json={
        "model": "dots-studio/dots.tts-mf",
        "input": "Have a nice day and enjoy south california sunshine.",
        "references": [{
            "audio_path": "docs/_static/audio/male-voice.wav",
            "text": "Hey, Adam here. Let's create something that feels real, sounds human, and connects every time.",
        }],
        "seed": 42,
    },
)
resp.raise_for_status()
with open("output.wav", "wb") as f:
    f.write(resp.content)
```

`ref_audio` / `ref_text` are accepted as a shorthand for `references[0].audio_path` / `references[0].text`.

### Streaming

Streaming lets you play audio while generation is still running, which cuts time-to-first-audio. dots.tts streams raw 48 kHz PCM: the AudioVAE decoder emits a waveform chunk every few latent patches instead of waiting for the whole utterance.

Set `"stream": true` and `"response_format": "pcm"`:

```bash
curl -N -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Get the trust fund to the bank early.",
    "references": [{
      "audio_path": "docs/_static/audio/female-voice.wav",
      "text": "By repeating what students say, teachers can demonstrate that they are listening. By extending what students say."
    }],
    "stream": true,
    "response_format": "pcm",
    "seed": 42
  }' \
  --output output.pcm
```

The `-N` flag disables curl's output buffering so chunks are written as they arrive. The response is 16-bit mono PCM at 48 kHz with no in-band JSON framing; convert it with:

```bash
ffmpeg -f s16le -ar 48000 -ac 1 -i output.pcm output.wav
```

### Request parameters

Top-level fields:

| Parameter | Type | Default | Description |
|---|---|---|---|
| `model` | string | served model | Served dots.tts model identifier |
| `input` | string | (required) | Text to synthesize |
| `references` | list | (required) | Reference audio for cloning. Each item has `audio_path` (local path, file URL, data URL, or HTTP URL) and `text` (transcript). Exactly one reference is accepted |
| `ref_audio` / `ref_text` | string | `null` | Shorthand for `references[0].audio_path` / `references[0].text` |
| `response_format` | string | `"wav"` | Output audio format (`wav`, `mp3`, `flac`, `opus`, `aac`, `pcm`) |
| `stream` | bool | `false` | Enable raw PCM streaming |
| `seed` | int | `null` | Seed for the flow sampler. Fixes the output for a given request |
| `language` | string | `null` | Language tag for the prompt text; `auto` or `auto_detect` detects it from the input |
| `instructions` | string | `null` | Style instructions; switches the prompt template to `instruction_tts` |

Solver knobs go under `stage_params.latent_engine`. They are **not** top-level fields:
`CreateSpeechRequest` drops unknown top-level keys, so sending `num_steps` next to
`input` changes nothing and reports no error.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `speaker_scale` | float | `1.5` | Scales the speaker x-vector before conditioning. Higher values push harder toward the reference timbre |
| `guidance_scale` | float | `1.2` | Classifier-free guidance strength for the flow sampler |
| `eos_threshold` | float | `0.8` | Probability above which the EOS head ends generation. Lower values cut utterances earlier |
| `num_steps` | int | `4` (MF), `10` (SOAR) | Flow solver steps. MeanFlow fixes this engine-wide: another value fails the request. SOAR and base run one request at a time and honour it |
| `ode_method` | string | `"euler"` | Flow solver. MeanFlow accepts only `euler` |
| `max_generate_length` | int | `500` | Cap on generated latent patches (≈ 160 ms each), bounded by the engine's `max_generate_length` |
| `normalize_text` | bool | `false` | Run the upstream text normalizer (numbers, symbols) before tokenizing |

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dots-studio/dots.tts-mf",
    "input": "Have a nice day and enjoy south california sunshine.",
    "references": [{
      "audio_path": "docs/_static/audio/male-voice.wav",
      "text": "Hey, Adam here. Let'\''s create something that feels real, sounds human, and connects every time."
    }],
    "seed": 42,
    "stage_params": {"latent_engine": {"speaker_scale": 2.0, "eos_threshold": 0.6}}
  }' \
  --output output.wav
```

A rejected solver value comes back as HTTP 500 with the engine's message, for example `dots.tts num_steps is fixed for continuous batching`. It is a validation failure, not a server fault.

`temperature`, `top_p`, and `top_k` do not apply. The backbone token logits are unused; the acoustic tail is a deterministic flow solve once the seed is fixed.

### Performance

We report throughput on Seed-TTS EN. Client `--max-concurrency` sweep against a single dots.tts server started from `examples/configs/dots_tts.yaml` (`max_running_requests=16`, bf16, `num_steps=4`, backbone decode CUDA graph and graph-captured acoustic tail on). Each row is the mean of two runs, seed 42. Hardware: **1x H100**.

| Concurrency | Throughput (req/s) | Mean latency | RTF (per-req) | audio_s/s | WER |
|---:|---:|---:|---:|---:|---:|
| 1 | 0.90 | 1.11 s | 0.284 | 3.58 | 1.06% |
| 2 | 1.48 | 1.35 s | 0.329 | 6.16 | 1.25% |
| 4 | 2.43 | 1.64 s | 0.399 | 10.14 | 1.36% |
| 8 | 3.99 | 2.00 s | 0.486 | 16.65 | 1.31% |
| 16 | 4.64 | 3.43 s | 0.830 | 19.36 | 1.31% |
| 32 | 4.43 | 7.15 s | 1.797 | 18.47 | 1.27% |

Zero failed requests in every run, and no sample above 50% WER. c=1 is a 50-sample latency probe; the other rows use the full 1,088-sample set. WER is measured on the first run of each row.

- **Concurrency** — Maximum number of in-flight client requests (`--max-concurrency`).
- **Throughput (req/s)** — Completed requests divided by total benchmark wall-clock time.
- **Mean latency** — Average end-to-end time per request (send to full response received).
- **RTF (per-req)** — Average ratio of processing time to generated audio duration per request. `<1` is faster than real time.
- **audio_s/s** — Total seconds of audio produced divided by total benchmark wall-clock time.
- **WER** — Corpus word error rate of the generated speech, transcribed with `Qwen/Qwen3-ASR-1.7B`.

To reproduce, start the server as in [Prerequisites](#prerequisites) and run the benchmark against it:

```bash
python -m benchmarks.eval.benchmark_tts_seedtts \
  --meta zhaochenyang20/seed-tts-eval-arrow \
  --model dots-studio/dots.tts-mf \
  --ref-format references \
  --base-url http://127.0.0.1:8000 --port 8000 \
  --lang en --max-concurrency 16 --warmup 8 --seed 42 \
  --generate-only --use-existing-server \
  --output-dir results/dots-seedtts-en-c16

python -m benchmarks.eval.benchmark_tts_seedtts \
  --meta zhaochenyang20/seed-tts-eval-arrow \
  --model dots-studio/dots.tts-mf \
  --ref-format references --lang en --seed 42 \
  --transcribe-only --port 8000 \
  --output-dir results/dots-seedtts-en-c16
```