Reasoning Parser

Reasoning Parser#

SGLang supports parsing reasoning content out from “normal” content for reasoning models such as DeepSeek R1.

Supported Models & Parsers#

Model

Reasoning tags

Parser

Notes

DeepSeek‑R1 series

<think> … </think>

deepseek-r1

Supports all variants (R1, R1-0528, R1-Distill)

DeepSeek‑V3 series

<think> … </think>

deepseek-v3

Including DeepSeek‑V3.2. Supports thinking parameter

Standard Qwen3 models

<think> … </think>

qwen3

Supports enable_thinking parameter

Qwen3-Thinking m odels

<think> … </think>

qwen3 or qwen3-thinking

Always generates thinking content

Kimi K2 Thinking

◁think▷ … ◁/think▷

kimi_k2

Uses special thinking delimiters. Also requires --tool-call-parser kimi_k2 for tool use.

GPT OSS

<\|channel\|>analysis<\|message\|> … <\|end\|>

gpt-oss

N/A

Model-Specific Behaviors#

DeepSeek-R1 Family:

  • DeepSeek-R1: No <think> start tag, jumps directly to thinking content

  • DeepSeek-R1-0528: Generates both <think> start and </think> end tags

  • Both are handled by the same deepseek-r1 parser

DeepSeek-V3 Family:

  • DeepSeek-V3.1/V3.2: Hybrid model supporting both thinking and non-thinking modes, use the deepseek-v3 parser and thinking parameter (NOTE: not enable_thinking)

Qwen3 Family:

  • Standard Qwen3 (e.g., Qwen3-2507): Use qwen3 parser, supports enable_thinking in chat templates

  • Qwen3-Thinking (e.g., Qwen3-235B-A22B-Thinking-2507): Use qwen3 or qwen3-thinking parser, always thinks

Kimi K2:

  • Kimi K2 Thinking: Uses special ◁think▷ and ◁/think▷ tags. For agentic tool use, also specify --tool-call-parser kimi_k2.

GPT OSS:

  • GPT OSS: Uses special <|channel|>analysis<|message|> and <|end|> tags

Usage#

Launching the Server#

Specify the --reasoning-parser option.

[1]:

import requests
from openai import OpenAI
from sglang.test.doc_patch import launch_server_cmd
from sglang.utils import wait_for_server, print_highlight, terminate_process

server_process, port = launch_server_cmd(
    "python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-7B --host 0.0.0.0 --reasoning-parser deepseek-r1 --log-level warning"
)

wait_for_server(f"http://localhost:{port}", process=server_process)

/actions-runner/_work/sglang/sglang/python/sglang/launch_server.py:54: UserWarning: 'python -m sglang.launch_server' is still supported, but 'sglang serve' is the recommended entrypoint.
  Example: sglang serve --model-path <model> [options]
  warnings.warn(
Multi-thread loading shards: 100% Completed | 2/2 [00:02<00:00,  1.42s/it]
Compiling num tokens (num_tokens=4): 100%|██████████| 58/58 [00:07<00:00,  7.56it/s]
Capturing num tokens (num_tokens=4 avail_mem=36.94 GB): 100%|██████████| 58/58 [00:03<00:00, 14.60it/s]
/usr/local/lib/python3.10/dist-packages/fastapi/routing.py:120: FastAPIDeprecationWarning: ORJSONResponse is deprecated, FastAPI now serializes data directly to JSON bytes via Pydantic when a return type or response model is set, which is faster and doesn't need a custom response class. Read more in the FastAPI docs: https://fastapi.tiangolo.com/advanced/custom-response/#orjson-or-response-model and https://fastapi.tiangolo.com/tutorial/response-model/
  response = await f(request)


NOTE: Typically, the server runs in a separate terminal.
In this notebook, we run the server and notebook code together, so their outputs are combined.
To improve clarity, the server logs are displayed in the original black color, while the notebook outputs are highlighted in blue.
To reduce the log length, we set the log level to warning for the server, the default log level is info.
We are running those notebooks in a CI environment, so the throughput is not representative of the actual performance.

Note that --reasoning-parser defines the parser used to interpret responses.

OpenAI Compatible API#

Using the OpenAI compatible API, the contract follows the DeepSeek API design established with the release of DeepSeek-R1:

  • reasoning_content: The content of the CoT.

  • content: The content of the final answer.

[2]:

# Initialize OpenAI-like client
client = OpenAI(api_key="None", base_url=f"http://0.0.0.0:{port}/v1")
model_name = client.models.list().data[0].id

messages = [
    {
        "role": "user",
        "content": "What is 1+3?",
    }
]

Non-Streaming Request#

[3]:

response_non_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.6,
    top_p=0.95,
    stream=False,  # Non-streaming
    extra_body={"separate_reasoning": True},
)
print_highlight("==== Reasoning ====")
print_highlight(response_non_stream.choices[0].message.reasoning_content)

print_highlight("==== Text ====")
print_highlight(response_non_stream.choices[0].message.content)
==== Reasoning ====
First, I recognize that the problem is asking for the sum of 1 and 3.

Next, I add the two numbers together: 1 plus 3 equals 4.

Therefore, the final answer is 4.
==== Text ====


**Solution:**

We are asked to find the sum of 1 and 3.

\[
1 + 3 = 4
\]

Therefore, the final answer is \(\boxed{4}\).

Streaming Request#

[4]:

response_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.6,
    top_p=0.95,
    stream=True,  # Non-streaming
    extra_body={"separate_reasoning": True},
)

reasoning_content = ""
content = ""
for chunk in response_stream:
    if chunk.choices[0].delta.content:
        content += chunk.choices[0].delta.content
    if chunk.choices[0].delta.reasoning_content:
        reasoning_content += chunk.choices[0].delta.reasoning_content

print_highlight("==== Reasoning ====")
print_highlight(reasoning_content)

print_highlight("==== Text ====")
print_highlight(content)
==== Reasoning ====
First, I recognize that the problem is asking for the sum of 1 and 3.

Next, I add the two numbers together: 1 plus 3 equals 4.

Therefore, the final answer is 4.
==== Text ====


**Solution:**

We need to find the sum of 1 and 3.

\[
1 + 3 = 4
\]

Therefore, the final answer is \(\boxed{4}\).

Optionally, you can buffer the reasoning content to the last reasoning chunk (or the first chunk after the reasoning content).

[5]:

response_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.6,
    top_p=0.95,
    stream=True,  # Non-streaming
    extra_body={"separate_reasoning": True, "stream_reasoning": False},
)

reasoning_content = ""
content = ""
for chunk in response_stream:
    if chunk.choices[0].delta.content:
        content += chunk.choices[0].delta.content
    if chunk.choices[0].delta.reasoning_content:
        reasoning_content += chunk.choices[0].delta.reasoning_content

print_highlight("==== Reasoning ====")
print_highlight(reasoning_content)

print_highlight("==== Text ====")
print_highlight(content)
==== Reasoning ====
I recognize that the problem is asking for the sum of 1 and 3.

I will add the two numbers together.

The result of adding 1 and 3 is 4.
==== Text ====


**Solution:**

We need to find the sum of 1 and 3.

\[
1 + 3 = 4
\]

**Answer:** \boxed{4}

The reasoning separation is enable by default when specify . To disable it, set the ``separate_reasoning`` option to ``False`` in request.

[6]:

response_non_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.6,
    top_p=0.95,
    stream=False,  # Non-streaming
    extra_body={"separate_reasoning": False},
)

print_highlight("==== Original Output ====")
print_highlight(response_non_stream.choices[0].message.content)
==== Original Output ====
To solve the problem of adding 1 and 3, I first identify the two numbers involved.

Next, I perform the addition of these two numbers to find the sum.

Finally, I conclude that the result of 1 plus 3 is 4.


To solve the addition of 1 and 3, follow these simple steps:

1. **Identify the numbers to add:**
\[
1 \quad \text{and} \quad 3
\]

2. **Perform the addition:**
\[
1 + 3 = 4
\]

3. **Present the final answer:**
\[
\boxed{4}
\]

SGLang Native API#

[7]:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
input = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, return_dict=False
)

gen_url = f"http://localhost:{port}/generate"
gen_data = {
    "text": input,
    "sampling_params": {
        "skip_special_tokens": False,
        "max_new_tokens": 1024,
        "temperature": 0.6,
        "top_p": 0.95,
    },
}
gen_response = requests.post(gen_url, json=gen_data).json()["text"]

print_highlight("==== Original Output ====")
print_highlight(gen_response)

parse_url = f"http://localhost:{port}/separate_reasoning"
separate_reasoning_data = {
    "text": gen_response,
    "reasoning_parser": "deepseek-r1",
}
separate_reasoning_response_json = requests.post(
    parse_url, json=separate_reasoning_data
).json()
print_highlight("==== Reasoning ====")
print_highlight(separate_reasoning_response_json["reasoning_text"])
print_highlight("==== Text ====")
print_highlight(separate_reasoning_response_json["text"])
==== Original Output ====
I recognize that the problem is asking for the sum of 1 and 3.

I start by identifying the two numbers involved in the addition.

Next, I perform the addition operation by combining these numbers.

Finally, I calculate the total to arrive at the answer.


Sure! Let's solve the problem step by step.

**Question:** What is \(1 + 3\)?

**Solution:**

1. **Identify the numbers to add:**
\[
1 \quad \text{and} \quad 3
\]

2. **Perform the addition:**
\[
1 + 3 = 4
\]

**Answer:** \(\boxed{4}\)
/usr/local/lib/python3.10/dist-packages/fastapi/routing.py:328: FastAPIDeprecationWarning: ORJSONResponse is deprecated, FastAPI now serializes data directly to JSON bytes via Pydantic when a return type or response model is set, which is faster and doesn't need a custom response class. Read more in the FastAPI docs: https://fastapi.tiangolo.com/advanced/custom-response/#orjson-or-response-model and https://fastapi.tiangolo.com/tutorial/response-model/
  return await dependant.call(**values)
==== Reasoning ====
I recognize that the problem is asking for the sum of 1 and 3.

I start by identifying the two numbers involved in the addition.

Next, I perform the addition operation by combining these numbers.

Finally, I calculate the total to arrive at the answer.
==== Text ====


Sure! Let's solve the problem step by step.

**Question:** What is \(1 + 3\)?

**Solution:**

1. **Identify the numbers to add:**
\[
1 \quad \text{and} \quad 3
\]

2. **Perform the addition:**
\[
1 + 3 = 4
\]

**Answer:** \(\boxed{4}\)
[8]:

terminate_process(server_process)

Offline Engine API#

[9]:

import sglang as sgl
from sglang.srt.parser.reasoning_parser import ReasoningParser
from sglang.utils import print_highlight

llm = sgl.Engine(model_path="deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
input = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, return_dict=False
)
sampling_params = {
    "max_new_tokens": 1024,
    "skip_special_tokens": False,
    "temperature": 0.6,
    "top_p": 0.95,
}
result = llm.generate(prompt=input, sampling_params=sampling_params)

generated_text = result["text"]  # Assume there is only one prompt

print_highlight("==== Original Output ====")
print_highlight(generated_text)

parser = ReasoningParser("deepseek-r1")
reasoning_text, text = parser.parse_non_stream(generated_text)
print_highlight("==== Reasoning ====")
print_highlight(reasoning_text)
print_highlight("==== Text ====")
print_highlight(text)

Multi-thread loading shards: 100% Completed | 2/2 [00:02<00:00,  1.32s/it]
Compiling num tokens (num_tokens=4): 100%|██████████| 58/58 [00:07<00:00,  7.53it/s]
Capturing num tokens (num_tokens=4 avail_mem=45.73 GB): 100%|██████████| 58/58 [00:03<00:00, 14.67it/s]
==== Original Output ====
I need to add the numbers 1 and 3.

First, I recognize that 1 is a single unit, and 3 represents three units.

By combining these two quantities, I get a total of 4 units.

Therefore, the sum of 1 and 3 is 4.


Sure! Let's solve the addition step by step.

**Problem:**
What is \(1 + 3\)?

**Solution:**
1. Start with the first number:
\(1\)

2. Add the second number:
\(1 + 3\)

3. Combine the quantities:
\(1 + 3 = 4\)

**Answer:**
\(\boxed{4}\)
==== Reasoning ====
I need to add the numbers 1 and 3.

First, I recognize that 1 is a single unit, and 3 represents three units.

By combining these two quantities, I get a total of 4 units.

Therefore, the sum of 1 and 3 is 4.
==== Text ====


Sure! Let's solve the addition step by step.

**Problem:**
What is \(1 + 3\)?

**Solution:**
1. Start with the first number:
\(1\)

2. Add the second number:
\(1 + 3\)

3. Combine the quantities:
\(1 + 3 = 4\)

**Answer:**
\(\boxed{4}\)
[10]:

llm.shutdown()

Supporting New Reasoning Model Schemas#

For future reasoning models, you can implement the reasoning parser as a subclass of BaseReasoningFormatDetector in python/sglang/srt/reasoning_parser.py and specify the reasoning parser for new reasoning model schemas accordingly.

Contents