Skip to content

[Bug] DeepSeekV3 series model chat_template contains redundant tojson in tool_calls, affecting Multi-Round FunctionCall #11700

@lcc970311-afk

Description

@lcc970311-afk

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

  1. Discrepancy in chat_template for tool_calls:
    A comparison between SGLang's DeepSeekV3 chat_template and Hugging Face's chat_template reveals an additional tojson applied to tool['function']['arguments'] in SGLang's implementation:
    Sglang(./examples/chat_template/tool_chat_template_deepseekv31.jinja):
    {{'<|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments']|tojson + '<|tool▁call▁end|>'}}.
    HuggingFace:
    {{'<|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments'] + '<|tool▁call▁end|>'}}

  2. Impact on Multi-Round FunctionCall (BFCL_V3 Evaluation):
    The extra tojson call causes double-escaping of function arguments, leading to incorrect tool_calls and abnormal FunctionCall scores in multi-round BFCL_V3 evaluations.
    Abnormal Output Example:
    normal_text="Now I'll get the current stock information for Zeta Corp (symbol: ZETA):" calls=[ToolCallItem(tool_index=-1, name='get_stock_info', parameters='"{\\"symbol\\": \\"ZETA\\"}"')]
    Normal Output Example:
    normal_text="I'll retrieve comprehensive information about NVIDIA (NVDA) stock for you." calls=[ToolCallItem(tool_index=-1, name='get_stock_info', parameters='{"symbol": "NVDA"}')]

  3. Proposed Solution/Recommendation:
    It is recommended to refer to other models, such as kimi, and add an arguments type judgment:
    <|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>{% if tool_call['function']['arguments'] is string %}{{ tool_call['function']['arguments'] }}{% else %}{{ tool_call['function']['arguments'] | tojson }}{% endif %}<|tool_call_end|>

Reproduction

Model:DeepSeekV3 & V31 & V32
Dataset:BFCL_V3 Multi-round FunctionCall

python3 -m sglang.launch_server
--model deepseek-ai/DeepSeek-V3.1
--tp 8
--port 30000
--host 0.0.0.0
--mem-fraction-static 0.9
--tool-call-parser deepseekv31
--chat-template ./examples/chat_template/tool_chat_template_deepseekv31.jinja

Environment

H20 + SGlang 053

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions