-
Couldn't load subscription status.
- Fork 3.2k
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 5. Please use English, otherwise it will be closed.
Describe the bug
-
Discrepancy in chat_template for tool_calls:
A comparison between SGLang's DeepSeekV3 chat_template and Hugging Face's chat_template reveals an additional tojson applied to tool['function']['arguments'] in SGLang's implementation:
Sglang(./examples/chat_template/tool_chat_template_deepseekv31.jinja):
{{'<|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments']|tojson + '<|tool▁call▁end|>'}}.
HuggingFace:
{{'<|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments'] + '<|tool▁call▁end|>'}} -
Impact on Multi-Round FunctionCall (BFCL_V3 Evaluation):
The extra tojson call causes double-escaping of function arguments, leading to incorrect tool_calls and abnormal FunctionCall scores in multi-round BFCL_V3 evaluations.
Abnormal Output Example:
normal_text="Now I'll get the current stock information for Zeta Corp (symbol: ZETA):" calls=[ToolCallItem(tool_index=-1, name='get_stock_info', parameters='"{\\"symbol\\": \\"ZETA\\"}"')]
Normal Output Example:
normal_text="I'll retrieve comprehensive information about NVIDIA (NVDA) stock for you." calls=[ToolCallItem(tool_index=-1, name='get_stock_info', parameters='{"symbol": "NVDA"}')] -
Proposed Solution/Recommendation:
It is recommended to refer to other models, such as kimi, and add an arguments type judgment:
<|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>{% if tool_call['function']['arguments'] is string %}{{ tool_call['function']['arguments'] }}{% else %}{{ tool_call['function']['arguments'] | tojson }}{% endif %}<|tool_call_end|>
Reproduction
Model:DeepSeekV3 & V31 & V32
Dataset:BFCL_V3 Multi-round FunctionCall
python3 -m sglang.launch_server
--model deepseek-ai/DeepSeek-V3.1
--tp 8
--port 30000
--host 0.0.0.0
--mem-fraction-static 0.9
--tool-call-parser deepseekv31
--chat-template ./examples/chat_template/tool_chat_template_deepseekv31.jinja
Environment
H20 + SGlang 053