chat : add Granite 4.0 chat template with correct tool_call role mapping (#20804)

* chat : add Granite 4.0 chat template with correct tool_call role mapping

Introduce `LLM_CHAT_TEMPLATE_GRANITE_4_0` alongside the existing Granite
3.x template (renamed `LLM_CHAT_TEMPLATE_GRANITE_3_X`).

The Granite 4.0 Jinja template uses `<tool_call>` XML tags and maps the
`assistant_tool_call` role to `<|start_of_role|>assistant<|end_of_role|><|tool_call|>`.
Without a matching C++ handler, the fallback path emits the literal role
`assistant_tool_call` which the model does not recognize, breaking tool
calling when `--jinja` is not used.

Changes:
- Rename `LLM_CHAT_TEMPLATE_GRANITE` to `LLM_CHAT_TEMPLATE_GRANITE_3_X`
  (preserves existing 3.x behavior unchanged)
- Add `LLM_CHAT_TEMPLATE_GRANITE_4_0` enum, map entry, and handler
- Detection: `<|start_of_role|>` + (`<tool_call>` or `<tools>`) → 4.0,
  otherwise → 3.x
- Add production Granite 4.0 Jinja template
- Add tests for both 3.x and 4.0 template paths (C++ and Jinja)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Code review: follow standard format and use common logic in test-chat-template.cpp

* Rename custom_conversation variable for extra_conversation to give it a more meaningful name

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jesus Talavera
2026-04-02 11:28:56 +02:00
committed by GitHub
parent 17193cce34
commit 6137c325a1
5 changed files with 189 additions and 9 deletions
+16
View File
@@ -1929,6 +1929,22 @@ static void test_template_output_peg_parsers(bool detailed_debug) {
// .run();
}
{
// IBM Granite 4.0 (production template shared by h-tiny, h-small, micro)
// Uses <tool_call> XML tags for tool calls, tools in system message
auto tst = peg_tester("models/templates/ibm-granite-granite-4.0.jinja", detailed_debug);
tst.test("Hello, world!\nWhat's up?").expect(message_assist).run();
tst.test(
"<tool_call>\n"
"{\"name\": \"special_function\", \"arguments\": {\"arg1\": 1}}\n"
"</tool_call>")
.tools({ special_function_tool })
.expect(message_assist_call)
.run();
}
{
// ByteDance-Seed-OSS (reasoning and tool calling model)
auto tst = peg_tester("models/templates/ByteDance-Seed-OSS.jinja", detailed_debug);