How We Updated Our Fix For TranslateGemma Bug in llama.cpp

i can haz fix update!

Session Summary

Date: July 5, 2026 (original fix) / September 8, 2026 (re-implementation after upstream PR #27764) Issue: ggml-org/llama.cpp#20305 PR: ggml-org/llama.cpp/pull/27764 Branch: ali0une-fixes

Discovery

The bug was discovered while running the translategemma-12B-Instruct model through the llama.cpp router. The server would fail at startup with:

common_chat_verify_template: failed to apply template:
While executing CallExpression at line 601, column 31 in source:
Error: Jinja Exception: User role must provide `content` as an iterable with exactly one item.
That item must be a `mapping(type:'text' | 'image', source_lang_code:string, target_lang_code:string, text:string | none, image:string | none)`.

The model had worked fine at build 8226 (commit 34df42f7b) but failed at build 8461 (commit cea560f48). The initial suspicion was commit 34df42f7b itself ("hexagon: add f32 ssm_conv op"), but that commit is unrelated to chat template handling.

Investigation

Root Cause Analysis

Tracing the 235 commits between 34df42f7b and cea560f48, the breaking change was identified as 566059a26 ("Autoparser - complete refactoring of parser architecture", PR #18675). This commit removed the dedicated TranslateGemma handler (common_chat_params_init_translate_gemma) from common/chat.cpp. The old code path was:

common_chat_templates_apply_jinja
  -> render_message_to_json (plain string content)
  -> detect [source_lang_code] / [target_lang_code] in template source
  -> common_chat_params_init_translate_gemma (transforms messages to required schema)
  -> apply Jinja template with transformed messages -> works

After the refactoring:

common_chat_templates_apply_jinja
  -> render_message_to_json (plain string content)
  -> common_chat_try_specialized_template (no TranslateGemma detection)
  -> autoparser fallback
  -> apply Jinja template with untransformed messages -> fails

The TranslateGemma Jinja template requires user message content to be an array with objects containing type, text, source_lang_code, and target_lang_code fields. The new autoparser passes plain string content, so the template's validation check throws.

Why The Old Handler Worked

The removed handler transformed messages before applying the template:

For each user message with string content, it wrapped that string into an array item
Added source_lang_code and target_lang_code fields (defaulting to en-GB, overridable via chat_template_kwargs)
Applied the Jinja template with the transformed messages

This transformation is not something the generic autoparser can do -- it requires model-specific knowledge of the expected schema.

Related Upstream References

Issue #20305 -- the upstream issue tracking this bug, closed as "stale"; fix shared with maintainers
PR #18675 -- the refactoring that removed the handler (commit 566059a26)
PR #20956 -- open, attempting to fix by supporting extra fields on content parts
PR #27764 -- "chat : split specialized parsers into common/parsers" (merged as 895c045), moved all 14 dedicated template parsers out of common/chat.cpp into one file each under common/parsers/. Kept the template detection in common_chat_try_specialized_template() inside chat.cpp. This PR is what broke the original fix.

Fix (Original, July 5, 2026)

Commit 9f6d3a4c2 restored the handler directly in common/chat.cpp: function body inserted before common_chat_params_init_ministral_3, detection added as the last check in common_chat_try_specialized_template(). This worked, but it lived in a region that upstream was about to restructure.

Upstream Conflict (PR #27764) and Re-implementation (September 8, 2026)

Before updating ali0une-fixes to the new upstream master, we audited all 56 local commits against PR #27764. Only two touched common/chat.cpp:

9f6d3a4c2 (TranslateGemma handler) -- conflicted: its function-body hunk anchored on common_chat_params_init_ministral_3, which the PR deleted from chat.cpp. Its detection hunk was clean (the PR kept the detection function intact).
71cea10a0 (Qwen3-Coder detection token) -- no conflict: it changes a line inside common_chat_try_specialized_template(), a region the PR did not touch.

Resolution:

Archived 9f6d3a4c2 on the ali0une-deprecated branch (as 9e6c6fa38) -- a graveyard for superseded fixes.
Rebased ali0une-fixes without it, then onto upstream master (post-#27764, commit ca86fb222). Zero conflicts.
Re-implemented the fix in the new layout (commit 6bc3a527f), which is non-conflicting with #27764 by construction: its only chat.cpp hunk lands in the detection function that the PR deliberately left alone.

New Implementation (post-#27764 layout)

File	Change
`common/parsers/translate-gemma.cpp`	New file. Handler body verbatim from the original fix, minus `static`, parameter renamed `params` -> `inputs` to match the other parsers, includes `parsers.h`.
`common/parsers/parsers.h`	Declaration of `common_chat_params_init_translate_gemma` (alphabetical, after `qwen3_coder`).
`common/parsers/sources.cmake`	New source added to `LLAMA_CHAT_PARSERS_SOURCES` (explicit list, no globbing).
`common/chat.cpp`	Detection block only: `[source_lang_code]` / `[target_lang_code]` in the template source, after MiniCPM5, before Qwen3-Coder.

The two helper calls (common_chat_template_direct_apply_impl / common_chat_template_generation_prompt_impl) remain valid: #27764 moved them to parsers.h, dropped static, and gave them default arguments there.

Testing

Metric	Before Fix (`cea560f48`)	After Original Fix (`5740bd414`)	After Re-implementation (build.sh, upstream `ca86fb222`)
Startup	Fails with Jinja exception	Clean, no errors	Clean, no errors
Template verification	`common_chat_verify_template: failed`	Passes	Passes
Example format	N/A (crashes before)	Renders correctly with language codes	Renders correctly with language codes
Translation response	N/A	Model returns correct translation output	Verified with a real translation request

Test with translategemma-12B-Instruct-Q4_K_M.gguf:

Server starts cleanly, no template errors
Translation request returns: model\nJohn est dans la cuisine. (correct French translation)
The model\n prefix is expected template output (turn delimiter)

Deliverables

Files Created/Modified

common/parsers/translate-gemma.cpp -- new parser file (47 lines)
common/parsers/parsers.h -- declaration (+2 lines)
common/parsers/sources.cmake -- source list entry (+1 line)
common/chat.cpp -- detection block only (+7 lines)
add-translategemma-specialized-template-parser.diff -- clean unified diff for maintainers
translategemma-12B-Instruct.jinja -- custom Jinja template file with language mappings (in ../translategemma-bug/)

Git Commits

9f6d3a4c2 -- original fix, archived on ali0une-deprecated as 9e6c6fa38
6bc3a527f -- common: add TranslateGemma specialized template parser (current, on ali0une-fixes)

How It Started

The -fit sleep/wake fix taught us to look for state being overwritten by refactoring. This time, the pattern was simpler: a dedicated handler that was removed during the autoparser overhaul and never replaced. The failing log made it obvious -- the same "Neither string content nor typed content is supported" warning appeared in both working and failing builds, but only the working build had the handler to transform the messages before the template saw them. What made this fix straightforward was recognizing that TranslateGemma's message schema requirement (array with language codes) is fundamentally different from any other template -- it cannot be handled by generic parsing rules alone. The second round (post-#27764) added a new lesson: audit local fixes against upstream PRs before updating, not after. The preventive audit caught the #27764 conflict before the rebase, so the update itself was zero-conflict. Archiving the old commit on ali0une-deprecated kept the history of the original fix without dragging a conflicting hunk along.

Key Takeaways

Refactoring can lose model-specific handlers: The autoparser overhaul was comprehensive but dropped TranslateGemma's dedicated handler without a replacement.
Detection strings matter: [source_lang_code] in the template source is unique enough to serve as a reliable fingerprint for this model family.
Message transformation belongs in specialized handlers: When a template expects non-standard message schemas, the autoparser cannot adapt -- a dedicated handler is needed.
Upstream restructuring invalidates placement, not logic: PR #27764 did not change what the fix does, only where it may live. The handler body carried over verbatim; only the file layout changed.
Preventive conflict audits make updates painless: Checking each local commit against an incoming upstream PR (files touched + hunk anchors) before rebasing turned a potential multi-hunk conflict into a clean no-op rebase plus a small, well-placed new commit.
AI as assistive tool: Bug discovery, investigation, and solution design were human-led. AI helped trace commit history, verify API compatibility, audit conflicts, and format documentation.

Non-native English speaker, French Assisted-by: llama.cpp:local pi