# i can haz fix again!

## Session Summary

**Date:** July 5, 2026  
**Issue:** [ggml-org/llama.cpp#20305](https://github.com/ggml-org/llama.cpp/issues/20305)  
**Branch:** `ali0une-fixes`

---

## Discovery

The bug was discovered while running the `translategemma-12B-Instruct` model through the llama.cpp router. The server would fail at startup with:

```
common_chat_verify_template: failed to apply template:
While executing CallExpression at line 601, column 31 in source:
Error: Jinja Exception: User role must provide `content` as an iterable with exactly one item.
That item must be a `mapping(type:'text' | 'image', source_lang_code:string, target_lang_code:string, text:string | none, image:string | none)`.
```

The model had worked fine at build 8226 (commit `34df42f7b`) but failed at build 8461 (commit `cea560f48`). The initial suspicion was commit `34df42f7b` itself ("hexagon: add f32 ssm_conv op"), but that commit is unrelated to chat template handling.

## Investigation

### Root Cause Analysis

Tracing the 235 commits between `34df42f7b` and `cea560f48`, the breaking change was identified as **`566059a26`** ("Autoparser - complete refactoring of parser architecture", PR #18675).

This commit removed the dedicated TranslateGemma handler (`common_chat_params_init_translate_gemma`) from `common/chat.cpp`. The old code path was:

```
common_chat_templates_apply_jinja
  -> render_message_to_json (plain string content)
  -> detect [source_lang_code] / [target_lang_code] in template source
  -> common_chat_params_init_translate_gemma (transforms messages to required schema)
  -> apply Jinja template with transformed messages -> works
```

After the refactoring:

```
common_chat_templates_apply_jinja
  -> render_message_to_json (plain string content)
  -> common_chat_try_specialized_template (no TranslateGemma detection)
  -> autoparser fallback
  -> apply Jinja template with untransformed messages -> fails
```

The TranslateGemma Jinja template requires user message `content` to be an array with objects containing `type`, `text`, `source_lang_code`, and `target_lang_code` fields. The new autoparser passes plain string content, so the template's validation check throws.

### Why The Old Handler Worked

The removed handler transformed messages before applying the template:

1. For each user message with string content, it wrapped that string into an array item
2. Added `source_lang_code` and `target_lang_code` fields (defaulting to `en-GB`, overridable via `chat_template_kwargs`)
3. Applied the Jinja template with the transformed messages

This transformation is not something the generic autoparser can do -- it requires model-specific knowledge of the expected schema.

### Related Upstream References

- **Issue #20305** -- the upstream issue tracking this bug, closed as "not planned"; [fix shared with maintainers](https://github.com/ggml-org/llama.cpp/issues/20305#issuecomment-4885937797)
- **PR #18675** -- the refactoring that removed the handler (commit `566059a26`)
- **PR #20956** -- open, attempting to fix by supporting extra fields on content parts

## Fix

### Approach

Restore the TranslateGemma handler in `common_chat_try_specialized_template()` so it intercepts the template before the autoparser fallback.

**Detection:** Check for `[source_lang_code]` and `[target_lang_code]` in the template source string. These substrings appear in `languages[source_lang_code]` / `languages[target_lang_code]` within the TranslateGemma template and are unique to it.

**Transformation:** For each user message with string content, wrap into the required array format:

```json
{
  "type": "text",
  "text": "<original content>",
  "source_lang_code": "<from chat_template_kwargs or en-GB>",
  "target_lang_code": "<from chat_template_kwargs or en-GB>"
}
```

**Placement:** Function defined before `common_chat_try_specialized_template` (line ~975) so no forward declaration is needed. Detection added as the last check in the specialized template chain, just before `return std::nullopt`.

### Safety Verification

| Check | Result |
|-------|--------|
| Detection strings unique to TranslateGemma | Yes -- no other handler or template uses `[source_lang_code]` / `[target_lang_code]` |
| No conflict with Gemma4 detection | Yes -- TranslateGemma template does not contain `'<\|tool_call\>call:'` |
| Pattern matches existing handlers | Yes -- same pattern as LFM2 (build `adjusted_messages`, pass as `messages_override`) |
| API compatible | Yes -- `common_chat_template_direct_apply_impl(tmpl, params, adjusted_messages)` is the current signature |
| Format enum valid | Yes -- uses `COMMON_CHAT_FORMAT_PEG_NATIVE` like all other specialized handlers |

## Testing

| Metric | Before Fix (`cea560f48`) | After Fix (`5740bd414`) |
|--------|------------------------|------------------------|
| Startup | Fails with Jinja exception | Clean, no errors |
| Template verification | `common_chat_verify_template: failed` | Passes |
| Example format | N/A (crashes before) | Renders correctly with language codes |
| Translation response | N/A | Model returns correct translation output |

Test with `translategemma-12B-Instruct-Q4_K_M.gguf`:
- Server starts cleanly, no template errors
- Translation request returns: `<start_of_turn>model\nJohn est dans la cuisine.` (correct French translation)
- The `<start_of_turn>model\n` prefix is expected template output (turn delimiter)

## Deliverables

### Files Created/Modified
- `common/chat.cpp` -- fix patch (+51 lines, 2 hunks: handler function + detection in specialized template chain)
- `llama.cpp-how-we-fixed-translategemma-bug.md` -- this file
- `restore-translategemma-specialized-template-handler.diff` -- clean unified diff for maintainers
- `translategemma-llama.cpp-34df42f7b.log` -- working log (build 8226) for reference
- `translategemma-llama.cpp-cea560f48.log` -- failing log (build 8461) for reference
- `translategemma-12B-Instruct.jinja` -- custom Jinja template file with language mappings

### Git Commit
`5740bd414` -- common : restore TranslateGemma specialized template handler

## How It Started

The `-fit` sleep/wake fix and the checkpoint clustering fix both taught us to look for state being overwritten by refactoring. This time, the pattern was simpler: a dedicated handler that was removed during the autoparser overhaul and never replaced. The failing log made it obvious -- the same `"Neither string content nor typed content is supported"` warning appeared in both working and failing builds, but only the working build had the handler to transform the messages before the template saw them.

What made this fix straightforward was recognizing that TranslateGemma's message schema requirement (array with language codes) is fundamentally different from any other template -- it cannot be handled by generic parsing rules alone.

## Key Takeaways

- **Refactoring can lose model-specific handlers:** The autoparser overhaul was comprehensive but dropped TranslateGemma's dedicated handler without a replacement.
- **Detection strings matter:** `[source_lang_code]` in the template source is unique enough to serve as a reliable fingerprint for this model family.
- **Message transformation belongs in specialized handlers:** When a template expects non-standard message schemas, the autoparser cannot adapt -- a dedicated handler is needed.
- **AI as assistive tool:** Bug discovery, investigation, and solution design were human-led. AI helped trace commit history, verify API compatibility, and format documentation.

---

*Non-native English speaker, French*  
*Assisted-by: llama.cpp:local pi*