Fix: model fails to load when chat template uses HuggingFace generation tags (#2226)
* chat-format: ignore HuggingFace's {% generation %} chat-template tag
HuggingFace's transformers chat-template extension adds {% generation %}
and {% endgeneration %} tags so trainers can mark generation spans for
loss masking. The tags ship in GGUF tokenizer.chat_template metadata
(SmolLM3 et al), but jinja2's default environment doesn't recognize
them, so Llama() raises TemplateSyntaxError at init for any affected
GGUF, even when the caller passes an explicit chat_format override.
Register a minimal Jinja extension that treats both tags as inert
wrappers: the body between them renders as-is, the markers themselves
emit nothing. No behavioral change for templates that don't use the
tags.
Prior art: PR #2082 attempted the same approach but referenced an
unimported 'nodes' module and didn't consume the body or closing tag.
* fix: simplify generation tag handling
* refactor: rename generation tag extension
---------
Co-authored-by: abetlen <abetlen@gmail.com> T
Tobias committed
f160bf7a3c3d896d17de464094918d30d4e2504c
Parent: 2c455a5
Committed by GitHub <noreply@github.com>
on 5/31/2026, 5:17:58 AM