SIGN IN SIGN UP

Fix: model fails to load when chat template uses HuggingFace generation tags (#2226)

* chat-format: ignore HuggingFace's {% generation %} chat-template tag

HuggingFace's transformers chat-template extension adds {% generation %}
and {% endgeneration %} tags so trainers can mark generation spans for
loss masking. The tags ship in GGUF tokenizer.chat_template metadata
(SmolLM3 et al), but jinja2's default environment doesn't recognize
them, so Llama() raises TemplateSyntaxError at init for any affected
GGUF, even when the caller passes an explicit chat_format override.

Register a minimal Jinja extension that treats both tags as inert
wrappers: the body between them renders as-is, the markers themselves
emit nothing. No behavioral change for templates that don't use the
tags.

Prior art: PR #2082 attempted the same approach but referenced an
unimported 'nodes' module and didn't consume the body or closing tag.

* fix: simplify generation tag handling

* refactor: rename generation tag extension

---------

Co-authored-by: abetlen <abetlen@gmail.com>
T
Tobias committed
f160bf7a3c3d896d17de464094918d30d4e2504c
Parent: 2c455a5
Committed by GitHub <noreply@github.com> on 5/31/2026, 5:17:58 AM