SIGN IN SIGN UP

convert: add MiniCPM5 tokenizer support (#23384)

Add minicpm5 pre-tokenizer hash via convert_hf_to_gguf_update.py and
implement hardcoded regex handling in llama-vocab.cpp, consistent with
other BPE pre-tokenizers.

Co-authored-by: zhangtao <zhangtao2@modelbest.cn>
Z
zhangtao2-1 committed
9777256c3130fa3201327bfab44bae187f7caea2
Parent: 7085492
Committed by GitHub <noreply@github.com> on 5/27/2026, 5:08:33 AM