convert: add MiniCPM5 tokenizer support (#23384)
Add minicpm5 pre-tokenizer hash via convert_hf_to_gguf_update.py and implement hardcoded regex handling in llama-vocab.cpp, consistent with other BPE pre-tokenizers. Co-authored-by: zhangtao <zhangtao2@modelbest.cn>
Z
zhangtao2-1 committed
9777256c3130fa3201327bfab44bae187f7caea2
Parent: 7085492
Committed by GitHub <noreply@github.com>
on 5/27/2026, 5:08:33 AM