Morph MORPH
SIGN IN SIGN UP
ggml-org / llama.cpp UNCLAIMED

LLM inference in C/C++

0 0 58 C++

Add byte token type when tokenizer.model is not exists (#4641)

* Add byte token type to hf format

* remove unused variable
W
wonjun Jang committed
f56d6077d0c37e6606ac0a4fa3169de70593acfe
Parent: dc68f00
Committed by GitHub <noreply@github.com> on 12/27/2023, 8:37:25 AM