COMMITS
/ tests/test-tokenizer-1-bpe.cpp December 22, 2025
J
tool/ex/tests: consistently free ctx, then model (#18168)
Johannes Gäßler committed
April 24, 2025
G
cmake : do not include ./src as public for libllama (#13062)
Georgi Gerganov committed
January 12, 2025
G
llama : add `llama_vocab`, functions -> methods, naming (#11110)
Georgi Gerganov committed
January 6, 2025
G
llama : update llama_model API names (#11063)
Georgi Gerganov committed
October 10, 2024
D
common : use common_ prefix for common library functions (#9805)
Diego Devesa committed
July 5, 2024
J
Detokenizer fixes (#8039)
jaime-m-p committed
May 11, 2024
H
llama : lookup word in vocab before doing BPE merges (#7193)
Haoxiang Fei committed
March 11, 2024
G
llama : refactor unicode stuff (#5992)
Georgi Gerganov committed
February 16, 2024
B
ggml : add numa options (#5377)
bmwl committed
February 13, 2024
G
tests : multi-thread the tokenizer tests (#5474)
Georgi Gerganov committed
October 24, 2023
G
Add more tokenizer tests (#3742)
Galunid committed
October 3, 2023
G
Work on the BPE tokenizer (#3252)
goerch committed