COMMITS
/ scripts/gen-unicode-data.py March 30, 2026
S
ci : bump ty to 0.0.26 (#21156)
Sigbjørn Skjæret committed
July 7, 2024
C
py : type-check all Python scripts with Pyright (#8341)
compilade committed
June 18, 2024
J
tokenizer : BPE fixes (#7530)
jaime-m-p committed
May 17, 2024
J
Unicode codepoint flags for custom regexs (#7245)
jaime-m-p committed
May 9, 2024
J
llama3 custom regex split (#6965)
jaime-m-p committed
May 5, 2024
B
py : logging and flake8 suppression refactoring (#7081)
Brian committed
May 4, 2024
G
tests : add test-tokenizer-0.sh + fix some tokenizers (#7036)
Georgi Gerganov committed