COMMITS
May 26, 2026
S
Merge pull request #83 from olewinther/fix/transform-range-bounds-check
sugarme committed
May 20, 2026
S
Merge pull request #76 from ajroetker/fix/transform-range-bounds-check
sugarme committed
S
Merge pull request #81 from alrs/wordpiece-nil
sugarme committed
May 19, 2026
O
Fix idx out-of-bounds panic in TransformRange when removals fall at end of string
Ole Winther committed
April 22, 2026
L
fix(model/wordpiece): prevent assignment to nil map
Lars Lehtonen committed
January 1, 2026
A
Fix index out of range panic in TransformRange
AJ Roetker committed
November 27, 2025
S
Merge pull request #74 from kshvakov/master
sugarme committed
S
Merge branch 'master' into master
sugarme committed
S
Merge pull request #72 from clems4ever/expose-raw-input-seq
sugarme committed
November 26, 2025
K
getCurrentPart[T]
kshvakov committed
K
rm reflect
kshvakov committed
September 26, 2025
C
expose NewRawInputSequence to prevent costly calls to the reflect library
Clément Michaud committed
September 18, 2025
S
Merge pull request #70 from gengzongjie/master
sugarme committed
G
update pkg name to sugarme
gengzongjie committed
G
修复unigram缓存带来的内存增长
gengzongjie committed
September 16, 2025
A
Merge pull request #1 from humf/patch-1
Andy committed
September 15, 2025
M
Update unigram.go
Michael committed
September 9, 2025
G
update pkg name
gengzongjie committed
September 4, 2025
G
fix bug: index out of range panic, when normalizing chinese punctuation marks
gengzongjie committed
August 6, 2025
S
Merge pull request #63 from knights-analytics/pretrained
sugarme committed
S
Merge pull request #62 from knights-analytics/unigram
sugarme committed
August 5, 2025
R
Enable Precompiled Normalizer
Rob Keevil committed
R
Add rust unit tests, and tokenize with byte_fallback and fuse_unk
Rob Keevil committed
R
Revert "fix(normalizer): Correct index calculation for multi-byte chars"
Rob Keevil committed
S
Merge pull request #61 from knights-analytics/unigram
sugarme committed
August 4, 2025
R
Add unigram tokenizer
Rob Keevil committed
July 16, 2025
S
Merge pull request #59 from kensinxie/fix_panic
sugarme committed
July 4, 2025
K
fix(normalizer): Correct index calculation for multi-byte chars
kensinxie committed
June 2, 2025
S
Merge pull request #58 from knights-analytics/cache-dir
sugarme committed
June 1, 2025
R
Initialise CachedDir on first use
Rob Keevil committed