Morph MORPH
SIGN IN SIGN UP
dhruvhead / tokenizers UNCLAIMED
0 0 37 Rust

COMMITS

main
May 14, 2026
B
Update daachorse from 1.0.1 to 3.0.0 (#2050)
Ben Beasley committed
1da56fa
April 27, 2026
A
update dev version to 0.23.2-dev.0
Arthur committed
22d54d3
A
ci: release workflow fixes (node + python) (#2043)
Arthur committed
bbe43ad
A
Fix node release (#2034)
Arthur committed
ab0c5d8
A
bindings/python: free-threaded Python (3.14t) support (#2041)
Arthur committed
decd8e0
April 24, 2026
A
update for release (#2033)
Arthur committed
3992692
April 23, 2026
S
BPE cache: per-thread read-through cache to avoid RwLock atomics on hits (#2028)
Sebastian Pop committed
bcdd25b
April 20, 2026
D
Bump follow-redirects in /tokenizers/examples/unstable_wasm/www (#2024)
dependabot[bot] committed
618eb38
April 15, 2026
R
chore: bump doc-builder SHA for PR upload workflow (#2025)
Remy committed
b6b1688
April 10, 2026
A
fix: use uvx --with cairosvg instead of uv pip install --system (#2021)
Arthur committed
19015d6
A
Ci benchmarks (#2019)
Arthur committed
efbcc68
A
🚨 Refactor a bit add_tokens logic: fix bytelevel decode of added tokens + less memory deserialization 🚨 (#1995)
Arthur committed
d863e6e
L
feat: remove string fmt allocation in hot loop in `BPEBuilder::build` (#2010)
Luc Georges committed
51a6e82
A
fix: replace wget/norvig with HF Hub downloads in node and python Makefiles (#2018)
Arthur committed
be55f0c
H
fix(security): remediate workflow vulnerability in .github/workflows/benchmark-trigger.yml (#2016)
hf-security-analysis[bot] committed
fb2ce8d
H
fix(security): remediate workflow vulnerability in .github/workflows/python-release.yml (#2017)
hf-security-analysis[bot] committed
0b413d8
N
🚨 Update release workflow for Python 3.14, dropping 3.9 (#1952)
Nathan Goldbaum committed
765f857
A
feat: CI benchmark regression detection on PRs (#2013)
Arthur committed
5e35576
April 8, 2026
A
bench: improve added_vocab_deserialize to reflect real-world workloads (#2000)
Arthur committed
bd497e3
K
Add unigram sampling (alpha, nbest_size) (#1994)
Ken Sible committed
2827113
April 7, 2026
A
perf: replace aho-corasick with daachorse for added vocabulary matching (#1999)
Arthur committed
fd1df33
April 2, 2026
P
🔒 Pin GitHub Actions to commit SHAs (#2004)
Pauline Bailly-Masson committed
307cb55
L
feat(tokenizer): add early exit for truncation (#1990)
Luc Georges committed
f5fde3d
H
fix(security): remediate workflow vulnerability in .github/workflows/node-release.yml (#2005)
hf-security-analysis[bot] committed
213c69a
A
Fix csec (#2006)
Arthur committed
fb8d126
A
Fix node-release: all platforms, zig cross-compilation, universal macOS (#1970)
Aaron Paterson committed
47c2fbc
March 31, 2026
A
no more s3 (#2002)
Arthur committed
e33e8bf
March 30, 2026
D
Bump serialize-javascript, copy-webpack-plugin and terser-webpack-plugin (#1998)
dependabot[bot] committed
f19a3ca
A
Update doc to be fully automated (#1997)
Arthur committed
abfd5f0
March 27, 2026
D
Bump node-forge in /tokenizers/examples/unstable_wasm/www (#1993)
dependabot[bot] committed
911b9aa