Allow tokenizer_py to run in oss
Summary: Changes to let tokenizer run in OSS env Reviewed By: larryliu0820 Differential Revision: D53676371 fbshipit-source-id: 96178ff52c74633a570b6fa45f45874d98cd07c3
L
Lucy Qiu committed
3b623f97164a7f46f7641d2244b73200f0c31dae
Parent: 084abda
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 2/13/2024, 3:27:18 AM