0 0 1 Python

Add tokenizer test into OSS (#3926)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/3926

As titled.

Reviewed By: lucylq, kirklandsign

Differential Revision: D58396343

fbshipit-source-id: 6105ee0fdd0c812b4eedfbe54e1bbd10b713fbf2

Mengwei Liu committed 2y ago

76fa85b6a8eabd4b9776f38a3159a6f8cd23909d

Parent: 2152d79

Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 6/11/2024, 6:48:50 PM