SIGN IN SIGN UP

feat: Add endpoints for tokenize, detokenize and count tokens (#1136)

* Add endpoint to count tokens

* Add tokenize and detokenize endpoints

* Change response key to tokens for tokenize endpoint

* Fix dependency bug

* Cleanup

* Remove example added by mistake

* Move tokenize, detokenize, and count to Extras namespace. Tag existing endpoints

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
F
Felipe Lorenz committed
c139f8b5d50f6f416a24c0ba65983a3fb84bf2f3
Parent: 1f3156d
Committed by GitHub <noreply@github.com> on 3/9/2024, 2:09:00 AM