Issue #25317: Converted doctests in test_tokenize to unittests.

Made test_tokenize discoverable.