Revert "Refactor lexer to use regular expressions"

We came across a major performance regression due to this change.
I'm reverting it until we find a better solution.

This reverts commit 23fc73081feb640164615930b36ef185c23a3526.