GH-98831: Add some tests for generate_cases.py (#100763)

- This doesn't cover everything (far from it) but it's a start.
- This uses pytest, which isn't ideal, but was quickest to get started.

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>