Remove unnecessary flowconfig ignore paths (#26159)

## Summary

By removing them, the flowconfig file will be cleaner and easier to
maintain.

## How did you test this change?

ci green