catch nasty exception classes with __new__ that doesn't return a exception (closes #11627)

Patch from Andreas Stührk.