Improve transient_internet() again to detect more network errors,

and use it in test_robotparser. Fixes #8574.