Partially revert r78719 - it removed a check that is still needed in some cases (i.e. this will allow Michael to add the test.__main__ support that broke the buildbots previously)