gh-79638: Treat an unreachable robots.txt as "disallow all" (GH-138555)
Disallow all access in urllib.robotparser if the robots.txt file is unreachable due to server or network errors.
S
Serhiy Storchaka committed
310fe88994249a5a02e20d1211b8fc067e34aa78
Parent: fbba343
Committed by GitHub <noreply@github.com>
on 5/7/2026, 7:06:57 PM