bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)
Co-Authored-By: Tal Einat <taleinat+github@gmail.com>
R
Rémi Lapeyre committed
8047e0e1c620f69cc21f9ca48b24bf2cdd5c3668
Parent: 3a1d50e
Committed by Tal Einat <taleinat@gmail.com>
on 6/16/2019, 6:48:57 AM