Issue #25400: RobotFileParser now correctly returns default values for crawl_delay and request_rate
Initial patch by Peter Wirtz.
B
Berker Peksag committed
9a7bbb2e3f12faaf4110ecd15fb739e94f4bc8f6
Parent: 85c98bf
Initial patch by Peter Wirtz.