Skip to content

Commit 4ad7782

Browse files
Fix: Musi-cal to python.org
1 parent 70748bd commit 4ad7782

File tree

1 file changed

+8
-11
lines changed

1 file changed

+8
-11
lines changed

Doc/library/urllib.robotparser.rst

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -91,16 +91,13 @@ class::
9191

9292
>>> import urllib.robotparser
9393
>>> rp = urllib.robotparser.RobotFileParser()
94-
>>> rp.set_url("http://www.musi-cal.com/robots.txt")
94+
>>> rp.set_url("https://www.python.org/robots.txt")
9595
>>> rp.read()
96-
>>> rrate = rp.request_rate("*")
97-
>>> rrate.requests
98-
3
99-
>>> rrate.seconds
100-
20
101-
>>> rp.crawl_delay("*")
102-
6
103-
>>> rp.can_fetch("*", "http://www.musi-cal.com/cgi-bin/search?city=San+Francisco")
104-
False
105-
>>> rp.can_fetch("*", "http://www.musi-cal.com/")
96+
>>> print(rp.request_rate("*"))
97+
None
98+
>>> print(rp.crawl_delay("*"))
99+
None
100+
>>> rp.can_fetch("*", "https://www.python.org/psf/")
106101
True
102+
>>> rp.can_fetch("*", "https://www.python.org/webstats/")
103+
False

0 commit comments

Comments
 (0)