Skip to content

Commit cc57fff

Browse files
kovanclaude
andcommitted
gh-141444: Replace dead URL in urllib.robotparser example
Replace the defunct musi-cal.com URL with python.org in the RobotFileParser documentation example. The old URL is no longer active and the example output was specific to that site's robots.txt file. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 7e2c9bd commit cc57fff

File tree

1 file changed

+2
-11
lines changed

1 file changed

+2
-11
lines changed

Doc/library/urllib.robotparser.rst

Lines changed: 2 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -91,16 +91,7 @@ class::
9191

9292
>>> import urllib.robotparser
9393
>>> rp = urllib.robotparser.RobotFileParser()
94-
>>> rp.set_url("http://www.musi-cal.com/robots.txt")
94+
>>> rp.set_url("http://www.python.org/robots.txt")
9595
>>> rp.read()
96-
>>> rrate = rp.request_rate("*")
97-
>>> rrate.requests
98-
3
99-
>>> rrate.seconds
100-
20
101-
>>> rp.crawl_delay("*")
102-
6
103-
>>> rp.can_fetch("*", "http://www.musi-cal.com/cgi-bin/search?city=San+Francisco")
104-
False
105-
>>> rp.can_fetch("*", "http://www.musi-cal.com/")
96+
>>> rp.can_fetch("*", "http://www.python.org/")
10697
True

0 commit comments

Comments
 (0)