Skip to content

gh-84232: Fix pydoc docs.python.org link generation #139995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hugovk merged 7 commits into from
Dec 23, 2025
Merged

gh-84232: Fix pydoc docs.python.org link generation #139995

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
da40768
Commit
StanFromIreland Oct 12, 2025
6603b19
Mark files as generated
StanFromIreland Oct 12, 2025
4eed3d3
Fix when `module_docs` is not available, remove exclusions list.
StanFromIreland Oct 13, 2025
33b4f2e
Merge branch 'main' into pydoc-link
StanFromIreland Dec 15, 2025
9166ca7
Update Doc/tools/extensions/pydoc_topics.py
StanFromIreland Dec 15, 2025
d037057
Revert Éric's change
StanFromIreland Dec 15, 2025
ca13520
Remove `--check` from pre-commit config
StanFromIreland Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ Include/opcode_ids.h generated
Include/token.h generated
Lib/_opcode_metadata.py generated
Lib/keyword.py generated
Lib/pydoc_data/topics.py generated
Lib/pydoc_data/module_docs.py generated
Lib/test/certdata/*.pem generated
Lib/test/certdata/*.0 generated
Lib/test/levenshtein_examples.json generated
Expand Down
3 changes: 2 additions & 1 deletion Doc/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ doctest:
pydoc-topics: BUILDER = pydoc-topics
pydoc-topics: build
@echo "Building finished; now run this:" \
"cp build/pydoc-topics/topics.py ../Lib/pydoc_data/topics.py"
"cp build/pydoc-topics/topics.py ../Lib/pydoc_data/topics.py" \
"&& cp build/pydoc-topics/module_docs.py ../Lib/pydoc_data/module_docs.py"

.PHONY: gettext
gettext: BUILDER = gettext
Expand Down
26 changes: 26 additions & 0 deletions Doc/tools/extensions/pydoc_topics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ class PydocTopicsBuilder(TextBuilder):
def init(self) -> None:
super().init()
self.topics: dict[str, str] = {}
self.module_docs: dict[str, str] = {}

def get_outdated_docs(self) -> str:
# Return a string describing what an update build will build.
Expand All @@ -130,6 +131,15 @@ def write_documents(self, _docnames: Set[str]) -> None:
continue
doc_labels.setdefault(docname, []).append((topic_label, label_id))

py_domain = env.domains['py']
for module_name, module_info in py_domain.data['modules'].items():
docname = module_info[0]
if docname.startswith('library/'):
doc_file = docname.replace('library/', '')
self.module_docs[module_name] = (
doc_file + f"#module-{module_name}"
)
Comment on lines 139 to 141
Copy link
Member

@merwok merwok Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.module_docs[module_name] = (
doc_file + f"#module-{module_name}"
)
self.module_docs[module_name] = f"{doc_file}#module-{module_name}"

Copy link
Member Author

@StanFromIreland StanFromIreland Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to revert this I'm afraid, ruff complains and I don't think it's worth adding the ignore.

Copy link
Member

@merwok merwok Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the complaint?

Don’t you find it unpleasant to mix string addition and f-string?

Copy link
Member Author

@StanFromIreland StanFromIreland Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately pre-commit does not give much information:

 Run Ruff (format) on Doc/................................................Failed
- hook id: ruff-format
- exit code: 1

Would reformat: Doc/tools/extensions/pydoc_topics.py
1 file would be reformatted, 13 files already formatted

If you feel strongly than I can apply the change and mark it as ignored?

Copy link
Member

@hugovk hugovk Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove --check from the Ruff formats in .pre-commit-config.yaml so it actually makes the change. It's not much help for it say it would make a change but keep it a secret :)

Copy link
Member Author

@StanFromIreland StanFromIreland Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, what is currently on this branch is what ruff formatted the line to.

Copy link
Member

@hugovk hugovk Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's now like this, which looks good to me:

                self.module_docs[module_name] = (
                    f"{doc_file}#module-{module_name}"
                )


for docname, label_ids in status_iterator(
doc_labels.items(),
"building topics... ",
Expand Down Expand Up @@ -161,6 +171,22 @@ def finish(self) -> None:
"""
self.outdir.joinpath("topics.py").write_text(topics, encoding="utf-8")

module_docs_repr = "\n".join(
f" '{module}': '{doc_file}',"
for module, doc_file in sorted(self.module_docs.items())
)
module_docs = f"""\
# Autogenerated by Sphinx on {asctime()}
# as part of the release process.

module_docs = {{
{module_docs_repr}
}}
"""
self.outdir.joinpath("module_docs.py").write_text(
module_docs, encoding="utf-8"
)


def _display_labels(item: tuple[str, Sequence[tuple[str, str]]]) -> str:
_docname, label_ids = item
Expand Down
19 changes: 14 additions & 5 deletions Lib/pydoc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,12 +577,21 @@ def getdocloc(self, object, basedir=sysconfig.get_path('stdlib')):
'marshal', 'posix', 'signal', 'sys',
'_thread', 'zipimport') or
(file.startswith(basedir) and
not file.startswith(os.path.join(basedir, 'site-packages')))) and
object.__name__ not in ('xml.etree', 'test.test_pydoc.pydoc_mod')):
if docloc.startswith(("http://", "https://")):
docloc = "{}/{}.html".format(docloc.rstrip("/"), object.__name__.lower())
not file.startswith(os.path.join(basedir, 'site-packages'))))):

try:
from pydoc_data import module_docs
except ImportError:
module_docs = None

if module_docs and object.__name__ in module_docs.module_docs:
doc_name = module_docs.module_docs[object.__name__]
if docloc.startswith(("http://", "https://")):
docloc = "{}/{}".format(docloc.rstrip("/"), doc_name)
else:
docloc = os.path.join(docloc, doc_name)
else:
docloc = os.path.join(docloc, object.__name__.lower() + ".html")
docloc = None
else:
docloc = None
return docloc
Expand Down
Loading
Loading