Skip to content

Commit 0ec7ef7

Browse files
authored
Merge pull request #42 from ayhammouda/release/v0.2.0
chore(release): v0.2.0
2 parents cc0145b + 8cc3ab1 commit 0ec7ef7

6 files changed

Lines changed: 264 additions & 4 deletions

File tree

.github/TEST-STRATEGY.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Test Strategy
2+
3+
Canonical map of **what we test, at which layer, and where the gaps are**.
4+
Pairs with the how-to-run instructions in `CONTRIBUTING.md` and the manual
5+
client runbook in `.github/INTEGRATION-TEST.md`.
6+
7+
Last verified: 2026-05-29 — 284 tests passing, ruff clean, pyright 0 errors.
8+
9+
## 1. The pyramid (current shape)
10+
11+
```
12+
/ stdio E2E \ 9 tests — real MCP process over stdio
13+
/ integration \ ~38 tests — multi-version, publish, cache, phase1
14+
/ unit / service \ ~230 tests — services, retrieval, ingestion, compare
15+
/ contract + regression \ 38 tests — schema snapshots, stability, curated cases
16+
```
17+
18+
This is the right shape: a wide, fast base and a thin, slow top. Keep new
19+
tests pushed **down** the pyramid — only add an stdio E2E test when a bug can
20+
*only* manifest across the process boundary (framing, lifespan DI, stdout
21+
hygiene).
22+
23+
## 2. Expected features → coverage map
24+
25+
The server exposes **6 MCP tools**. Every tool must have at least one
26+
behavioral test and appear in the schema snapshot.
27+
28+
| Tool / feature | Primary tests | Layer | Status |
29+
|------------------------|----------------------------------------------------------------------|------------------|--------|
30+
| `search_docs` | `test_services`, `test_retrieval`, `test_synonyms`, `test_stability` | unit + regression| STRONG |
31+
| `get_docs` | `test_services`, `test_retrieval`, `test_persistent_docs_cache`, `test_mcp_get_docs_cache_smoke` | unit + integration | STRONG |
32+
| `list_versions` | `test_services`, `test_multi_version` | unit + integration| GOOD |
33+
| `compare_versions` | `test_compare_versions` (15), `test_services` | unit | GOOD |
34+
| `lookup_package_docs` | `test_package_docs` (8) | unit only | THIN |
35+
| `detect_python_version`| `test_detection` (12) | unit | GOOD |
36+
37+
Cross-cutting coverage:
38+
39+
- **Schema contract**: `test_schema.py`, `test_schema_snapshot.py` — input/output
40+
JSON schemas for each tool are frozen as fixtures; a wire-shape change fails CI.
41+
- **Multi-version routing**: `test_multi_version.py` — version param resolution and
42+
default fallback across indexed doc sets.
43+
- **Regression**: `test_retrieval_regression.py` (curated query→expected cases) and
44+
`test_stability.py` (property-based invariants that survive CPython doc revisions).
45+
- **Process hygiene**: `test_stdio_smoke.py`, `test_stdio_hygiene.py` — confirm a real
46+
stdio server starts, answers, and keeps stdout free of non-protocol noise.
47+
- **Packaging / CI**: `test_packaging.py`, `test_ci_workflows.py` — installable
48+
artifact + workflow file invariants.
49+
50+
## 3. What to test, by component type
51+
52+
- **Services** (`services/`): business logic in isolation against a `tmp_path`
53+
SQLite fixture. Cover the happy path, every error branch (`DocsServerError`
54+
subclasses), and token-budget trimming.
55+
- **Retrieval/ranking** (`retrieval/`): query parsing, FTS5 behavior, ranker
56+
ordering. Use property assertions (`>= 1 result`, substring match) over exact
57+
content so upstream doc edits don't break the suite.
58+
- **Ingestion** (`ingestion/`): parse valid + deliberately broken `.fjson`
59+
fixtures; assert idempotency on re-publish.
60+
- **Server layer** (`server.py`): thin — it only delegates to services and maps
61+
`DocsServerError → ToolError`. Cover that mapping via stdio smoke, not unit tests.
62+
- **Detection** (`detection.py`): pure environment probing — see gap below.
63+
64+
## 4. Coverage targets
65+
66+
No line-coverage gate is enforced (no `pytest-cov` in the dev deps). The bar is
67+
**behavioral**, not numeric:
68+
69+
- Every public tool has ≥1 happy-path + ≥1 error-path test.
70+
- Every `errors.py` exception type is raised by at least one test.
71+
- Every wire-facing model is pinned by a schema snapshot.
72+
73+
Adopt these as the definition of done for new tools. A line-coverage gate is
74+
optional future work; if added, target the `services/` and `retrieval/`
75+
packages, not `server.py` (intentionally thin) or `__main__.py`.
76+
77+
## 5. Known gaps
78+
79+
1. **`detection.py` — CLOSED (2026-05-29).** `tests/test_detection.py` now
80+
covers all three branches of the fallback chain (`.python-version` file →
81+
`python3` in PATH → `sys.version_info`), `_parse_major_minor` parsing, and
82+
`match_to_indexed` — 12 tests. The isolation pattern (`monkeypatch.chdir` to
83+
escape a real `.python-version`, `monkeypatch.setattr` on `subprocess.run`)
84+
is the reference for testing order-dependent environment probing.
85+
2. **`lookup_package_docs` has no stdio smoke (LOW).** Covered at the service
86+
layer only; the PyPI-allowlist trust boundary is never exercised end-to-end.
87+
3. **No negative version-resolution E2E (LOW).** Unknown-version errors are
88+
unit-tested but not asserted over the stdio boundary.
89+
90+
## 6. Reference cases — `detection.py` (now implemented in `test_detection.py`)
91+
92+
| Case | Expectation |
93+
|----------------------------------------|------------------------------------------|
94+
| `.python-version` file present in cwd | returns `(version, ".python-version file")` |
95+
| `.python-version` malformed / empty | falls through to next source, no crash |
96+
| no file, `python3` on PATH | returns `(version, "python3 in PATH")` |
97+
| no file, no `python3` | returns runtime `(X.Y, "server runtime")`|
98+
| `_parse_major_minor("Python 3.13.2")` | `"3.13"` |
99+
| `_parse_major_minor("no digits here")` | `None` |
100+
| `match_to_indexed("3.13", ["3.13"])` | `"3.13"` |
101+
| `match_to_indexed("3.9", ["3.13"])` | `None` |

CHANGELOG.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,40 @@ All notable changes to `python-docs-mcp-server` are documented here.
44
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
55
this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [0.2.0] — 2026-05-29
8+
9+
### Added
10+
11+
- **New MCP tool: `compare_versions(symbol, v1, v2)`** (Phase 09). Diffs a Python
12+
stdlib symbol between two indexed versions and returns a structured result with
13+
`change=added|removed|changed|unchanged` plus optional `new_in`, `changed_in`,
14+
`deprecated_in`, `signature_delta` (advisory heuristic), `see_also_added`,
15+
`see_also_removed`, `section_diff`, and `note` fields. Token-frugal by design —
16+
emits only changed fields, not full page content. Both versions must be indexed;
17+
an unknown version raises an actionable error naming the available versions. This
18+
brings the server to a **six-tool surface**. ([#41](https://github.com/ayhammouda/python-docs-mcp-server/pull/41))
19+
20+
### Security
21+
22+
- Bumped two transitive dependencies to patched releases:
23+
- `idna` 3.13 → 3.17 — resolves CVE-2026-45409 (ReDoS in `idna.encode()`).
24+
- `starlette` 1.0.0 → 1.2.0 — resolves PYSEC-2026-161 ("BadHost", a `Host`-header
25+
auth bypass that explicitly affects MCP servers).
26+
Both arrive via the `mcp` / `sse-starlette` chain; no direct-dependency or API
27+
changes. `pip-audit` reports no known vulnerabilities after the bump.
28+
29+
### Changed
30+
31+
- `services/compare.py` extractors simplified — precompiled the four Sphinx-directive
32+
regexes and collapsed three near-identical `_extract_*` helpers into one.
33+
34+
### Docs
35+
36+
- README tools table and `.github/INTEGRATION-TEST.md` updated to document the full
37+
six-tool surface including `compare_versions`.
38+
- Added `.github/TEST-STRATEGY.md` — canonical map of test layers, the feature→coverage
39+
matrix, and known gaps.
40+
741
## [0.1.6] — 2026-05-14
842

943
### Fixed

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
44

55
[project]
66
name = "python-docs-mcp-server"
7-
version = "0.1.6"
7+
version = "0.2.0"
88
description = "The canonical Python stdlib oracle for AI coding agents — exact symbols, exact sections, exact versions, offline, always free, always MIT, token-frugal."
99
readme = "README.md"
1010
license = "MIT"

server.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,13 @@
88
"url": "https://github.com/ayhammouda/python-docs-mcp-server",
99
"source": "github"
1010
},
11-
"version": "0.1.6",
11+
"version": "0.2.0",
1212
"packages": [
1313
{
1414
"registryType": "pypi",
1515
"registryBaseUrl": "https://pypi.org",
1616
"identifier": "python-docs-mcp-server",
17-
"version": "0.1.6",
17+
"version": "0.2.0",
1818
"runtimeHint": "uvx",
1919
"transport": {
2020
"type": "stdio"

tests/test_detection.py

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
"""Unit tests for environment Python-version detection (detection.py).
2+
3+
Closes the long-standing gap: ``detect_python_version`` backs 1 of the 6
4+
public MCP tools but had no dedicated coverage. See ``.github/TEST-STRATEGY.md``
5+
section 5/6.
6+
7+
Detection is a *fallback chain*:
8+
1. ``.python-version`` file in cwd
9+
2. ``python3 --version`` on PATH
10+
3. ``sys.version_info`` (server runtime)
11+
12+
To test any branch in isolation we must neutralize the branches *above* it:
13+
escape the dev machine's real ``.python-version`` with ``monkeypatch.chdir``,
14+
and control the ``python3`` probe by patching ``subprocess.run``.
15+
"""
16+
from __future__ import annotations
17+
18+
import subprocess
19+
import sys
20+
21+
import pytest
22+
23+
from mcp_server_python_docs import detection
24+
from mcp_server_python_docs.detection import (
25+
_parse_major_minor,
26+
detect_python_version,
27+
match_to_indexed,
28+
)
29+
30+
# ── _parse_major_minor: pure regex extraction ──────────────────────
31+
32+
@pytest.mark.parametrize(
33+
"raw, expected",
34+
[
35+
("3.13.2", "3.13"),
36+
("Python 3.13.2", "3.13"),
37+
("cpython-3.13", "3.13"),
38+
("3.9", "3.9"),
39+
("no digits here", None),
40+
("", None),
41+
],
42+
)
43+
def test_parse_major_minor(raw: str, expected: str | None) -> None:
44+
assert _parse_major_minor(raw) == expected
45+
46+
47+
# ── match_to_indexed: only return exact, indexed matches ───────────
48+
49+
def test_match_to_indexed_returns_exact_match() -> None:
50+
assert match_to_indexed("3.13", ["3.12", "3.13"]) == "3.13"
51+
52+
53+
def test_match_to_indexed_returns_none_when_absent() -> None:
54+
assert match_to_indexed("3.9", ["3.12", "3.13"]) is None
55+
56+
57+
# ── detect_python_version: the fallback chain ──────────────────────
58+
59+
def test_detects_from_python_version_file(tmp_path, monkeypatch) -> None:
60+
"""Branch 1: a .python-version file in cwd wins over everything else."""
61+
monkeypatch.chdir(tmp_path)
62+
(tmp_path / ".python-version").write_text("3.11.4\n")
63+
64+
version, source = detect_python_version()
65+
66+
assert version == "3.11"
67+
assert source == ".python-version file"
68+
69+
70+
def test_malformed_version_file_falls_through(tmp_path, monkeypatch) -> None:
71+
"""Branch 1 with no parseable version must NOT crash — it falls through.
72+
73+
We stub the PATH probe so the assertion is deterministic regardless of
74+
what ``python3`` the host actually has.
75+
"""
76+
monkeypatch.chdir(tmp_path)
77+
(tmp_path / ".python-version").write_text("not-a-version\n")
78+
79+
def fake_run(*args, **_kwargs):
80+
return subprocess.CompletedProcess(args, 0, stdout="Python 3.12.1\n", stderr="")
81+
82+
monkeypatch.setattr(detection.subprocess, "run", fake_run)
83+
84+
version, source = detect_python_version()
85+
86+
assert version == "3.12"
87+
assert source == "python3 in PATH"
88+
89+
90+
def test_detects_from_path_probe(tmp_path, monkeypatch) -> None:
91+
"""Branch 2: no .python-version file, so the python3 PATH probe wins.
92+
93+
chdir to an empty tmp dir to neutralize branch 1 (any real
94+
.python-version on the host), then stub the probe deterministically.
95+
"""
96+
monkeypatch.chdir(tmp_path)
97+
98+
def fake_run(*args, **_kwargs):
99+
return subprocess.CompletedProcess(args, 0, stdout="Python 3.10.9\n", stderr="")
100+
101+
monkeypatch.setattr(detection.subprocess, "run", fake_run)
102+
103+
version, source = detect_python_version()
104+
105+
assert version == "3.10"
106+
assert source == "python3 in PATH"
107+
108+
109+
def test_falls_back_to_runtime_when_no_python3(tmp_path, monkeypatch) -> None:
110+
"""Branch 3: no file and no python3 on PATH -> server's own interpreter.
111+
112+
Neutralize branch 1 (empty cwd) and force branch 2 to fail by making the
113+
probe raise FileNotFoundError, exactly as a missing python3 would.
114+
"""
115+
monkeypatch.chdir(tmp_path)
116+
117+
def boom(*args, **_kwargs):
118+
raise FileNotFoundError("python3 not found")
119+
120+
monkeypatch.setattr(detection.subprocess, "run", boom)
121+
122+
version, source = detect_python_version()
123+
124+
assert source == "server runtime"
125+
assert version == f"{sys.version_info.major}.{sys.version_info.minor}"

uv.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)