Skip to content

Commit 6784c8a

Browse files
committed
fix(docs): guard links.json in generate.sh, narrow cache key
Fix two bugs found during review: (1) generate.sh crashes on missing links.json after the pre-render change removed its generation — add if-guard around the sed/mv. (2) V2 cache key included test files via scripts/docs/** glob, causing unnecessary cache busts — narrow to scripts/docs/*.py + templates/**. Also add scripts/docs/README.md documenting how the three documentation workflows operate. JIRA: trivial risk: high
1 parent f86cb78 commit 6784c8a

File tree

3 files changed

+93
-5
lines changed

3 files changed

+93
-5
lines changed

.github/workflows/netlify-deploy-v2.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ jobs:
4444
uses: actions/cache@v4
4545
with:
4646
path: docs/versioned_docs/${{ matrix.version.section }}
47-
key: version-docs-${{ hashFiles('scripts/docs/**', 'docs/*_template.md') }}-${{ matrix.version.section }}-${{ steps.sha.outputs.sha }}
47+
key: version-docs-${{ hashFiles('scripts/docs/*.py', 'scripts/docs/templates/**', 'docs/*_template.md') }}-${{ matrix.version.section }}-${{ steps.sha.outputs.sha }}
4848
- name: Checkout
4949
if: steps.cache.outputs.cache-hit != 'true'
5050
uses: actions/checkout@v4

scripts/docs/README.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Documentation Generation
2+
3+
This directory contains Python scripts that generate the API reference pages for
4+
the GoodData Python SDK documentation site.
5+
6+
## How it works
7+
8+
The docs site is built with Hugo. API reference pages are generated in two steps:
9+
10+
1. **`json_builder.py`** — introspects the installed `gooddata_sdk` and
11+
`gooddata_pandas` packages using `inspect`, producing a `data.json` file with
12+
class/function/property metadata and parsed docstrings.
13+
2. **`python_ref_builder.py`** — reads `data.json` and `api_spec.toml`, then
14+
generates markdown files with pre-rendered HTML for each module, class, and
15+
function. Jinja2 templates in `templates/` replicate the output of the former
16+
Hugo shortcodes, but at build time instead of at Hugo render time.
17+
18+
The generated markdown files contain Hugo front matter (title, linkTitle, weight)
19+
and a `CONTENT` block with the full HTML. Hugo serves them as-is without needing
20+
shortcodes, `data.json`, or `links.json`.
21+
22+
## Workflows
23+
24+
There are three documentation deployment workflows, in order of
25+
preference:
26+
27+
### 1. V2 parallel workflow (`netlify-deploy-v2.yaml`) — recommended
28+
29+
Triggered manually via `workflow_dispatch`.
30+
31+
```
32+
discover-versions ──> generate-version (matrix, parallel) ──> build-and-deploy
33+
```
34+
35+
- `discover-versions.sh` finds the latest N release branches.
36+
- Each version runs in its own runner: checks out the version's SDK packages,
37+
installs master's `script-requirements.txt`, runs `json_builder.py` +
38+
`python_ref_builder.py` via `generate-single-version.sh`.
39+
- Per-version results are cached by `(scripts hash + templates hash + branch SHA)`.
40+
- `assemble-versions.sh` merges all version artifacts, promotes the highest
41+
numbered version to `/latest`, then Hugo builds the final site.
42+
- Deploys as a Netlify **draft** (no `--prod`).
43+
44+
### 2. Legacy single-job workflow (`netlify-deploy.yaml`) — production
45+
46+
Triggered manually via `workflow_dispatch`.
47+
48+
- Uses `hugo-build-versioned-action` which downloads `generate.sh` from master.
49+
- `generate.sh` runs all versions sequentially in one job.
50+
- Deploys to Netlify with `--prod`.
51+
52+
### 3. PR preview (`netlify-deploy-preview.yaml`)
53+
54+
Triggered automatically on PRs that change `docs/**`.
55+
56+
- Only builds the current branch's docs content (no multi-version).
57+
- Uses the simpler `hugo-build-action` (no API reference generation).
58+
- Deploys to a Netlify preview URL (`preview-{PR}--{site}.netlify.app`).
59+
60+
## Local development
61+
62+
Build and serve docs locally using Docker:
63+
64+
```bash
65+
cd docs
66+
docker build -t python-sdk-docs -f Dockerfile ..
67+
docker run --rm -p 1313:1313 python-sdk-docs
68+
# Open http://localhost:1313/latest/
69+
```
70+
71+
## Key files
72+
73+
| File | Purpose |
74+
|------|---------|
75+
| `json_builder.py` | Introspects SDK packages into `data.json` |
76+
| `python_ref_builder.py` | Generates markdown + HTML from `data.json` |
77+
| `templates/*.html.j2` | Jinja2 templates (replicate old Hugo shortcodes) |
78+
| `tests/test_python_ref_builder.py` | Unit tests (`make test-docs-scripts`) |
79+
| `../script-requirements.txt` | Python dependencies for both scripts |
80+
| `../../docs/api_spec.toml` | Maps package names to output directories |
81+
| `../../docs/*_template.md` | Markdown front matter templates (module/class/function) |
82+
83+
## Running tests
84+
85+
```bash
86+
make test-docs-scripts
87+
```

scripts/generate.sh

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,10 @@ highest_version=$(ls -v1 ./versioned_docs/ | grep -E '^[0-9]+.[0-9]+$' | sort -V
119119
echo "Moving ${highest_version} to /latest"
120120
mv -f ./versioned_docs/$highest_version ./versioned_docs/latest
121121

122-
# Replace "/${highest_version}/" with "/latest/" using sed
123-
sed "s|${highest_version}|latest|g" ./versioned_docs/latest/links.json > temp.json
124-
125-
mv temp.json ./versioned_docs/latest/links.json
122+
# Replace "/${highest_version}/" with "/latest/" in links.json (if it exists)
123+
if [ -f "./versioned_docs/latest/links.json" ]; then
124+
sed "s|${highest_version}|latest|g" ./versioned_docs/latest/links.json > temp.json
125+
mv temp.json ./versioned_docs/latest/links.json
126+
fi
126127

127128
popd

0 commit comments

Comments
 (0)