Skip to content

Commit 2497c2a

Browse files
tdhopperclaude
tdhopper
and
claude
committed
Update documentation for modernized infrastructure
README.md: - Add modern development setup instructions with uv - Document system R installation requirements - Add build commands and CI/CD information - Clarify plotly no longer needs authentication CLAUDE.md: - Update all dependency versions and requirements - Document new uv-based setup process - Add comprehensive CI/CD section - Update technical constraints and configuration files - Add GitHub Actions workflow documentation Both files now reflect Python 3.11+, uv package manager, and GitHub Actions deployment to Netlify. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 291f9f4 commit 2497c2a

File tree

2 files changed

+233
-3
lines changed

2 files changed

+233
-3
lines changed

‎CLAUDE.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
pythonplot.com is a static website that provides a visual comparison of different Python plotting libraries (pandas, matplotlib, seaborn, plotnine, plotly, altair) and R's ggplot2 for exploratory data analysis. It serves as a "Rosetta Stone" showing how to create the same plots across different libraries.
8+
9+
## Architecture
10+
11+
The site is generated from a Jupyter notebook (Examples.ipynb) that contains tagged code cells. The build process:
12+
13+
1. Executes the notebook using `jupyter nbconvert`
14+
2. Extracts tagged cells using `render.py`
15+
3. Generates static HTML using Jinja2 templates
16+
4. Outputs PNG images and an HTML file to the `web/` directory
17+
18+
### Key Components
19+
20+
- **Examples.ipynb**: Source notebook containing plot examples for each library
21+
- **render.py**: Core build script that:
22+
- Extracts cells tagged with metadata (ex, name:*, package:*)
23+
- Extracts base64 PNG images from cell outputs and saves them to web/img/plots/
24+
- Parses cell source code and optional markdown comments
25+
- Renders the final HTML using Jinja2
26+
- **templates/t_index.html**: Jinja2 template for the website
27+
- **web/**: Output directory for generated site
28+
- **INTRO.md**: Markdown content for the site introduction
29+
30+
### Cell Tagging System
31+
32+
Notebook cells must be tagged with metadata for the render script to process them:
33+
34+
```
35+
ex # Marks cell as an example
36+
name:scatter-plot # Maps to names dict in render.py
37+
package:seaborn # Maps to packages dict in render.py
38+
```
39+
40+
Valid plot names and packages are defined in dictionaries at the top of render.py (lines 19-43).
41+
42+
## Development Commands
43+
44+
### Setup Environment
45+
46+
```bash
47+
make dev_environment # Installs Python and R dependencies using uv
48+
make setup # Alternative command (same as dev_environment)
49+
```
50+
51+
**Prerequisites:**
52+
- Python 3.11+ installed on system
53+
- R 4.0+ installed separately (via Homebrew, apt, or CRAN)
54+
- uv package manager (auto-installed by make targets if missing)
55+
56+
**Manual Setup:**
57+
```bash
58+
# Install uv
59+
curl -LsSf https://astral.sh/uv/install.sh | sh
60+
61+
# Install Python dependencies
62+
uv pip install -r requirements.txt
63+
64+
# Install R packages (ggplot2, mgcv)
65+
./setup_r.sh
66+
```
67+
68+
### Build Site
69+
70+
```bash
71+
make qrender # Quick render from Examples.ipynb without executing
72+
make render # Full build: execute notebook, then render (creates timestamped .ipynb)
73+
make # Full build + S3 upload
74+
```
75+
76+
The render process:
77+
1. `make render` runs the notebook with `jupyter nbconvert --execute`
78+
2. Creates a git-commit-stamped copy (Examples.<hash>.ipynb)
79+
3. Runs `python render.py` to extract cells and generate web/index.html
80+
4. PNG images are extracted and saved to web/img/plots/ with MD5-based filenames
81+
82+
### Testing
83+
84+
```bash
85+
make test # Run pytest tests
86+
python -m pytest tests/
87+
```
88+
89+
The test suite (tests/test_plots.py) validates that Examples.ipynb contains all expected plot/package combinations defined in the defined_plots dictionary.
90+
91+
### Local Development
92+
93+
After rendering, serve locally:
94+
95+
```bash
96+
cd web && python -m http.server
97+
```
98+
99+
### Image Rendering
100+
101+
All plots are rendered to static PNG images:
102+
- **Plotly**: Uses Kaleido for local rendering (no authentication required with v5+)
103+
- **Altair**: Uses native rendering or selenium/geckodriver
104+
- **R/ggplot2**: Uses rpy2 to interface with system R installation
105+
106+
**Note**: Plotly authentication is no longer needed with plotly v5+. The old authentication code in `.travis/authenticate_plotly.py` is deprecated.
107+
108+
## Adding New Plots
109+
110+
1. Add the plot name to the `names` dictionary in render.py
111+
2. Add a new cell in Examples.ipynb with code that produces a PNG output
112+
3. Tag the cell with: `ex`, `name:<plot-name>`, `package:<library-name>`
113+
4. Keep code lines under ~46 characters to avoid horizontal scrolling in the UI
114+
5. Optionally add a markdown comment in triple quotes on the first line
115+
6. Update tests/test_plots.py to include the new plot in defined_plots
116+
7. Run `make qrender` to test (or `make render` for full rebuild)
117+
118+
## Technical Constraints
119+
120+
- Plot code must generate PNG output in the notebook cell
121+
- For plotly, images must be generated via their server (requires credentials)
122+
- R code cells must start with `%%R` magic command
123+
- Altair cells must start with `%%altair` magic command
124+
- Code lines should wrap to ~46 characters for proper display
125+
- All image paths are MD5 hashes of the base64-encoded PNG data
126+
127+
## Dependencies
128+
129+
### Python Environment
130+
- **Python**: 3.11+ (specified in runtime.txt and pyproject.toml)
131+
- **Package Manager**: uv (modern, fast alternative to pip)
132+
- **Jupyter**: For notebook execution
133+
134+
### Plotting Libraries (all latest versions)
135+
- pandas 2.0+
136+
- matplotlib 3.7+
137+
- seaborn 0.13+
138+
- plotnine 0.13+
139+
- plotly 5.24+ (with Kaleido for image export)
140+
- altair 5.0+
141+
- statsmodels 0.14+
142+
143+
### R Environment
144+
- **R**: 4.0+ (system installation required)
145+
- **R Packages**: ggplot2, mgcv
146+
- **Python-R Bridge**: rpy2 3.5+
147+
148+
### Other Key Dependencies
149+
- Jinja2 with jinja2-highlight for templating
150+
- selenium 4.15+ with geckodriver for browser automation
151+
- markdown for text processing
152+
153+
**Configuration Files:**
154+
- `pyproject.toml`: Modern Python project metadata and dependencies
155+
- `requirements.txt`: Pin-free dependency list
156+
- `setup_r.sh`: R package installation script
157+
158+
## CI/CD
159+
160+
The project uses **GitHub Actions** for continuous integration and deployment (migrated from Travis CI).
161+
162+
### Workflow (.github/workflows/deploy.yml)
163+
164+
**On Every Push:**
165+
1. Setup R 4.3 using r-lib/setup-r action
166+
2. Install R packages (ggplot2, mgcv)
167+
3. Setup Python 3.11
168+
4. Install uv and Python dependencies
169+
5. Setup Firefox and geckodriver for selenium
170+
6. Run pytest tests with xvfb (virtual display)
171+
7. Execute notebook and render site
172+
8. Deploy to Netlify:
173+
- **master branch**: Production deployment
174+
- **Other branches**: Preview deployments
175+
176+
**Required Secrets:**
177+
- `NETLIFY_AUTH_TOKEN`: Netlify authentication token
178+
- `NETLIFY_SITE_ID`: Netlify site identifier
179+
180+
**Environment Variables:**
181+
- `R_HOME`: Set to R installation path (e.g., `/opt/R/4.3.3/lib/R` on GitHub runners)
182+
183+
### Legacy CI Files
184+
- `.travis.yml`: Old Travis CI config (deprecated)
185+
- `.travis/`: Old CI scripts (mostly deprecated)
186+
- `.travis/authenticate_plotly.py`: No longer needed with plotly v5+

‎README.md‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,28 @@ Available at [pythonplot.com](http://pythonplot.com/)
4949

5050
The site is generated from plots in the `Examples.ipynb` Python 3, Jupyter notebook.
5151

52-
You can create a [Conda](https://conda.io/docs/install/quick.html) dev environment to run the notebook with `make dev_environment`.
52+
#### Development Setup
53+
54+
**Requirements:**
55+
- Python 3.11+
56+
- R 4.0+ (install via [Homebrew](https://brew.sh/) on macOS: `brew install r`, or from [CRAN](https://cran.r-project.org/))
57+
- [uv](https://github.com/astral-sh/uv) for Python package management
58+
59+
**Quick Start:**
60+
```bash
61+
# Set up development environment (installs Python & R dependencies)
62+
make dev_environment
63+
64+
# Or manually:
65+
# 1. Install uv
66+
curl -LsSf https://astral.sh/uv/install.sh | sh
67+
68+
# 2. Install Python dependencies
69+
uv pip install -r requirements.txt
70+
71+
# 3. Install R packages
72+
./setup_r.sh
73+
```
5374

5475
Each plot in the notebook is tagged with metadata using Jupyter cell tags. You can view the cell tags in your notebook with `View > Cell Toolbar > Tags`.
5576

@@ -76,8 +97,31 @@ If you are using Jupyter lab, these are available through the `Cell Tools` left
7697

7798
The site layout current allows for ~46 characters in the code window. Please keep this in mind and wrap your code to avoid too much side scrolling.
7899

79-
At the moment, the code must return a png image into the output cell. To generate image files with `plotly`, using their server is currently the only way. Please see the [getting started page](https://plot.ly/python/getting-started/) for instructions on the `.credentials` file.
100+
At the moment, the code must return a png image into the output cell. With modern versions of plotly (v5+), images are rendered locally using Kaleido - no authentication required.
80101

81102
A Markdown comment can be added within triple quotes on the first line of the cell. (This currently doesn't work for `R` cells.)
82103

83-
You can render the images to `web/img/plots` by running `$ make`. Afterward, launch a local server (e.g. `python -m http.server`) from the `web` directory.
104+
#### Building the Site
105+
106+
```bash
107+
# Quick render (uses existing notebook without re-execution)
108+
make qrender
109+
110+
# Full build (executes notebook, then renders)
111+
make render
112+
113+
# Run tests
114+
make test
115+
```
116+
117+
After rendering, launch a local server from the `web` directory:
118+
```bash
119+
cd web && python -m http.server
120+
```
121+
122+
#### CI/CD
123+
124+
The project uses GitHub Actions for continuous integration and deployment:
125+
- **Pushes to `master`**: Automatically deployed to production on Netlify
126+
- **Other branches**: Create preview deployments on Netlify
127+
- Requires secrets: `NETLIFY_AUTH_TOKEN` and `NETLIFY_SITE_ID`

0 commit comments

Comments
 (0)