Migrate from curl-based skill to Python plugin architecture

[nickwinder]

Why

The existing curl-based skill used inline curl commands with complex JSON payloads, which required agents to construct nested instructions payloads, debug multipart form encoding, and chain multiple curl calls with intermediate file handling. This migration replaces that with Python scripts behind simple CLI flags, dramatically reducing agent turns and token usage on complex tasks.

Summary

• Replace curl-based REFERENCE.md (598 lines of API docs) with 16 focused Python scripts in nutrient-document-processing/scripts/, each exposing a single operation via CLI flags
• Add shared library (scripts/lib/common.py) handling API auth, file upload, async execution, and error handling
• Add custom workflow template (assets/templates/custom-workflow-template.py) for multi-step pipelines — agents copy and customize it at runtime instead of committing new scripts
• Add testing guide (tests/testing-guide.md) with benchmark methodology
• Update README with Python 3.10+/uv requirements and updated paths
• Rewrite SKILL.md for zero-exploration agent execution — complete uv run invocations with all args for every script, inline page range format and multi-step workflow rules

SKILL.md design

The SKILL.md is written so an agent can execute ANY operation with zero exploration turns:

• Complete uv run invocations with all args for all 16 scripts
• Grouped into Single-Operation Scripts and Multi-Step Workflow Rule sections
• Page range format, PDF requirements, and custom pipeline instructions inlined
• No --help lookups or reference file reads required

Scripts (16)

Convert & Transform: convert.py, merge.py, split.py, ocr.py, rotate.py, optimize.py
Extract & Analyze: extract-text.py, extract-table.py, extract-key-value-pairs.py
Edit & Secure: watermark-text.py, redact-ai.py, sign.py, password-protect.py
Page Operations: add-pages.py, delete-pages.py, duplicate-pages.py

Benchmark Results

Original 3 tests (convert, merge, OCR+watermark pipeline)

Comparing v1 (curl-based) and v2 (Python scripts with improved SKILL.md):

Metric	v1 (curl)	v2 (Python)	v2 vs v1
Pass rate	3/3	3/3	—
Avg wall time	35.2s	33.1s	6% faster
Total output tokens	3,754	3,164	16% fewer
Total cost	$1.17	$1.06	9% cheaper

The pipeline test showed the largest improvement: v1 needed 12 turns / 52s, v2 completed in 8 turns / 44s.

Extended benchmark: 10 complex document workflows

10 additional complex workflows covering extraction, security, page manipulation, and multi-step pipelines. Each test was run on both v1 and v2 using claude -p with Sonnet, --dangerously-skip-permissions, 360s timeout.

Tests: extract text (page-scoped), split into individual pages, AI-redact emails, watermark with custom params, sign with metadata, OCR+optimize pipeline, convert to markdown, extract key-value pairs, redact+watermark pipeline, merge+split pipeline.

Metric	v1 (curl)	v2 (Python)	Improvement
Pass rate	10/10	10/10	—
Avg wall time	198.9s	71.2s	64% faster
Avg turns	20.7	10.4	50% fewer turns
Total output tokens	47,084	23,163	51% fewer tokens
Total cost	$8.57	$4.61	46% cheaper

v2 won 9 of 10 tests. Per-test highlights:

Test	v1	v2	Speedup
sign-with-metadata	297.8s (45 turns)	35.6s (5 turns)	8.4x
redact-then-watermark	360.0s (timeout)	60.9s (5 turns)	5.9x
extract-key-value-pairs	273.0s (28 turns)	47.4s (6 turns)	5.8x
ai-redact-emails	314.7s (28 turns)	65.6s (8 turns)	4.8x
convert-to-markdown	152.0s (19 turns)	41.8s (6 turns)	3.6x
ocr-then-optimize	222.2s (25 turns)	126.2s (18 turns)	1.8x

The only test v1 won was merge-then-split (61s vs 74s), where v1's single curl call with inline JSON is marginally faster than chaining two Python scripts.

Why v2 is faster on complex tasks

v1's curl-based SKILL.md has copy-paste examples for simple operations, but complex tasks require the agent to:

1. Construct nested JSON instructions payloads from memory
2. Debug multipart form encoding for multi-file operations
3. Chain multiple curl calls with intermediate file handling

v2's Python scripts encapsulate all API complexity behind CLI flags. The SKILL.md gives the agent the exact invocation for every script, so it can go from prompt to execution in 1–2 turns.