Syndicate Hugo blog posts to dev.to and Hashnode#4872
Open
shai-almog wants to merge 11 commits intomasterfrom
Open
Syndicate Hugo blog posts to dev.to and Hashnode#4872shai-almog wants to merge 11 commits intomasterfrom
shai-almog wants to merge 11 commits intomasterfrom
Conversation
Daily GitHub Action that picks the oldest blog post under docs/website/content/blog dated after 2026-04-30, at least 7 days old, and not yet syndicated to a given platform. The script absolutizes relative links/images, inserts a one-sentence "What is Codename One" blurb after the fold, and POSTs to each platform with canonical_url pointing back to the original on www.codenameone.com. Per-platform state in scripts/website/syndication-state.json so partial failures retry only the failed side. Requires repo secrets: DEVTO_API_KEY, HASHNODE_TOKEN, HASHNODE_PUBLICATION_ID. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
✅ Continuous Quality ReportTest & Coverage
Static Analysis
Generated automatically by the PR CI workflow. |
Contributor
Cloudflare Preview
|
Adds foojay.io as a third syndication target. Unlike dev.to and Hashnode the foojay flow creates a WP draft via /wp-json/wp/v2/posts so the foojay editors can review before publishing. The canonical link is surfaced as a visible note at the top of the draft (rather than a meta field) so the reviewer can wire it up using whichever SEO plugin foojay runs. Side effects: - platforms with missing credentials are now skipped at startup with a note instead of failing the whole run, so adding a new platform later does not strand the candidate selector - requests now send a real User-Agent and Accept header (Cloudflare in front of foojay rejected the default Python-urllib UA with error 1010) - foojay credentials (FOOJAY_USER / FOOJAY_PASSWORD) wired through the workflow as optional secrets; the script auto-skips foojay until both are configured Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
foojay runs Yoast SEO, so the canonical URL is now sent as meta._yoast_wpseo_canonical on the WP draft. Yoast registers that key as a REST-exposed post meta, so the standard /wp-json/wp/v2/posts payload carries it through. The visible "originally published" line at the top of the draft body is dropped — Yoast handles the SEO directive and the "What is Codename One" blurb still provides reader-facing attribution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
foojay.io has Wordfence configured to disable WordPress Application Passwords, so there is no usable Basic Auth path for the WP REST API from the syndication script. Removing the foojay code path until / unless foojay editorial offers an alternative auth method (JWT, per-user API key, etc.). The User-Agent header and skip-when-unconfigured behaviour introduced alongside the foojay work are kept — they are useful for the remaining platforms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds scripts/website/syndicate_browser_posts.py — a Playwright-based counterpart to the API syndicator. Each target site has its own adapter (login + draft submission). State and post selection are shared with the API script via syndication-state.json, so a post is "candidate" until all configured platforms — API and browser — have a record. Adapters: - foojay: hybrid path. Playwright drives wp-login.php to obtain a real session (Wordfence has Application Passwords disabled, so token auth is out), then the script POSTs the draft via /wp-json/wp/v2/posts using the session cookies + X-WP-Nonce. Pure UI submission was attempted but Cloudflare in front of foojay challenges form POSTs and drops the payload, so drafts never landed. Yoast canonical isn't REST-writable on this Yoast install, so the canonical is surfaced as a visible note at the top of the draft body for the editor reviewer. Validated end- to-end against the live site (draft #123656). - hackernoon, dzone, medium: standard browser flow. Selectors are best-effort and need a one-time validation pass against each live site via --validate-only --headed. medium has no password login, so it relies on a base64-encoded MEDIUM_STORAGE_STATE secret exported from a manually logged-in browser session. Workflow additions: - Detects whether any browser-syndication secret is configured; only installs Playwright + Chromium when something will actually run. - Uploads the Playwright screenshot directory as a CI artifact on any outcome (kept for 14 days), so selector failures are debuggable. - Screenshots dir is gitignored. Per-platform secrets (all optional; missing = platform skipped): FOOJAY_USER, FOOJAY_PASSWORD HACKERNOON_USER, HACKERNOON_PASSWORD DZONE_USER, DZONE_PASSWORD MEDIUM_STORAGE_STATE Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
scripts/website/export_medium_storage.py captures a logged-in Medium session as a base64 blob suitable for the MEDIUM_STORAGE_STATE secret that the browser syndicator's MediumAdapter requires. Three modes: - --from-firefox-profile (no second login): reads cookies.sqlite from the user's existing Firefox profile and builds the storage state JSON directly. Auto-detects the most recently used profile under ~/Library/Application Support/Firefox/Profiles/. Refuses to write state if the profile is not actually logged in (uid cookie missing or prefixed with `lo_`). - --browser firefox|chrome|chromium|msedge: launches Playwright with the requested browser, opens medium.com/m/signin, and polls cookies every 3s until a non-`lo_` uid appears. 10-minute timeout default. - --interactive: same launch but waits on stdin instead of polling (useful when running attached to a real terminal). Output is written as JSON to --output and (unless --no-base64) printed as a base64 blob ready to paste as a repo secret. The local JSON file is gitignored. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two structural changes to the browser syndicator:
1. DZone — switch from password-based login to a saved Playwright
storageState loaded from a DZONE_STORAGE_STATE secret. DZone's
AngularJS doLogin() requires a reCAPTCHA token (visible in
scope.credentials.recaptchaToken) that Google's invisible reCAPTCHA
does not issue to headless browsers; the auth request is never sent.
Same approach as the existing MediumAdapter.
2. HackerNoon — replace .fill() with .press_sequentially() because the
login inputs are React-controlled. .fill() set DOM .value but never
updated React's internal state, so doLogin() ran with empty fields.
With per-character typing the form actually submits; HackerNoon's
"Invalid email or password" message now surfaces (instead of a
silent no-op) when credentials don't match. Also fail-fast on a
stuck-on-/login URL with the explicit error text.
Helper script renamed export_medium_storage.py ->
export_storage_state.py and generalized to support multiple sites via
--site {medium,dzone}, with per-site cookie host filter and login
detector. Browser-launch path picks Playwright Firefox when --browser
firefox is requested.
Workflow updated for the new DZONE_STORAGE_STATE secret name; gitignore
generalized to exclude all *-storage-state.json scratch files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous commit only captured the file rename; this one carries the actual code changes for DZone (storage-state auth via DZONE_STORAGE_STATE), HackerNoon (React-friendly press_sequentially typing + fail-fast on stuck-on-/login), the multi-site export helper (--site, profiles for medium and dzone, Firefox cookie host filter), the workflow secret rename (DZONE_USER/PASSWORD -> DZONE_STORAGE_STATE), and the broader *-storage-state.json gitignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three groups of changes based on review of foojay draft #123656:
1. Body rendering (applies to all syndication targets):
- Strip the trailing Hugo "## Discussion" + giscus shortcode block
so the syndicated copy ends at the actual article body.
- Strip any remaining {{< shortcode >}} forms.
- Replace the markdown-blockquote "What is Codename One?" with an
HTML <aside> styled as a left-bordered callout so it reads as a
sidebar instead of a quote.
2. FoojayAdapter post creation now sets:
- categories=[1722] (Java)
- a `codenameone` tag (created lazily via /wp/v2/tags if missing)
- featured_media: downloads the post's cover image from
www.codenameone.com and uploads it to /wp/v2/media, then assigns
the returned id as the post's featured image
- excerpt from the post's `description` front-matter
- meta._yoast_wpseo_canonical / _title / _metadesc are sent in the
payload as a best-effort; Yoast on foojay does not register these
for REST writes, so they are silently dropped. The canonical URL
is also kept as a hidden HTML comment at the top of the body so
the editor reviewer can paste it into Yoast's metabox.
3. Refactor the WP REST plumbing into _rest_get / _rest_post helpers
and centralise the User-Agent string used by both Playwright and
urllib calls.
Verified against draft #123658: categories, tags, featured_media,
excerpt, sidebar, and footer-strip all confirmed via /wp/v2/posts/...
?context=edit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
REST silently drops _yoast_wpseo_* meta keys (not registered for REST
writes) and Cloudflare blocks both new-post and update form submits to
/wp-admin/post.php with a JS challenge that loses the form payload.
WordPress XML-RPC is unprotected on foojay (Wordfence's app-password
block does not extend to xmlrpc.php), accepts the user's normal
password, and wp.editPost's custom_fields parameter lets us write the
underscore-prefixed Yoast meta keys directly.
After REST creates the draft, the foojay adapter now follows up with
an XML-RPC wp.editPost that sets:
_yoast_wpseo_canonical -> the original codenameone.com/blog/... URL
_yoast_wpseo_title -> the post title
_yoast_wpseo_metadesc -> the post description, trimmed to 155 chars
on a word boundary
Verified end-to-end against draft #123664: Yoast metabox now shows the
canonical, SEO title, and meta description correctly. The visible
canonical HTML comment at the top of the body content is kept as a
secondary signal for the editor reviewer.
Also bumps the cover-image download timeout to 120s after a transient
60s timeout on the prior run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HackerNoon
----------
The /login page form is decorative — submitting it does nothing
useful. The actually-working login lives in a drawer that opens from
the header "Login" button on hackernoon.com. Switched the adapter to:
- navigate to hackernoon.com home page
- dismiss the Iubenda cookie banner
- click header "Login" button to open the drawer
- press_sequentially the email + password into the drawer's React-
controlled inputs (fill() doesn't update React state)
- click drawer "Log In" button
- confirm login by polling for the .hackernoon.com `hasAuthCookie`
Editor flow:
- navigate to hackernoon.com/new
- click "Start Draft" (which routes to app.hackernoon.com/articles/new)
- fill title (textarea[name='title'][placeholder='Title']) and the
SEO description textarea
- paste body into the Quill rich-text editor (div.ql-editor) with
a leading "Originally published at <canonical>" line for the
editorial reviewer
- in normal mode, click "Submit Story for Review!"; in --validate-only
mode, screenshot and exit
dev.to / Hashnode
-----------------
Adds --draft-mode flag to syndicate_blog_posts.py so the API path can
be verified without going live. dev.to switches to published=false;
Hashnode switches from publishPost to createDraft. Production cron
runs without the flag and publishes as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| data = response.get("data") or {} | ||
| if draft: | ||
| node = data.get("createDraft", {}).get("draft", {}) | ||
| slug = node.get("slug") |
| try: | ||
| field = _find_first(page, self.CANONICAL_SELECTORS, timeout=3000) | ||
| field.fill(ctx.post.canonical_url) | ||
| except AdapterError: |
| field.fill(ctx.post.canonical_url) | ||
| # Close the settings panel | ||
| page.keyboard.press("Escape") | ||
| except AdapterError: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/workflows/blog-syndication.yml) that syndicates new blog posts to dev.to and Hashnode with acanonical_url/originalArticleURLback towww.codenameone.com.scripts/website/syndicate_blog_posts.py(Python stdlib only) picks the oldest post dated after 2026-04-30, at least 7 days old, and not yet syndicated to a given platform; absolutizes relative links/images; and inserts a one-sentence "What is Codename One" blurb right after the fold.scripts/website/syndication-state.jsontracks per-slug, per-platform results so partial failures retry only the failed side. Action commits state updates back tomaster.Setup
Repo secrets required (already added):
DEVTO_API_KEYHASHNODE_TOKENHASHNODE_PUBLICATION_IDThe workflow runs daily at 13:00 UTC and supports
workflow_dispatchwith a dry-run toggle. The first eligible post isliquid-glass-material-3-modern-native-themes(2026-05-01), which becomes a candidate from 2026-05-08 onward.Test plan
python3 scripts/website/syndicate_blog_posts.py --dry-runon 2026-05-06 / 2026-05-07 reports no candidate.--dry-run --today 2026-05-08selects the May 1 post.dry_run=trueafter merge to confirm secrets are wired.🤖 Generated with Claude Code