Skip to content

Fetch known-plugins.json from our own server with daily refresh + stats#78

Draft
ilicfilip wants to merge 15 commits into
developfrom
filip/remote-known-plugins
Draft

Fetch known-plugins.json from our own server with daily refresh + stats#78
ilicfilip wants to merge 15 commits into
developfrom
filip/remote-known-plugins

Conversation

@ilicfilip

@ilicfilip ilicfilip commented May 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Two related changes that let the recognized-plugins list grow without waiting for a plugin release, plus a way for users to help expand it.

Hosting update: the recognized-plugins file is now served from our own Cloudflare worker (option-optimizer-api.progressplanner.com), not from wordpress.org. The WP.org plugin team asked us not to fetch the data file from wordpress.org (ps.w.org) and to host it on our own systems instead. Fetching remote data is fine with them as long as the user knows what's happening and why, so the disclosure below is part of this PR.

1. Remote known-plugins list with daily refresh

  • The plugin loads its recognized-plugins mapping from a cached copy, refreshed once a day in the background from https://option-optimizer-api.progressplanner.com/known-plugins.json, with the JSON bundled in the plugin as a fallback when no fresh copy is available.
  • The refresh is a POST that sends the plugin version and WordPress version so the maintainers can keep anonymous usage statistics. No site identity is sent.
  • The fetch URL is filterable via aaa_option_optimizer_known_plugins_url, so installs can redirect or disable the refresh.
  • Map_Plugin_To_Options reads through the new Known_Plugins loader and exposes is_known() so callers can tell a recognized source from an unknown one.

2. "Report origin" for unknown options

  • In the options tables, any row whose source is Unknown shows a Report action that opens a popover.
  • The user pastes a wp.org plugin slug or URL. The plugin verifies it against the wp.org plugin directory and shows the official plugin name for confirmation before the report can be submitted.
  • On submit, the report (option name + the slug the user supplied + the site hostname) is sent to a configurable endpoint, where it's recorded as a GitHub issue for a maintainer to review and add to the recognized-plugins list. No auto-merge — a maintainer is always the gate.
  • The submission URL is filterable via aaa_option_optimizer_report_url, so installs can redirect or disable the feature.
  • REST responses include a plugin_known flag per option so the UI knows which rows get the Report action.

Server side (separate repo)

  • The data file is served, and fetch stats recorded, by the option-optimizer-api Cloudflare worker. It serves known-plugins.json from R2 and records each fetch (plugin + WordPress version, no site identity) to Analytics Engine. See ProgressPlanner/aaa-option-optimizer-api branch filip/known-plugins-endpoint.

External services / privacy

  • readme.txt's External services section discloses: the daily mapping fetch from option-optimizer-api.progressplanner.com (sends plugin + WP version, no site identity), the wp.org slug-verification call (api.wordpress.org, sends only the typed slug), and the report-submission endpoint.
  • For reports: only the option name and the slug the user typed are sent. The site hostname is hashed before storage and never published. The option name and slug appear in the resulting public GitHub issue. Nothing is sent unless the user explicitly clicks Submit.
  • Changelog entry added under 1.7.0 (version bump itself lands in the release PR).

Test plan

  • PHPStan + PHPCS pass on touched files
  • php bin/validate-known-plugins.php exits 0 against current JSON
  • On a test site, confirm the daily refresh cron event is registered and Map_Plugin_To_Options::get_plugin_name() still resolves a known prefix (e.g. aioseo_* → AIOSEO)
  • Confirm the refresh POST reaches the worker and a fresh mapping is cached
  • Simulate a 404 / error on the remote URL and confirm fallback to the bundled JSON
  • On an Unknown row, confirm the Report popover verifies a real slug, rejects a bogus one, and submits successfully

Outstanding before merge

  • Remove .github/workflows/sync-known-plugins.yml — it still pushes the file to the wp.org SVN /assets/ directory, which is exactly the hosting the WP.org team asked us to stop. The publish target is now the worker's R2 bucket, handled in the worker repo.
  • Deploy the worker (create R2 bucket, seed/publish known-plugins.json, wrangler deploy) so the endpoint is live before this ships. Until then, installs use the bundled fallback.

Notes for reviewer

  • The validator surfaces one warning on current data: siteground_data_ is claimed by both sg-optimizer and sg-security. Treated as a warning, not an error.

🤖 Generated with Claude Code

ilicfilip and others added 2 commits May 7, 2026 17:02
@ilicfilip @claude
Fetch known-plugins.json from wp.org with daily refresh
f4f89fa 
Adds a Known_Plugins loader that caches a remote copy of the mapping
and falls back to the bundled JSON. A daily WP-Cron event refreshes
the cache from https://ps.w.org/aaa-option-optimizer/assets/known-plugins.json.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Sync known-plugins.json to wp.org SVN assets daily
0e9a69f 
Daily GitHub Action validates known-plugins.json and publishes it to
the plugin's wp.org /assets/ directory so installs can fetch updates
without waiting for a release. Validator rejects malformed JSON,
missing fields, and dangerously generic prefixes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented May 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Test on Playground
Test this pull request on the Playground
or download the zip

ilicfilip and others added 9 commits May 7, 2026 17:54
@ilicfilip @claude
Exclude bin/ CLI scripts from PHPCS
85b4d79 
The validator runs in GitHub Actions, not in WordPress, so
WordPress-specific sniffs (escaping, $-prefix globals, WP_Filesystem)
don't apply.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Drop redundant activation-hook scheduling, clean up on uninstall
267468e 
The runtime guard in Plugin::register_hooks() already self-heals the
schedule for every install. Activation-time scheduling was redundant.
Uninstall now also removes the cached mapping option and clears the
cron event as defense-in-depth.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Add Report origin button + popover for unknown options
931786e 
For rows where no plugin matched, the Source column becomes a button
that opens a popover. User pastes a wp.org slug or URL; the plugin
verifies it via api.wordpress.org and shows the official plugin name.
On submit, the report is POSTed to the configurable submission endpoint
(filterable via aaa_option_optimizer_report_url) for the maintainer
to triage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Tighten Report trigger styling
b1a65e3 
Drop dashed-border button look. Render label and action as plain text
with the action subtly underlined. Force single-line so the row
doesn't grow when the trigger is shown.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Stack Report link under the Unknown label
a43c89c 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Tighten Report popover and SVN sync workflow
dc7198f 
- Per-popover verify-debounce timer instead of a single shared one,
  so concurrently-open popovers don't cancel each other's lookup.
- Stable popover ids via a counter, eliminating collisions when two
  unknown option names normalize to the same slug.
- Drop dead svn-mkdir fallback and redundant mkdir in the SVN sync
  workflow; assets/ always exists on wp.org plugin SVN.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Drop submission-worker gitignore entry
82c5ea7 
The submission worker is now maintained in its own separate repository,
so it no longer needs a local exclusion in this repo's gitignore.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Lock SVN sync workflow to main
4128603 
Cron, push, and workflow_dispatch all check out main now. Develop work
no longer propagates to wp.org SVN — main is the source of truth for
what installs fetch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Document external services and changelog for the new feature
95e5595 
Adds an "External services" section disclosing wp.org and the
report-submission endpoint, as required by wp.org plugin review.
Adds a 1.7.0 changelog entry covering the Report-origin feature
and the daily known-plugins refresh. Version bump itself lands in
the release PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ilicfilip ilicfilip force-pushed the filip/remote-known-plugins branch from 9df50a8 to 95e5595 Compare June 1, 2026 08:29
@ilicfilip @claude
Host known-plugins mapping off wordpress.org
84564d2 
The WP.org plugin team asked us not to fetch the data file from
wordpress.org (ps.w.org). Move the fetch to our own Cloudflare worker
at option-optimizer-api.progressplanner.com, and POST the plugin and
WordPress versions with the request so the maintainers can keep
anonymous usage statistics. No site identity is sent.

Add the aaa_option_optimizer_known_plugins_url filter so the fetch can
be redirected or disabled, and update the External services disclosure
in readme.txt accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ilicfilip ilicfilip changed the title Fetch known-plugins.json from wp.org with daily SVN sync Fetch known-plugins.json from our own server with daily refresh + stats Jun 2, 2026
ilicfilip and others added 3 commits June 2, 2026 11:29
@ilicfilip @claude
Remove wp.org SVN sync workflow
0a7dc4e 
The known-plugins mapping is now served from our own Cloudflare worker
(option-optimizer-api.progressplanner.com), not from wordpress.org, so
the workflow that published the file to the wp.org SVN /assets/ directory
is no longer used.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Add filter to override the option-to-plugin mapping
186809c 
Add the aaa_option_optimizer_plugin_name filter to Map_Plugin_To_Options
so integrations can recognize options the bundled known-plugins list does
not, or override an existing match. Returning null leaves the option
unrecognized.

The filter is applied in find_match(), the single lookup used by both
get_plugin_name() and is_known(), so a filtered match is treated as known
everywhere (e.g. it suppresses the "Report origin" prompt). The filter
name follows the existing aaa_option_optimizer_* convention. Requested by
a user.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ilicfilip @claude
Require opt-in consent before contacting our servers
32a7413 
The daily known-plugins refresh contacts our server and sends the plugin
and WordPress version for anonymous stats. The WP.org plugin team allows
this only with the user's knowledgeable consent, so make it opt-in:

- Add a remote_data_consent setting (default off). Known_Plugins::refresh()
  bails without it, and the daily cron is only scheduled while it is on
  (and cleared when it is turned off).
- Add a checkbox on the settings tab to grant/revoke consent, with a
  description of what is fetched and sent.
- In the Report popover (which also contacts our server), show an inline
  consent checkbox when consent has not been granted; Submit stays
  disabled until it is ticked. Granting it there persists consent via a
  new /set-consent REST route, so the daily refresh starts and the user
  is not asked again.
- Update the readme External services disclosure and changelog to note the
  fetch is opt-in.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ilicfilip