Skip to content

Update Impact Tab to use Cloudflare Edge Data#3551

Open
isTravis wants to merge 13 commits intomainfrom
tr/impact2
Open

Update Impact Tab to use Cloudflare Edge Data#3551
isTravis wants to merge 13 commits intomainfrom
tr/impact2

Conversation

@isTravis
Copy link
Copy Markdown
Member

@isTravis isTravis commented Apr 2, 2026

Replaces the Metabase/Redshift analytics pipeline with per-community analytics sourced directly from Cloudflare's edge data. Staged as "Impact2" alongside the existing Impact tab so nothing is removed while we test.

What this adds

Server

  • server/utils/cloudflareAnalytics.ts: Cloudflare GraphQL Analytics API client. Single combined query fetches daily traffic, top paths, countries, devices, and referrers per hostname. Includes date chunking (CF max ~31 days per query), contiguous span grouping, and per-day breakdown caching.
  • server/analyticsDailyCache/model.ts: Sequelize model for Postgres-backed daily cache. Past days are cached permanently; today's data has a 1-hour TTL. Auto-prunes rows older than 90 days (hourly throttle).
  • server/impact2/api.ts: API routes (GET /api/impact2, /test, /debug). Resolves the Cloudflare-facing hostname by querying the Community model directly (bypasses getInitialData's localhost domain overwrite in dev). Debug endpoint is disabled in production.
  • Registered in server/apiRoutes.ts, server/routes/index.ts, server/models.ts.

Client

  • client/containers/DashboardImpact2/: React component + SCSS. Top row with stat cards (color-coded left border) and area chart side by side. Four-column data grid below: Top Pages (with clickable links), Countries, Referrers, Devices (as percentage table). Responsive down to single-column on mobile.
  • Date range picker: Today / 7 days / 30 days.
  • Graceful degradation: friendly "not available" state when env vars are missing, stale-data callout when CF is unreachable but cache exists, standard error state with retry for transient failures.
  • Registered in client/containers/App/paths.ts, client/containers/index.ts, utils/dashboard.ts.
  • Nav button added to ScopeDropdown.

Filtering and noise reduction

Two layers of filtering to get numbers closer to real human readership:

  1. Cloudflare query filters: Only counts requests that are successful (2xx/3xx), serve HTML content, use GET method, and come from eyeball sources (not cloudflare internal routing).
  2. Server-side noise path filter: Removes /wp-*, /cdn-cgi/, /api/, /static/, /login, /robots.txt, .xml, etc. from Top Pages list. Subtracts noise path page views from totals and proportionally scales visits, countries, devices, referrers, and daily chart data to match.

Raw (pre-adjustment) totals are included in the API response and shown in the footer for transparency.

Caching strategy

  • 0 Cloudflare API calls when all requested days are cached and within TTL.
  • Past days: permanent cache (data is final).
  • Today: 1-hour TTL, then re-fetched from CF on next request.
  • Cache is Postgres-backed, so it persists across swarm deploys.

Env vars required

CLOUDFLARE_ANALYTICS_API_TOKEN # API token with Analytics:Read permission
CLOUDFLARE_ZONE_TAG # Zone ID for the PubPub traffic zone

If missing, the server logs a warning and the client shows a "not available" message instead of erroring.

@isTravis isTravis requested review from gabestein and tefkah April 2, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant