realm-server: HTTPS+HTTP/2 in local dev

[habdelra]

⚠️ Required dev setup after pulling this branch

The local stack is now HTTPS+HTTP/2 end-to-end (realm-server on :4201, vite on :4200). Three one-time manual steps are required before mise run dev / mise run dev-all will work; skipping any of them leaves the browser stuck in CORS / mixed-content / cert errors that look like the app is broken.

1. Trust the dev TLS cert (one-time, prompts sudo)

The realm-server and vite both terminate TLS with a mkcert leaf. Chromium / Firefox read the system NSS DB (~/.pki/nssdb on Linux), not /etc/ssl/certs, so libnss3-tools is mandatory on Linux for the browser to actually trust the cert.

Install the tools and run the trust task:

# Linux (Debian/Ubuntu)
sudo apt install -y mkcert libnss3-tools
# Linux (Fedora/RHEL)
sudo dnf install -y mkcert nss-tools
# macOS
brew install mkcert nss

mise run infra:trust-dev-cert

mise run infra:trust-dev-cert runs mkcert -install (sudo prompt) and seeds ~/.pki/nssdb so Chromium picks up the root CA. mise run dev / dev-all invoke infra:ensure-dev-cert as a preflight that fails fast with a copy-paste message if you skipped this. CI does the equivalent via passwordless sudo in .github/actions/init — devs don't need to think about it there.

2. Migrate your matrix account_data (one-time)

Every Boxel user keeps their realm workspace list in matrix app.boxel.realms account_data. Existing entries still point at http://localhost:42XX/.... The realm-server's HTTP→HTTPS dispatcher 301-redirects those requests, but CORS preflight forbids following redirects, so the host bundle's first realm fetch will fail with:

Access to fetch at http://localhost:4201/... from origin https://localhost:4200 has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.

Run the migration once after mise run infra:trust-dev-cert:

mise run infra:migrate-matrix-account-data-http-to-https

The script logs in as the local synapse admin, walks every user, impersonates each via /_synapse/admin/v1/users/<id>/login, reads app.boxel.realms, rewrites every http://localhost:4201/... / http://localhost:4202/... to https://..., and PUTs the result back. Safe to re-run — users already on https are skipped.

3. Clear your browser's localStorage

The host bundle caches realm-scoped JWTs and session metadata keyed by realm URL. Those entries are still keyed on the old http://localhost:42XX/ form, and there is no server-side migration path — the only way to evict them is in your browser:

DevTools → Application → Local Storage → right-click https://localhost:4200 (and the old http://localhost:4200 entry if present) → Clear

After that, log in fresh — new tokens will be keyed under the https origin.

When in doubt: full-reindex a realm

If something still looks broken after the three steps above — stale rows, missing types, a card that won't render — kick off a from-scratch index for the affected realm. The realm-server exposes the same _grafana-reindex endpoint we use from Grafana in production. In local dev the shared secret is literally shhh! it's a secret:

curl -X POST \
  -H "Authorization: Bearer shhh! it's a secret" \
  'https://localhost:4201/_grafana-reindex?realm=/user/prudent-octopus/'

The request blocks until the from-scratch job finishes and prints the result stats. Swap the realm= query param for whichever realm you need (/base/, /test/, /user/<your-realm>/, etc.); the path is relative to the realm-server origin.

And when even that doesn't help, the nuclear option is still available:

mise run infra:full-reset

That drops and recreates the Postgres DB, clears dynamic realms, and restarts matrix — i.e. you start over from a clean slate.

Rolling back

If you ever need to go back to the previous http-canonical state (e.g. you're bisecting against this branch, or you pnpm migrate down the postgres migration), the rewrite is symmetric in both directions:

# 1. Roll back the postgres canonical-url migration
pnpm --filter @cardstack/postgres migrate down 1
# 2. Flip account_data back so the host bundle matches
mise run infra:migrate-matrix-account-data-https-to-http
# 3. Clear localStorage again so JWTs get re-minted under the http origin

Both migrations gate on realm_user_permissions containing localhost canonicals in the relevant scheme, so they're no-ops on staging / production.

---

Auto-applied: postgres data migration

The first mise run dev after pulling runs a Postgres migration (1779100257124_canonical-url-http-to-https.js) that rewrites every text/varchar/jsonb column on every public table from http://localhost:42XX/… to https://localhost:42XX/… in place — index rows, realm registry, permissions, JSONB documents inside pristine_doc/search_doc/etc. The migration is idempotent and gated on a cheap realm_registry pre-check, so re-runs and production environments are no-ops.

If you have stale http://localhost:42XX/… URLs in personal-realm card .json files (in realms/localhost_4201/**), the dispatcher's 301-redirect resolves them at runtime so cards still work — no on-disk rewrite is required. To clean the data anyway:

find realms/localhost_4201 -name '*.json' -exec sed -i 's|http://localhost:4201|https://localhost:4201|g' {} +

Navigation

Visit https://localhost:4200/ (vite host) as the manual-browser entry point. Both vite and the realm-server speak HTTPS+HTTP/2 now, so the host bundle's realm fetches multiplex over a single h2 connection — no mixed-content warnings.

Summary

Local dev's realm-server now speaks HTTPS+HTTP/2 on a single canonical origin (https://localhost:4201, plus https://localhost:4202 for test-realms). This unblocks the heavy aggregator-card prerender bottleneck described in CS-11114 — cohort and dashboard renders today fan out 80+ federated-search requests inside one Chromium tab, get throttled by Chrome's HTTP/1.1 6-per-origin connection ceiling, and take minutes; HTTP/2 multiplexes them over one connection and the same render finishes in seconds.

Following @lukemelia's suggestion in #4787, this PR ships the single-origin design rather than the dual-listen alternative that #4787 had been carrying. There is no separate h2 alias port, no per-page __realmH2OriginMappings__ injection, no alias-host rewrite middleware — the wire protocol and the canonical realm URL agree.

Design

Realm-server: same-port HTTPS+HTTP/2 dispatcher

• RealmServer.listen(port): when REALM_SERVER_TLS_CERT_FILE/_KEY_FILE are set, binds a single net.Server that peeks the first byte of every connection. 0x16 (TLS ClientHello) routes to an http2.createSecureServer for h2; anything else routes to a plain http.Server that 301-redirects to https://<host><path>. Same listener, no extra port. When the cert is absent (in-process test fixtures), falls back to plain http.createServer — unchanged behavior.
• The dispatcher tracks every accepted socket and exposes closeAllConnections() so shutdown can force-close in-flight TLS / HTTP/2 / keep-alive sessions rather than waiting for peers. main.ts's existing typeof feature-detect picks it up unchanged.
• Defensive cert load: readFileSync + createSecureServer wrapped in try/catch so a malformed cert downgrades to plain HTTP with a warning rather than killing boot.
• patchKoaResponseForH2Head() on the Koa response prototype: Node's http2 compat layer leaves Http2Stream.writable === false on HEAD streams, which short-circuits Koa's respond() and hangs every HEAD request indefinitely. The patch returns true for HEAD streams so the response actually flushes.
• middleware/index.ts:
  • fullRequestURL detects ctx.req.socket.encrypted for the scheme and falls back to the HTTP/2 :authority pseudo-header when headers.host is absent, so URL-keyed realm lookup matches the HTTPS canonical.
  • fetchRequestFromContext strips :-prefixed pseudo-headers before constructing new Request(...) — WHATWG Headers rejects them.
  • setContextResponse filters HTTP/1-only connection / keep-alive / transfer-encoding / upgrade / proxy-connection / http2-settings response headers that Node's h2 compat layer would otherwise reject.
  • proxyAsset was reimplemented as a hand-rolled forwarder (replacing koa-proxies + http-proxy) so pseudo-headers and the request host get filtered before the upstream call — http-proxy.setHeader(':path', …) throws ERR_INVALID_HTTP_TOKEN.
• main.ts defaults --serverURL to https://localhost:${port} (was http://). The realm-server stamps serverURL into the realmServerURL claim of every JWT it mints, so an http default leaks into tokens and the host's assertOwnRealmServer rejects them as a "different realm server".
• Prerender (prerender/browser-manager.ts, scripts/wait-for-host-standby.ts) launches puppeteer with --ignore-certificate-errors when BOXEL_HOST_URL / REALM_BASE_URL is https.

Vite host (packages/host)

• vite.config.mjs reads REALM_SERVER_TLS_CERT_FILE/_KEY_FILE and sets both server.https (dev) and preview.https (built) so vite terminates TLS on :4200. Browsers refuse HTTP/2 over cleartext, so vite has to speak HTTPS for the h2 connection-pool win to apply on the host origin.
• packages/host/scripts/vite-with-traefik.js adds a same-port http→https redirect dispatcher for vite (dev) only — vite binds an internal port, the dispatcher owns :4200, peeks the first byte, and either pipes raw bytes to vite (TLS) or 301-redirects (plain HTTP). vite preview skips the dispatcher and binds :4200 directly (the byte-peek + cross-process TCP pipe pattern doesn't survive chrome's TLS+h2 handshake under load in CI; preview doesn't need browser-bar UX anyway).
• Host config/environment.js defaults flip to https://localhost:4201 for realmServerURL / baseRealmURL / catalogRealmURL / legacyCatalogRealmURL / skillsRealmURL / openRouterRealmURL.

Mise tasks and env-vars

• mise run infra:ensure-dev-cert provisions the mkcert leaf at $HOME/.local/share/boxel/dev-certs/. Idempotent; auto-runs infra:trust-dev-cert when passwordless sudo is available (CI), otherwise fails fast with a copy-paste install message. mise run dev / dev-all invoke it as a preflight.
• mise run infra:trust-dev-cert runs mkcert -install and (on Linux) verifies libnss3-tools is installed so Chromium picks up the root CA from ~/.pki/nssdb.
• mise-tasks/lib/env-vars.sh: defaults REALM_BASE_URL/REALM_TEST_URL/HOST_URL to https; exports REALM_SERVER_TLS_CERT_FILE/_KEY_FILE + NODE_EXTRA_CA_CERTS when the mkcert leaf is present. Also auto-detects system chrome (/usr/bin/google-chrome, Chromium.app, etc.) and sets PUPPETEER_EXECUTABLE_PATH — puppeteer's bundled Chrome 143 has an h2 stream-window bug that hangs the prerender on cold vite optimizer; Chrome 148+ is fine.
• services/{realm-server,realm-server-base,worker-base,prerender,test-realms} were updated to use the https canonical URLs.
• Matrix helpers/isolated-realm-server.ts and realm-test-harness/src/{support-services,isolated-realm-stack}.ts strip REALM_SERVER_TLS_CERT_FILE/_KEY_FILE before spawning child processes, so the matrix-isolated and software-factory stacks stay plain HTTP on their dynamic ports regardless of the outer dev env.

CI

• .github/actions/init/action.yml installs mkcert + libnss3-tools via apt and runs mise run infra:ensure-dev-cert so realm-servers in CI come up HTTPS+HTTP/2 the same as local.
• tests/index.ts (realm-server test bootstrap) deletes the TLS env vars before any in-process fixture realm-server is spun up — supertest connects plain HTTP to those fixtures on random 127.0.0.1:444X ports.
• Wait-on probes against https://localhost:42XX (in ci/serve-test-assets, ci/cache-index, test-services/{host,realm-server,matrix}) use https-get:// (start-server-and-test's default https:// is HEAD, which vite preview behind h2 doesn't reliably answer) and pass START_SERVER_AND_TEST_INSECURE=1 to disable wait-on's strictSSL check.
• ci-software-factory.yaml no longer starts host-dist on :4200 — the realm-test-harness is hermetic and brings up its own vite preview on dynamic ports. Only services:icons (port 4206) is started externally.
• packages/postgres/scripts/ensure-db-exists.sh forces -h localhost -p 5432 (TCP) inside docker exec, since the postgres:16.3 image doesn't reliably create /var/run/postgresql/.s.PGSQL.5432. set -e makes a failed CREATE DATABASE actually exit non-zero instead of fabricating a success line.

Data migrations

Two migrations cover the http→https flip and are both reversible:

Postgres — packages/postgres/migrations/1779100257124_canonical-url-http-to-https.js:

• Walks information_schema.columns for every text/varchar/jsonb column on every public table (excludes modules, pgmigrations/migrations, generated columns).
• For each column, runs in-place REPLACE(...)-based UPDATEs for http://localhost:4201 → https://localhost:4201 and http://localhost:4202 → https://localhost:4202. WHERE filter restricts the touch set to rows that still contain the old URL — idempotent.
• down is symmetric (https → http) — same realm_user_permissions pre-check on the source scheme.
• Both directions gate on realm_user_permissions containing localhost URLs, so production / staging (real hostnames, never localhost) is a no-op either way.
• Runs automatically: mise run dev passes --migrateDB to the realm-server, so the migration fires on the first post-pull boot.

Matrix account_data — packages/matrix/scripts/migrate-account-data-http-to-https.ts:

• Logs in as the local synapse admin, admin-impersonates every user, rewrites app.boxel.realms account_data entries from http://localhost:42XX/... to https://..., PUTs back.
• --reverse flips the direction (pnpm migrate-account-data-https-to-http / mise run infra:migrate-matrix-account-data-https-to-http) for symmetry with the postgres migrate-down.
• Safe to re-run; users already on the target scheme are skipped.

Tests

• packages/realm-server/tests/listener-dispatcher-test.ts covers the dispatcher: TLS h2, ALPN HTTP/1.1 fallback, TLS h2 HEAD (the patched-writable path), plain-HTTP 301, no-Host-header raw-socket path, malformed-cert downgrade, and no-cert-env-vars plain HTTP.
• The rest of the realm-server qunit/mocha suite continues to run plain HTTP via the test-bootstrap env-var delete; the per-test URL fixtures (card-endpoints-test.ts, types-endpoint-test.ts, module-syntax-test.ts) were updated to https where they reference port 4202.
• Host integration test fixtures (realm-indexing-test.gts, realm-test.gts) updated for the new https canonical and the alphanumeric URL sort order that follows from it.

Test plan

• mise run infra:ensure-dev-cert succeeds with mkcert installed; emits clean install hints + exits 1 when missing.
• curl -kI --http2 https://localhost:4201/_alive returns HTTP/2 200.
• curl -kI --http1.1 https://localhost:4201/_alive returns HTTP/1.1 200 (ALPN fallback for h1 clients).
• curl -sI http://localhost:4201/_alive returns HTTP/1.1 301 with Location: https://localhost:4201/_alive.
• curl -skI -X HEAD --http2 https://localhost:4201/_alive returns HTTP/2 200 (HEAD over h2 doesn't hang).
• Pull with existing local realm data → first mise run dev runs the URL-rewrite migration → realm-server boots clean on https.
• pnpm --filter @cardstack/postgres migrate down 1 && pnpm --filter @cardstack/postgres migrate up round-trips cleanly (the Postgres Migration CI job validates this).
• Trigger a base-realm reindex via curl -X POST -H "Authorization: Bearer shhh! it's a secret" 'https://localhost:4201/_grafana-reindex?realm=/base/' — completes without errors.
• After mise run infra:trust-dev-cert + matrix account_data migration + localStorage clear, open https://localhost:4200/, log in, click a workspace — index card populates and DevTools shows h2 for realm fetches.
• mise run dev shutdown closes the listener cleanly.
• pnpm lint passes on packages/{realm-server,host,matrix,postgres,realm-test-harness} (lint:js + prettier; pre-existing lint:types errors in ../base/*.gts are unrelated).

Closes #4787 (dual-listen approach abandoned in favor of this single-origin design).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1655a6f2df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files  ± 0      1 suites  ±0   1h 18m 34s ⏱️ + 8m 51s
2 137 tests  - 40  2 123 ✅  - 39  14 💤  - 1  0 ❌ ±0 
2 153 runs   - 40  2 139 ✅  - 39  14 💤  - 1  0 ❌ ±0 

Results for commit de5129a. ± Comparison against earlier commit f5de14e.

Realm Server Test Results

    1 files  ±0      1 suites  ±0   7m 54s ⏱️ - 5m 20s
1 372 tests ±0  1 372 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 451 runs  ±0  1 451 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit de5129a. ± Comparison against earlier commit f5de14e.

[Copilot]

Pull request overview

Enables local-dev realm-server to serve a single canonical HTTPS origin with HTTP/2 (plus same-port HTTP→HTTPS redirect) to remove Chrome’s HTTP/1.1 per-origin connection bottleneck during heavy prerender/search fan-outs, and migrates local indexed data from http://localhost:42xx to https://localhost:42xx.

Changes:

• Add TLS-capable listener that multiplexes HTTPS/HTTP2 and HTTP redirect on the same port; update URL construction to recognize TLS sockets.
• Default local dev URLs/config/docs to https://localhost:4201 (+ :4202 for test realms) and add mkcert-based cert provisioning.
• Add a Postgres migration to rewrite persisted localhost canonical URLs from http→https.

Reviewed changes

Copilot reviewed 45 out of 46 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
README.md	Document local HTTPS/HTTP2 setup, migration, and updated local URLs.
QUICKSTART.md	Update quickstart URLs to https://localhost:4201.
packages/realm-server/tests/types-endpoint-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/search-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/search-prerendered-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/info-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/index-responses-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/helpers.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/federated-types-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/server-endpoints/authentication-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/request-forward-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints/user-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints/reindex-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints/markdown-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints/info-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints/dependencies-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/realm-endpoints-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/publish-unpublish-realm-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/prerender-manager-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/openrouter-passthrough-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/module-cache-race-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/helpers/index.ts	Update close helpers/types to tolerate non-http.Server server handles.
packages/realm-server/tests/get-boxel-claimed-domain-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/file-watcher-events-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/delete-boxel-claimed-domain-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/claim-boxel-domain-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/card-source-endpoints-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/card-endpoints-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/card-dependencies-endpoint-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/boxel-domain-availability-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/tests/atomic-endpoints-test.ts	Use RealmHttpServer type alias for server handle.
packages/realm-server/server.ts	Add TLS/http2+redirect dispatcher and export RealmHttpServer type; update listen logging.
packages/realm-server/prerender/browser-manager.ts	Add --ignore-certificate-errors for prerender Chromium when using https.
packages/realm-server/middleware/index.ts	Treat TLS sockets as https for fullRequestURL() computation.
packages/realm-server/main.ts	Make shutdown tolerant of non-http.Server handles lacking closeAllConnections().
packages/realm-server/lib/dev-service-registry.ts	Broaden registry typing to net.Server.
packages/postgres/migrations/1779200000000_canonical-url-http-to-https.js	Add migration to rewrite localhost canonical URLs from http→https.
packages/host/config/schema/1779200000000_schema.sql	Add regenerated host sqlite schema snapshot.
packages/host/config/environment.js	Flip local default realm URLs to https.
mise-tasks/services/test-realms	Ensure dev cert task runs before test realms.
mise-tasks/services/realm-server-base	Ensure dev cert task runs before base realm server.
mise-tasks/services/realm-server	Ensure dev cert task runs before realm server.
mise-tasks/lib/env-vars.sh	Flip default realm URLs to https and export TLS cert/CA env vars.
mise-tasks/lib/dev-common.sh	Use https readiness probes when realm URLs are https.
mise-tasks/infra/ensure-dev-cert	New task to provision mkcert leaf cert for local HTTPS/HTTP2.
.claude/skills/indexing-diagnostics/SKILL.md	Update localhost URLs and markdown formatting in diagnostics skill doc.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 50 changed files in this pull request and generated 11 comments.