Skip to content

Conversation

@backspace
Copy link
Contributor

@backspace backspace commented Jan 30, 2026

screencast 2026-01-30 23-11-49

@backspace backspace self-assigned this Jan 30, 2026
@backspace backspace added the enhancement New feature or request label Jan 30, 2026
@github-actions
Copy link

Preview deployments

@github-actions
Copy link

github-actions bot commented Jan 30, 2026

Host Test Results

    1 files  ±0      1 suites  ±0   1h 43m 54s ⏱️ +14s
1 925 tests ±0  1 908 ✅ ±0  17 💤 ±0  0 ❌ ±0 
1 940 runs  ±0  1 923 ✅ ±0  17 💤 ±0  0 ❌ ±0 

Results for commit 099ce9f. ± Comparison against base commit 397e601.

♻️ This comment has been updated with latest results.

There’s a mysterious syntax error, this isn’t helping it
seems but…?
@backspace backspace force-pushed the download-realm-cs-10134 branch from 13c79a8 to 099ce9f Compare February 2, 2026 17:29
@backspace backspace marked this pull request as ready for review February 2, 2026 18:20

import RealmDropdown from '@cardstack/host/components/realm-dropdown';

// These were inline but caused the template to have spurious Glint errors
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can see the errors here. I tried for a while to come up with a better way, including splitting the template, moving the functions outside the class… this was the only way I found that worked.

@backspace backspace requested a review from a team February 2, 2026 18:26
@lukemelia
Copy link
Contributor

@codex review this. in particular, I'm curious if the implementation in this PR requires the zip file for the full realm to be assembled in memory on the realm server. If so, could it be adjusted to stream instead?

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@lukemelia
Copy link
Contributor

@codex Does the implementation in this PR mean that the entire zip file will be assembled in memory on the realm server? If so, could it be adjusted to stream as it adds/compresses instead?

@chatgpt-codex-connector
Copy link

To use Codex here, create an environment for this repo.

@lukemelia
Copy link
Contributor

The answer to my question:

The server-side implementation already streams the zip — it does not assemble it in memory. Here's the key mechanism in packages/realm-server/handlers/handle-download-realm.ts:

  • Line 149: An archiver stream is created: archiver('zip', { zlib: { level: 9 } })
  • Line 161: ctxt.respond = false disables Koa's default response handling, giving manual control of the
    response stream
  • Line 163: archive.pipe(ctxt.res) pipes the archive directly to the HTTP response — compressed chunks are
    written to the socket as they're produced
  • Line 166: archive.directory(realmPath, false) reads files from disk incrementally and feeds them into
    the archive stream

So archiver reads files from the filesystem, compresses them, and writes chunks directly to the HTTP
response. The full zip is never materialized in a single buffer on the server.

However, the client side does buffer the entire response in memory. In
packages/host/app/components/operator-mode/code-submode/left-panel-toggle.gts:139:

let blob = await response.blob();

This collects the entire streamed response into a Blob in browser memory before triggering the download
via a temporary link. For very large realms this could be a concern in the browser, but it's a common
pattern and browser Blob handling is generally efficient. An alternative would be to use the
https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream piped to a
https://github.com/nickersgmh/streamsaver.js-style approach, or more simply, to just navigate the browser
directly to the download URL (via window.location or an tag with the href set to the endpoint) and let
the browser handle the download natively — which would avoid holding it all in JS memory. The auth token
would need to be handled differently in that case (e.g., via a short-lived query parameter or cookie
rather than an Authorization header).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants