From d9bc201d421cfbb14e27f72b6765b23e44a81fdb Mon Sep 17 00:00:00 2001 From: Val Redchenko Date: Fri, 13 Feb 2026 23:36:28 +0000 Subject: [PATCH] docs: add frontend redesign ADR, design spec, and updated guidelines Record the monorepo restructure decision (ADR-0017), publish the frontend design specification refined from working documents, commit the functional requirements spec with updated route paths and terminology, and rewrite REPO-GUIDELINES.md for the npm workspaces monorepo layout. --- .../smartem-frontend/REPO-GUIDELINES.md | 148 +++-- docs/decision-records/decisions.md | 1 + ...7-smartem-frontend-monorepo-restructure.md | 100 ++++ .../smartem-frontend-design.md | 359 +++++++++++ .../smartem-frontend-requirements.md | 559 ++++++++++++++++++ 5 files changed, 1115 insertions(+), 52 deletions(-) create mode 100644 docs/decision-records/decisions/0017-smartem-frontend-monorepo-restructure.md create mode 100644 docs/decision-records/smartem-frontend-design.md create mode 100644 docs/decision-records/smartem-frontend-requirements.md diff --git a/claude-code/smartem-frontend/REPO-GUIDELINES.md b/claude-code/smartem-frontend/REPO-GUIDELINES.md index 7561ca4..a6a4469 100644 --- a/claude-code/smartem-frontend/REPO-GUIDELINES.md +++ b/claude-code/smartem-frontend/REPO-GUIDELINES.md @@ -2,60 +2,85 @@ ## Development Environment -- **Node.js**: 20+ (strict requirement) -- **Package Manager**: npm +- **Node.js**: 22+ (strict requirement) +- **Package Manager**: npm (with workspaces) - **Code Style**: Biome for TS/JS, Prettier for YAML/MD - **Git Hooks**: Lefthook (auto-installed via npm install) ## Code Standards -- **No Emojis**: Consistent with backend - no emojis in code, commits, or docs +- **No Emojis**: Consistent with backend — no emojis in code, commits, or docs - **British English**: Use British spelling in documentation and UI text - **TypeScript**: Strict mode, no `any` types without justification - **Imports**: Let Biome handle import sorting automatically +## Project Structure (Monorepo) + +The repository uses npm workspaces. See [ADR-0017](../../docs/decision-records/decisions/0017-smartem-frontend-monorepo-restructure.md) for the decision record. + +``` +smartem-frontend/ +├── apps/ +│ ├── legacy/ # Existing application (@smartem/legacy) +│ │ ├── src/ +│ │ │ ├── components/ # Reusable React components +│ │ │ ├── hooks/ # Custom React hooks +│ │ │ ├── routes/ # Route components +│ │ │ └── root.tsx # Application root with providers +│ │ ├── package.json +│ │ └── vite.config.ts +│ └── smartem/ # New application (@smartem/app) +│ ├── src/ +│ ├── package.json +│ └── vite.config.ts +├── packages/ +│ ├── api/ # Shared API client (@smartem/api) +│ │ ├── src/ +│ │ │ ├── generated/ # Orval output (DO NOT EDIT) +│ │ │ ├── mutator.ts # Axios configuration +│ │ │ ├── stubs.ts # Development stubs +│ │ │ └── index.ts # Barrel export +│ │ ├── openapi.json # OpenAPI spec (version controlled) +│ │ └── orval.config.ts +│ └── ui/ # Shared UI component library (@smartem/ui) +│ └── src/ +├── package.json # Workspace root +├── biome.json # Shared lint/format config +├── tsconfig.base.json # Shared TS config (apps extend) +├── tsconfig.json # Root project references +└── lefthook.yml # Shared git hooks +``` + ## Common Commands ```bash # Development -npm install # Install dependencies -npm run dev # Start dev server (port 5174) -npm run dev:mock # Dev with mock API data +npm install # Install all workspace dependencies +npm run dev # Legacy app dev server +npm run dev:mock # Legacy app with mock API data +npm run dev:smartem # New app dev server +npm run dev:smartem:mock # New app with mock API data # API Client -npm run api:update # Fetch OpenAPI spec + regenerate client -npm run api:fetch:local # Fetch from local backend (localhost:8000) -npm run api:generate # Regenerate client from current spec +npm run api:update # Fetch OpenAPI spec + regenerate client +npm run api:fetch:local # Fetch from local backend (localhost:8000) +npm run api:generate # Regenerate client from current spec # Code Quality -npm run check # Format + lint + imports (recommended) -npm run check:fix # Auto-fix all issues -npm run lint # Lint only -npm run format # Format only -npm run typecheck # TypeScript checking +npm run check # Biome lint + format (recommended) +npm run check:fix # Biome auto-fix all issues +npm run lint # Lint only +npm run format # Format (Biome + Prettier) +npm run typecheck # TypeScript checking across all workspaces # Build -npm run build # Production build -npm start # Run production server -``` - -## Project Structure - -``` -app/ -├── api/ -│ ├── generated/ # Auto-generated API client (DO NOT EDIT) -│ ├── mutator.ts # Axios configuration -│ └── openapi.json # OpenAPI spec (version controlled) -├── components/ # Reusable React components -├── hooks/ # Custom React hooks -├── routes/ # Route components -└── root.tsx # Application root with providers +npm run build # Build legacy app +npm run build:smartem # Build new app ``` ## API Client Workflow -The frontend uses Orval to generate a type-safe API client from the backend OpenAPI spec. +The frontend uses Orval to generate a type-safe API client from the backend OpenAPI spec. The client lives in `packages/api/` and is shared across both apps as `@smartem/api`. ### When Backend API Changes @@ -63,7 +88,7 @@ The frontend uses Orval to generate a type-safe API client from the backend Open # 1. Regenerate client npm run api:update -# 2. Check for type errors +# 2. Check for type errors across all workspaces npm run typecheck # 3. Update components using changed endpoints @@ -72,7 +97,7 @@ npm run typecheck ### Using Generated Hooks ```typescript -import { useGetAcquisitionsAcquisitionsGet } from '../api/generated/default/default' +import { useGetAcquisitionsAcquisitionsGet } from '@smartem/api' function MyComponent() { const { data, isLoading, error } = useGetAcquisitionsAcquisitionsGet() @@ -96,7 +121,7 @@ npm run api:update ### Biome (TS/JS/JSON) - Linting + formatting + import sorting -- Config: `biome.json` +- Config: `biome.json` (workspace root, shared) - Faster than ESLint + Prettier combined ### Prettier (YAML/MD) @@ -125,37 +150,56 @@ git commit --no-verify -m "message" ## Technology Stack -| Category | Technology | -|----------|------------| -| Framework | React 19 | -| Routing | React Router 7 (SSR) | -| State | TanStack Query | -| UI Components | Material-UI | -| Styling | Tailwind CSS | -| API Client | Orval (generated) | -| Build | Vite | -| Linting | Biome | -| Git Hooks | Lefthook | +| Category | Technology | Notes | +|----------|------------|-------| +| Framework | React 19 | | +| Routing (legacy) | React Router 7 | Legacy app only | +| Routing (new) | TanStack Router | New app — type-safe, SPA | +| Server State | TanStack Query | Auto-generated hooks via Orval | +| Tables | TanStack Table | New app — headless, full styling control | +| UI Components | Material UI | | +| Component Library | sci-react-ui + @smartem/ui | sci-react-ui for legacy; @smartem/ui for new | +| API Client | Orval (generated) | Shared via @smartem/api package | +| Build | Vite | Per-app config | +| Linting | Biome | Workspace root config | +| Git Hooks | Lefthook | | ## Environment Configuration | Environment | API Endpoint | |-------------|--------------| -| Development | Vite proxy `/api` -> `localhost:8000` | +| Development | Vite proxy `/api` → `localhost:8000` | +| Mock | MSW (Mock Service Worker) with client-side stubs | | Production | `VITE_API_ENDPOINT` env var or `localhost:8000` | +## Route Structure + +### Legacy app (`apps/legacy`) + +| Route | Page | +|-------|------| +| `/` | Session List (Home) | +| `/acquisitions/:acqId` | Session Dashboard | +| `/acquisitions/:acqId/grids/:gridId` | Grid Explorer | +| `/acquisitions/:acqId/grids/:gridId/atlas` | Atlas View | +| `/acquisitions/:acqId/grids/:gridId/gridsquares` | Grid Square Gallery | +| `/acquisitions/:acqId/grids/:gridId/workspace` | Workspace | +| `/acquisitions/:acqId/grids/:gridId/squares/:squareId` | Grid Square Detail | +| `/acquisitions/:acqId/grids/:gridId/square/:squareId/predictions` | Quality Analytics | +| `/models` | Models Overview | +| `/models/:modelName/grids/:gridId/weights` | Model Weights | + +### New app (`apps/smartem`) + +Uses session-anchored nesting. Route structure is under active design — see the [design specification](../../docs/decision-records/smartem-frontend-design.md) for the current plan. + ## Claude Workflow 1. Make changes to components/code 2. Run `npm run check:fix` to auto-fix formatting/lint issues -3. Run `npm run typecheck` to verify types +3. Run `npm run typecheck` to verify types across all workspaces 4. If API-related changes, ensure `npm run api:update` was run first -## Routes - -- `/` - Main application (acquisition view) -- `/admin` - System administration (separate concerns, may be split later) - ## Contributing Checklist - [ ] No TypeScript errors (`npm run typecheck`) diff --git a/docs/decision-records/decisions.md b/docs/decision-records/decisions.md index a8ac9f8..1ddfd37 100644 --- a/docs/decision-records/decisions.md +++ b/docs/decision-records/decisions.md @@ -20,5 +20,6 @@ Architectural decisions are made throughout a project's lifetime. As a way of ke - [ADR-0014: JSON Config with TypeScript Types](/docs/explanations/decisions/0014-json-config-with-typescript-types) - [ADR-0015: SmartEM Release Architecture](/docs/explanations/decisions/0015-smartem-release-architecture) - [ADR-0016: Facility Connector Fork Synchronization](/docs/explanations/decisions/0016-facility-connector-fork-sync) (Proposed) +- [ADR-0017: SmartEM Frontend Monorepo Restructure](/docs/explanations/decisions/0017-smartem-frontend-monorepo-restructure) For more information on ADRs see this [blog by Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions). diff --git a/docs/decision-records/decisions/0017-smartem-frontend-monorepo-restructure.md b/docs/decision-records/decisions/0017-smartem-frontend-monorepo-restructure.md new file mode 100644 index 0000000..d7abe14 --- /dev/null +++ b/docs/decision-records/decisions/0017-smartem-frontend-monorepo-restructure.md @@ -0,0 +1,100 @@ +# 17. SmartEM Frontend Monorepo Restructure + +Date: 2026-02-13 + +## Status + +Accepted + +## Context + +The smartem-frontend repository was a single-app React project with all source code under `src/`. This layout created several problems: + +1. **Parallel development blocked**: Building a redesigned frontend alongside the existing application was impractical within a single `src/` tree. Shared dependencies (API client, UI components) had no clean separation from application-specific code. +2. **API client coupling**: The auto-generated API client (Orval + React Query hooks) lived inside the application source, making it impossible to share between applications without circular imports or copy-paste. +3. **UI component isolation**: The planned `smartem-ui` component library (a local complement to sci-react-ui) needed its own package boundary to enforce clean interfaces and enable future extraction. + +A frontend design specification identified the monorepo restructure as a prerequisite for all further development. The existing application needed to remain fully functional during and after the transition, with the new application developed in parallel. + +## Decision + +Restructure smartem-frontend into an npm workspaces monorepo with the following layout: + +``` +smartem-frontend/ +├── apps/ +│ ├── legacy/ # Existing application (moved from src/) +│ │ ├── src/ +│ │ ├── package.json # @smartem/legacy +│ │ └── vite.config.ts +│ └── smartem/ # New application (fresh scaffold) +│ ├── src/ +│ ├── package.json # @smartem/app +│ └── vite.config.ts +├── packages/ +│ ├── api/ # Shared API client +│ │ ├── src/ +│ │ │ ├── generated/ # Orval output (DO NOT EDIT) +│ │ │ ├── mutator.ts # Axios configuration +│ │ │ ├── stubs.ts # Development stubs +│ │ │ └── index.ts # Barrel export +│ │ ├── orval.config.ts +│ │ └── package.json # @smartem/api +│ └── ui/ # Shared UI component library +│ ├── src/ +│ └── package.json # @smartem/ui +├── package.json # Workspace root (npm workspaces config) +├── biome.json # Shared lint/format config +├── tsconfig.base.json # Shared TypeScript config (apps extend) +├── tsconfig.json # Root project references +└── lefthook.yml # Shared git hooks +``` + +### Workspace packages + +| Package | Name | Purpose | +|---------|------|---------| +| `apps/legacy` | `@smartem/legacy` | Existing application, moved from `src/` | +| `apps/smartem` | `@smartem/app` | New application with redesigned shell and routes | +| `packages/api` | `@smartem/api` | Auto-generated API client (Orval config, hooks, mutator, types, stubs) | +| `packages/ui` | `@smartem/ui` | SmartEM UI component library (local, eventually syncs with sci-react-ui) | + +### Key patterns + +- **One app runs at a time**: `npm run dev` for legacy, `npm run dev:smartem` for new +- **Barrel exports**: `@smartem/api` re-exports all generated hooks via `index.ts`, so consumers import from `'@smartem/api'` rather than reaching into generated internals +- **Shared tooling at root**: Biome, Lefthook, Prettier, and TypeScript base config live at the workspace root. Each app extends `tsconfig.base.json`. +- **SPA-only**: Both applications are pure client-side SPAs (no SSR, no server components) +- **sci-react-ui**: Remains a dependency of the legacy app. `packages/ui` is the new local library that will eventually sync with sci-react-ui (take from and contribute back). + +### Root-level scripts + +```bash +npm run dev # Legacy app dev server +npm run dev:smartem # New app dev server +npm run typecheck # TypeScript checking across all workspaces +npm run api:update # Fetch OpenAPI spec + regenerate client +npm run check # Biome lint + format +npm run check:fix # Biome auto-fix +``` + +## Consequences + +### Positive + +- **Parallel development**: Legacy and redesigned applications coexist cleanly, sharing the API client and UI components without interference +- **Shared packages**: `@smartem/api` is imported identically from both apps, eliminating duplication and ensuring API client changes propagate automatically +- **Clean separation**: Application-specific code, shared libraries, and tooling configuration each have clear boundaries +- **Incremental migration**: Users can continue using the legacy app while the new application is developed; switching requires only changing which app is built/deployed +- **Future extraction**: `@smartem/ui` has a clean package boundary from day one, simplifying eventual contribution back to sci-react-ui + +### Negative + +- **CI complexity**: Build, lint, and typecheck workflows must be updated to handle workspaces (workspace-aware scripts, potentially per-app build matrix) +- **Workspace learning curve**: Contributors unfamiliar with npm workspaces may need guidance on hoisting, cross-package imports, and workspace-specific commands +- **Dockerfile updates**: The container build needs updating to handle the monorepo layout (separate concern, tracked separately) + +### Neutral + +- Node.js minimum version raised from 20 to 22 (aligns with current LTS) +- React Router 7 (legacy) and TanStack Router (new app) coexist in separate workspaces without conflict diff --git a/docs/decision-records/smartem-frontend-design.md b/docs/decision-records/smartem-frontend-design.md new file mode 100644 index 0000000..218e072 --- /dev/null +++ b/docs/decision-records/smartem-frontend-design.md @@ -0,0 +1,359 @@ +# SmartEM Frontend: Design Specification + +**Status**: Working draft — route structure and several UI decisions are marked as temporary or deferred + +This document describes the architectural and UX design of the SmartEM frontend: how the system is structured, the principles that guide it, and the patterns it follows. It complements the [functional requirements specification](smartem-frontend-requirements.md), which describes *what* the system should do. + +--- + +## 1. Overview + +SmartEM Frontend is a single-page application for monitoring and reviewing cryo-electron microscopy (cryo-EM) data acquisition sessions at Diamond Light Source. It consumes a backend REST API, an SSE event stream (via RabbitMQ), and will integrate with ARIA's GraphQL API for data deposition. + +An existing system called **Pato** (with its own backend and frontend) already provides much of the core monitoring and analysis functionality. SmartEM is being built alongside Pato, not replacing it. Some functionality will initially proxy through the SmartEM API to Pato's backend. Users will migrate naturally as SmartEM offers features Pato does not: real-time ML recommendations, live monitoring with SSE, operator advisory controls, and ARIA deposition integration. + +### Design ambition + +The frontend aims to be significantly above typical internal-tool standard — not merely functional but carefully crafted in its visual design, interaction patterns, and attention to detail. Typography, spacing, transitions, and information density are treated as first-class design concerns. The goal is a professional scientific instrument interface that rewards sustained use. + +--- + +## 2. Technology Stack + +| Category | Technology | Notes | +|----------|------------|-------| +| Framework | React 19 | | +| UI Components | Material UI (MUI) | Deep theme overrides beyond basic `createTheme` | +| Routing | TanStack Router | Type-safe, search param serialisation, loader patterns | +| Server State | TanStack Query | Auto-generated hooks via Orval | +| Tables | TanStack Table | Headless — full styling control | +| Build | Vite | | +| Linting/Formatting | Biome | | +| Git Hooks | Lefthook | | +| Component Library | sci-react-ui (DLS shared) + @smartem/ui (local) | | + +### API integration patterns + +The frontend consumes three distinct API surfaces: + +1. **REST / OpenAPI** — CRUD operations for the full data domain (sessions, grids, squares, foil holes, predictions, models). The backend is FastAPI and publishes an OpenAPI spec. Typed React Query hooks are auto-generated via Orval. Goal: near-zero hand-written fetch calls. + +2. **SSE (Server-Sent Events)** — Live event stream from RabbitMQ for real-time data during active sessions (agent events, new entities, prediction updates). The SSE client pushes cache invalidations into React Query so views update transparently regardless of whether data arrived via fetch or event stream. This is the only bespoke data integration piece. + +3. **GraphQL** — ARIA deposition API for scientific dataset submission (external system). Thin client layer via `graphql-codegen` with React Query plugin or a lightweight wrapper (urql or fetch-based). + +--- + +## 3. Repository Structure + +The repository is structured as an npm workspaces monorepo (see [ADR-0017](decisions/0017-smartem-frontend-monorepo-restructure.md)): + +``` +smartem-frontend/ +├── apps/ +│ ├── legacy/ # Existing application (moved from src/) +│ └── smartem/ # New application — fresh routes, new shell +├── packages/ +│ ├── api/ # @smartem/api — Orval config, generated hooks, mutator, types, stubs +│ └── ui/ # @smartem/ui — SmartEM UI library (local, eventually syncs with sci-react-ui) +├── package.json # npm workspaces root +├── biome.json # Shared lint/format config +├── tsconfig.base.json # Shared TypeScript config (apps extend) +└── lefthook.yml # Shared git hooks +``` + +- Both apps import from `@smartem/api` and `@smartem/ui` +- One app runs at a time (`npm run dev` for legacy, `npm run dev:smartem` for new) +- sci-react-ui remains a dependency of the legacy app; `@smartem/ui` is the new local library +- `@smartem/ui` will eventually sync with sci-react-ui (take from and contribute back) +- Mock dev mode (MSW) is carried forward to the new app +- The command palette / omnibox component lives in `@smartem/ui` for potential sharing + +--- + +## 4. Configuration Layers + +| Layer | Purpose | Example | Mechanism | +|-------|---------|---------|-----------| +| **Env config** | Deploy-time infrastructure | API base URL, auth endpoint | `import.meta.env.VITE_*` | +| **App config** | Runtime behaviour | Default density mode, default theme | Runtime config file (JSON) loaded at startup | +| **Feature flags** | Toggle incomplete/experimental features | Show agent logs tab, enable deposition route | Keys in app config, checked in code | + +Unsettled UI choices are hidden behind feature flags until confirmed. This avoids premature commitment to interaction patterns that may change. + +--- + +## 5. Data Domain Hierarchy + +The cryo-EM data domain has a strict parent-child hierarchy that drives both navigation and information architecture: + +``` +Visit (scientist visiting facility) +└── Session (user working with microscope) + └── Acquisition (data-model specific, tied to a microscope) + └── Grid (physical grid/sample, up to 12+ per cassette) + ├── Atlas (spatial overview image + tiles) + ├── GridSquare (region at medium magnification) + │ └── FoilHole (individual hole in carbon film) + │ └── Micrograph (high-resolution image) + └── Prediction Models (ML quality scores at square/hole level) +``` + +### Terminology clarification + +- **Session** is the user-facing term for working with the microscope. This is what users think in terms of. +- **Acquisition** is the data-model term, specific to a microscope. Usually 1:1 with session, but a microscope swap mid-session creates a new acquisition. +- **Visit** is a first-class citizen for ARIA depositions and facility administration. A visit contains sessions. + +### Open domain questions + +- **Visit → Session → Acquisition relationship**: How does the URL/navigation handle the case where session and acquisition are not 1:1? Does the user think in terms of sessions or acquisitions? +- **Cassette reloading**: Up to 12 grids per cassette, but reloading is possible. Is there a meaningful "cassette" concept in the UI, or is it a flat list of grids under an acquisition? +- **Remote visits**: Users can visit remotely. Does this change anything about the UI (different controls, video feed, latency considerations)? +- **Visit metadata**: Where does visit info come from — SmartEM's data model or an external system (e.g. DLS user office, ISPyB)? + +--- + +## 6. Shell Chrome + +### 6.1 Header (fixed, always visible) + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ [Logo] [Sessions] [Models] [ Search or jump to... ⌘K ] [◐ ☰] │ +└─────────────────────────────────────────────────────────────────┘ +``` + +| Zone | Content | Notes | +|------|---------|-------| +| **Left** | SmartEM logo/wordmark | Click navigates home | +| **Left-centre** | Top-level nav links: Sessions, Models | Kept visible for now — few top-level sections. Easy to remove later if the command palette makes them redundant. | +| **Centre** | Omnibox bar | Compact search field; opens command palette on focus/click or `Cmd+K` | +| **Right** | Density toggle, colour scheme toggle | Later: user avatar/menu, live indicator | +| **Right** | Login/logout placeholder | UI placeholder only — auth not implemented yet | + +No footer. Any links that would traditionally live in a footer move to an about/settings page. + +### 6.2 Context Strip (session-scoped, below header) + +Only renders when inside a `/sessions/:id/**` route. Provided by the session layout route. + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Session: EM-2026-0042 ● Live [atlas thumb] → [sq thumb] → [hole thumb] │ +└─────────────────────────────────────────────────────────────────┘ +``` + +| Element | Description | +|---------|-------------| +| **Session name + status** | Name and colour-coded status badge (green=completed, blue=started, yellow=paused, grey=planned, red=abandoned) | +| **Live indicator** | Pulse dot + "Live" or "Last updated: Xs ago" for active sessions | +| **Spatial thumbnail chain** | Miniature thumbnails showing the zoom path: atlas → selected square → selected hole. Each clickable to navigate back to that level. Thumbnails highlight the currently selected entity. Accumulate left-to-right as the user drills deeper. | + +The context strip replaces traditional breadcrumbs with spatial meaning — the user can *see* where they are on the specimen rather than reading a text path. + +### 6.3 Command Palette / Omnibox + +Overlay triggered by `Cmd+K` or clicking the omnibox bar. Renders above all content. + +**Implementation**: Open decision — use the `cmdk` library or a bespoke omnibox component. Either way, the component lives in `@smartem/ui` for potential sharing with sci-react-ui. + +**Search targets**: +- Sessions — by name, date, instrument, status +- Grids — by name, ID +- Squares, holes — by ID (deep link jump) +- Models — by name +- Commands — "go to latest session", "switch to dark mode", "toggle compact density", "toggle feature X" + +**Behaviour**: +- Results grouped by category (Sessions, Grids, Models, Commands) +- Recent items shown when opened with empty query +- Keyboard navigation (arrows, enter, esc) +- Fuzzy matching +- Selected result navigates to entity or executes command + +### 6.4 Density Modes + +Three levels: **compact** / **comfortable** / **spacious**. Toggle in header cycles through them. + +Affects spacing, font sizes, table row heights, card padding, and chart dimensions. All via CSS custom properties on the root element. Every component respects density from the start — this is not a retrofit. + +### 6.5 Theme + +- **Light mode first** — dark mode will be designed separately later +- Colour scheme toggle hidden until dark mode is ready (feature flag) +- Deep MUI theme overrides beyond basic `createTheme` (transitions, elevation, state layers, density) +- Typography and vertical rhythm tuned for data-heavy views +- Accessible colour scales for predictions and heatmaps (colour is never the only indicator) + +### 6.6 Complete Shell Wireframe + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ [Logo] [Sessions] [Models] [ Search... ⌘K ] [◐] [user] │ ← Header (always) +├─────────────────────────────────────────────────────────────────┤ +│ Session: EM-2026-0042 ● Live [atlas] → [sq] → [hole] │ ← Context strip (session routes) +├─────────────────────────────────────────────────────────────────┤ +│ [Atlas] [Squares] [Predictions] [Workspace] │ ← View switcher (grid routes) +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ │ +│ Route content │ ← Outlet +│ │ +│ │ +└─────────────────────────────────────────────────────────────────┘ + +Command palette: full-screen overlay on ⌘K, above everything +``` + +--- + +## 7. Route Structure + +**Status**: Temporary — using Option B (session-anchored nesting) as the working structure. Will be revisited once auth, ARIA deposition, and admin views are better defined. + +### Spatial hierarchy routes + +``` +/ → Landing page (placeholder) +/visits/:visitId → Visit overview (content TBD) +/sessions/:sessionId → Session overview (layout route) +/sessions/:sessionId/grids/:gridId → Grid level (layout route, atlas default) +/sessions/:sessionId/grids/:gridId/atlas → Atlas spatial view +/sessions/:sessionId/grids/:gridId/squares → GridSquare table/gallery +/sessions/:sessionId/grids/:gridId/predictions → Grid-level prediction dashboard +/sessions/:sessionId/grids/:gridId/workspace → Configurable panel layout +/sessions/:sessionId/grids/:gridId/squares/:squareId → Square detail (layout route) +/sessions/:sessionId/grids/:gridId/squares/:squareId/holes/:holeId + → Foil hole detail (TBD) +/sessions/:sessionId/grids/:gridId/squares/:squareId/holes/:holeId/micrographs/:micrographId + → Micrograph detail (TBD) +``` + +### Cross-cutting routes + +``` +/models → ML model catalogue +/models/:modelName → Model detail + cross-grid performance +/sessions/:sessionId/logs → Agent log viewer (future, feature-flagged) +/sessions/:sessionId/deposition → ARIA deposition workflow (future, feature-flagged) +``` + +### Layout routes + +- **Root (`__root`)**: Header + command palette overlay + outlet +- **Session (`/sessions/:sessionId`)**: Context strip (session name, live status, spatial thumbnail chain) + outlet +- **Grid (`/sessions/:sessionId/grids/:gridId`)**: View switcher (Atlas | Squares | Predictions | Workspace) + outlet +- **Square (`/.../squares/:squareId`)**: Parallel structure to grid level (spatial default + view switcher) + +### Navigation within grid level + +A view switcher (tabs, segmented control, or chips) enables lateral navigation between views of the same grid without returning to the parent: + +- **Atlas** (default) — spatial map with prediction overlays +- **Squares** — tabular/gallery view with stats +- **Predictions** — grid-level prediction charts +- **Workspace** — configurable multi-panel layout + +Square level mirrors this structure with fewer tabs. + +### Open routing questions + +- Should `/visits/:visitId` show sessions for that visit, or be a richer visit-level view (metadata, ARIA deposition status, linked proposals)? +- Where does acquisition sit? Hidden inside session context, or exposed in the URL when a session has multiple acquisitions? +- Do admin/facility routes live in the same namespace or under an `/admin` prefix? +- How do "my sessions" vs "all sessions" vs "visit sessions" relate in navigation when auth is in place? + +--- + +## 8. UX Principles + +### No modals + +With the rarest exceptions (destructive confirmation dialogs, and even those should prefer inline confirmation with undo), nothing becomes a modal. Everything that would lazily become a modal should be an inline expansion, slide-over panel, dedicated route, or in-context reveal. + +### Spatial navigation metaphor + +The data hierarchy (session → grid → square → foil hole → micrograph) maps to decreasing physical scale on the specimen. Navigation should feel like zooming into a specimen, not clicking through a file tree. The context strip reinforces this with spatial thumbnails rather than text breadcrumbs. + +### Motion as information + +Layout animations communicate state changes rather than serving as decoration. Filter results flow rather than snap. Form sections settle rather than shove. New entities appear with brief highlight animations. Prediction overlays update smoothly when new data arrives. + +### Keyboard-first + +The command palette (`Cmd+K`) is the highest-impact power user feature. Keyboard shortcuts are provided for major interactions throughout the application. The interface is usable without a mouse for common workflows. + +### Accessible colour scales + +Colour is never the sole indicator of quality or status. Predictions and heatmaps use size, shape, and labels in addition to colour. Colour scales are chosen for accessibility (sufficient contrast ratios, distinguishable in common forms of colour vision deficiency). WCAG 2.1 AA compliance as a baseline. + +### Data density as a design parameter + +The application is data-heavy by nature. Rather than fighting this, density modes (compact/comfortable/spacious) treat information density as a first-class design parameter. Typography, spacing, and vertical rhythm are tuned for sustained use with tabular and chart-heavy views. + +--- + +## 9. State Management + +### Approach + +| Concern | Owner | Mechanism | +|---------|-------|-----------| +| Server state | React Query | Auto-generated hooks via Orval, cache invalidation via SSE | +| View state | URL search params | TanStack Router serialisation (deep links, shareability) | +| App-wide UI state | React context | Theme, command palette visibility, SSE connection status, density mode | +| Complex cross-view coordination | Deferred | Add Zustand/Jotai only if a specific problem demands it | + +### SSE integration pattern + +The SSE client maintains a persistent connection during active session monitoring. When events arrive, the client invalidates the relevant React Query cache entries rather than managing a parallel state tree. This means views update transparently — they do not need to know whether data arrived via an initial fetch or a live event. + +The most likely candidate for a dedicated client-side store is coordinating linked selections across workspace panels (e.g. clicking a point in a latent space scatter plot highlights the corresponding grid square on the atlas). This will be evaluated when the workspace view is implemented. + +### What is explicitly avoided + +- No centralised Redux-style store by default +- No parallel state trees that duplicate server data +- No prop drilling for cross-cutting concerns (use context instead) + +--- + +## 10. Reference: Existing System Patterns + +The existing Pato system provides useful reference for features SmartEM will need: + +- **Atlas view**: Two-panel layout — grid overlay left, foil hole heatmap right with metric toggles (resolution, astigmatism, particle count) +- **Processing views**: CTF graphs, motion correction drift plots, particle picking stats, 2D/3D classification galleries, refinement — as expandable accordion sections +- **Micrograph viewing**: Before/after image flipper, dark image detection, movie-by-movie navigation +- **Heatmaps**: Foil hole overlays with configurable metrics and colour scales +- **Calendar view**: Calendar component for browsing sessions by date +- **Alerts**: User-configurable metric thresholds with email notifications +- **Navigation**: Proposal → Session → Data Collection Groups → type-dispatched views + +SmartEM's differentiation: real-time ML recommendations, live monitoring with SSE, operator advisory controls, ARIA deposition integration. + +--- + +## 11. Deferred Scope + +These are acknowledged as important but deferred from the initial shell implementation. All can be feature-flagged in when ready. + +| Feature | Notes | +|---------|-------| +| **Auth implementation** | Placeholders for login/logout only. Design accommodates route guards, token management, conditional UI. | +| **Dark mode** | Light mode first. Dark mode will be designed separately. | +| **SSE live data integration** | Architecture designed for it; implementation deferred. | +| **ARIA GraphQL deposition** | Needs its own route and multi-step form workflow. | +| **Agent log viewer** | Structured log events with correlation to atlas view. Backend work also needed. | +| **Bug reporting widget** | One-click capture of URL, screenshot, cache state, browser info → GitHub Issues. | +| **Mobile version** | Different functionality from desktop (session status, failure alerts, quality trends). Needs separate design discussion for build strategy, structure, and activation. | +| **Operator controls** | Approve/reject ML recommendations, manual square selection, session pause/resume, parameter adjustment, annotations. These collectively require a toolbar/action bar in the atlas/grid view. | +| **Processing pipeline view** | Per-micrograph processing status (motion correction → CTF → particle picking → selection). Orthogonal to the spatial hierarchy. | +| **Foil hole + micrograph detail views** | Routes exist in the structure; content needs domain input to specify. | +| **CI pipeline** | Monorepo CI (lint, typecheck, build per app). Tracked separately. | +| **Bun.js evaluation** | Evaluate as alternative to Node.js for dev tooling and package management. | +| **Command palette implementation** | `cmdk` library vs bespoke component — compare capabilities, styling flexibility, maintenance burden. | +| **State management review** | The "no centralised store by default" approach needs validation against specific cross-view coordination scenarios. | +| **Minimum supported screen size** | Identify the smallest viewport that fits the desktop content. | +| **Release versioning** | Strategy for matching frontend versions with backend, agent, and other system components. | diff --git a/docs/decision-records/smartem-frontend-requirements.md b/docs/decision-records/smartem-frontend-requirements.md new file mode 100644 index 0000000..5eaa519 --- /dev/null +++ b/docs/decision-records/smartem-frontend-requirements.md @@ -0,0 +1,559 @@ +# SmartEM Frontend: UX and Functional Requirements + +## 1. Introduction and Users + +SmartEM Frontend is a web application for monitoring and reviewing cryo-electron microscopy (cryo-EM) data acquisition sessions at Diamond Light Source. It provides real-time visibility into automated data collection, displays ML-driven quality predictions at multiple levels of the sample hierarchy, and supports both live session monitoring and post-session analysis. The frontend is a pure SPA that consumes the SmartEM backend REST API and is deployed in proximity to the backend infrastructure. + +### 1.1 Primary Users + +| User | Role | Primary Needs | +|------|------|---------------| +| **Facility operators / microscopists** | Run and monitor acquisition sessions at the beamline | Live session status, quality feedback during collection, ability to assess whether collection is proceeding well | +| **Researchers** | Review data quality after sessions, plan follow-up experiments | Post-session analysis, quality distributions, comparison across grids and models | +| **Facility managers** | Oversee utilisation and throughput | High-level session summaries, completion rates, time-on-task | + +### 1.2 Usage Modes + +- **Live monitoring**: Operator observes an active acquisition session. Data arrives incrementally as the microscope collects. ML predictions update as processing completes. The operator needs to see progress and quality signals without manually refreshing. +- **Post-session review**: Researcher or operator reviews a completed session. All data is available. The user explores quality across grids, squares, and foil holes. They compare ML models, examine spatial patterns, and identify regions of interest. + +--- + +## 2. Information Architecture + +### 2.1 Data Hierarchy + +The domain has a strict parent-child hierarchy that drives navigation: + +``` +Acquisition (session) +└── Grid (one per physical grid/sample) + ├── Atlas (spatial overview of the entire grid) + │ └── Atlas Tiles (stitched image segments) + ├── Grid Square (region of the grid imaged at medium magnification) + │ ├── Foil Hole (individual hole in the carbon support film) + │ │ └── Micrograph (high-resolution image of the specimen) + │ └── Quality Predictions (ML scores at square or foilhole level) + └── Prediction Models (ML models that produce quality scores) +``` + +### 2.2 Page Hierarchy and Navigation Model + +Navigation follows the data hierarchy with lateral views at each level. + +> **Note**: Route paths are being redesigned as part of the frontend redesign. The paths below reflect the legacy application. The new route structure uses session-anchored nesting (`/sessions/:sessionId/grids/:gridId/...`) — see the [design specification](smartem-frontend-design.md) for the current route plan. + +``` +Session List (Home) +├── Session Dashboard (/sessions/:sessionId) +│ ├── Grid Explorer (/sessions/:sessionId/grids/:gridId) +│ ├── Atlas View (/sessions/:sessionId/grids/:gridId/atlas) +│ ├── Grid Square Gallery (/sessions/:sessionId/grids/:gridId/squares) +│ ├── Workspace (/sessions/:sessionId/grids/:gridId/workspace) +│ ├── Grid Square Detail (/sessions/:sessionId/grids/:gridId/squares/:squareId) +│ └── Quality Analytics (/sessions/:sessionId/grids/:gridId/squares/:squareId/predictions) +└── Models Overview (/models) + └── Model Weights (/models/:modelName) +``` + +A **spatial context strip** replaces traditional breadcrumbs, showing miniature thumbnails of the zoom path (atlas → square → hole) that serve the same "where am I" function with spatial meaning. See the [design specification](smartem-frontend-design.md) for details. + +**Lateral navigation** allows switching between views of the same grid (atlas, table, gallery, workspace) without returning to parent. A view selector (tabs or segmented control) at the grid level enables this. + +--- + +## 3. Views and Functional Requirements + +### 3.1 Session List (Home) + +**Purpose**: Entry point. Shows all acquisition sessions so the user can find the one they care about. + +**What it shows**: +- Table of acquisitions with columns: + - Session name + - Status (planned, started, paused, completed, abandoned) + - Start time, end time (if applicable) + - Instrument model / ID + - Grid count (number of grids in the session) +- Live indicator (pulsing dot or badge) for sessions with status `started` +- Quick stats per session: number of grids, number of grid squares collected, progress estimate + +**User actions**: +- Click a session row to navigate to Session Dashboard +- Sort and filter by status, date range, instrument +- Search by session name + +**Data sources**: +- `GET /acquisitions` (list all acquisitions) + +**Open questions**: +- Should completed sessions be shown by default, or should there be a recency filter? +- Should the home page show a summary dashboard (total sessions, active sessions) above the table? + +--- + +### 3.2 Session Dashboard + +**Purpose**: Overview of a single acquisition session. Shows session metadata and all grids within it. + +**What it shows**: +- Session metadata: name, status, instrument, start/end time, clustering mode/radius, storage path +- Status badge with colour coding (green=completed, blue=started, yellow=paused, grey=planned, red=abandoned) +- Grid list as cards or table rows, each showing: + - Grid name + - Grid status (none, scan started, scan completed, grid squares decision started/completed) + - Scan start/end time + - Number of grid squares + - Quick quality summary (if predictions available) +- Session-level quality summary card (aggregated from `GET /quality_metrics`) +- Progress indicator for active sessions + +**User actions**: +- Click a grid to navigate to the default grid view (Atlas View or Grid Explorer) +- Return to Session List via breadcrumb + +**Data sources**: +- `GET /acquisitions/{acquisition_uuid}` (session detail) +- `GET /acquisitions/{acquisition_uuid}/grids` (grids in session) +- `GET /quality_metrics` (aggregate quality stats) + +--- + +### 3.3 Grid Explorer + +**Purpose**: Tabular view of grid squares within a grid, with expandable detail rows. + +**What it shows**: +- Table of grid squares with columns: + - Grid square ID + - Status (none, registered, foil holes decision started/completed) + - Selected (boolean - whether the square was selected for collection) + - Foil hole count + - Quality prediction score (latest, per selected model) + - Acquisition datetime + - Defocus, magnification, pixel size +- Expandable rows revealing foil holes within each square: + - Foil hole ID + - Quality score + - Position (x, y) + - Diameter + - Near grid bar flag + - Micrograph count (if available) + +**User actions**: +- Expand/collapse rows to see foil holes +- Sort by any column (foil hole count, quality score, status) +- Filter by status, by "selected only", by quality threshold +- Click Insights icon on a row to navigate to Quality Analytics for that square +- Click a square row to navigate to Grid Square Detail + +**Data sources**: +- `GET /grids/{grid_uuid}/gridsquares` (all squares in grid) +- `GET /gridsquares/{gridsquare_uuid}/foilholes` (foil holes per square, on expand) +- `GET /prediction_model/{name}/grid/{grid_uuid}/prediction` (predictions for overlay) + +--- + +### 3.4 Atlas View (Spatial Visualisation) + +**Purpose**: Spatial visualisation of the entire grid. The atlas image is an overview image of the physical grid, and grid squares are overlaid as interactive regions. + +**What it shows**: +- Atlas image (rendered from MRC data via `GET /grids/{grid_uuid}/atlas_image`) +- Interactive grid square overlays: + - Circles (or rectangles) positioned at each square's center coordinates + - Size scaled by grid square physical size and/or prediction value + - Colour-coded by quality prediction (green > 0.5, red <= 0.5, purple = no prediction) + - Blue highlight for ML-suggested squares +- Model selector dropdown to choose which prediction model colours the overlay +- Toggle button to show/hide prediction overlay +- Latent space panel (optional, toggleable): + - Scatter plot of 2D latent representations (from dimensionality reduction) + - Points coloured by cluster index + - Click a point to highlight the corresponding grid square on the atlas + - Model selector for latent representation model + - Toggle to show/hide unselected squares + +**User actions**: +- Hover over a grid square to see tooltip (ID, prediction value, status) +- Click a grid square to freeze selection and see detail summary +- Click a grid square to navigate to Grid Square Detail +- Toggle predictions on/off +- Switch prediction model +- Open/close latent space panel +- Pan and zoom the atlas image + +**Data sources**: +- `GET /grids/{grid_uuid}/atlas_image` (atlas image, with optional crop params x, y, w, h) +- `GET /grids/{grid_uuid}/gridsquares` (square positions and metadata) +- `GET /prediction_model/{name}/grid/{grid_uuid}/prediction` (quality predictions) +- `GET /prediction_model/{name}/grid/{grid_uuid}/latent_representation` (latent space data) +- `GET /grid/{grid_uuid}/prediction_model/{name}/latent_rep/{name}/suggested_squares` (suggested squares) +- `GET /prediction_models` (available models for selector) + +--- + +### 3.5 Grid Square Detail + +**Purpose**: Detailed view of a single grid square. Shows the square image with foil hole overlays, analogous to Atlas View but one level deeper. + +**What it shows**: +- Grid square image (rendered from MRC/TIFF via `GET /gridsquares/{uuid}/gridsquare_image`) +- Interactive foil hole overlays: + - Circles at each foil hole position + - Size proportional to diameter and/or prediction value + - Colour-coded by quality prediction + - Grey/excluded for foil holes near grid bar +- Model selector dropdown +- Latent space panel (optional): + - Scatter plot of foil hole latent representations for this square + - Points coloured by cluster index + +**User actions**: +- Hover over a foil hole to see tooltip (ID, prediction, quality, diameter) +- Click a foil hole to see its micrograph information +- Toggle predictions on/off +- Switch prediction model +- Navigate to Quality Analytics via Insights button + +**Data sources**: +- `GET /gridsquares/{uuid}/gridsquare_image` (square image) +- `GET /gridsquares/{uuid}/foilholes?on_square_only=true` (foil holes) +- `GET /prediction_model/{name}/gridsquare/{uuid}/prediction` (foilhole predictions) +- `GET /prediction_model/{name}/gridsquare/{uuid}/latent_representation` (latent space) +- `GET /gridsquare/{uuid}/overall_prediction` (overall combined prediction) +- `GET /prediction_models` (available models) + +--- + +### 3.6 Quality Analytics + +**Purpose**: Time-series and distribution views of quality predictions for a specific grid square and its foil holes. Answers "how has quality evolved?" and "what is the distribution of quality?" + +**What it shows**: +- Grid of cards, each corresponding to a prediction model: + - **Grid square prediction time series**: Line chart showing prediction value over time + - Y-axis: quality value (0-1) + - X-axis: timestamp + - Statistics panel: mean, current value, standard deviation, time range + - **Foil hole prediction distribution**: Bar chart (histogram) of foilhole prediction values + - 19 bins from 0 to 1 + - Temporal slider to scrub through prediction snapshots (most recent to oldest) +- Metric selector dropdown at the top to filter by quality metric name (e.g., default quality vs specific metrics like CTF resolution) + +**User actions**: +- Select a prediction model to focus on +- Adjust time range via slider +- Switch between quality metrics +- Compare distributions across time snapshots + +**Data sources**: +- `GET /gridsquares/{uuid}/quality_predictions` (time series, grouped by model) +- `GET /gridsquares/{uuid}/foilhole_quality_predictions` (foilhole distributions, grouped by model and foilhole) +- `GET /quality_metric/{name}/gridsquares/{uuid}/foilhole_quality_predictions` (metric-filtered) +- `GET /quality_metrics` (available metrics for selector) + +--- + +### 3.7 Models Overview + +**Purpose**: Catalogue of available ML prediction models. Shows what models exist, their metadata, and provides access to model weight evolution. + +**What it shows**: +- Grid of model cards, each showing: + - Model name + - Description + - Level (gridsquare or foilhole) + - Avatar/icon +- Click through to model weight time series for a specific grid + +**Model weights view** (sub-page): +- Time series chart of model weight evolution for a given grid +- Shows how the model's confidence/calibration has changed during the session +- Statistics: mean weight, current weight, standard deviation + +**User actions**: +- Browse available models +- Click a model card to see detail +- Select a grid to view model weight evolution +- Navigate to Atlas View with a specific model pre-selected + +**Data sources**: +- `GET /prediction_models` (list all models) +- `GET /prediction_models/{name}` (model detail) +- `GET /grid/{grid_uuid}/model_weights` (weight time series, grouped by model) + +--- + +### 3.8 Workspace (Configurable Dashboard) + +**Purpose**: Flexible layout for power users who want to see multiple views side-by-side during a session. + +**What it shows**: +- Expandable/collapsible panels +- Add menu to add components: + - Atlas view panel + - Grid Explorer panel + - Quality Analytics panel + - Model Weights panel +- Each panel can be expanded to full width or collapsed + +**User actions**: +- Add panels from a menu +- Expand/collapse individual panels +- Remove panels +- Future: drag-and-drop reordering, save layout presets + +**Data sources**: Same as underlying components (Atlas View, Grid Explorer, etc.) + +**Open questions**: +- Should layout persist across sessions (local storage)? +- What is the maximum number of panels before performance degrades? +- Should this replace or complement the individual view pages? + +--- + +## 4. Real-Time Requirements + +### 4.1 Live Data Integration + +The backend currently supports SSE for agent-to-backend communication (`GET /agent/{agent_id}/session/{session_id}/instructions/stream`). The frontend does not yet consume SSE, but the architecture is designed for it. + +**Required real-time behaviours**: + +| Data Type | Update Frequency | Mechanism | +|-----------|-----------------|-----------| +| Acquisition status changes | On state transition | Polling or SSE | +| New grid squares appearing | As agent ingests EPU data | Polling or SSE | +| New foil holes registered | As agent processes square metadata | Polling or SSE | +| New micrographs detected | As high-res images are acquired | Polling or SSE | +| Quality predictions | As ML models produce scores | Polling or SSE | +| Model weight updates | As models retrain on new data | Polling or SSE | +| Processing status (motion correction, CTF, particle picking) | As cryoem-services completes stages | Polling or SSE | + +**Current approach**: React Query polling with 5-minute stale time and 1 retry. For live monitoring, this should be reduced to a shorter interval (5-30 seconds) when viewing an active session, or replaced with SSE push. + +### 4.2 Session Lifecycle States + +The frontend must represent these acquisition states visually: + +| Status | Visual Indicator | Behaviour | +|--------|-----------------|-----------| +| `planned` | Grey badge | Static, no live updates expected | +| `started` | Blue/green badge + pulse animation | Live updates active, auto-refresh | +| `paused` | Yellow/amber badge | Paused state, may resume | +| `completed` | Green badge (solid) | All data available, no further updates | +| `abandoned` | Red/grey badge | Session terminated early | + +### 4.3 Visual Indicators for Data Arrival + +During live monitoring: +- New entities (squares, holes, micrographs) should appear with a brief highlight animation +- Prediction overlay colours should update when new predictions arrive +- A "last updated" timestamp or "live" indicator should be visible +- Optional: toast/notification for significant events (new grid started, predictions complete) + +--- + +## 5. Future: Operator Control Features + +These features are not currently implemented but are planned. The backend has the infrastructure (SSE instruction stream, agent sessions, instruction acknowledgements) to support them. + +### 5.1 Approve/Reject ML Recommendations + +- When ML models suggest grid squares for collection, the operator can review and approve/reject individual suggestions +- Approved suggestions are sent as instructions to the agent via the existing SSE instruction mechanism +- Rejection removes the square from the suggested collection set + +### 5.2 Manual Square Selection/Deselection + +- On the Atlas View, the operator can manually select or deselect grid squares for collection +- Selected squares are sent as instructions to the agent +- Visual distinction between ML-suggested and manually selected squares + +### 5.3 Session Pause/Resume + +- Operator can pause a running session from the frontend +- Backend sets `paused_time` on the acquisition, agent receives pause instruction +- Resume restarts collection from where it left off + +### 5.4 Parameter Adjustment + +- Adjust collection parameters (defocus, clustering radius) during a session +- Parameters sent as instructions to agent + +### 5.5 Annotation and Notes + +- Operator can add text notes to acquisitions, grids, or individual squares +- Notes persist in the database and are visible during post-session review +- Useful for flagging interesting regions or recording observations + +--- + +## 6. Cross-Cutting Concerns + +### 6.1 Responsive Design + +- Primary target: desktop (1920x1080 and above) - this is the beamline monitoring use case +- Secondary target: tablet (1024px+) for facility walk-arounds +- Mobile not a priority but basic usability should be maintained + +### 6.2 Accessibility + +- Colour is not the only indicator of quality (use size, shape, labels in addition to colour) +- Keyboard navigation for major interactions +- Screen reader support for tables and navigation +- Sufficient contrast ratios (WCAG 2.1 AA) + +### 6.3 Theme + +- Light mode first; dark mode will be designed separately +- Colour scheme toggle hidden until dark mode is ready (feature flag) +- Deep MUI theme overrides (transitions, elevation, state layers, density) +- DLS branding (Diamond Light Source logo, eBIC links) + +### 6.4 Performance + +- Atlas images can be large (4000x4000+ pixels). Use progressive loading and crop parameters where supported. +- Grids can have hundreds of squares with thousands of foil holes. Virtualise long lists. +- Prediction data can be voluminous for time series. Paginate or aggregate server-side. +- Image conversion (MRC to PNG) happens server-side; cache results. + +### 6.5 Error States and Offline Behaviour + +- Show clear error messages when API calls fail (connection refused, 404, 500) +- Distinguish between "no data yet" (empty state) and "error loading data" +- Stale data indicator when real-time updates are interrupted +- Graceful degradation: if prediction endpoints fail, still show structural data (squares, holes) + +--- + +## 7. API Coverage Matrix + +### 7.1 View-to-Endpoint Mapping + +| View | Endpoint | Status | Notes | +|------|----------|--------|-------| +| **Session List** | `GET /acquisitions` | Implemented | Missing: grid count per session (requires join or separate call) | +| **Session Dashboard** | `GET /acquisitions/{uuid}` | Implemented | | +| | `GET /acquisitions/{uuid}/grids` | Implemented | | +| | `GET /quality_metrics` | Implemented | Global metrics, not per-session | +| **Grid Explorer** | `GET /grids/{uuid}/gridsquares` | Implemented | | +| | `GET /gridsquares/{uuid}/foilholes` | Implemented | Supports `on_square_only` param | +| | `GET /prediction_model/{name}/grid/{uuid}/prediction` | Implemented | | +| **Atlas View** | `GET /grids/{uuid}/atlas_image` | Implemented | Supports crop via x,y,w,h params | +| | `GET /grids/{uuid}/gridsquares` | Implemented | | +| | `GET /prediction_model/{name}/grid/{uuid}/prediction` | Implemented | Frontend currently uses stubs | +| | `GET /prediction_model/{name}/grid/{uuid}/latent_representation` | Implemented | Frontend currently uses stubs | +| | `GET /grid/{uuid}/prediction_model/{name}/latent_rep/{name}/suggested_squares` | Implemented | | +| | `GET /prediction_models` | Implemented | | +| **Grid Square Gallery** | `GET /grids/{uuid}/gridsquares` | Implemented | | +| | `GET /grids/{uuid}/atlas_image` (cropped) | Implemented | Used for thumbnails | +| **Grid Square Detail** | `GET /gridsquares/{uuid}/gridsquare_image` | Implemented | | +| | `GET /gridsquares/{uuid}/foilholes` | Implemented | | +| | `GET /prediction_model/{name}/gridsquare/{uuid}/prediction` | Implemented | | +| | `GET /prediction_model/{name}/gridsquare/{uuid}/latent_representation` | Implemented | | +| | `GET /gridsquare/{uuid}/overall_prediction` | Implemented | | +| **Quality Analytics** | `GET /gridsquares/{uuid}/quality_predictions` | Implemented | Time series, grouped by model | +| | `GET /gridsquares/{uuid}/foilhole_quality_predictions` | Implemented | Nested: model -> foilhole -> predictions | +| | `GET /quality_metric/{name}/gridsquares/{uuid}/foilhole_quality_predictions` | Implemented | Metric-filtered variant | +| | `GET /quality_metrics` | Implemented | For metric selector | +| **Models Overview** | `GET /prediction_models` | Implemented | Frontend uses stub data | +| | `GET /prediction_models/{name}` | Implemented | | +| **Model Weights** | `GET /grid/{uuid}/model_weights` | Implemented | Time series, grouped by model | +| **Workspace** | (Composite - uses endpoints from other views) | Implemented | | + +### 7.2 Identified Gaps + +| Need | Current State | Proposed Solution | +|------|--------------|-------------------| +| Grid count per acquisition in session list | Requires `GET /acquisitions/{uuid}/grids` per row (N+1 problem) | Add `grid_count` field to `AcquisitionResponse` or a summary endpoint | +| Per-session quality metrics | `GET /quality_metrics` is global, not scoped to a session | Add `GET /acquisitions/{uuid}/quality_metrics` | +| Acquisition completion percentage | Not computed server-side | Derive from grid statuses client-side, or add server-side calculation | +| Foil hole count per grid square | Requires expanding each square | Add `foilhole_count` to `GridSquareResponse` | +| Micrograph count per foil hole | Not surfaced in foilhole response | Add `micrograph_count` to `FoilHoleResponse` | +| Frontend SSE for live updates | SSE exists only for agent communication | Add frontend-facing SSE endpoint or configure polling intervals | +| Session search/filter | No server-side filtering on acquisitions | Add query params (status, date range, name search) to `GET /acquisitions` | + +### 7.3 Frontend Stub Usage (to be migrated to real endpoints) + +The frontend currently uses client-side stubs (`packages/api/src/stubs.ts`) for: +- `getPredictionModels` (4 mock models) +- `getPredictions` (grid-level predictions) +- `getLatentRep` (latent representations) + +These stubs should be replaced with the real auto-generated React Query hooks once the frontend-backend integration is verified end-to-end. + +--- + +## Appendix A: Entity Status Reference + +### Acquisition Status +| Value | Meaning | +|-------|---------| +| `planned` | Session created but not started | +| `started` | Active data collection | +| `paused` | Temporarily paused | +| `completed` | Collection finished normally | +| `abandoned` | Session cancelled | + +### Grid Status +| Value | Meaning | +|-------|---------| +| `none` | Initial state | +| `scan started` | Grid scanning in progress | +| `scan completed` | All grid square positions identified | +| `grid squares decision started` | ML model evaluating grid squares | +| `grid squares decision completed` | ML recommendations available for grid squares | + +### Grid Square Status +| Value | Meaning | +|-------|---------| +| `none` | Initial state | +| `all foil holes registered` | All foil holes at this square have been detected | +| `foil holes decision started` | ML model evaluating foil holes | +| `foil holes decision completed` | ML recommendations available for foil holes | + +### Foil Hole Status +| Value | Meaning | +|-------|---------| +| `none` | Initial state | +| `micrographs detected` | High-resolution images acquired at this hole | + +### Micrograph Status +| Value | Meaning | +|-------|---------| +| `none` | Initial state | +| `motion correction started` | Motion correction processing begun | +| `motion correction completed` | Motion correction done | +| `ctf started` | CTF estimation begun | +| `ctf completed` | CTF estimation done | +| `particle picking started` | Particle picking begun | +| `particle picking completed` | Particle picking done | +| `particle selection started` | Particle selection/filtering begun | +| `particle selection completed` | Particle selection done | + +### Model Level +| Value | Meaning | +|-------|---------| +| `gridsquare` | Model produces predictions at grid square level | +| `foilhole` | Model produces predictions at foil hole level | + +## Appendix B: Frontend Route Summary + +> **Note**: These routes reflect the legacy application (`apps/legacy`). The new application (`apps/smartem`) uses session-anchored nesting — see the [design specification](smartem-frontend-design.md) for the new route structure. + +### Legacy application routes + +| Route | Page | Status | +|-------|------|--------| +| `/` | Session List (Home) | Implemented (basic table) | +| `/acquisitions/:acqId` | Session Dashboard | Implemented (grid table) | +| `/acquisitions/:acqId/grids/:gridId` | Grid Explorer | Implemented (collapsible table) | +| `/acquisitions/:acqId/grids/:gridId/atlas` | Atlas View | Implemented (full) | +| `/acquisitions/:acqId/grids/:gridId/gridsquares` | Grid Square Gallery | Implemented (paginated cards) | +| `/acquisitions/:acqId/grids/:gridId/workspace` | Workspace | Partial (basic panels) | +| `/acquisitions/:acqId/grids/:gridId/squares/:squareId` | Grid Square Detail | Implemented (full) | +| `/acquisitions/:acqId/grids/:gridId/square/:squareId/predictions` | Quality Analytics | Implemented (charts) | +| `/models` | Models Overview | Stub data only | +| `/models/:modelName/grids/:gridId/weights` | Model Weights | Implemented (time series) |