[7257] Add proper pagination to Hosted and Remote Instances pages + N+1 fix for Hosted Instances by n-lark · Pull Request #7353 · FlowFuse/flowfuse

n-lark · 2026-05-27T19:59:57Z

Description

See #7257 (comment)

Related Issue(s)

Resolves #7257

Checklist

Labels

Includes a DB migration? -> add the area:migration label

…Remote Instances

codecov · 2026-05-27T20:17:27Z

Codecov Report

❌ Patch coverage is 98.57143% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 76.71%. Comparing base (77cdb58) to head (b5cbd25).

Files with missing lines	Patch %	Lines
forge/db/models/Device.js	95.45%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7353      +/-   ##
==========================================
+ Coverage   76.65%   76.71%   +0.06%     
==========================================
  Files         410      410              
  Lines       20860    20920      +60     
  Branches     5061     5094      +33     
==========================================
+ Hits        15990    16049      +59     
- Misses       4870     4871       +1

Flag	Coverage Δ
backend	`76.71% <98.57%> (+0.06%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…t to show total count on search, update specs

…s and filter tests for server-side pagination

…keep totals in sync

…ia allOf, matching the codebase convention

cstns · 2026-06-03T11:22:27Z

                    includeSettings = false,
                    includeMeta = false,
                    limit = null,
+                    offset = null,


Should we switch to a generic pagination object to house these params, similar to the device model? This method's arguments are getting out of hand really fast.

maybe set it up as an object with default values?

Done. byTeam now takes a pagination object alongside the other opts. Defaults are inline so callers can pass a partial object or skip it entirely.

cstns · 2026-06-03T11:27:35Z

                    orderByMostRecentFlows = false,
-                    excludeApplications = null
+                    excludeApplications = null,
+                    withTotal = false


with total could be derived off if we're sending in the pagination object or not

Yup, gone. withTotal now derives from pagination.page != null inside byTeam. Callers no longer pass it explicitly.

cstns · 2026-06-03T11:30:17Z

+                        properties: {
+                            sort: {
+                                type: 'string',
+                                enum: ['name', 'createdAt', 'updatedAt', 'application.name', 'flowLastUpdatedAt']


pagination params already include sort as an option, was this only so we can type the endpoint more tightly?

Yeah PaginationParams.sort is just a string, so unknown values would silently fall through to the default order. The enum makes the route 400 instead.

cstns · 2026-06-03T11:31:24Z

        }

-        const projects = await app.db.models.Project.byTeam(request.params.teamId, options)
+        const queryResult = await app.db.models.Project.byTeam(request.params.teamId, options)


this is where we could send the pagination param in

Yup refactored.

cstns · 2026-06-03T11:36:33Z

        const options = {
            includeSettings: true,
-            limit: request.query.limit,
+            limit,


we could extract the pagination options similarly to how we do it for devices?
const paginationOptions = app.db.controllers.Device.getDevicePaginationOptions(request)

Done. New app.db.controllers.Project.getProjectPaginationOptions(request) mirrors the device pattern. ?limit= alone still gets the legacy array.

cstns · 2026-06-03T12:23:10Z

                        Alerts.emit('Successfully deleted the device', 'confirmation')
                        this.deleteLocalCopyOfDevice(device)
                        this.$emit('delete-device')
+                        if (typeof this.loadDevices === 'function') {


this has me thinking on why we need to do this if we're already deleting the local copy of the device (which should in turn remove it from the ui)

deleteLocalCopyOfDevice only removes the row from the in-memory map. Refetching keeps totalRows (server-side count) honest and pulls in a backfill row from the next page so we don't show "29 of 30" with a missing row. Skipping the refetch leaves the footer drifting and rows missing on small pages.

cstns · 2026-06-03T12:25:39Z

        return {
            loading: false,
+            // Gates the full-page spinner to first paint so refetches don't unmount the search input.
+            hasLoadedOnce: false,


this is hurting my brain. I'm not going to block it, but we need to understand why we need to resort to these sorts of things and address the underlying issue

Yah issue was the table would remount so it felt glitchy flashy. So I went down the path of a skeleton loader first but ended up just gating the load on loading && !instancesMap.size so we don't flashy glitch on page change.

cstns · 2026-06-03T12:28:23Z

                            pendingStateChange: 'pendingStateChange',
                            optimisticStateChange: 'optimisticStateChange',
-                            status: 'meta.state'
+                            status: 'status'


I may be wrong, but status and meta.status are not the same

I think i'm getting what you're doing. you're now pre-loading every instance status by default, so we won't need to re-query all instances iteratively/in a cascade. I can't place exactly where you're doing that just yet

nvm, found it, you're including includeMeta: true in the initial fetch data request

the problem with that is that the byTeam method returns the db entries of the statuses (which do not include the transitory states instances can be in, only the stable ones - see https://flowfuse.com/docs/user/instance-states/#flowfuse-node-red-instance-states). the old getInstanceMeta was calling '/:instanceId/status' which in turn was getting the actual live state of the instance by calling await instance.liveState({ omitStorageFlows: true }) which retrieved it's data twofold, corroborating from the db and from the cache. the byTeam endpoint is not doing that

the result is that the db instance status can be running (which it well may be), but its actual state could be (installing,pending etc)

This little bit needs a rethink. My thinking is that the best way about it would be to make the byTeam method use the project's actual liveState. I'm sure there were reasons why this was not done this way in the first place, but now maybe that we have proper caching in place for statuses, we might get away with it.

@hardillb I'd appreciate your input on the above proposition

For the liveState thing, isn't that what this does in Project.js? It runs through the same liveState method in the view layer the old endpointed used.

if (includeMeta) { const liveState = await project.liveState() result.meta = liveState?.meta result.flowLastUpdatedAt = liveState?.flowLastUpdatedAt }

So its like byTeam -> Project.js liveState in view -> we get back the correct pending, installing, starting state etc.

I also tested this for starting in the UI and it returns the pending state so it is working.

I may be wrong, but status and meta.status are not the same

They hold the same value on hosted-instance rows, we normalize meta.state onto r.status in api/team.js:223-224. The reason we map the column to status instead of meta.state is that the dashboard-role endpoint (/dashboard-instances) returns top-level status and no meta object, so the old meta.state mapping gave dashboard users undefined and a forever spinner. Reading status works for both response shapes.

cstns · 2026-06-03T12:29:51Z

+            // Gates the full-page spinner to first paint so refetches don't unmount the search input.
+            hasLoadedOnce: false,
            instancesMap: new Map(),
+            page: 1,


this is the 2nd time i've seen these params (granted without the default sort) but i'd say we could have a default object that we could instantiate where needed with these props and their defaults

cstns · 2026-06-03T13:00:00Z

+        paginationProps () {
+            if (this.dashboardRoleOnly) return null
+            // Hide only when no page size could ever split the result — keeps the size selector reachable otherwise.
+            if (this.totalRows <= 10) return null


again, I feel this should be handled within the component and is a leak

…lteredDevices

n-lark · 2026-06-03T20:42:24Z

Hey @cstns all feedback should be addressed - thanks!

Add server-side pagination, search, and sort to Hosted Instances and …

548ec55

…Remote Instances

n-lark self-assigned this May 27, 2026

n-lark temporarily deployed to staging May 27, 2026 20:03 — with GitHub Actions Inactive

Add loading overlay to prevent table looking glitchy on page change

db3cbbe

n-lark temporarily deployed to staging May 27, 2026 20:24 — with GitHub Actions Inactive

Fix dashboard only users seeing pagination footer, update devices lis…

d4dba5e

…t to show total count on search, update specs

n-lark temporarily deployed to staging May 27, 2026 20:54 — with GitHub Actions Inactive

Add Cypress test coverage for pagination flows; update spec intercept…

8c9e889

…s and filter tests for server-side pagination

n-lark temporarily deployed to staging May 28, 2026 16:30 — with GitHub Actions Inactive

Fix dashboard role status reads and refetch device list on delete to …

1fc84e9

…keep totals in sync

n-lark temporarily deployed to staging May 28, 2026 17:08 — with GitHub Actions Inactive

n-lark marked this pull request as ready for review May 28, 2026 17:20

n-lark requested a review from cstns May 28, 2026 17:20

Merge branch 'main' into 7257-pagination

ff8f0cd

cstns temporarily deployed to staging June 2, 2026 19:40 — with GitHub Actions Inactive

Refactor /teams/:teamId/projects schema to use : 'PaginationParams' v…

93e2ca8

…ia allOf, matching the codebase convention

n-lark temporarily deployed to staging June 2, 2026 20:11 — with GitHub Actions Inactive

Remove loading overlay from devices and instances

c8eca28

n-lark temporarily deployed to staging June 2, 2026 20:18 — with GitHub Actions Inactive

Merge branch 'main' into 7257-pagination

ed63ccf

cstns temporarily deployed to staging June 3, 2026 06:54 — with GitHub Actions Inactive

cstns requested changes Jun 3, 2026

View reviewed changes

Address PR feedback: typed Pagination.vue, prune dead nextCursor + fi…

35ab9dc

…lteredDevices

n-lark temporarily deployed to staging June 3, 2026 18:27 — with GitHub Actions Inactive

PR feedback: pagination self-hides when total <= pageSize

10120d0

n-lark temporarily deployed to staging June 3, 2026 18:38 — with GitHub Actions Inactive

PR feedback: collapse byTeam/getInstances args into a pagination object

8372ee7

n-lark temporarily deployed to staging June 3, 2026 19:04 — with GitHub Actions Inactive

Restore ?limit handling for unpaginated /projects requests

07b93f0

n-lark temporarily deployed to staging June 3, 2026 19:19 — with GitHub Actions Inactive

Add server-side sort for deviceGroup.name on device list

2c346e6

n-lark temporarily deployed to staging June 3, 2026 19:35 — with GitHub Actions Inactive

Drop hasLoadedOnce flag — derive first-paint from data presence

23e2812

n-lark temporarily deployed to staging June 3, 2026 19:40 — with GitHub Actions Inactive

Merge remote-tracking branch 'origin/main' into 7257-pagination

b558e47

n-lark temporarily deployed to staging June 3, 2026 20:02 — with GitHub Actions Inactive

Fix old heroicons import

b5cbd25

n-lark temporarily deployed to staging June 3, 2026 20:39 — with GitHub Actions Inactive

n-lark requested a review from cstns June 3, 2026 20:42

Conversation

n-lark commented May 27, 2026

Description

Related Issue(s)

Checklist

Labels

Uh oh!

codecov Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n-lark Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n-lark Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n-lark Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n-lark commented Jun 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented May 27, 2026 •

edited

Loading

n-lark Jun 3, 2026 •

edited

Loading

n-lark Jun 3, 2026 •

edited

Loading

n-lark Jun 3, 2026 •

edited

Loading