-
Notifications
You must be signed in to change notification settings - Fork 0
Development to Main Diff #431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
* Adding endpoint to save changes from the layer * Adding New Layer * No Empty Label and no Annotations * Updated new Layer * Adding Items to partOf * Changing id convention * Removing updating layer * Annotation Change * Adding Delete Endpoint * not sure... This type of thing? * Label Change * Added Layer Class * Adding rerum ids to delete and add * Changing tests * Update exists.test.mjs * Changing LayerLabel * Changes AnnotationCollection Structure * Update Pages API * Updating partOf Ids in case of any change * Adding Layer Metadata Label Change * example results as a base for comments, delete these files before merge. * Adding AddLayer Commenting Rest of the APIs * Added DeleteLayer Back * Deleting json files * Updates to the comments * Making Project to Layer Changes * Renaming layerAnnotationCollection * Removing UpdateOne() changes * Removing UpdateOne() changes * Removing UpdateOne() changes * Error Handled for no ProjectID or LayerID * Clear Code * send() instead of json() --------- Co-authored-by: cubap <cubap@slu.edu> Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
GitHub per @mepripri
…pleenv-file 91 create sampleenv file
* determine data type by content * Removing type dependencies - Took controller and type out of controller - Added a function to detect the data types and assign the correct collection * matching tests to code move
* Removing fileSystem from Github API * Update ProjectFactory.mjs --------- Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
* file dump init * services for feedback * /feedback/feedback is not a good route * Update cd_dev.yaml * stop if things are missing * adding verify on main * test results * added docs and ES6 conversion * what the Golden AI hell is adding imports from my docs?
* file dump init * services for feedback * /feedback/feedback is not a good route * Update cd_dev.yaml * stop if things are missing * adding verify on main * test results * added docs and ES6 conversion * what the Golden AI hell is adding imports from my docs? * explicitly adding CORS
* API call to Update Profile * existingEmail and existingName * Changes to comments * Update User.mjs
* saveCollection to RERUM * Adding Tinypen to Create RERUM Object * Update exists.test.mjs * Update exists.test.mjs * Update Page.mjs * Update cd_dev.yaml * stop if things are missing * adding verify on main * starting some adjustments * better the tests * headed home * multiple ways to extract the data * retest with suggestions * layer/page halos * percolating deletes * setting up routes --------- Co-authored-by: Patrick Cuba <cubap@slu.edu>
* issues * 204 here * upgrade _sub property when user in no longer temp
* Go with minimax m2 * Another kind of line uri pattern * cleanup * polish * polish * polish * polish * Touch up so we don't resolve objects that are already resolved
* invite flow * upgrades and rejections when temp user was invited to multiple projects before doing either. * upgrades and rejections when temp user was invited to multiple projects before doing either. * cleanup * copilot suggestion * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* opus 4.5 unattended nice * cleanup * consolodate tests * polish * A deeper equality check for comparing JSON objects * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* dang this needs to go to development as a hotfix * Let a logged in user leave a project they are a part of (same business rules apply as for removing a user) * refactor a bit. Send users an email when they are removed from a project by admins, or when they leave a group voluntarily * cleanup * polish * Apply suggestion from @Copilot whitespace in email Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Move stuff around based on review. Allow removal of the only leader. * unused * rename for separation of actions * changes from testing * Move stuff around * changes from testing * indentation * id instead of projectid for consistency * id instead of projectid for consistency * id instead of projectid for consistency * id instead of projectid for consistency --------- Co-authored-by: Patrick Cuba <cubap@slu.edu> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Create a New Column * Delete classes/Column/__tests__/Column.test.js This was all skipped, so let's just avoid it * Implement code review recommendations for Column feature This commit addresses all the critical and recommended improvements from the code review: - Added TPENCOLUMNS environment variable to config.env - Fixed variable scope errors in checkAndCreateUnorderedColumn function - Added null safety checks for page.columns in line/index.js (lines 135, 184, 238) - Added comprehensive error handling to all Column class methods - Added JSDoc documentation for Column class and helper functions - Added input validation to createNewColumn method - Refactored checkAndCreateUnorderedColumn into smaller, more maintainable helper functions: - getRemainingAnnotations() - createUnorderedColumn() - deleteUnorderedColumn() - updateUnorderedColumn() - Updated CLAUDE.md and copilot-instructions.md with TPENCOLUMNS documentation These changes improve code reliability, maintainability, and documentation quality. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Patrick Cuba <cubap@users.noreply.github.com> * Merge and Extend endpoints * Prev and Next addition, sus check * dup labels * headers issue resolved * code cleanup * existingUnorderedColumn guard * true * suggested changes * Self review * No comments * whitespace check * remove UnorderedColumn totally * PrevNext Fix * docs * fixes * 400 projectID pageId * space * label col fix * delete column if empty * cols in index * variable name change * line fix * column guards --------- Co-authored-by: Patrick Cuba <cubap@slu.edu> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Patrick Cuba <cubap@users.noreply.github.com> Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
* Columns Update on Annotator * remove update * back to start * Column Update on Line Parsing Change * deletes * bounds and post line * bounds fix * logs * itemsProvided * line fix * delete fix * remove comment
#428) * Initial plan * Fix my/projects endpoint to return 200 with empty array for users with no projects Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> * Update API documentation to reflect new empty projects response * Revert unintended package-lock.json changes Co-authored-by: thehabes <3287006+thehabes@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> Co-authored-by: thehabes <3287006+thehabes@users.noreply.github.com>
* Runtime improvement on the /text amd /bounds routes * remove debug logging * whitespace * whitespace * one user fetch * not a diff * Change from standup, catch errors from the page and project update * Change from standup, catch errors from the page and project update * fix for copilot comment
* Error catching and logging * touchup * small tweak from testing and reviewing * Stop this unhandled 500 * Stop this 500 * 500s from other routes * Error handler as a middleware util * stop this 500 * Stop that 500 * returns * returns * use respondWithError in app.js too * touchup * touchup * touchup * touchup from copilot review * touchup from copilot review * touchup from copilot review * oof too much of a logic change, this is what we actually wanted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR merges substantial development work into main, introducing new features, improving error handling consistency, and enhancing the user authentication flow. The changes span across multiple core areas of the TPEN Services API including user management, project collaboration, page/line annotations, and column management.
Key Changes:
- New Column feature for organizing line annotations within pages with full CRUD endpoints
- Enhanced temporary user handling with automatic merging when invited users sign up
- Improved error handling consistency with standardized
respondWithErrorusage across all routes - New resolved page endpoint for fetching complete annotation data in a single request
Reviewed changes
Copilot reviewed 39 out of 39 changed files in this pull request and generated 34 comments.
Show a summary per file
| File | Description |
|---|---|
| utilities/shared.js | Added new utility functions (resolveReferences, hasAnnotationChanges, resolveReference), updated updatePageAndProject with column handling logic, changed findLineInPage to use page.items instead of page.lines |
| utilities/routeErrorHandler.js | New centralized error handling middleware for Express with colored logging |
| utilities/proxy.js | Standardized error responses using respondWithError, added protocol validation |
| utilities/mailer/index.js | Changed forEach to for...of for env validation, made transporter.verify async |
| userProfile/privateProfile.js | Changed empty projects response from 404 to 200 with empty array |
| project/projectToolsRouter.js | Added request body validation, standardized error handling with return statements |
| project/projectReadRouter.js | Standardized all .all() handlers to return respondWithError |
| project/projectCreateRouter.js | Removed error.code fallback, added body validation, simplified error responses |
| project/projectCopyRouter.js | Removed error.code fallback, moved body destructuring after validation |
| project/memberRouter.js | Added body validation, changed Project instantiation to getById for consistency |
| project/memberLeaveRouter.js | New router for voluntary project leave functionality |
| project/memberDeclineInviteRouter.js | Fixed spelling error in comment, standardized error handling |
| project/index.js | Registered new memberLeaveRouter |
| page/index.js | Major refactor: added Column support, new column CRUD endpoints, split filter IDs logic, resolved page endpoint |
| line/index.js | Added Column updates when line IDs change, improved error handling, added body validation |
| layer/index.js | Added body validation, improved pages array validation, standardized error responses |
| classes/Column/Column.js | New Column class for managing annotation columns with MongoDB persistence |
| classes/Line/Line.js | Added hasAnnotationChanges check to skip unnecessary RERUM updates, new updateTargetXYWH method, fixed body default to empty array |
| classes/Layer/Layer.js | Preserved columns property when converting pages to project format |
| classes/Group/Group.js | Added transferMembership and getGroupsByMember methods, enhanced removeMember validation |
| classes/Project/Project.js | Enhanced invite flow to reuse temp users, added email notifications for member removal |
| classes/User/User.js | Added mergeFromTemporaryUser method, duplicate email validation, fixed getSelf async pattern |
| auth/index.js | Enhanced authentication to handle temp user merging and _sub tracking |
| app.js | Integrated routeErrorHandler middleware, standardized service down response |
| config.env | Added TPENCOLUMNS environment variable |
| API.md | Updated /my/projects response documentation, added /page/:pageId/resolved endpoint documentation |
| feedback/feedbackController.js | Added body validation for both submitFeedback and submitBug |
| tests/mount.test.js | Added test for new member leave route |
| utilities/tests/shared.test.js | New comprehensive tests for hasAnnotationChanges utility function |
| classes/User/tests/unit.test.js | Added skipped test for empty projects response |
| /** | ||
| * Splits a list of items into groups based on non-date IDs. | ||
| * | ||
| * @param {Array} items - The list of items to be split | ||
| * @returns {Array} An array of objects, each containing a non-date ID as the key and an array of date IDs as the value | ||
| */ | ||
| function splitFilterIds(items) { | ||
| const isDateId = (id) => /^\d+$/.test(id) | ||
| let result = [] | ||
| let currentKey = null | ||
| let currentDates = [] | ||
|
|
||
| for (const item of items) { | ||
| const id = item.id | ||
| if (!isDateId(id)) { | ||
| if (currentKey && currentDates.length > 0) { | ||
| result.push({ [currentKey]: currentDates }) | ||
| } | ||
| currentKey = id | ||
| currentDates = [] | ||
| } else { | ||
| currentDates.push(id) | ||
| } | ||
| } | ||
| if (currentKey && currentDates.length > 0) { | ||
| result.push({ [currentKey]: currentDates }) | ||
| } | ||
| return result | ||
| } |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The splitFilterIds function uses a regex /^\d+$/ to determine if an ID is a "date ID" based on whether it's all digits. This naming and logic is unclear - why are numeric IDs called "date IDs"? The function groups items by non-numeric IDs with their following numeric IDs, but the purpose and expected input format should be documented. Add comprehensive documentation explaining the expected input format and use case.
| } | ||
| this.data.members[memberId] = { roles: [] } | ||
| this.setMemberRoles(memberId, roles) | ||
| await this.setMemberRoles(memberId, roles) |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The setMemberRoles call is now awaited, but immediately after on line 64, the method returns without waiting for the result to be used. If the caller expects the member to be fully set up before this method returns, this is correct. However, if there were side effects expected from not awaiting previously, this could change behavior. Verify this change is intentional and all callers handle the async behavior correctly.
| } catch (error) { | ||
| console.error(`Failed to resolve annotation ${item}:`, error) | ||
| // Return the ID string if fetching fails | ||
| return { id: item, error: error.message } |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error object returned when fetching fails includes both an id field (from the failed string ID) and an error field with the error message. This creates an inconsistent data structure - items in the array could be either full annotation objects OR error objects with this shape. Consumers need to check for the presence of the error field to distinguish. Consider using a more explicit error indicator or throwing an error to fail fast rather than returning partial results.
| return respondWithError(res, 400, "Request body is required") | ||
| } | ||
| const { label, annotations } = req.body | ||
| if (typeof label !== 'string' || !label?.trim() || !Array.isArray(annotations)) { |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The condition checks both typeof label !== 'string' and !label?.trim(). The optional chaining ?. will return undefined if label is not a string, so the typeof check is redundant. Simplify to just check !label?.trim() or keep the typeof check and use regular property access !label.trim() for clarity.
| if (typeof label !== 'string' || !label?.trim() || !Array.isArray(annotations)) { | |
| if (!label?.trim() || !Array.isArray(annotations)) { |
| const hasPixel = /xywh=pixel/.test(target) | ||
| const prefix = hasPixel ? "#xywh=pixel" : "#xywh=pixel" | ||
| if (target.includes("#xywh")) { | ||
| return target.replace(/#xywh(=pixel)?:?.*/, `${prefix}:${x},${y},${w},${h}`) | ||
| } | ||
| return `${target}#xywh=pixel:${x},${y},${w},${h}` |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The updateTargetXYWH method has inconsistent behavior for string targets. When target.includes("#xywh") is true, it replaces the fragment, but when false, it always uses #xywh=pixel prefix. However, line 169-170 sets prefix based on whether "pixel" exists in the target, but then line 170 hardcodes #xywh=pixel regardless. This means the hasPixel check on line 169 has no effect on the actual prefix used. Fix the logic to respect the original format.
| const hasPixel = /xywh=pixel/.test(target) | |
| const prefix = hasPixel ? "#xywh=pixel" : "#xywh=pixel" | |
| if (target.includes("#xywh")) { | |
| return target.replace(/#xywh(=pixel)?:?.*/, `${prefix}:${x},${y},${w},${h}`) | |
| } | |
| return `${target}#xywh=pixel:${x},${y},${w},${h}` | |
| const hasPixel = /#xywh=pixel/.test(target) | |
| const prefix = hasPixel ? "#xywh=pixel" : "#xywh" | |
| if (target.includes("#xywh")) { | |
| return target.replace(/#xywh(=pixel)?:?.*/, `${prefix}:${x},${y},${w},${h}`) | |
| } | |
| return `${target}${prefix}:${x},${y},${w},${h}` |
| let error_out | ||
| const layerIndex = project.data.layers.findIndex(l => l.pages.some(p => p.id.split('/').pop() === page.id.split('/').pop())) | ||
| if (layerIndex < 0 || layerIndex === undefined || layerIndex === null) throw new Error("Cannot update Page. Its Layer was not found.") | ||
| if (layerIndex < 0 || layerIndex === undefined || layerIndex === null) { | ||
| error_out = new Error("Cannot update Page. Its Layer was not found.") | ||
| error_out.status = 500 | ||
| throw error_out |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error_out variable is declared with let but doesn't need reassignment. It could be declared with const at the point where it's assigned. Consider refactoring to throw the error directly at each error point rather than assigning to a variable first, which would be clearer and more idiomatic.
| label: page.label, | ||
| target: page.target, | ||
| items: page.items ?? [], | ||
| columns: layer.pages[pageIndex].columns |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The columns property is copied from the old page reference (layer.pages[pageIndex].columns) to the formatted page, but this may include stale data. Consider whether columns should be updated based on the actual page state rather than copying from the existing reference.
| return respondWithError(res, 400, 'Items must be an array') | ||
| } | ||
| if (Array.isArray(update.items) && update.items.some(item => (typeof item !== 'object' && typeof item !== 'string') || item === null)) { | ||
| return respondWithError(res, 400, 'Each item must be an object') |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The validation Array.isArray(update.items) && update.items.some(item => (typeof item !== 'object' && typeof item !== 'string') || item === null) allows strings in the items array. However, the error message says "Each item must be an object", which is misleading since strings are explicitly allowed by the validation. Update the error message to reflect that items can be either objects or strings.
| return respondWithError(res, 400, 'Each item must be an object') | |
| return respondWithError(res, 400, 'Each item must be an object or a string') |
| if (itemsProvided) { | ||
| for (const updatePair of updatedItemsList) { | ||
| const oldId = Object.keys(updatePair)[0] | ||
| const newId = updatePair[oldId] | ||
| if (pageInProject.columns && pageInProject.columns.length > 0) { | ||
| for (const column of pageInProject.columns) { | ||
| if (column.lines.includes(oldId)) { | ||
| const columnDB = new Column(column.id) | ||
| const columnData = await columnDB.getColumnData() | ||
| const lineIndex = columnData.lines.indexOf(oldId) | ||
| if (lineIndex !== -1 && newId !== null && newId !== oldId) { | ||
| columnData.lines[lineIndex] = newId | ||
| } | ||
| if (newId === null) { | ||
| columnData.lines = columnData.lines.filter(lineId => lineId !== oldId) | ||
| } | ||
| columnDB.data = columnData | ||
| await columnDB.update() | ||
| pageColumnsUpdate = pageColumnsUpdate.map(col => { | ||
| if (col.id === column.id) { | ||
| return { | ||
| id: column.id, | ||
| label: column.label, | ||
| lines: columnData.lines | ||
| } | ||
| } | ||
| return col | ||
| }) | ||
| if (columnData.lines.length === 0) { | ||
| await columnDB.delete() | ||
| pageColumnsUpdate = pageColumnsUpdate.filter(col => col.id !== column.id) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| if (!oldId.startsWith(process.env.RERUMIDPREFIX)) { | ||
| const splitIdsEntry = splitIds.find(pair => Object.keys(pair)[0] === newId) | ||
| const newColumnRecord = await Column.createNewColumn(pageId, projectId, null, [newId, ...(splitIdsEntry ? splitIdsEntry[newId] : [])]) | ||
| const newColumn = { | ||
| id: newColumnRecord._id, | ||
| label: newColumnRecord.label, | ||
| lines: newColumnRecord.lines | ||
| } | ||
| pageColumnsUpdate = pageColumnsUpdate ? [...pageColumnsUpdate, newColumn] : [newColumn] | ||
| } else { | ||
| const columnToUpdate = pageColumnsUpdate.find(col => col.lines.includes(newId)) | ||
| if (columnToUpdate) { | ||
| const columnDB = new Column(columnToUpdate.id) | ||
| const columnData = await columnDB.getColumnData() | ||
| const splitIdsEntry = splitIds.find(pair => Object.keys(pair)[0] === newId) | ||
| if (splitIdsEntry) { | ||
| const dateIds = splitIdsEntry[newId] | ||
| columnData.lines.push(...dateIds) | ||
| columnDB.data = columnData | ||
| await columnDB.update() | ||
| pageColumnsUpdate = pageColumnsUpdate.map(col => { | ||
| if (col.id === columnToUpdate.id) { | ||
| return { | ||
| id: columnToUpdate.id, | ||
| label: columnToUpdate.label, | ||
| lines: columnData.lines | ||
| } | ||
| } | ||
| return col | ||
| }) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The complex column management logic (lines 172-241) involves multiple database operations including fetches, updates, and deletes within a loop. This could lead to performance issues with many columns or items. Consider batching database operations or refactoring to reduce the number of individual database calls. Additionally, this logic lacks transaction support - if an operation fails midway, the data could be left in an inconsistent state.
| jest.restoreAllMocks() | ||
| }) | ||
|
|
||
| it.skip("should return 200 with empty projects array when user has no projects", async () => { |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test is marked as skip, which means it won't run in the test suite. If this test is important for validating the new behavior (returning 200 with empty projects instead of 404), it should be enabled. Remove the .skip and ensure the test passes, or add a comment explaining why it's skipped.
* 'pixel' in fragment selector hotfix * Changes from testing
* Update packages, npm versions, and node versions * changes from testing * oh boy * undo localhost * undo localhost * the force * the force
No description provided.