-
Notifications
You must be signed in to change notification settings - Fork 4
Cluster Caching #226
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cluster Caching #226
Conversation
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…e now, so long as the token is active and the account is paid and in good standing.
…meters (#227) - Remove X-Worker-ID debug headers from middleware response interceptors - Remove unused timing variables (startTime, workerId, duration, clusterGetDuration) from delete() and invalidateByObject() methods - Remove deprecated countAsInvalidation parameter from delete() method signature - Remove countAsInvalidation arguments from all 6 call sites (2 in index.js, 4 in middleware.js) All changes are code cleanup only - no functional changes. Tests: 54/54 passing in cache.test.js Co-authored-by: Claude <noreply@anthropic.com>
…italHumanities/rerum_server_nodejs into 224-cluster-caching
|
|
||
| const versionIds = [objIdShort, previousId, primeId].filter(id => id && id !== 'root').join('|') | ||
| if (versionIds) { | ||
| const regex = new RegExp(`^(history|since):(${versionIds})`) |
Check failure
Code scanning / CodeQL
Regular expression injection High
user-provided value
This regular expression is constructed from a
user-provided value
This regular expression is constructed from a
user-provided value
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 2 months ago
To fix this issue, all user input that is joined/interpolated into a regular expression must be sanitized to escape regex special characters. The best-known solution is to use lodash’s escapeRegExp function, which safely escapes all regex metacharacters in a given string. Thus, before interpolating the values (objIdShort, previousId, and primeId) into the regex, each value must be passed through _.escapeRegExp.
Make the following changes in cache/middleware.js:
- Import lodash (
_) at the top of the file. (If other parts of your project already use lodash, you may already have it, but add it if not.) - In the section where
versionIdsis constructed, apply_.escapeRegExpto each of the three IDs before joining them. - No changes to project functionality, just ensures any regex metacharacters in user-controlled input are escaped.
-
Copy modified line R10 -
Copy modified lines R278-R281
| @@ -7,6 +7,7 @@ | ||
|
|
||
| import cache from './index.js' | ||
| import { getAgentClaim } from '../controllers/utils.js' | ||
| import _ from 'lodash'; | ||
|
|
||
| const sendCacheHit = (res, data, includeCacheControl = false) => { | ||
| res.set('Content-Type', 'application/json; charset=utf-8') | ||
| @@ -274,7 +275,10 @@ | ||
| // Also invalidate based on NEW object in case it matches different queries | ||
| await cache.invalidateByObject(updatedObject, invalidatedKeys) | ||
|
|
||
| const versionIds = [objIdShort, previousId, primeId].filter(id => id && id !== 'root').join('|') | ||
| const versionIds = [objIdShort, previousId, primeId] | ||
| .filter(id => id && id !== 'root') | ||
| .map(id => _.escapeRegExp(id)) | ||
| .join('|'); | ||
| if (versionIds) { | ||
| const regex = new RegExp(`^(history|since):(${versionIds})`) | ||
| await cache.invalidate(regex, invalidatedKeys) |
-
Copy modified lines R40-R41
| @@ -37,7 +37,8 @@ | ||
| "express-urlrewrite": "~2.0.3", | ||
| "mongodb": "~6.20.0", | ||
| "morgan": "~1.10.1", | ||
| "pm2-cluster-cache": "^2.1.7" | ||
| "pm2-cluster-cache": "^2.1.7", | ||
| "lodash": "^4.17.21" | ||
| }, | ||
| "devDependencies": { | ||
| "@jest/globals": "^30.2.0", |
| Package | Version | Security advisories |
| lodash (npm) | 4.17.21 | None |
| const primeId = extractId(releasedObject?.__rerum?.history?.prime) | ||
| const versionIds = [objIdShort, previousId, primeId].filter(id => id && id !== 'root').join('|') | ||
| if (versionIds) { | ||
| const regex = new RegExp(`^(history|since):(${versionIds})`) |
Check failure
Code scanning / CodeQL
Regular expression injection High
user-provided value
This regular expression is constructed from a
user-provided value
This regular expression is constructed from a
user-provided value
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 2 months ago
To fix the problem, every value that is interpolated into the regular expression must be escaped in a way that prevents it from affecting the regular expression's structure. The best approach is to use a well-tested library function, such as _.escapeRegExp from lodash, to escape any regex special characters in each component ID before constructing the regex pattern.
Specifically, in cache/middleware.js:
- Import lodash's
escapeRegExp. - When creating
versionIds, escape each id in the array using_.escapeRegExp, before joining them and inserting them into the regex pattern.
This ensures that no part of user-controlled input can introduce regex metacharacters or modify the meaning of the expression.
-
Copy modified line R10 -
Copy modified lines R343-R346
| @@ -7,7 +7,7 @@ | ||
|
|
||
| import cache from './index.js' | ||
| import { getAgentClaim } from '../controllers/utils.js' | ||
|
|
||
| import { escapeRegExp } from 'lodash'; | ||
| const sendCacheHit = (res, data, includeCacheControl = false) => { | ||
| res.set('Content-Type', 'application/json; charset=utf-8') | ||
| res.set('X-Cache', 'HIT') | ||
| @@ -340,7 +340,10 @@ | ||
| // Invalidate version chain caches | ||
| const previousId = extractId(releasedObject?.__rerum?.history?.previous) | ||
| const primeId = extractId(releasedObject?.__rerum?.history?.prime) | ||
| const versionIds = [objIdShort, previousId, primeId].filter(id => id && id !== 'root').join('|') | ||
| const versionIdSegments = [objIdShort, previousId, primeId] | ||
| .filter(id => id && id !== 'root') | ||
| .map(id => escapeRegExp(id)); | ||
| const versionIds = versionIdSegments.join('|') | ||
| if (versionIds) { | ||
| const regex = new RegExp(`^(history|since):(${versionIds})`) | ||
| await cache.invalidate(regex, invalidatedKeys) |
| const currentVersionTS = originalObject.__rerum?.isOverwritten ?? "" | ||
|
|
||
| if (expectedVersion !== undefined && expectedVersion !== currentVersionTS) { | ||
| console.log(`RERUM v1 says 'If-Overwritten-Version' header value '${expectedVersion}' does not match current version '${currentVersionTS}'`) |
Check warning
Code scanning / CodeQL
Log injection Medium
user-provided value
Log entry depends on a
user-provided value
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 2 months ago
To fix the log injection risk, any user-supplied value that is written to logs should be sanitized to remove newlines and other characters that can create log injection vulnerabilities. In the affected line (console.log on line 65), the potentially dangerous value is expectedVersion, which is taken directly from an HTTP header. The best and simplest fix is to sanitize expectedVersion to remove any newline sequences before logging.
- Before logging, replace all occurrences of
\rand\ninexpectedVersion(the user-supplied header value) with empty strings. - The string interpolation in the log message remains unchanged, except now using the sanitized version.
- This should be done only for the log entry, not for processing/logic elsewhere (for auditability of errors as they occurred).
What to change:
- In file
controllers/overwrite.js, on or just before line 65, add a line that creates a sanitized version ofexpectedVersionby removing\rand\n(using.replace(/\r|\n/g, "")). - Use this sanitized variable in the log statement on line 65.
No additional imports are needed.
-
Copy modified lines R65-R67
| @@ -62,7 +62,9 @@ | ||
| const currentVersionTS = originalObject.__rerum?.isOverwritten ?? "" | ||
|
|
||
| if (expectedVersion !== undefined && expectedVersion !== currentVersionTS) { | ||
| console.log(`RERUM v1 says 'If-Overwritten-Version' header value '${expectedVersion}' does not match current version '${currentVersionTS}'`) | ||
| // Sanitize user-provided value to avoid log injection | ||
| const sanitizedExpectedVersion = typeof expectedVersion === "string" ? expectedVersion.replace(/\r|\n/g, "") : expectedVersion; | ||
| console.log(`RERUM v1 says 'If-Overwritten-Version' header value '${sanitizedExpectedVersion}' does not match current version '${currentVersionTS}'`) | ||
| console.log("overwrite 409") | ||
| res.status(409) | ||
| res.json({ |
No description provided.