feat: update pnpm and fix stale dependency cache after dep config changes#9541
feat: update pnpm and fix stale dependency cache after dep config changes#9541zkochan merged 7 commits intoteambit:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR claims to "update express and replace request with postman-request", but the actual changes only consist of version updates to various @pnpm packages. The PR has a critical discrepancy between its title/intent and the actual code changes.
Changes:
- Updated multiple @pnpm package versions (client, config, core, default-reporter, dependency-path, fetch, lockfile packages, modules-yaml, npm-resolver, package-store, plugin-commands, sort-packages, store-connection-manager, types, worker, workspace.pkgs-graph)
| "@pnpm/client": "1001.1.20", | ||
| "@pnpm/colorize-semver-diff": "1.0.1", | ||
| "@pnpm/config": "1004.9.0", | ||
| "@pnpm/core": "1016.1.0", | ||
| "@pnpm/default-reporter": "1002.1.5", | ||
| "@pnpm/dependency-path": "1001.1.8", | ||
| "@pnpm/config": "1004.10.2", | ||
| "@pnpm/core": "1016.1.8", | ||
| "@pnpm/default-reporter": "1002.1.11", | ||
| "@pnpm/dependency-path": "1001.1.10", | ||
| "@pnpm/error": "1000.0.5", | ||
| "@pnpm/fetch": "1000.2.10", | ||
| "@pnpm/fetch": "1001.0.0", | ||
| "@pnpm/list": "^1000.3.1", | ||
| "@pnpm/lockfile.filtering": "^1001.0.28", | ||
| "@pnpm/lockfile.fs": "^1001.1.28", | ||
| "@pnpm/lockfile.types": "^1002.0.8", | ||
| "@pnpm/lockfile.filtering": "^1001.0.30", | ||
| "@pnpm/lockfile.fs": "^1001.1.30", | ||
| "@pnpm/lockfile.types": "^1002.0.9", | ||
| "@pnpm/logger": "1001.0.1", | ||
| "@pnpm/modules-yaml": "1001.0.1", | ||
| "@pnpm/modules-yaml": "1002.0.1", | ||
| "@pnpm/network.ca-file": "3.0.3", | ||
| "@pnpm/node-fetch": "^1.0.0", | ||
| "@pnpm/npm-resolver": "1005.1.0", | ||
| "@pnpm/package-store": "1007.1.0", | ||
| "@pnpm/npm-resolver": "1005.2.1", | ||
| "@pnpm/package-store": "1007.1.4", | ||
| "@pnpm/parse-overrides": "1001.0.3", | ||
| "@pnpm/plugin-commands-publishing": "1000.3.3", | ||
| "@pnpm/plugin-commands-rebuild": "1008.0.1", | ||
| "@pnpm/plugin-commands-publishing": "1000.3.11", | ||
| "@pnpm/plugin-commands-rebuild": "1008.0.9", | ||
| "@pnpm/plugin-trusted-deps": "0.2.2", | ||
| "@pnpm/registry-mock": "3.48.0", | ||
| "@pnpm/reviewing.dependencies-hierarchy": "^1001.3.1", | ||
| "@pnpm/semver-diff": "1.1.0", | ||
| "@pnpm/sort-packages": "1000.0.15", | ||
| "@pnpm/store-connection-manager": "1002.3.9", | ||
| "@pnpm/types": "1001.2.0", | ||
| "@pnpm/worker": "1000.6.1", | ||
| "@pnpm/workspace.pkgs-graph": "1000.0.33", | ||
| "@pnpm/sort-packages": "1000.0.16", | ||
| "@pnpm/store-connection-manager": "1002.3.15", | ||
| "@pnpm/types": "1001.3.0", | ||
| "@pnpm/worker": "1000.6.5", | ||
| "@pnpm/workspace.pkgs-graph": "1000.0.37", |
There was a problem hiding this comment.
The PR title states "fix: update express and replace request with postman-request", but the actual changes only update @pnpm package versions. There are no changes to express (which remains at version 4.21.2 in the workspace) and no changes related to replacing the request package with postman-request. Either the PR title is incorrect, or the PR is incomplete and missing the actual changes described in the title.
setPeer, unsetPeer, setDependency, removeDependency, reset, and eject all modify dependency configuration in .bitmap but never invalidated the filesystem dependency cache. This caused stale cached data to persist across commands — e.g. after setPeer, the next install would still treat the component as a regular dependency instead of a peer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…change The node-modules-linker only cleared the dependency cache when a component's package.json was brand new. When the file already existed but its content changed (e.g. bit.peer flag added after set-peer), the stale cache persisted. This caused dependency resolution during install to re-cache stale data before the linker wrote the updated package.json. Now the linker compares the existing package.json content with the new one and triggers cache invalidation when they differ. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…omponents Also removes redundant deleteAllDependenciesDataCache() calls from setDependency/removeDependency/reset/eject — those methods write .bitmap and invalidateDependenciesCacheIfNeeded already detects .bitmap mtime changes to clear the cache between processes. The cache flush is only necessary in setPeer/unsetPeer because the node_modules package.json (bit.peer field) is the source of truth read during dep resolution, and the linker updates it AFTER dep resolution runs during install. The explicit cache flush before install and the linker's content-change detection together close that gap. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
scopes/dependencies/dependencies/dependencies.main.runtime.ts:128
- The PR description states that the bug affects "set-peer, unset-peer, deps set, deps remove, deps reset, and deps eject", but only setPeer and unsetPeer received the cache-clearing fix.
The comment on lines 125-127 explains why: "Other dep mutations (setDependency, removeDependency, etc.) don't need this because they only change the component's own .bitmap policy, which dep resolution reads directly; the normal invalidateDependenciesCacheIfNeeded mechanism (checking .bitmap mtime) is sufficient there."
This is architecturally correct - the setPeer/unsetPeer commands are special because they set the "peer: true" config which gets written to the bit.peer field in package.json AFTER dependency resolution runs. Other commands (deps set, deps remove, deps reset, deps eject) only modify the policy field in .bitmap, which is read directly during dependency resolution.
Consider updating the PR description to be more accurate: "Fix a bug where set-peer and unset-peer modified dependency configuration in .bitmap without invalidating the filesystem dependency cache."
// Other dep mutations (setDependency, removeDependency, etc.) don't need this because they only
// change the component's own .bitmap policy, which dep resolution reads directly; the normal
// invalidateDependenciesCacheIfNeeded mechanism (checking .bitmap mtime) is sufficient there.
await this.workspace.consumer.componentFsCache.deleteAllDependenciesDataCache();
scopes/workspace/modules/node-modules-linker/node-modules-linker.ts:281
- There's a logic issue with the packageJsonCreated flag. On line 269, the condition checks
!this.packageJsonCreated && packageJsonExist. However, if the package.json doesn't exist initially (packageJsonExist is false), the code on line 223 sets packageJsonCreated to true. This means the new comparison block (lines 269-281) will never execute in the case where a package.json needs to be created from scratch.
This is actually fine for the current use case, but the flag name is misleading. The flag is now being used to track "should we invalidate the cache" rather than just "was the package.json created". Consider renaming it to shouldInvalidateCache or packageJsonChanged for clarity.
if (!this.packageJsonCreated && packageJsonExist) {
try {
const existingPkgJson = await fs.readJson(packageJsonPath);
const newPkgJsonStr = JSON.stringify(packageJson.packageJsonObject);
const existingPkgJsonStr = JSON.stringify(existingPkgJson);
if (newPkgJsonStr !== existingPkgJsonStr) {
this.packageJsonCreated = true;
}
} catch {
// if we can't read the existing package.json, treat it as changed
this.packageJsonCreated = true;
}
}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
scopes/workspace/modules/node-modules-linker/node-modules-linker.ts:276
- The package.json change detection compares
JSON.stringify()output, which is sensitive to object key insertion order. If the on-disk JSON property order differs but the data is semantically identical, this will incorrectly mark it as changed and trigger broader cache invalidation. Prefer a deep structural equality check (e.g.lodash/isEqual) or a stable stringify (sorted keys) to avoid unnecessary cache clears.
if (!this.packageJsonCreated && packageJsonExist) {
try {
const existingPkgJson = await fs.readJson(packageJsonPath);
const newPkgJsonStr = JSON.stringify(packageJson.packageJsonObject);
const existingPkgJsonStr = JSON.stringify(existingPkgJson);
if (newPkgJsonStr !== existingPkgJsonStr) {
this.packageJsonCreated = true;
}
| "@pnpm/client": "1001.1.20", | ||
| "@pnpm/colorize-semver-diff": "1.0.1", | ||
| "@pnpm/config": "1004.9.0", | ||
| "@pnpm/core": "1016.1.0", | ||
| "@pnpm/default-reporter": "1002.1.5", | ||
| "@pnpm/dependency-path": "1001.1.8", | ||
| "@pnpm/config": "1004.10.2", | ||
| "@pnpm/core": "1016.1.8", | ||
| "@pnpm/default-reporter": "1002.1.11", | ||
| "@pnpm/dependency-path": "1001.1.10", |
There was a problem hiding this comment.
PR description says only @pnpm/core and @pnpm/modules-yaml were updated, but this change updates many additional @pnpm/* packages (client/config/default-reporter/dependency-path/fetch/lockfile/*/npm-resolver/package-store/etc.). Please update the PR description to reflect the broader pnpm dependency bump (or reduce the diff if only two packages are intended).
| await this.workspace.bitMap.write(`set-peer (${componentId})`); | ||
| // Peer status is determined by reading the `bit.peer` field from the component's node_modules | ||
| // package.json, which the linker writes AFTER dep resolution runs during `bit install`. This | ||
| // means dep resolution during install re-populates the cache with stale data (no bit.peer yet), | ||
| // and the linker's subsequent package.json update doesn't clear the cache on its own unless the | ||
| // content actually changed (see node-modules-linker). Clearing here ensures the cache is empty | ||
| // going into install so that after the linker writes the correct package.json, any subsequent | ||
| // `bit show` will compute deps fresh. | ||
| // Other dep mutations (setDependency, removeDependency, etc.) don't need this because they only | ||
| // change the component's own .bitmap policy, which dep resolution reads directly; the normal | ||
| // invalidateDependenciesCacheIfNeeded mechanism (checking .bitmap mtime) is sufficient there. | ||
| await this.workspace.consumer.componentFsCache.deleteAllDependenciesDataCache(); | ||
| } |
There was a problem hiding this comment.
After clearing the filesystem dependencies cache, the workspace may still serve already-loaded components (and their dependency data) from the in-memory workspace/component loader caches in long-running processes (e.g. IDE server). Consider also calling this.workspace.clearAllComponentsCache() (or at least clearing the affected components) after deleteAllDependenciesDataCache() so subsequent bit show/dependency queries in the same process can't reuse stale component instances.
Summary
@pnpm/core1016.1.0 → 1016.1.8,@pnpm/modules-yaml1001.0.1 → 1002.0.1)set-peer,unset-peermodified dependency configuration without invalidating the filesystem dependency cache. This caused stale cached data to persist — e.g. afterset-peer, the nextinstallwould still treat the component as a regular dependency instead of a peer.Test plan
e2e/functionalities/peer-dependency-component.e2e.ts— "set-peer for existing component" passes