Resolve database lock contention w/ vscode-codeql

[data-douser]

Resolves #117

Summary of Changes

Fixes a known compatibility issue for databases added, and therefore locked, via the GitHub.vscode-codeql extension.

The vscode-codeql query server creates .lock files in the cache directory of every registered CodeQL database, preventing the ql-mcp server from running CLI commands (codeql_query_run, codeql_database_analyze) against those same databases.

Add a DatabaseCopier that syncs databases from vscode-codeql storage into a managed directory under the vscode-codeql-development-mcp-server extension's globalStorage, stripping .lock files from the copy. The EnvironmentBuilder now sets CODEQL_DATABASES_BASE_DIRS to this managed directory by default (configurable via codeql-mcp.copyDatabases).

• New DatabaseCopier class with incremental sync (skips unchanged databases)
• StoragePaths.getManagedDatabaseStoragePath() for the managed databases/ dir
• EnvironmentBuilder accepts injectable DatabaseCopierFactory for testability
• codeql-mcp.copyDatabases setting (default: true)
• 11 unit tests for DatabaseCopier (real filesystem operations)
• 15 unit tests for EnvironmentBuilder (updated for copy mode + fallback)
• 3 bridge integration tests (managed dir structure, no .lock files)
• 4 E2E integration tests: inject .lock → copy → codeql_query_run + codeql_database_analyze succeed against the lock-free copy

Outline of Changes

Database Management and Copying:

• Introduced a new DatabaseCopier class (database-copier.ts) that copies CodeQL databases from the vscode-codeql extension storage into a managed directory, removing .lock files to prevent lock contention with the query server. Databases are only re-copied if the source is newer or missing in the destination. ([extensions/vscode/src/bridge/database-copier.tsR1-R147](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-f95b7c227745cc63fd413aca3bb3ab92f38fa6c5913c5764f99d1a1d5d90e7e5R1-R147))
• Updated the EnvironmentBuilder to use the DatabaseCopier when the copyDatabases configuration is enabled (default), so the MCP server uses lock-free database copies. Added a factory for testability. ([[1]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-39d2037d09f583e2b747bb6205aba05e4fafaba6ee23a1ea2b7449aff3b11d10R7-R13), [[2]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-39d2037d09f583e2b747bb6205aba05e4fafaba6ee23a1ea2b7449aff3b11d10R29-R39), [[3]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-39d2037d09f583e2b747bb6205aba05e4fafaba6ee23a1ea2b7449aff3b11d10L86-R111))
• Added a new storage path method to provide the managed database directory location. ([extensions/vscode/src/bridge/storage-paths.tsR91-R98](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-f43625f47e526f8ce69d6f5987ff4aac2f148f2b37b4b250c27cb16e37023491R91-R98))

Configuration and Settings Refactor:

• Refactored and reordered configuration options in package.json for clarity, including introducing copyDatabases (default: true) to control whether databases are copied and lock files removed. Reordered and clarified other related options. ([[1]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-288f1ebddf6afc6081e2ca910029e86e55c9cd7871b425c2a97a82a2cd77ac9eL51-R57), [[2]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-288f1ebddf6afc6081e2ca910029e86e55c9cd7871b425c2a97a82a2cd77ac9eL87-R114))

Testing and Test Infrastructure:

• Added a comprehensive test suite for the new DatabaseCopier class, covering copying logic, lock file removal, error handling, and edge cases. ([extensions/vscode/test/bridge/database-copier.test.tsR1-R193](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-1eb1f847db90fa157e74eb68805d0d11d6e5c6dfb824bff742dc102a7eba9c52R1-R193))
• Updated environment builder tests to verify the new managed directory logic and integration with the copier factory. ([[1]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-a1d34577b9157e066782fd383cecd056a13b7aca3b5cfd22c30be830b26aabecL3-R4), [[2]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-a1d34577b9157e066782fd383cecd056a13b7aca3b5cfd22c30be830b26aabecR26), [[3]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-a1d34577b9157e066782fd383cecd056a13b7aca3b5cfd22c30be830b26aabecR51-R71), [[4]](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-a1d34577b9157e066782fd383cecd056a13b7aca3b5cfd22c30be830b26aabecL90-R108))
• Included a new end-to-end test file in the test suite configuration. ([extensions/vscode/esbuild.config.jsR38](https://github.com/advanced-security/codeql-development-mcp-server/pull/118/files#diff-59325d32f2d0a8314cd0d7ed9a3a25ae5ebde8d46f2264104deafd3629fd2ef9R38))

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[Copilot]

Pull request overview

This PR addresses CodeQL database lock contention when the GitHub.vscode-codeql extension has registered databases (and created persistent .lock files), by introducing a “managed copy” database directory for the MCP server to use by default.

Changes:

• Added a DatabaseCopier that syncs databases into the MCP extension’s own managed databases/ directory and strips .lock files.
• Updated EnvironmentBuilder to set CODEQL_DATABASES_BASE_DIRS to the managed directory when codeql-mcp.copyDatabases is enabled (default), with an injectable factory for tests.
• Expanded unit/integration/E2E coverage and added the new E2E suite to the extension test bundle.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
extensions/vscode/src/bridge/database-copier.ts	New copier for lock-free database copies into a managed directory.
extensions/vscode/src/bridge/environment-builder.ts	Uses copier + managed dir when copyDatabases is enabled; adds factory injection.
extensions/vscode/src/bridge/storage-paths.ts	Adds getManagedDatabaseStoragePath() for managed databases/ dir.
extensions/vscode/package.json	Adds codeql-mcp.copyDatabases setting (default true) and reorders config entries.
extensions/vscode/esbuild.config.js	Includes new E2E test entrypoint in the integration bundle.
extensions/vscode/test/bridge/database-copier.test.ts	New unit tests for DatabaseCopier.
extensions/vscode/test/bridge/environment-builder.test.ts	Updates tests for default copy mode + disabled-copy fallback.
extensions/vscode/test/bridge/storage-paths.test.ts	Adds test for managed database storage path computation.
extensions/vscode/test/suite/bridge.integration.test.ts	Updates assertions for managed-dir mode + adds copyDatabases integration checks.
extensions/vscode/test/suite/workspace-scenario.integration.test.ts	Updates workspace scenario assertions + adds copyDatabases scenario test.
extensions/vscode/test/suite/copydb-e2e.integration.test.ts	New E2E coverage for lock removal + running tools against a copied DB.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 14 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

extensions/vscode/src/bridge/database-copier.ts:91

• If cp() fails partway through, the destination directory may be left in a partially-copied state, and future runs might treat it as a valid database (because codeql-database.yml could already exist). After a failed copy, it’d be safer to delete dest (best-effort) so consumers never see a corrupt/incomplete database copy.

  private async copyDatabase(src: string, dest: string): Promise<void> {
    this.logger.info(`Copying database ${src} → ${dest}`);
    try {
      // Remove stale destination if present
      if (existsSync(dest)) {
        await rm(dest, { recursive: true, force: true });
      }

      await cp(src, dest, { recursive: true });
      await removeLockFiles(dest);
      this.logger.info(`Database copied successfully: ${dest}`);
    } catch (err) {
      this.logger.error(
        `Failed to copy database ${src}: ${err instanceof Error ? err.message : String(err)}`,
      );
    }
  }

[Copilot]

This test checks for a managed directory via dirs.includes('/databases'), which is POSIX-specific and will fail on Windows (backslashes). Prefer splitting on path.delimiter and checking path.basename(p) === 'databases' (similar to the later test), or using path.sep-aware logic.

Suggested change

	const dirs = env.CODEQL_DATABASES_BASE_DIRS;
	if (folders && folders.length > 0) {
	// With copyDatabases enabled (default), CODEQL_DATABASES_BASE_DIRS points
	// to a managed databases/ directory under our globalStorage instead of the
	// original workspaceStorage paths. Accept either layout.
	const hasManagedDir = dirs.includes('/databases');
	const hasWorkspaceStorage = dirs.includes('workspaceStorage');
	const dirs = env.CODEQL_DATABASES_BASE_DIRS ?? '';
	if (folders && folders.length > 0) {
	// With copyDatabases enabled (default), CODEQL_DATABASES_BASE_DIRS points
	// to a managed databases/ directory under our globalStorage instead of the
	// original workspaceStorage paths. Accept either layout.
	const dirList = String(dirs)
	.split(path.delimiter)
	.filter((p) => p);
	const hasManagedDir = dirList.some((p) => path.basename(p) === 'databases');
	const hasWorkspaceStorage = dirList.some((p) => path.basename(p) === 'workspaceStorage');

[Copilot]

The top-level comment now says these env vars use the platform-native list delimiter, but later JSDoc in this file still says “colon-separated” for CODEQL_*_DIRS. Please update those remaining references so the documentation is consistent with the new parsing behavior.

[Copilot]

DatabaseCopier.syncAll() can throw if mkdir(this.destinationBase) fails (e.g. permissions / disk issues). Right now EnvironmentBuilder.build() doesn’t catch that, so enabling copyDatabases could prevent the extension from starting at all. Consider catching errors in syncAll (log + return []) and/or catching around await copier.syncAll(...) in EnvironmentBuilder and falling back to source dirs when the copy step fails.

This issue also appears on line 75 of the same file.