feat(cli): polite deep merge for settings.json and support JSONC

[RbBtSn0w]

Supersedes #1863

Description

This PR addresses issue #1799, where the Specify CLI would overwrite the user's existing .vscode/settings.json file during initialization or upgrades, leading to data loss of personal preferences.

Key Changes & Architectural Decisions

• JSONC Parsing Refactor (The json5 library):
  Initially, a custom state-machine tokenizer was implemented to handle VS Code's JSONC format (comments and trailing commas). However, given the complexity of edge cases (e.g., trailing commas immediately followed by block comments before a closing brace) and to future-proof the parser against other JSON5/JSONC quirks, the custom parser was entirely removed.
  Instead, the robust json5 library has been integrated. It natively, effortlessly, and safely parses files with single/multi-line comments, trailing commas, and BOMs without the maintenance overhead of a custom tokenizer.
• Polite Deep Merge: Implemented deep_merge_polite which strictly ensures that:
  • New recommended settings from templates are added.
  • Existing user settings are preserved and are never overwritten by template defaults.
  • Nested dictionaries are merged recursively to maintain complex structures.
• Non-Destructive Fallbacks (Zero Data Loss Guarantee):
  Added extreme defensive programming to the merge pipeline. If the user's existing settings.json is completely unparseable (e.g., severe syntax error) or is not a JSON object (dictionary), the merge_json_files function immediately returns None. The CLI catches this and gracefully skips the file writing step entirely, guaranteeing that the user's original (even if invalid) configuration is never wiped by an empty dictionary or overridden by the template.
• Atomic write + metadata preservation:
  Rewrites now use atomic temp-file replacement and preserve existing mode bits (and owner/group best-effort), reducing corruption risk while avoiding unexpected permission changes.
• Unit Tests: Added comprehensive tests in tests/test_merge.py that specifically target the deep-merge preservation rules, type mismatch handling, secure fallback behavior, and file-mode preservation on rewrite.

Verification

• Verified that .vscode/settings.json containing comments and trailing commas can be parsed cleanly via json5.
• Verified that user-defined keys (e.g., editor.fontSize) remain strictly intact.
• Verified that unparseable user files do not result in a destructive overwrite.
• Verified all tests pass with uv run pytest -q.

Fixes #1799

[Copilot]

Pull request overview

This PR updates the Specify CLI’s .vscode/settings.json handling to avoid overwriting user configuration during init/upgrade by parsing VS Code JSONC and performing a “polite” deep merge that preserves existing user values.

Changes:

• Integrates json5 to parse JSONC (comments, trailing commas, BOM) for VS Code settings files.
• Implements a “polite” deep merge that only adds missing keys (recursively for nested dicts) and returns None to skip rewrites when unsafe or unnecessary.
• Adds atomic rewrite logic with best-effort metadata preservation and introduces unit tests covering merge behavior and mode preservation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
src/specify_cli/__init__.py	Adds JSONC parsing via json5, implements polite deep merge with non-destructive fallbacks, and switches settings rewrites to atomic replace with permission preservation.
tests/test_merge.py	Adds targeted tests for merge semantics (preservation, deep nesting, JSONC/BOM parsing, safe no-op behavior) plus atomic-write mode preservation.
pyproject.toml	Adds json5 as a runtime dependency.
CHANGELOG.md	Documents the new VS Code settings merge behavior under Unreleased additions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Prevents data loss by changing how the CLI handles .vscode/settings.json during init/upgrade: it now parses VS Code JSONC reliably and only adds missing recommended settings without overwriting user preferences.

Changes:

• Switch .vscode/settings.json parsing to json5 to support JSONC (comments/trailing commas/BOM).
• Implement a “polite” deep merge that preserves existing user values and skips rewrites when no changes are needed (or when parsing is unsafe).
• Add atomic rewrite with best-effort metadata preservation and introduce targeted unit tests.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/specify_cli/init.py	Adds json5 parsing, polite deep merge + safe None return contract, and atomic write preserving mode/ownership best-effort.
tests/test_merge.py	Adds unit coverage for merge preservation rules, JSONC/BOM parsing, unsafe fallbacks, and mode preservation on rewrite.
pyproject.toml	Adds json5 runtime dependency.
CHANGELOG.md	Documents the new CLI behavior in Unreleased notes.

Comments suppressed due to low confidence (1)

src/specify_cli/init.py:788

• In deep_merge_polite, the inline comment says "Key already exists and is not a dict", but this else branch also runs when the existing value is a dict and the incoming value is not (i.e., types mismatch). Consider rewording the comment to match the actual condition (e.g., "key exists and values are not both dicts").

            elif isinstance(result[key], dict) and isinstance(value, dict):
                # Recursively merge nested dictionaries
                result[key] = deep_merge_polite(result[key], value)
            else:
                # Key already exists and is not a dict, PRESERVE existing value
                # This ensures user settings aren't overwritten by template defaults
                pass

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR improves the Specify CLI’s handling of .vscode/settings.json by switching to JSONC/JSON5 parsing via json5 and introducing a “polite” deep-merge strategy that preserves existing user settings and avoids destructive rewrites.

Changes:

• Add JSONC-capable parsing for VS Code settings using the json5 library.
• Implement a polite deep merge that never overwrites existing user values and skips rewriting when no changes are needed or parsing is unsafe.
• Add unit tests covering merge behavior, JSONC/BOM parsing, no-op behavior, and atomic write mode preservation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/specify_cli/__init__.py	Use json5 for parsing and introduce polite deep-merge behavior + atomic rewrite with metadata preservation.
tests/test_merge.py	Add coverage for polite merge rules, JSONC parsing, safety fallbacks, and mode preservation.
pyproject.toml	Add json5 as a runtime dependency.
CHANGELOG.md	Document the new VS Code settings merge behavior under Unreleased.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.