Make `AGENT_DATA_DIR` default cross-platform safe (avoid `/data/agents` on non-container local runs)

[Congregalis]

Summary

AGENT_DATA_DIR currently defaults to /data/agents.
This works in Docker (with a mounted volume), but it breaks local non-container setups (especially macOS), where /data is often unavailable or read-only.

This causes runtime failures in endpoints that create directories under AGENT_DATA_DIR (example: enterprise knowledge base file listing/upload paths).

Problem

On local macOS development with no explicit AGENT_DATA_DIR override:

• The app falls back to /data/agents.
• Code attempts mkdir(parents=True, exist_ok=True) under /data/....
• The request fails with OSError: [Errno 30] Read-only file system: '/data'.
• API returns 500 Internal Server Error.

Observed failing path:

• /api/enterprise/knowledge-base/files
• Internally creates: Path(settings.AGENT_DATA_DIR) / "enterprise_info"

Why this matters

• Fresh local setup can fail unexpectedly even before users upload files.
• The default is environment-specific (container-oriented) but used globally.
• This creates avoidable friction for contributors and local operators.

Proposed behavior

Resolve AGENT_DATA_DIR with environment-aware defaults:

1. If AGENT_DATA_DIR is explicitly set, always use it.
2. If running in a container, default to /data/agents (current behavior).
3. Otherwise (local host), default to a user-writable path:
   • Option A (simple and explicit): ~/.clawith/data/agents
   • Option B (OS-conventional): use platformdirs user data directory + /agents

Either option is fine; the key is avoiding a root-level path by default on host environments.

Additional recommendations

• Expand ~ safely (do not treat it as a literal string path).
• Ensure the directory is created on startup or first use with clear error reporting.
• Log the resolved AGENT_DATA_DIR at startup for easier diagnostics.
• Update .env.example and docs to clarify:
  • local default path
  • Docker path (/data/agents) with volume mapping
  • production recommendation (e.g. /var/lib/clawith/agents when managed by ops)

Acceptance criteria

• Local macOS/Linux run with no AGENT_DATA_DIR no longer writes to /data/....
• GET /api/enterprise/knowledge-base/files?path= returns 200 (or expected empty list) on fresh local setup.
• Docker deployment behavior remains unchanged.
• Documentation reflects the new default resolution strategy.

[Congregalis]

Additional reproducibility detail for local (non-Docker) runs, including environment:

Environment

• Host OS: macOS 14.7.4 (Build 23H420), Darwin 23.6.0, arm64
• Python: 3.12.13
• FastAPI: 0.135.1
• Starlette: 0.52.1
• Uvicorn: 0.41.0
• Run mode: local (bash restart.sh), not Docker

Reproduction

1. Keep default .env state where AGENT_DATA_DIR is not set.
2. Start services via bash restart.sh (local mode). This launches uvicorn directly from backend/ and does not explicitly set AGENT_DATA_DIR.
3. Call GET /api/enterprise/knowledge-base/files?path=.

Expected

• 200 with an empty list on a fresh setup.

Actual

• 500 Internal Server Error
• OSError: [Errno 30] Read-only file system: '/data'

This indicates the issue is easy to hit in the documented local quick-start path, not only in custom setups.