Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
51 changes: 51 additions & 0 deletions .agents/skills/deepchat-data-import/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
name: deepchat-data-import
description: Help developers build third-party tools that import, inspect, migrate, or analyze DeepChat data. Use when Codex needs to work with DeepChat provider configuration, model configuration, MCP/app settings, sessions, messages, legacy chat data, `agent.db`, `chat.db`, SQLCipher encrypted SQLite, Electron safeStorage wrapped passwords, Tauri importers, or native macOS/Windows/Linux data access.
---

# DeepChat Data Import

## Overview

Use this skill to design or implement importers for DeepChat local data. Treat DeepChat's SQLite
schema as an internal but documentable contract: inspect the current schema when precision matters,
prefer read-only snapshots, and avoid writing to a live profile.

## Workflow

1. Identify the source: live DeepChat profile, copied profile, sync backup, exported `agent.db`, or
legacy `chat.db`.
2. Read [references/data-locations.md](references/data-locations.md) to locate `agent.db`,
sidecar files, encryption metadata, and backup paths.
3. Read [references/sqlite-access.md](references/sqlite-access.md) before opening SQLite. Decide
whether the database is unencrypted, can be unlocked through Electron safeStorage, or must ask
the user for the SQLite password.
4. Read [references/schema-reference.md](references/schema-reference.md) for provider config,
settings, session, message, and legacy table relationships.
5. Read [references/import-recipes.md](references/import-recipes.md) when writing extractor code,
mapping DeepChat data to another app, or creating a compatibility import.

## Safety Rules

- Get explicit user consent before reading local DeepChat data. Provider keys, OAuth tokens, MCP
env vars, prompt text, message traces, and chat content may be sensitive.
- Never open the active `agent.db` read-write from a third-party tool. Copy `agent.db`,
`agent.db-wal`, and `agent.db-shm`, or use SQLite backup APIs through DeepChat itself.
- If DeepChat is running, either ask the user to quit it or make a WAL-aware snapshot before import.
- Prefer parameterized key APIs for SQLCipher passwords. Do not interpolate passwords into SQL.
- Redact secrets by default in logs, telemetry, previews, and generated sample output.
- When the schema has changed, inspect `schema_versions`, `sqlite_master`, and the table classes
under `src/main/presenter/sqlitePresenter/tables/` before assuming column availability.

## Source Files

Use these repository files as the current source of truth when updating the skill or answering
version-sensitive questions:

- `src/main/presenter/sqlitePresenter/index.ts`
- `src/main/presenter/sqlitePresenter/connectionConfig.ts`
- `src/main/presenter/databaseSecurityPresenter/index.ts`
- `src/main/presenter/sqlitePresenter/tables/*.ts`
- `src/main/presenter/agentRuntimePresenter/messageStore.ts`
- `src/main/presenter/agentRuntimePresenter/sessionStore.ts`
- `src/main/presenter/configPresenter/**`
4 changes: 4 additions & 0 deletions .agents/skills/deepchat-data-import/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "DeepChat Data Import"
short_description: "Import DeepChat config and chat data"
default_prompt: "Use $deepchat-data-import to help a third-party tool import DeepChat provider config, sessions, messages, and encrypted or unencrypted agent.db data."
87 changes: 87 additions & 0 deletions .agents/skills/deepchat-data-import/references/data-locations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Data Locations

Use this reference when an importer must locate DeepChat data on disk or inside a backup.

## Primary Files

DeepChat stores the current main database at:

```text
<electron userData>/app_db/agent.db
<electron userData>/app_db/agent.db-wal
<electron userData>/app_db/agent.db-shm
```

The legacy database, when present, is:

```text
<electron userData>/app_db/chat.db
```

Database encryption metadata is outside SQLite:

```text
<electron userData>/database-security.json
```

The file is an ElectronStore JSON file. The relevant shape is:

```json
{
"metadata": {
"version": 1,
"enabled": true,
"cipher": "sqlcipher",
"passwordStorage": "safeStorage",
"wrappedPassword": "base64-electron-safeStorage-blob",
"safeStorageBackend": "basic_text",
"lastMigrationAt": 1770000000000,
"lastMigrationDirection": "enable"
}
}
```

`passwordStorage` can be `safeStorage`, `manual`, or `none`. If `enabled` is false, open
`agent.db` as normal SQLite.

## Default UserData Paths

Electron derives the profile path from the packaged product name `DeepChat` unless the runtime
overrides `app.getPath('userData')`.

```text
macOS: ~/Library/Application Support/DeepChat
Windows: %APPDATA%\DeepChat
Linux: ~/.config/DeepChat
```

Treat these as defaults. Portable builds, development builds, tests, or user overrides can point
elsewhere.

## Sync Backup Layout

DeepChat sync backups use `database/agent.db` as the primary database payload in current backup
versions. Some compatibility backups may also contain `database/chat.db` or old JSON settings. When
both `agent.db` and `chat.db` exist, prefer `agent.db`.

## Snapshot Rules

- If DeepChat is running, copy `agent.db`, `agent.db-wal`, and `agent.db-shm` together.
- If DeepChat is closed, `agent.db` alone is usually enough, but copying sidecars is still harmless.
- For high-integrity import, open a read-only source connection and run a SQLite backup into a temp
file, then import from the temp file.
- Do not delete `*.migration-tmp`, `*.migration-rollback`, `agent.db-wal`, or `agent.db-shm` from a
user profile. DeepChat owns those lifecycle decisions.

## Related Files Still Outside Agent.db

Most sensitive configuration has moved into SQLite. Some lightweight or compatibility JSON files may
still exist in userData, including:

- `app-settings.json`
- `custom_prompts.json`
- `system_prompts.json`
- `mcp-settings.json`

Prefer SQLite tables for current provider, MCP, app setting, prompt, and knowledge config imports.
Use JSON files only as legacy fallback.
153 changes: 153 additions & 0 deletions .agents/skills/deepchat-data-import/references/import-recipes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# Import Recipes

Use this reference when turning DeepChat data into an importer or migration tool.

## Minimal Read-Only Export

1. Locate and snapshot `agent.db`.
2. Open it with the flow in `sqlite-access.md`.
3. Read provider config from `providers`, `provider_models`, `model_status`, and `model_configs`.
4. Read sessions with:

```sql
SELECT ns.*, ds.*
FROM new_sessions ns
LEFT JOIN deepchat_sessions ds ON ds.id = ns.id
WHERE ns.session_kind = 'regular'
ORDER BY ns.updated_at DESC, ns.id DESC;
```

5. Read messages per session:

```sql
SELECT *
FROM deepchat_messages
WHERE session_id = ?
ORDER BY order_seq ASC, id ASC;
```

6. Hydrate user messages from `deepchat_user_messages`, `deepchat_user_message_files`, and
`deepchat_user_message_links`.
7. Hydrate assistant messages from `deepchat_assistant_blocks`; fall back to parsing
`deepchat_messages.content`.
8. Export to the target app's format, redacting secrets unless the user explicitly chooses to
include them.

## Provider Config Import

When importing providers into another tool, preserve:

- provider id, name, API type, base URL, enabled flag, custom flag.
- API key and OAuth token only with explicit user consent.
- capability provider id for model capability lookup.
- model rows from `provider_models`, split by `source`.
- enabled/disabled state from `model_status`.
- model config from `model_configs.config_json`.

Do not rely only on `provider_json`; DeepChat deliberately stores common scalar fields in columns
for queryability and migration.

## Session And Message Import

Recommended target shape:

```json
{
"session": {
"id": "session-id",
"title": "Session title",
"agentId": "deepchat",
"projectDir": "/path/to/project",
"providerId": "openai",
"modelId": "gpt-4.1",
"createdAt": 1770000000000,
"updatedAt": 1770000000000
},
"messages": [
{
"id": "message-id",
"orderSeq": 1,
"role": "user",
"status": "sent",
"content": {
"text": "hello",
"files": [],
"links": [],
"search": false,
"think": false
},
"metadata": {},
"createdAt": 1770000000000,
"updatedAt": 1770000000000
}
]
}
```

For assistant messages, keep the assistant block array when possible instead of flattening to text.
Tool calls, tool responses, reasoning blocks, image data, action prompts, and error blocks may all
be represented as assistant blocks.

## Handling Partial Or Old Rows

- If structured user rows are missing, parse `deepchat_messages.content`.
- If structured assistant blocks are missing, parse `deepchat_messages.content`.
- If `new_sessions` is missing but `conversations` exists, import through the legacy path.
- If `agent.db` is missing and `chat.db` exists, open `chat.db` as legacy data.
- If a column is missing, check `schema_versions` and use the nearest fallback rather than failing
the whole import.

## Writing Back Into DeepChat

Avoid third-party direct writes to a user's live DeepChat database.

If a tool must generate data for DeepChat:

- Prefer creating an export file or backup package that DeepChat can import through its own code.
- If implementing inside DeepChat, use Presenter/table helpers instead of raw SQL.
- If writing a copied database for controlled migration tests, use one transaction per session and
keep table groups consistent:
- `new_sessions`
- `deepchat_sessions`
- `deepchat_messages`
- structured user or assistant tables
- optional search, trace, usage, pending input, and tape rows
- Keep `deepchat_messages.content` compatible even when structured tables are populated, because it
remains the fallback path.
- Do not update `database-security.json` manually after rekeying; use DeepChat's migration flow.

## Secret Handling Checklist

Redact or require explicit opt-in for:

- `providers.api_key`
- OAuth tokens in `providers.provider_json`
- MCP server `env` and custom headers
- `app_settings` rows marked `sensitive = 1`
- `deepchat_message_traces.headers_json` and `body_json`
- file paths in user message files
- chat content, system prompts, summaries, and project paths

## Useful Consistency Checks

Run these after import from a copied database:

```sql
PRAGMA quick_check;

SELECT COUNT(*) FROM new_sessions;
SELECT COUNT(*) FROM deepchat_sessions;
SELECT COUNT(*) FROM deepchat_messages;

SELECT m.session_id
FROM deepchat_messages m
LEFT JOIN new_sessions s ON s.id = m.session_id
WHERE s.id IS NULL
LIMIT 20;
```

For encrypted databases, validate the password before any import work:

```sql
SELECT name FROM sqlite_master LIMIT 1;
```
Loading
Loading