Add unified query DSL with MongoDB-style operators by Copilot · Pull Request #52 · objectstack-ai/framework

Copilot · 2026-01-21T04:33:37Z

Implements a JSON-based, database-agnostic query filter syntax following Prisma/Strapi/TypeORM conventions for frontend-backend data filtering.

Changes

New schema: filter.zod.ts with recursive filter structure
- Comparison: $eq, $ne, $gt, $gte, $lt, $lte
- Set/Range: $in, $nin, $between
- String: $contains, $startsWith, $endsWith
- Logical: $and, $or, $not
- Special: $null, $exist
Type-safe generics: Filter<T> with conditional type constraints
Dual syntax: Implicit equality (field: value) and explicit operators (field: { $op: value })
Nested relations: JSON structure maps to JOIN operations
Normalization layer: NormalizedFilterSchema for adapter implementation

Example

import { Filter, QueryFilter } from '@objectstack/spec';

interface User {
  age: number;
  email: string;
  profile: { verified: boolean };
}

const filter: Filter<User> = {
  age: { $gte: 18 },                    // Explicit operator
  email: { $contains: '@company.com' }, // String operator
  profile: { verified: true },          // Nested relation
  $or: [                                // Logical combination
    { status: 'active' },
    { role: { $in: ['admin', 'editor'] } }
  ]
};

The schema generates JSON schemas and MDX documentation automatically via existing build pipeline.

Original prompt

This section details on the original issue you should resolve

<issue_title>统一查询语言设计规范 (Unified Query DSL Specification)</issue_title>
<issue_description>这是一份基于行业最佳实践（参考 Prisma, Strapi, TypeORM, LoopBack）起草的通用查询语言（Generic Query DSL）设计规范文档。
你可以直接将这份文档作为你团队内部的技术规格说明书（Spec）使用。
统一查询语言设计规范 (Unified Query DSL Specification)
版本: 1.0.0
状态: 草案 (Draft)
目标: 定义一套基于 JSON 的、跨数据库兼容的查询语法标准，用于前端与后端 API 的数据筛选交互。

核心设计原则

声明式 (Declarative): 前端只描述“要什么数据”，不描述“如何查询数据库”。

后端无关性 (Database Agnostic): 语法不包含特定数据库（如 SQL 或 Mongo）的专用指令。所有指令在后端通过适配器层（Adapter Layer）转换。

类型安全 (Type Safety): 结构必须能被 TypeScript 静态推断。

默认约定优于配置 (Convention over Configuration): 提供隐式写法以简化常见查询。

语法结构定义
查询对象（Filter Object）是一个递归的树状结构。
2.1 基础结构
一个合法的查询对象由以下三种元素组成：

隐式相等 (Implicit Equality): key: value

显式操作符 (Explicit Operators): key: { $op: value }

逻辑组合 (Logical Groups): $and, $or, $not
2.2 完整示例
{
"where": {
"status": "active", // 隐式相等 (AND)
"age": { "$gte": 18 }, // 显式比较 (AND)
"$or": [ // 逻辑分支
{ "role": "admin" },
{ "email": { "$contains": "@company.com" } }
],
"profile": { // 关联查询 (Relation)
"verified": true
}
}
}

操作符标准 (Operator Standards)
为了保证跨库兼容，必须严格限制支持的操作符清单。不要直接透传数据库指令。
3.1 比较操作符 (Comparison)
| 操作符 | 描述 | SQL 映射示例 | MongoDB 映射示例 | 数据类型限制 |
|---|---|---|---|---|
| $eq | 等于 (默认) | = | $eq | Any |
| $ne | 不等于 | <> 或 != | $ne | Any |
| $gt | 大于 | > | $gt | Number, Date |
| $gte | 大于等于 | >= | $gte | Number, Date |
| $lt | 小于 | < | $lt | Number, Date |
| $lte | 小于等于 | <= | $lte | Number, Date |
3.2 集合与区间 (Set & Range)
| 操作符 | 描述 | SQL 映射示例 | MongoDB 映射示例 |
|---|---|---|---|
| $in | 在列表中 | IN (?, ?, ?) | $in: [...] |
| $nin | 不在列表中 | NOT IN (...) | $nin: [...] |
| $between | 区间 (闭合) | BETWEEN ? AND ? | $gte AND $lte |
3.3 字符串专用 (String Specific)
注意：此处需在后端处理大小写敏感（Case Sensitivity）配置。
| 操作符 | 描述 | SQL 映射示例 | MongoDB 映射示例 |
|---|---|---|---|
| $contains | 包含 | LIKE %?% | $regex |
| $startsWith | 前缀匹配 | LIKE ?% | $regex |
| $endsWith | 后缀匹配 | LIKE %? | $regex |
3.4 逻辑操作符 (Logical)
| 操作符 | 描述 | SQL 映射 | MongoDB 映射 |
|---|---|---|---|
| $and | 逻辑与 | (A AND B) | $and |
| $or | 逻辑或 | (A OR B) | $or |
| $not | 逻辑非 | NOT (A) | $not |
3.5 特殊检查 (Special)
| 操作符 | 描述 | SQL 映射 | MongoDB 映射 |
|---|---|---|---|
| $null | 是否为空 | IS NULL (true) / IS NOT NULL (false) | field: null |
| $exist | 字段是否存在 | (通常用于 NoSQL) | $exists |

解析与执行流程 (Architecture)
这是实现跨数据库的核心架构逻辑。
阶段一：标准化 (Normalization Pass)
在进入适配器之前，必须将所有“语法糖”转换为“标准 AST 结构”。这能极大简化后续适配器的编写难度。
规则：

将所有 key: value 转换为 key: { $eq: value }。

将同级的所有 Key 合并入 $and 数组。
输入：
{ "age": 18, "role": "admin" }

标准化后输出 (Internal IR):
{
"$and": [
{ "age": { "$eq": 18 } },
{ "role": { "$eq": "admin" } }
]
}

阶段二：适配器转换 (Adapter Translation)
使用 Visitor Pattern 遍历标准化后的对象。
SQL Adapter 伪逻辑：

遍历对象。

遇到 $and/$or: 递归生成子 SQL，用 AND/OR 连接，并包裹括号。

遇到字段 { age: { $gt: 18 } }:

提取字段名 age (需做安全校验，防 SQL 注入)。

提取操作符 $gt -> 映射为 >。

提取值 18 -> 存入 Parameters 数组，SQL 中替换为占位符 ? 或 $1。

关联查询规范 (Relation / Join)
这是 ObjectQL 的核心优势，利用 JSON 的嵌套特性表达 SQL JOIN。
规则：
如果一个 Key 对应的值是对象（且不是操作符对象），则视为关联查询。
输入：
{
"department": {
"name": { "$eq": "IT" }
}
}

SQL Adapter 行为：

检测到 department 是关联字段。

自动执行 INNER JOIN department ON users.dept_id = department.id。

添加 WHERE 条件：department.name = 'IT'。

TypeScript 定义 (Interfaces)
直接提供给前端使用的类型定义。
// 基础标量类型
type Scalar = string | number | boolean | Date | null;

// 操作符定义
type FilterOperators = {
$eq?: T;
$ne?: T;
$in?: T[];
$nin?: T[];
$gt?: T; // 仅限 number/date
$lt?: T;
$contains?: string; // 仅限 string
$startsWith?: string;
$not?: FilterOperators;
};

// 递归过滤器定义
export type Filter = {
[K in keyof T]?:
| T[K] // 隐式相等
| FilterOperators<T[K]> // 显式操作符
| Filter<T[K]>; // 关联表嵌套 (Join)
} & {
$and?: Filter[];
$or?: Filter[];
};

安全性与边界限制 (Security Guardrails)
在实现解析器时，必须强制执行以下规则以防止恶意攻击：

最大深度限制 (Max Depth): 防止深度嵌套导致的堆栈溢出（建议限制为 5-6 层）。

字段白名单 (Field Whitelist): 解析器必须检查 Key 是否存在于定义的 Schema 中，禁止查询数据库中存在但未开放 API 的字段（如 password_hash, salt）。

数组长度限制: $in 操作符的数组长度不得超过 1000（防止 SQL 性能问题）。

禁止全表扫描: 强制要求查询必须包含索引字段（可选的高级配置）。
下一步行动建议

定义 Schema: 确定你的实体模型（Model）。

编写 Normalizer: 实现阶段一的“标准化函数”，这一步是通用的。

选择 ORM/QueryBuilder: 你的后端是用 TypeORM, Prisma, Knex 还是原生 SQL？这决定了 Adapter 的写法。</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes 统一查询语言设计规范 (Unified Query DSL Specification) #50

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

vercel · 2026-01-21T04:34:15Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
spec	Error		Jan 21, 2026 4:42am

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

github-actions · 2026-01-21T04:42:44Z

This PR is very large. Consider breaking it into smaller PRs for easier review.

Copilot

Pull request overview

This PR implements a JSON-based, MongoDB-style query filter DSL for database-agnostic data filtering. The design follows industry best practices from Prisma, Strapi, TypeORM, and LoopBack, providing both implicit equality syntax and explicit operator syntax with full TypeScript type safety.

Changes:

New filter.zod.ts schema with comprehensive operator support (comparison, set, range, string, logical, and special operators)
Type-safe generic Filter<T> type with conditional type constraints for proper operator-field matching
Dual syntax support: implicit equality (field: value) and explicit operators (field: { $op: value })
Normalization layer (NormalizedFilterSchema) to simplify adapter implementation
Comprehensive test suite with 622 lines covering all operators, edge cases, and real-world scenarios
Auto-generated JSON schemas and MDX documentation

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
packages/spec/src/index.ts	Exports the new filter module
packages/spec/src/data/filter.zod.ts	Core implementation with Zod schemas, TypeScript types, and operator definitions
packages/spec/src/data/filter.test.ts	Comprehensive test suite covering all operators and use cases
packages/spec/json-schema/*.json	Auto-generated JSON schemas for all filter-related types (9 files)
content/docs/references/data/*.mdx	Auto-generated MDX documentation files (9 files)

Copilot · 2026-01-21T05:04:45Z

packages/spec/src/data/filter.zod.ts

+
+  // Special
+  $null: z.boolean().optional(),
+  $exist: z.boolean().optional(),


The special operator is named $exist, but MongoDB's standard operator is $exists (with an 's'). For consistency with MongoDB conventions and industry standards, this should be $exists.

Copilot · 2026-01-21T05:04:45Z

packages/spec/src/data/filter.zod.ts

+        $startsWith?: T[K] extends string ? string : never;
+        $endsWith?: T[K] extends string ? string : never;
+        $null?: boolean;
+        $exist?: boolean;


The property should be $exists (with an 's') to match MongoDB's standard operator name.

Suggested change

$exist?: boolean;

$exists?: boolean;

Copilot · 2026-01-21T05:04:45Z

content/docs/references/data/SpecialOperator.mdx

+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **$null** | `boolean` | optional |  |
+| **$exist** | `boolean` | optional |  |


Copilot · 2026-01-21T05:04:46Z

content/docs/references/data/FieldOperators.mdx

+| **$startsWith** | `string` | optional |  |
+| **$endsWith** | `string` | optional |  |
+| **$null** | `boolean` | optional |  |
+| **$exist** | `boolean` | optional |  |


Copilot · 2026-01-21T05:04:46Z

packages/spec/src/data/filter.zod.ts

+ * ```
+ */
+export const QueryFilterSchema = z.object({
+  where: FilterConditionSchema.optional(),


Add .describe() call to the where property to populate documentation. For example: where: FilterConditionSchema.optional().describe('Filter conditions for the query').

Suggested change

where: FilterConditionSchema.optional(),

where: FilterConditionSchema.optional().describe('Filter conditions for the query'),

Copilot · 2026-01-21T05:04:48Z

packages/spec/src/data/filter.zod.ts

+export const SpecialOperatorSchema = z.object({
+  /** Is null check - SQL: IS NULL (true) / IS NOT NULL (false) | MongoDB: field: null */
+  $null: z.boolean().optional(),
+
+  /** Field exists check (primarily for NoSQL) - MongoDB: $exists */
+  $exist: z.boolean().optional(),
+});


Add .describe() calls to each property to populate the documentation descriptions. For example: $null: z.boolean().optional().describe('Is null check').

Copilot · 2026-01-21T05:04:48Z

packages/spec/src/data/filter.zod.ts

+  $between: z.tuple([
+    z.union([z.number(), z.date()]),
+    z.union([z.number(), z.date()])
+  ]).optional(),


Add .describe() call to the property to populate the documentation description. For example: $between: z.tuple([...]).optional().describe('Between (inclusive)').

Suggested change

]).optional(),

]).optional().describe('Between (inclusive)'),

Copilot · 2026-01-21T05:04:48Z

packages/spec/src/data/filter.zod.ts

+export const StringOperatorSchema = z.object({
+  /** Contains substring - SQL: LIKE %?% | MongoDB: $regex */
+  $contains: z.string().optional(),
+
+  /** Starts with prefix - SQL: LIKE ?% | MongoDB: $regex */
+  $startsWith: z.string().optional(),
+
+  /** Ends with suffix - SQL: LIKE %? | MongoDB: $regex */
+  $endsWith: z.string().optional(),
+});


Add .describe() calls to each property to populate the documentation descriptions. For example: $contains: z.string().optional().describe('Contains substring').

Copilot · 2026-01-21T05:04:49Z

packages/spec/src/data/filter.zod.ts

+      $and: z.array(FilterConditionSchema).optional(),
+      $or: z.array(FilterConditionSchema).optional(),
+      $not: FilterConditionSchema.optional(),


Add .describe() calls to the logical operator properties to populate documentation. For example: $and: z.array(FilterConditionSchema).optional().describe('Logical AND - combines all conditions that must be true').

Suggested change

$and: z.array(FilterConditionSchema).optional(),

$or: z.array(FilterConditionSchema).optional(),

$not: FilterConditionSchema.optional(),

$and: z

.array(FilterConditionSchema)

.optional()

.describe('Logical AND - combines all conditions that must be true'),

$or: z

.array(FilterConditionSchema)

.optional()

.describe('Logical OR - at least one condition must be true'),

$not: FilterConditionSchema

.optional()

.describe('Logical NOT - negates the condition'),

Copilot · 2026-01-21T05:04:49Z

packages/spec/src/data/filter.zod.ts

+  $null: z.boolean().optional(),
+
+  /** Field exists check (primarily for NoSQL) - MongoDB: $exists */
+  $exist: z.boolean().optional(),


The special operator is named $exist, but MongoDB's standard operator is $exists (with an 's'). For consistency with MongoDB conventions mentioned in the file header and to align with industry standards, this should be $exists.

Suggested change

$exist: z.boolean().optional(),

$exists: z.boolean().optional(),

Initial plan

6e77343

Copilot AI assigned Copilot and hotlong Jan 21, 2026

Copilot started work on behalf of hotlong January 21, 2026 04:34 View session

vercel bot had a problem deploying to Preview January 21, 2026 04:34 Failure

Implement unified query DSL specification with MongoDB-style operators

d7cc23d

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

vercel bot had a problem deploying to Preview January 21, 2026 04:40 Failure

Fix code review issues: remove incorrect FilterOperators type

810104f

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>

vercel bot had a problem deploying to Preview January 21, 2026 04:42 Failure

github-actions bot added documentation Improvements or additions to documentation protocol:data tests size/xl labels Jan 21, 2026

Copilot AI changed the title ~~[WIP] Add unified query DSL specification document~~ Add unified query DSL with MongoDB-style operators Jan 21, 2026

Copilot AI requested a review from hotlong January 21, 2026 04:43

Copilot finished work on behalf of hotlong January 21, 2026 04:43

hotlong approved these changes Jan 21, 2026

View reviewed changes

hotlong marked this pull request as ready for review January 21, 2026 04:58

Copilot AI review requested due to automatic review settings January 21, 2026 04:58

hotlong merged commit 1c22142 into main Jan 21, 2026
13 of 14 checks passed

Copilot started reviewing on behalf of hotlong January 21, 2026 04:59 View session

Copilot AI reviewed Jan 21, 2026

View reviewed changes

	\| $exist \| `boolean` \| optional \| \|
	\| $exists \| `boolean` \| optional \| \|

	where: FilterConditionSchema.optional(),
	where: FilterConditionSchema.optional().describe('Filter conditions for the query'),

	]).optional(),
	]).optional().describe('Between (inclusive)'),

	$exist: z.boolean().optional(),
	$exists: z.boolean().optional(),

Conversation

Copilot AI commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example

Comments on the Issue (you are @copilot in this section)

Uh oh!

vercel bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 21, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jan 21, 2026 •

edited

Loading

vercel bot commented Jan 21, 2026 •

edited

Loading