Add Object Storage and File Attachment Protocol schemas#383
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Created object-storage.zod.ts with comprehensive schemas - Added file attachment configuration to field.zod.ts - Added comprehensive tests for all new schemas - Generated documentation automatically Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
- Update terminology: ipWhitelist/ipBlacklist → allowedIps/blockedIps - Add validation: minSize <= maxSize - Add validation: virusScanProvider requires virusScan enabled - Add validation: transition action requires targetStorageClass - Add comprehensive tests for all validation rules - Remove incorrectly placed integration/object-storage.mdx - All 2238 tests passing Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
|
This PR is very large. Consider breaking it into smaller PRs for easier review. |
| import { | ||
| StorageProviderSchema, | ||
| StorageAclSchema, | ||
| StorageClassSchema, | ||
| LifecycleActionSchema, | ||
| ObjectMetadataSchema, | ||
| PresignedUrlConfigSchema, | ||
| MultipartUploadConfigSchema, | ||
| AccessControlConfigSchema, | ||
| LifecyclePolicyRuleSchema, | ||
| LifecyclePolicyConfigSchema, | ||
| BucketConfigSchema, | ||
| StorageConnectionSchema, | ||
| ObjectStorageConfigSchema, | ||
| type StorageProvider, | ||
| type StorageAcl, | ||
| type StorageClass, | ||
| type ObjectMetadata, | ||
| type PresignedUrlConfig, | ||
| type MultipartUploadConfig, | ||
| type AccessControlConfig, | ||
| type LifecyclePolicyRule, | ||
| type LifecyclePolicyConfig, | ||
| type BucketConfig, | ||
| type StorageConnection, | ||
| type ObjectStorageConfig, | ||
| } from './object-storage.zod'; |
Check notice
Code scanning / CodeQL
Unused variable, import, function or class Note test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 2 months ago
To fix the problem, remove the unused imported symbol LifecycleActionSchema from the import list on line 2. This aligns with the recommendation to delete unused program elements and will clear the CodeQL warning.
Concretely, in packages/spec/src/system/object-storage.test.ts, edit the import block at the top of the file so that LifecycleActionSchema is no longer listed among the imported schemas. No other code changes are needed, and no new imports or definitions are required. The rest of the tests and imports should remain unchanged.
| @@ -3,7 +3,6 @@ | ||
| StorageProviderSchema, | ||
| StorageAclSchema, | ||
| StorageClassSchema, | ||
| LifecycleActionSchema, | ||
| ObjectMetadataSchema, | ||
| PresignedUrlConfigSchema, | ||
| MultipartUploadConfigSchema, |
There was a problem hiding this comment.
Pull request overview
This PR implements metadata schemas for object storage providers and file attachment field configuration to enable storage-backed file uploads in ObjectStack applications.
Changes:
- Added comprehensive Object Storage Protocol schemas in
packages/spec/src/system/object-storage.zod.tssupporting 9 cloud providers (S3, Azure Blob, GCS, MinIO, R2, Spaces, Wasabi, Backblaze, Local) - Extended Field schema with FileAttachmentConfig for file/image/avatar field types with virus scanning, storage integration, and image validation capabilities
- Added comprehensive test suites for both protocols with 750+ lines of test coverage
- Generated JSON schemas and documentation files for the new protocols
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/spec/src/system/object-storage.zod.ts | Core object storage protocol with provider configs, bucket management, lifecycle policies, and multipart uploads |
| packages/spec/src/system/object-storage.test.ts | Comprehensive test suite covering all storage schemas and validation rules |
| packages/spec/src/data/field.zod.ts | Added FileAttachmentConfig schema with file type restrictions, virus scanning, and storage provider binding |
| packages/spec/src/data/field.test.ts | Extended tests with 370+ lines covering file attachment configuration scenarios |
| packages/spec/src/system/index.ts | Added exports for object storage protocol |
| packages/spec/json-schema/system/*.json | Generated JSON schemas for object storage types |
| packages/spec/json-schema/data/*.json | Generated JSON schemas for file attachment config |
| content/docs/references/system/object-storage.mdx | Documentation for system-level object storage protocol |
| content/docs/references/integration/object-storage.mdx | ISSUE: Incorrectly references non-existent integration file |
| content/docs/references/integration/misc.mdx | Updated to remove MultipartUploadConfig section but imports remain |
| content/docs/references/data/field.mdx | Added FileAttachmentConfig documentation |
| # Object Storage | ||
|
|
||
| <Callout type="info"> | ||
| **Source:** `packages/spec/src/integration/object-storage.zod.ts` |
There was a problem hiding this comment.
The documentation file content/docs/references/integration/object-storage.mdx incorrectly references a non-existent source file packages/spec/src/integration/object-storage.zod.ts. The actual object storage protocol is defined in packages/spec/src/system/object-storage.zod.ts. This documentation file should either be removed or moved to the system directory where it actually belongs.
| **Source:** `packages/spec/src/integration/object-storage.zod.ts` | |
| **Source:** `packages/spec/src/system/object-storage.zod.ts` |
| <Callout type="info"> | ||
| **Source:** `packages/spec/src/integration/object-storage.zod.ts` | ||
| </Callout> | ||
|
|
||
| ## TypeScript Usage | ||
|
|
||
| ```typescript | ||
| import { MultipartUploadConfigSchema } from '@objectstack/spec/integration'; | ||
| import type { MultipartUploadConfig } from '@objectstack/spec/integration'; | ||
|
|
||
| // Validate data | ||
| const result = MultipartUploadConfigSchema.parse(data); | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## MultipartUploadConfig | ||
|
|
||
| ### Properties | ||
|
|
||
| | Property | Type | Required | Description | | ||
| | :--- | :--- | :--- | :--- | | ||
| | **enabled** | `boolean` | optional | Enable multipart uploads | | ||
| | **partSize** | `number` | optional | Part size in bytes (min 5MB) | | ||
| | **maxConcurrentParts** | `number` | optional | Maximum concurrent part uploads | | ||
| | **threshold** | `number` | optional | File size threshold for multipart upload in bytes | | ||
|
|
There was a problem hiding this comment.
This file documents a MultipartUploadConfig schema that does not match the one defined in the actual source code. The integration layer already has a MultipartUploadConfigSchema defined in packages/spec/src/integration/connector/file-storage.zod.ts with different properties (partSize, maxConcurrentParts, threshold). This documentation file appears to be incorrectly created and should be removed, as the proper documentation for the system-level MultipartUploadConfig is already in content/docs/references/system/object-storage.mdx.
| <Callout type="info"> | |
| **Source:** `packages/spec/src/integration/object-storage.zod.ts` | |
| </Callout> | |
| ## TypeScript Usage | |
| ```typescript | |
| import { MultipartUploadConfigSchema } from '@objectstack/spec/integration'; | |
| import type { MultipartUploadConfig } from '@objectstack/spec/integration'; | |
| // Validate data | |
| const result = MultipartUploadConfigSchema.parse(data); | |
| ``` | |
| --- | |
| ## MultipartUploadConfig | |
| ### Properties | |
| | Property | Type | Required | Description | | |
| | :--- | :--- | :--- | :--- | | |
| | **enabled** | `boolean` | optional | Enable multipart uploads | | |
| | **partSize** | `number` | optional | Part size in bytes (min 5MB) | | |
| | **maxConcurrentParts** | `number` | optional | Maximum concurrent part uploads | | |
| | **threshold** | `number` | optional | File size threshold for multipart upload in bytes | | |
| <Callout type="warning"> | |
| This integration-level documentation for `MultipartUploadConfig` has been deprecated. | |
| The canonical reference for the system-level object storage configuration (including multipart uploads) is available at: | |
| - [`System / Object Storage` reference](../system/object-storage) | |
| Please refer to that page for the up-to-date schema and properties. | |
| </Callout> |
| import { AckModeSchema, ApiVersionConfigSchema, CdcConfigSchema, ConsumerConfigSchema, DatabaseConnectorSchema, DatabasePoolConfigSchema, DatabaseProviderSchema, DatabaseTableSchema, DeliveryGuaranteeSchema, DlqConfigSchema, FileAccessPatternSchema, FileFilterConfigSchema, FileMetadataConfigSchema, FileStorageConnectorSchema, FileStorageProviderSchema, FileVersioningConfigSchema, MessageFormatSchema, MessageQueueConnectorSchema, MessageQueueProviderSchema, ProducerConfigSchema, SaasConnectorSchema, SaasObjectTypeSchema, SaasProviderSchema, SslConfigSchema, StorageBucketSchema, TopicQueueSchema } from '@objectstack/spec/integration'; | ||
| import type { AckMode, ApiVersionConfig, CdcConfig, ConsumerConfig, DatabaseConnector, DatabasePoolConfig, DatabaseProvider, DatabaseTable, DeliveryGuarantee, DlqConfig, FileAccessPattern, FileFilterConfig, FileMetadataConfig, FileStorageConnector, FileStorageProvider, FileVersioningConfig, MessageFormat, MessageQueueConnector, MessageQueueProvider, ProducerConfig, SaasConnector, SaasObjectType, SaasProvider, SslConfig, StorageBucket, TopicQueue } from '@objectstack/spec/integration'; |
There was a problem hiding this comment.
The removal of the MultipartUploadConfig section from this file is correct, as it was previously documenting the schema from packages/spec/src/integration/connector/file-storage.zod.ts. However, the import statement on line 15 still includes MultipartUploadConfigSchema and MultipartUploadConfig, which should be removed from the imports if this section has been deleted.
| */ | ||
| export const PresignedUrlConfigSchema = z.object({ | ||
| operation: z.enum(['get', 'put', 'delete', 'head']).describe('Allowed operation'), | ||
| expiresIn: z.number().min(1).max(604800).describe('Expiration time in seconds (max 7 days)'), |
There was a problem hiding this comment.
The validation allows a minimum expiration of 1 second (min(1)), but the PR description states "60s - 7 days TTL", suggesting a minimum of 60 seconds. Consider whether the minimum should be 60 seconds instead of 1 second for security and practical reasons. Very short-lived presigned URLs (< 60 seconds) may cause issues with client clock skew and network delays.
Implements metadata schemas for object storage providers (S3, Azure Blob, GCS, MinIO, etc.) and file attachment field configuration to enable storage-backed file uploads in ObjectStack applications.
Object Storage Protocol (
system/object-storage.zod.ts)Comprehensive schemas for multi-cloud storage integration:
Schemas:
StorageProvider,BucketConfig,ObjectMetadata,AccessControlConfig,LifecyclePolicyConfig,PresignedUrlConfig,MultipartUploadConfigKey features:
File Attachment Configuration (
data/field.zod.ts)Extends
FieldSchemawithFileAttachmentConfigSchemafor file/image/avatar field types:Capabilities:
Validation constraints:
virusScanProviderrequiresvirusScan: trueminSizemust be ≤maxSizetargetStorageClassOriginal prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.