chore(devextreme): migrate transpile gulp task to nx, part 2 #32359
Conversation
chaosmirage
force-pushed
the
feat/migrate-devextreme-to-nx-part-4
branch
from
64e4e11 to
1f6aed3
Compare
There was a problem hiding this comment.
Pull request overview
Continues the migration of DevExtreme’s transpile/build pipeline from Gulp scripting to Nx targets by introducing new custom Nx executors and a dedicated “workflows” project to orchestrate dev/testing builds, and updates CI to use Nx configurations instead of global env flags.
Changes:
- Replace the
all:build-devflow with Nx-orchestrated targets in a newpackages/workflowsproject and update CI workflows accordingly. - Add new custom Nx executors (
babel-transform,concatenate-files,create-dual-mode-manifest) and extend existing infra executors (copy-files glob support, logging level adjustments, TS path alias resolution). - Expand
packages/devextreme/project.jsonwith granular Nx targets for transpilation, bundler-config generation, dual-mode packaging manifests, and CI/testing configuration support.
Reviewed changes
Copilot reviewed 53 out of 54 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/scripts/build-all.ts | Removes dev-mode branching and always runs production-style build orchestration for all:build. |
| packages/workflows/project.json | Adds Nx “workflows” project to orchestrate cross-package build commands for dev/testing. |
| packages/nx-infra-plugin/src/executors/prepare-submodules/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/prepare-package-json/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/pack-npm/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/localization/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/karma-multi-env/executor.ts | Reduces log verbosity in Karma runner output. |
| packages/nx-infra-plugin/src/executors/generate-components/executor.ts | Reduces log verbosity during component generation. |
| packages/nx-infra-plugin/src/executors/generate-components/angular-generator.ts | Reduces log verbosity during Angular generation steps. |
| packages/nx-infra-plugin/src/executors/generate-component-names/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/create-dual-mode-manifest/side-effect-finder.ts | Introduces side-effect import analysis for dual-mode manifests. |
| packages/nx-infra-plugin/src/executors/create-dual-mode-manifest/schema.ts | Adds TS schema for the new dual-mode manifest executor. |
| packages/nx-infra-plugin/src/executors/create-dual-mode-manifest/schema.json | Adds JSON schema for the new dual-mode manifest executor. |
| packages/nx-infra-plugin/src/executors/create-dual-mode-manifest/executor.ts | Implements generating per-subpath package.json for ESM+CJS dual-mode output. |
| packages/nx-infra-plugin/src/executors/create-dual-mode-manifest/executor.e2e.spec.ts | Adds E2E coverage for dual-mode manifest generation. |
| packages/nx-infra-plugin/src/executors/copy-files/schema.json | Documents glob support and adds schema metadata. |
| packages/nx-infra-plugin/src/executors/copy-files/executor.ts | Adds glob pattern support and improves diagnostics. |
| packages/nx-infra-plugin/src/executors/copy-files/executor.e2e.spec.ts | Adds E2E test for glob-copy behavior. |
| packages/nx-infra-plugin/src/executors/concatenate-files/schema.ts | Adds TS schema for new concatenate-files executor. |
| packages/nx-infra-plugin/src/executors/concatenate-files/schema.json | Adds JSON schema for new concatenate-files executor. |
| packages/nx-infra-plugin/src/executors/concatenate-files/executor.ts | Implements file concatenation with extraction/transforms for bundler-config generation. |
| packages/nx-infra-plugin/src/executors/concatenate-files/executor.e2e.spec.ts | Adds E2E coverage for concatenation use case. |
| packages/nx-infra-plugin/src/executors/clean/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/build-typescript/schema.ts | Adds options for path alias resolution support. |
| packages/nx-infra-plugin/src/executors/build-typescript/schema.json | Updates schema defaults and documents alias-resolution options. |
| packages/nx-infra-plugin/src/executors/build-typescript/executor.ts | Adds tsc-alias-based path rewriting and refactors compilation flow. |
| packages/nx-infra-plugin/src/executors/build-typescript/executor.e2e.spec.ts | Expands E2E coverage (CJS/ESM matrix + alias rewrite). |
| packages/nx-infra-plugin/src/executors/build-angular-library/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/src/executors/babel-transform/schema.ts | Adds TS schema for new babel-transform executor. |
| packages/nx-infra-plugin/src/executors/babel-transform/schema.json | Adds JSON schema for new babel-transform executor. |
| packages/nx-infra-plugin/src/executors/babel-transform/executor.ts | Implements Babel-based transpilation executor (debug stripping, extension renames). |
| packages/nx-infra-plugin/src/executors/babel-transform/executor.e2e.spec.ts | Adds E2E coverage for babel-transform behavior. |
| packages/nx-infra-plugin/src/executors/add-license-headers/executor.ts | Reduces log verbosity. |
| packages/nx-infra-plugin/package.json | Adds minimatch/tsc-alias deps and Babel-related peer deps metadata. |
| packages/nx-infra-plugin/jest.config.ts | Adds resolver and transformIgnorePatterns for pnpm layout compatibility. |
| packages/nx-infra-plugin/jest-resolver.js | Adds custom Jest resolver to better handle pnpm symlinked modules. |
| packages/nx-infra-plugin/executors.json | Registers the new executors. |
| packages/devextreme/project.json | Adds granular Nx targets for transpile steps and updates build orchestration. |
| packages/devextreme/package.json | Removes legacy build script in favor of Nx-driven build target. |
| packages/devextreme/gulpfile.js | Wires gulp transpile to Nx build:transpile target. |
| packages/devextreme/build/gulp/transpile.js | Removes legacy transpile pipelines; keeps watch/tests tasks. |
| packages/devextreme/build/gulp/side-effects-finder.js | Removes legacy side-effect finder (moved to Nx executor). |
| packages/devextreme/build/gulp/localization.js | Replaces legacy localization pipeline with Nx build:localization. |
| packages/devextreme-angular/project.json | Updates dependsOn/config forwarding and adds testing configuration env. |
| package.json | Switches all:build-dev to Nx workflows orchestration. |
| apps/demos/project.json | Forwards params when depending on devextreme:build. |
| .github/workflows/wrapper_tests_e2e.yml | Uses Nx workflows build target instead of all:build-dev + env flag. |
| .github/workflows/wrapper_tests.yml | Uses devextreme -c testing instead of global env flag. |
| .github/workflows/testcafe_tests.yml | Uses devextreme -c testing instead of global env flag. |
| .github/workflows/publish-demos.yml | Uses Nx workflows build target instead of all:build-dev + env flag. |
| .github/workflows/demos_visual_tests.yml | Uses Nx workflows build target and devextreme -c testing. |
| .github/workflows/default_workflow.yml | Uses -c ci shorthand for Nx configuration. |
| .github/copilot-instructions.md | Updates repo guidance for new Nx targets and custom executors. |
| "build:cjs:bundles": { | ||
| "executor": "devextreme-nx-infra-plugin:babel-transform", | ||
| "options": { | ||
| "babelConfigPath": "./build/gulp/transpile-config.js", | ||
| "configKey": "cjs", | ||
| "sourcePattern": "./build/bundle-templates/**/*.js", | ||
| "outDir": "./artifacts/transpiled/bundles", | ||
| "removeDebug": true | ||
| }, | ||
| "configurations": { | ||
| "production": { | ||
| "outDir": "./artifacts/transpiled-renovation-npm/bundles" | ||
| }, | ||
| "esm-npm": { | ||
| "outDir": "./artifacts/transpiled-esm-npm/bundles" | ||
| } | ||
| }, | ||
| "inputs": ["{projectRoot}/build/bundle-templates/**/*.js"], | ||
| "outputs": ["{projectRoot}/artifacts/transpiled/bundles"], | ||
| "cache": true |
There was a problem hiding this comment.
build:cjs:bundles changes outDir for both production and esm-npm configurations, but outputs stays {projectRoot}/artifacts/transpiled/bundles. This can cause Nx to cache/restore the wrong folder for non-default configurations. Consider making outputs depend on outDir (e.g. {options.outDir}) or specifying per-configuration outputs.
| const isIndex = fileName === 'index.js'; | ||
| const relative = path.join('./', fileDir.replace(srcDir, ''), fileName); | ||
| const currentPath = isIndex ? path.join(relative, '../') : relative; | ||
|
|
||
| const esmFile = path.relative(currentPath, path.join('./esm', relative)); | ||
| const cjsFile = path.relative(currentPath, path.join('./cjs', relative)); |
There was a problem hiding this comment.
createModuleConfig derives relative via fileDir.replace(srcDir, ''). When fileDir is under srcDir, the replacement typically leaves a leading path separator (e.g. /ui), and path.join('./', '/ui', ...) becomes an absolute path. That breaks main/module/typings path calculation for nested modules (and can even make them empty), producing invalid per-subpath package.json manifests. Use path.relative(srcDir, fileDir) (or strip any leading separators after replace) to build a truly relative module path before computing path.relative(...) values.
| const relativePath = fileDir.replace(srcDir, ''); | ||
| const baseName = path.basename(fileName, '.js'); | ||
| const isIndex = fileName === 'index.js'; | ||
|
|
||
| if (isIndex) { | ||
| return path.join(outputDir, relativePath, 'package.json'); | ||
| } else { | ||
| return path.join(outputDir, relativePath, baseName, 'package.json'); | ||
| } |
There was a problem hiding this comment.
getPackageJsonOutputPath uses fileDir.replace(srcDir, '') to compute relativePath. If that result starts with a path separator, path.join(outputDir, relativePath, ...) ignores outputDir and writes package.json outside the intended output folder (e.g. to /ui/package.json). Prefer path.relative(srcDir, fileDir) (and normalize to no leading separators) before joining with outputDir.
| const sourceFiles = await glob(globPattern, { | ||
| absolute: true, | ||
| ignore: options.excludePatterns || [], |
There was a problem hiding this comment.
glob() is executed with an absolute globPattern, but ignore is taken directly from options.excludePatterns (typically project-relative patterns like ./js/__internal/**/*). With the current implementation, these ignore patterns may not match the absolute paths returned by glob, so excluded files can still be transformed. Resolve excludePatterns against projectRoot (and normalize for Windows) or set cwd: projectRoot and keep both patterns relative.
| const sourceFiles = await glob(globPattern, { | |
| absolute: true, | |
| ignore: options.excludePatterns || [], | |
| const rawExcludePatterns = options.excludePatterns ?? []; | |
| const excludePatterns = rawExcludePatterns.map((pattern) => { | |
| const resolved = path.isAbsolute(pattern) ? pattern : path.join(projectRoot, pattern); | |
| return isWindowsOS() ? normalizeGlobPathForWindows(resolved) : resolved; | |
| }); | |
| const sourceFiles = await glob(globPattern, { | |
| absolute: true, | |
| ignore: excludePatterns, |
| import { PromiseExecutor, logger } from '@nx/devkit'; | ||
| import * as path from 'path'; | ||
| import * as fs from 'fs-extra'; | ||
| import * as babel from '@babel/core'; | ||
| import { glob } from 'glob'; |
There was a problem hiding this comment.
@babel/core is imported at module load time, but in this repo the Nx plugin is loaded via a path (nx.json plugins) rather than via a package dependency. With pnpm's isolated node_modules, peer dependencies are not guaranteed to be resolvable from packages/nx-infra-plugin at runtime, so this executor can fail to load before it even runs. Consider either making @babel/core (and any required Babel plugins) a direct dependency of devextreme-nx-infra-plugin, or switching to a lazy require resolved from the target project root (e.g., require.resolve(..., { paths: [projectRoot] })) and emitting a clear error if Babel is unavailable.
| async function emitWithAliasResolution( | ||
| program: ts.Program, | ||
| options: EmitOptions, | ||
| ): Promise<{ success: boolean; diagnostics: ts.Diagnostic[] }> { | ||
| const { aliasTranspileFunc, outDir, aliasPath } = options; | ||
| const emittedFiles: Array<{ path: string; content: string }> = []; | ||
|
|
||
| const result = program.emit(undefined, (filePath, fileData) => { | ||
| let finalContent = fileData; | ||
|
|
||
| if (aliasTranspileFunc && aliasPath) { | ||
| const normalizedFilePath = filePath.replace(outDir, aliasPath); | ||
| finalContent = aliasTranspileFunc(normalizedFilePath, fileData); | ||
| } | ||
|
|
||
| emittedFiles.push({ path: filePath, content: finalContent }); | ||
| }); | ||
|
|
||
| for (const file of emittedFiles) { | ||
| const dir = path.dirname(file.path); | ||
| await ensureDir(dir); | ||
| await writeFileText(file.path, file.content); | ||
| } |
There was a problem hiding this comment.
When resolvePaths is enabled, emitWithAliasResolution buffers all emitted outputs in memory (emittedFiles) before writing them to disk. For large TypeScript builds (e.g. DevExtreme js/__internal), this can significantly increase peak memory usage. Consider writing files as they are emitted (e.g., push write promises in the emit callback and await Promise.all(...) afterward, or use sync writes in the callback) to avoid keeping every output file’s contents in RAM at once.
| "build:cjs": { | ||
| "executor": "devextreme-nx-infra-plugin:babel-transform", | ||
| "options": { | ||
| "babelConfigPath": "./build/gulp/transpile-config.js", | ||
| "configKey": "cjs", | ||
| "sourcePattern": "./js/**/*.{js,jsx}", | ||
| "excludePatterns": ["./js/**/*.d.ts", "./js/__internal/**/*"], | ||
| "outDir": "./artifacts/transpiled" | ||
| }, | ||
| "configurations": { | ||
| "production": { | ||
| "outDir": "./artifacts/transpiled-renovation-npm", | ||
| "removeDebug": true | ||
| } | ||
| }, | ||
| "inputs": [ | ||
| "{projectRoot}/js/**/*.{js,jsx}", | ||
| "!{projectRoot}/js/**/*.d.ts", | ||
| "!{projectRoot}/js/__internal/**/*" | ||
| ], | ||
| "outputs": ["{projectRoot}/artifacts/transpiled"], | ||
| "cache": true |
There was a problem hiding this comment.
build:cjs changes outDir in the production configuration, but outputs is hard-coded to {projectRoot}/artifacts/transpiled. This makes Nx caching/artifact tracking incorrect for -c production runs (it will cache/restore the wrong folder and may leave stale files). Consider expressing outputs in terms of the option (e.g. {options.outDir}) or listing both possible output directories.
| "build:cjs:internal": { | ||
| "executor": "devextreme-nx-infra-plugin:babel-transform", | ||
| "options": { | ||
| "babelConfigPath": "./build/gulp/transpile-config.js", | ||
| "configKey": "tsCjs", | ||
| "sourcePattern": "./artifacts/dist_ts/__internal/**/*.{js,jsx}", | ||
| "outDir": "./artifacts/transpiled/__internal", | ||
| "renameExtensions": { ".jsx": ".js" } | ||
| }, | ||
| "configurations": { | ||
| "production": { | ||
| "outDir": "./artifacts/transpiled-renovation-npm/__internal", | ||
| "removeDebug": true | ||
| } | ||
| }, | ||
| "inputs": ["{projectRoot}/artifacts/dist_ts/__internal/**/*.{js,jsx}"], | ||
| "outputs": ["{projectRoot}/artifacts/transpiled/__internal"], | ||
| "cache": true |
There was a problem hiding this comment.
build:cjs:internal also varies outDir in the production configuration, but outputs remains {projectRoot}/artifacts/transpiled/__internal. For production builds this points to the wrong location (transpiled-renovation-npm/__internal), which can break Nx output caching/cleanup. Consider using {options.outDir} (or configuration-specific outputs) so Nx tracks the actual produced directory.
1 participant
This PR should be completed after #32309