Skip to content

Fallback to WASM tree-sitter parser#393

Open
kddejong wants to merge 6 commits intomainfrom
feature/wasmparser
Open

Fallback to WASM tree-sitter parser#393
kddejong wants to merge 6 commits intomainfrom
feature/wasmparser

Conversation

@kddejong
Copy link
Copy Markdown
Collaborator

@kddejong kddejong commented Jan 14, 2026

Issue #, if available:

Description of changes:

  • Fallback to WASM tree-sitter parser for environments where native bindings are unavailable (e.g., legacy Linux)

Changes:

  • ParserFactory detects legacy Linux via BUILD_TARGET=legacy (set at build time in webpack) and uses WASM tree-sitter instead of native bindings
  • WasmParserFactory provides WASM-based parsing with automatic fallback to native if WASM initialization fails
  • NativeParserFactory tries native bindings first, falls back to WASM if they fail to load
  • GrammarManager handles WASM grammar loading with retry logic and caching; resolves grammar paths from node_modules in unbundled environments
  • parserFactoryReady promise exported and awaited in server onInitialize to ensure WASM is fully initialized before use
  • Webpack configured to bundle .wasm files as assets
  • Fixed node reference equality (===) to use node.id comparison for WASM compatibility (web-tree-sitter creates new JS wrapper objects per access)

Performance (e2e/integration tests, cold cache, WASM confirmed via ParserAdapter):

Metric Native WASM
Test execution ~22.5s ~19.3s
Setup ~88.2s ~87.9s
Total duration ~16.2s ~15.6s

No measurable performance difference between native and WASM parsers.

Testing:

  • Unit tests: 2941 passed (189 files)
  • e2e/integration tests: 553 passed with both native and WASM parsers
  • Added test:wasm npm script and CI step to run e2e tests with BUILD_TARGET=legacy
  • Verified WASM is actually used: ParserAdapter constructor confirmed in WASM path

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@kddejong kddejong requested a review from a team as a code owner January 14, 2026 23:26
Comment thread webpack.config.js Outdated
Comment thread src/parser/ParserFactory.ts Outdated
Comment thread src/parser/ParserFactory.ts Outdated
Comment thread src/parser/ParserFactory.ts Outdated
…ests

- Fix webpack.config.js: [name][ext] -> [name].[ext]
- Replace shouldForceWasm() with const isLegacyLinux using BUILD_TARGET
- Add native fallback when WASM initialization fails
- Add test:wasm script and CI step for WASM e2e tests
- Update unit tests for new env var
- Export parserFactoryReady promise and await it in server onInitialize
- Fix GrammarManager to resolve WASM files from node_modules in unbundled env
- Await parserFactoryReady in TestExtension for e2e tests
- Confirmed via test: WASM path now returns ParserAdapter, not native Parser
web-tree-sitter creates new JS wrapper objects for each node access,
so reference equality (===) fails even for the same underlying node.
Use node.id comparison instead, which works for both native and WASM.

Fixes 3 JSON e2e test failures when using WASM parser.
@kddejong kddejong force-pushed the feature/wasmparser branch from 2170444 to a1ae3ce Compare April 22, 2026 17:24
Standalone ParserType module avoids circular dependency with LoggerFactory.
Every SyntaxTree @measure now emits parser.type=native|wasm so metrics
can be filtered/grouped by parser implementation in CloudWatch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants