Experimental: Unified LSP server in rewatch

.claude/skills/otel-investigate/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: investigate
+description: Investigate a rewatch OTEL trace report from a span ID.
+user-invocable: true
+---
+
+# OTEL Investigation Skill
+
+The user invokes this as `/investigate-otel <span_id>` where `<span_id>` is a 16-character hex string identifying an `lsp.llm_report` span in the local otel-viewer.
+
+## OTEL Viewer API (http://localhost:4707)
+
+Only these endpoints exist — do NOT invent others:
+
+- `GET /api/spans/{span_id}/context` — **start here** for `lsp.llm_report` spans. Returns the report's `message` attribute AND the session timeline (filtered `did_save`/`did_change`/`flush` siblings) in one call.
+- `GET /api/spans/{span_id}/flush` — structured flush summary with errors inlined. Use for flush spans found in the context.
+- `GET /api/spans/{span_id}` — full span detail (attributes, events). Use for drill-down.
+- `GET /api/spans/{span_id}/children` — direct children of a span. Use for drill-down.
+
+## Procedure
+
+1. **Fetch context** — this is the entry point, gives you the report message and session timeline:
+
+   ```
+   curl -s http://localhost:4707/api/spans/{SPAN_ID}/context | jq .
+   ```
+
+   The `target_span.attributes.message` is the user's problem description. `session_spans` is the chronological editing session.
+
+2. **Drill into flushes** — for `lsp.flush` spans in `session_spans` (especially with `has_error: true`):
+
+   ```
+   curl -s http://localhost:4707/api/spans/{FLUSH_SPAN_ID}/flush | jq .
+   ```
+
+3. **Drill deeper** if needed via `/api/spans/{id}` or `/api/spans/{id}/children`.
+
+4. **Read rewatch source** based on what the traces reveal:
+   - `rewatch/src/lsp/` — LSP logic
+   - `rewatch/src/build/` — build system
+   - `rewatch/src/watcher.rs` — file watching
+
+5. **Summarize**: what the report describes, session timeline, where things went wrong, root cause hypothesis, suggested fix.
+
+## Notes
+
+- Attributes like `code.filepath`, `code.namespace`, `code.lineno`, `thread.name` are instrumentation noise — ignore them.
+- The `/context` endpoint already filters to session-relevant spans with gap detection.
+- The `/flush` endpoint shows build stages with error texts inlined; collapsed subtrees have `span_id` for drill-down.

.github/workflows/ci.yml

@@ -324,18 +324,6 @@ jobs:
         if: runner.os != 'Windows'
         run: make -C tests/gentype_tests/typescript-react-example clean test
 
-      # On Windows, after running setup-ocaml (if it wasn't cached yet or the cache couldn't be restored),
-      # Cygwin bash is used instead of Git Bash for Windows, breaking the rewatch tests.
-      # So we need to adjust the path to bring back Git Bash for Windows.
-      - name: Rewatch tests need Git Bash for Windows
-        if: ${{ runner.os == 'Windows' }}
-        run: echo "C:\Program Files\Git\bin" >> $GITHUB_PATH
-        shell: bash
-
-      - name: Run rewatch tests
-        run: ./rewatch/tests/suite.sh rewatch/target/release/rescript
-        shell: bash
-
       - name: Run syntax benchmarks
         if: matrix.benchmarks
         run: |
@@ -401,14 +389,14 @@ jobs:
 
       - name: "Upload artifacts: binaries"
         if: matrix.upload_binaries
-        uses: actions/upload-artifact@v7
+        uses: actions/upload-artifact@v6
         with:
           name: binaries-${{ matrix.node-target }}
           path: packages/@rescript/${{ matrix.node-target }}/bin
 
       - name: "Upload artifacts: lib/ocaml"
         if: matrix.upload_libs
-        uses: actions/upload-artifact@v7
+        uses: actions/upload-artifact@v6
         with:
           name: lib-ocaml
           path: |
@@ -423,7 +411,7 @@ jobs:
       - name: "Upload artifacts: scripts/res/apiDocs"
         id: upload-api-docs
         if: ${{ matrix.generate_api_docs }}
-        uses: actions/upload-artifact@v7
+        uses: actions/upload-artifact@v6
         with:
           name: api
           path: scripts/res/apiDocs/
@@ -445,7 +433,7 @@ jobs:
           node-version-file: .nvmrc
 
       - name: Download artifacts
-        uses: actions/download-artifact@v8
+        uses: actions/download-artifact@v7
         with:
           pattern: "@(binaries-*|lib-ocaml)"
 
@@ -475,7 +463,7 @@ jobs:
           ssh-key: ${{ secrets.RESCRIPT_LANG_ORG_DEPLOY_KEY }}
 
       - name: Download artifacts
-        uses: actions/download-artifact@v8
+        uses: actions/download-artifact@v7
         with:
           artifact-ids: ${{ needs.build-compiler.outputs.api-docs-artifact-id }}
           path: data/api
@@ -544,7 +532,7 @@ jobs:
         uses: actions/setup-node@v6
         with:
           # Run integration tests with the oldest supported node version.
-          node-version: 20
+          node-version: 22
 
       - name: Make test directory
         id: tmp-dir
@@ -595,7 +583,7 @@ jobs:
         uses: actions/setup-node@v6
         with:
           # Run integration tests with the oldest supported node version.
-          node-version: 20
+          node-version: 22
 
       - name: Checkout
         uses: actions/checkout@v6
@@ -646,17 +634,20 @@ jobs:
         uses: actions/setup-node@v6
         with:
           # Run integration tests with the oldest supported node version.
-          node-version: 20
+          node-version: 22
+
+      - name: Install npm packages
+        run: yarn install
 
-      - name: Install ReScript package in rewatch/testrepo
+      - name: Install ReScript package in test fixture
         run: |
           COMMIT_SHA="${{ needs.pkg-pr-new.outputs.commit_sha }}"
           yarn add "rescript@https://pkg.pr.new/rescript-lang/rescript@${COMMIT_SHA}"
         shell: bash
-        working-directory: rewatch/testrepo
+        working-directory: tests/rewatch_tests/fixture
 
       - name: Run rewatch integration tests
-        run: ./rewatch/tests/suite.sh rewatch/testrepo/node_modules/.bin/rescript
+        run: node scripts/test.js -rewatch
         shell: bash
 
   publish:
@@ -680,7 +671,7 @@ jobs:
           registry-url: https://registry.npmjs.org # Needed to make auth work for publishing
 
       - name: Download artifacts
-        uses: actions/download-artifact@v8
+        uses: actions/download-artifact@v7
         with:
           pattern: "@(binaries-*|lib-ocaml)"
 

AGENTS.md

@@ -83,7 +83,13 @@ compiler/
 ├── ext/             # Extended utilities and data structures
 └── gentype/         # TypeScript generation
 
-analysis/            # Language server and tooling
+analysis/            # OCaml analysis binary (hover, completion, references)
+
+rewatch/src/
+├── build/           # Build system logic
+├── lsp/             # LSP server implementation (Rust, tower-lsp)
+└── ...              # CLI, config, watcher, etc.
+
 packages/@rescript/
 ├── runtime/         # Runtime and standard library
 └── <platform>/      # Platform-specific binaries
@@ -92,7 +98,9 @@ tests/
 ├── syntax_tests/    # Parser/syntax layer tests
 ├── tests/           # Runtime library tests
 ├── build_tests/     # Integration tests
-└── ounit_tests/     # Compiler unit tests
+├── ounit_tests/     # Compiler unit tests
+└── rewatch_tests/
+    └── tests/lsp/   # LSP integration tests
 ```
 
 ## Working on the Compiler
@@ -307,6 +315,7 @@ rewatch/src/
 │   ├── deps.rs         # Dependency analysis and module graph
 │   ├── clean.rs        # Build artifact cleanup
 │   └── logs.rs         # Build logging and error reporting
+├── lsp/               # LSP server (tower-lsp, shells out to analysis binary)
 ├── cli.rs              # Command-line interface definitions
 ├── config.rs           # rescript.json configuration parsing
 ├── watcher.rs          # File watching and incremental builds
@@ -387,21 +396,7 @@ make test-rewatch     # Run integration tests
 
 **Note**: The rewatch project is located in the `rewatch/` directory with its own `Cargo.toml` file. All cargo commands should be run from the project root using the `--manifest-path rewatch/Cargo.toml` flag, as shown in the CI workflow.
 
-**Integration Tests**: The `make test-rewatch` command runs bash-based integration tests located in `rewatch/tests/suite.sh`. These tests use the `rewatch/testrepo/` directory as a test workspace with various package configurations to verify rewatch's behavior across different scenarios.
-
-**Running Individual Integration Tests**: You can run individual test scripts directly by setting up the environment manually:
-
-```bash
-cd rewatch/tests
-export REWATCH_EXECUTABLE="$(realpath ../target/debug/rescript)"
-eval $(node ./get_bin_paths.js)
-export RESCRIPT_BSC_EXE
-export RESCRIPT_RUNTIME
-source ./utils.sh
-bash ./watch/06-watch-missing-source-folder.sh
-```
-
-This is useful for iterating on a specific test without running the full suite.
+**Integration Tests**: The `make test-rewatch` command runs Vitest-based integration tests located in `tests/rewatch_tests/`. These tests use a sandbox copy of `tests/rewatch_tests/fixture/` to verify rewatch's behavior across different scenarios (build, watch, clean, format, etc.).
 
 #### Debugging
 
@@ -410,6 +405,26 @@ This is useful for iterating on a specific test without running the full suite.
 - **Dependencies**: Inspect module dependency graph in `deps.rs`
 - **File Watching**: Monitor file change events in `watcher.rs`
 
+#### OpenTelemetry Tracing
+
+Rewatch supports OpenTelemetry (OTEL) tracing for build, watch, and LSP commands. Any OTLP-compatible viewer (Jaeger, Grafana, etc.) will work. For a lightweight option tailored to rewatch development, see `rewatch/otel-viewer/` (development notes in `rewatch/otel-viewer/AGENTS.md`) — it requires only `uv` (no Docker) and includes features like LLM export.
+
+```bash
+# Start the viewer (see rewatch/otel-viewer/README.md for setup)
+cd rewatch/otel-viewer
+uv run python server.py
+```
+
+Then run rewatch with the OTLP endpoint set:
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4707 cargo run --manifest-path rewatch/Cargo.toml -- build
+```
+
+Open http://localhost:4707 to browse traces.
+
+Note: Use `tracing::debug!` (not `log::debug!`) for events you want to appear in OTEL traces — they use separate logging systems.
+
 #### Running Rewatch Directly
 
 When running the rewatch binary directly (via `cargo run` or the compiled binary) during development, you need to set environment variables to point to the local compiler and runtime. Otherwise, rewatch will try to use the installed versions:
@@ -490,8 +505,16 @@ When clippy suggests refactoring that could impact performance, consider the tra
 3. Handle different file types (`.res`, `.resi`, etc.)
 4. Consider performance impact of watching many files
 
+## Working on the LSP Server
+
+The LSP server runs via `rescript lsp` — a single Rust process (tower-lsp) that owns the build state and speaks LSP over stdio. It shells out to the OCaml `rescript-editor-analysis` binary for features like hover, completion, and references.
+
+**Start here:** read `LSP.md` in the project root for architecture and protocol details.
+
+- **Source:** `rewatch/src/lsp/` — see `rewatch/src/lsp/AGENTS.md` for development notes
+- **Analysis binary:** `analysis/` — the OCaml binary that the LSP shells out to
+- **Tests:** `tests/rewatch_tests/tests/lsp/` — see `tests/rewatch_tests/AGENTS.md` for test infrastructure
+
 ## CI Gotchas
 
 - **`sleep` is fragile** — Prefer polling (e.g., `wait_for_file`) over fixed sleeps. CI runners are slower than local machines.
-- **`exit_watcher` is async** — It only signals the watcher to stop (removes the lock file), it doesn't wait for the process to exit. Avoid triggering config-change events before exiting, as the watcher may start a concurrent rebuild.
-- **`sed -i` differs across platforms** — macOS requires `sed -i '' ...`, Linux does not. Use the `replace` / `normalize_paths` helpers from `rewatch/tests/utils.sh` instead of raw `sed`.