WIP : Experiment [this repo only] : Fast and unsafe development time compilation out of FSharpChecker caches

.github/skills/fsharp-diagnostics/PLAN_FOR_WINDOWS.md

@@ -0,0 +1,100 @@
+# Windows Support Plan for FastBuildFromCache
+
+## Strategy
+
+Use **Unix Domain Sockets (UDS) on all platforms**, including Windows.  
+Windows 10 1803+ supports `AF_UNIX`. .NET's `UnixDomainSocketEndPoint` works cross-platform since .NET Core 3.0+.  
+This keeps the server transport code unchanged — the work is a new PowerShell client script and minor path fixes.
+
+**Why not Named Pipes?** Would require a transport abstraction in the server (different accept-loop lifecycle for `NamedPipeServerStream` vs `Socket`), doubling transport code for no benefit when UDS works.
+
+**Why not a .NET client tool?** Chicken-and-egg: the client must exist before FCS builds, but would itself need building first. Also adds ~150ms JIT startup per invocation.
+
+**Prerequisite:** `pwsh` (PowerShell 7+). Windows PowerShell 5.1 lacks `UnixDomainSocketEndPoint`. The MSBuild targets use `ContinueOnError="true"`, so missing `pwsh` gracefully falls back to normal `fsc`.
+
+---
+
+## Changes
+
+### 1. `eng/targets/FastBuildFromCache.targets`
+
+Replace hardcoded `bash` with OS-conditional properties and a single `<Exec>`:
+
+```xml
+<_FastBuildScript Condition="'$(OS)'!='Windows_NT'">...get-fsharp-errors.sh</_FastBuildScript>
+<_FastBuildScript Condition="'$(OS)'=='Windows_NT'">...get-fsharp-errors.ps1</_FastBuildScript>
+<_FastBuildInterpreter Condition="'$(OS)'!='Windows_NT'">bash</_FastBuildInterpreter>
+<_FastBuildInterpreter Condition="'$(OS)'=='Windows_NT'">pwsh -NoProfile -File</_FastBuildInterpreter>
+```
+
+Single Exec: `Command="$(_FastBuildInterpreter) &quot;$(_FastBuildScript)&quot; ..."`
+
+### 2. NEW: `scripts/get-fsharp-errors.ps1`
+
+PowerShell Core port of `get-fsharp-errors.sh` (~100-120 lines). Key translations:
+
+| Bash | PowerShell |
+|------|------------|
+| `shasum -a 256` | `[System.Security.Cryptography.SHA256]::HashData()` |
+| `nc -U "$sock"` | `[System.Net.Sockets.Socket]` + `[UnixDomainSocketEndPoint]` + `NetworkStream` + `StreamReader/Writer` |
+| `nohup dotnet run ... &` | `Start-Process dotnet -ArgumentList ... -NoNewWindow` |
+| `[ -S "$sock" ]` | `Test-Path $sock` |
+| `set -euo pipefail` | `$ErrorActionPreference = 'Stop'; Set-StrictMode -Version Latest` |
+| `$HOME/.fsharp-diag` | `Join-Path $env:USERPROFILE '.fsharp-diag'` |
+
+Same JSON protocol, same command-line interface (`--compile`, `--parse-only`, etc.).
+
+### 3. `server/Server.fs`
+
+Two one-line fixes:
+
+- **`File.SetUnixFileMode`** (throws `PlatformNotSupportedException` on Windows):
+  ```fsharp
+  if not (OperatingSystem.IsWindows()) then File.SetUnixFileMode(socketPath, ...)
+  ```
+
+- **`TrimEnd('/')`** (doesn't strip `\` on Windows paths):
+  ```fsharp
+  config.RepoRoot.TrimEnd('/', '\\') + "/"
+  ```
+
+### 4. `server/ProjectRouting.fs`
+
+- `TrimEnd('/')` → `TrimEnd('/', '\\')`
+- `StringComparison.Ordinal` → `StringComparison.OrdinalIgnoreCase` for path prefix checks
+- Normalize relative path: `.Replace('\\', '/')` before pattern matching against `"tests/"`, `"src/"` etc.
+
+### 5. `server/DiagnosticsFormatter.fs`
+
+- `TrimEnd('/')` → `TrimEnd('/', '\\')`
+- `StringComparison.OrdinalIgnoreCase` for `path.StartsWith(root)`
+
+---
+
+## Critical: Path Normalization Before Hashing
+
+The socket path is derived from `SHA256(repoRoot)`. Client and server **must hash the exact same string** or they'll look for different sockets.
+
+Problem: `git rev-parse --show-toplevel` returns `C:/Users/foo/fsharp` on Windows (forward slashes), but .NET's `Directory.GetCurrentDirectory()` returns `C:\Users\foo\fsharp` (backslashes).
+
+**Rule:** Before hashing, normalize to: forward slashes, no trailing separator.  
+Apply this in both the PS1 script and `deriveSocketPath` in Server.fs.
+
+---
+
+## What Does NOT Need Changing
+
+- **Socket transport in Server.fs** — `Socket(AddressFamily.Unix)` + `UnixDomainSocketEndPoint` works on Windows
+- **FileSystemWatcher** — cross-platform in .NET
+- **FSharpDiagServer.fsproj** — `net10.0` SDK project, fully cross-platform
+- **Program.fs** — no OS-specific code
+- **All product code** (`service.fs`, `FSharpCheckerResults.fs`, `CompilerImports.fs`, `fsc.fs`) — already cross-platform
+
+## Testing Checklist
+
+- [ ] `pwsh` can connect to server via UDS on Windows
+- [ ] Server spawns correctly via `Start-Process` from PS1
+- [ ] Socket path matches between PS1 client and server (hash normalization)
+- [ ] `dotnet test ... /p:FastBuildFromCache=true` works end-to-end on Windows
+- [ ] Graceful fallback when `pwsh` is not installed
+- [ ] No-change build is a no-op (MSBuild incremental skip works on Windows)

.github/skills/fsharp-diagnostics/SKILL.md

@@ -13,36 +13,32 @@ description: "After modifying any F# file, use this to get quick parse errors an
 GetErrors() { "$(git rev-parse --show-toplevel)/.github/skills/fsharp-diagnostics/scripts/get-fsharp-errors.sh" "$@"; }
 ```
 
-## Parse first, typecheck second
+## Rules
 
-```bash
-GetErrors --parse-only src/Compiler/Checking/CheckBasics.fs
-```
-If errors → fix syntax. Do NOT typecheck until parse is clean.
-```bash
-GetErrors src/Compiler/Checking/CheckBasics.fs
-```
+1. **After every edit** to a `src/Compiler/*.fs` file → typecheck it before proceeding. This catches errors in ~2s vs ~35s for a full build. Do NOT attempt `dotnet build` or `dotnet test` until the file typechecks clean.
+2. **Use `--find-refs` instead of grep** for finding usages of a symbol (function, type, member, field). Returns semantically resolved references — no false positives from comments, strings, or similarly-named symbols.
+3. **Use `--type-hints` to read code blocks** — F# infers most types, so bindings like `env`, `state`, `x` are opaque without it.
+   - ⚠️ Output has `// (name: Type)` annotations. These are **read-only overlays**. When editing, use `view` to get the real unannotated source.
+4. **Parse first, typecheck second** — fix `--parse-only` errors before running a full typecheck.
 
-## Find references for a single symbol (line 1-based, col 0-based)
+## Commands
 
-Before renaming or to understand call sites:
 ```bash
-GetErrors --find-refs src/Compiler/Checking/CheckBasics.fs 30 5
+GetErrors --parse-only src/Compiler/path/File.fs        # parse errors only
+GetErrors src/Compiler/path/File.fs                     # full typecheck
+GetErrors --find-refs src/Compiler/path/File.fs 30 5    # references (line 1-based, col 0-based)
+GetErrors --type-hints src/Compiler/path/File.fs 50 60  # annotated code (line range, 1-based)
+GetErrors --check-project                               # typecheck entire project
+GetErrors --ping                                        # server alive?
+GetErrors --shutdown                                    # stop server
 ```
 
-## Type hints for a range selection (begin and end line numbers, 1-based)
-
-To see inferred types as inline `// (name: Type)` comments:
-```bash
-GetErrors --type-hints src/Compiler/TypedTree/TypedTreeOps.fs 1028 1032
-```
+## Cached test runs
 
-## Other
+No separate `dotnet build` of FSharp.Compiler.Service needed — `dotnet test` builds all dependencies automatically.
 
 ```bash
-GetErrors --check-project   # typecheck entire project
-GetErrors --ping
-GetErrors --shutdown
+dotnet test tests/FSharp.Compiler.ComponentTests/FSharp.Compiler.ComponentTests.fsproj -c Release /p:FastBuildFromCache=true
 ```
 
 First call starts server (~70s cold start, set initial_wait=600). Auto-shuts down after 4h idle. ~3 GB RAM.

.github/skills/fsharp-diagnostics/scripts/get-fsharp-errors.sh

@@ -5,6 +5,7 @@ set -euo pipefail
 # Usage:
 #   get-fsharp-errors.sh [--parse-only] <file.fs>
 #   get-fsharp-errors.sh --check-project <project.fsproj>
+#   get-fsharp-errors.sh --compile <project.fsproj> <output.dll>
 #   get-fsharp-errors.sh --ping
 #   get-fsharp-errors.sh --shutdown
 
@@ -104,9 +105,19 @@ case "${1:-}" in
         ensure_server "$REPO_ROOT" "$SOCK_PATH"
         send_request "$SOCK_PATH" "{\"command\":\"typeHints\",\"file\":\"$FILE\",\"startLine\":$START_LINE,\"endLine\":$END_LINE}"
         ;;
+    --compile)
+        shift
+        PROJECT="$1"
+        OUTPUT="$2"
+        ensure_server "$REPO_ROOT" "$SOCK_PATH"
+        RESPONSE=$(send_request "$SOCK_PATH" "{\"command\":\"compile\",\"project\":\"$PROJECT\",\"output\":\"$OUTPUT\"}")
+        echo "$RESPONSE"
+        case "$RESPONSE" in ERROR*) exit 1 ;; esac
+        ;;
     -*)
         echo "Usage: get-fsharp-errors [--parse-only] <file.fs>" >&2
         echo "       get-fsharp-errors --check-project <project.fsproj>" >&2
+        echo "       get-fsharp-errors --compile <project.fsproj> <output.dll>" >&2
         echo "       get-fsharp-errors --ping | --shutdown" >&2
         exit 1
         ;;

.github/skills/fsharp-diagnostics/server/DesignTimeBuild.fs

@@ -5,7 +5,9 @@ open System.Diagnostics
 open System.IO
 open System.Text.Json
 
-type DtbResult = { CompilerArgs: string array }
+type DtbResult =
+    { CompilerArgs: string array
+      IntermediateOutputPath: string }
 
 type DtbConfig =
     {
@@ -27,15 +29,49 @@ let run (fsprojPath: string) (config: DtbConfig) =
             |> Option.defaultValue ""
 
         let projDir = Path.GetDirectoryName(fsprojPath)
+        let projName = Path.GetFileNameWithoutExtension(fsprojPath)
 
-        // /t:Build runs BeforeBuild (generates buildproperties.fs via CompileBefore).
-        // DesignTimeBuild=true skips dependency projects.
-        // SkipCompilerExecution=true + ProvideCommandLineArgs=true populates FscCommandLineArgs without compiling.
+        // Query IntermediateOutputPath to find and delete the intermediate assembly,
+        // defeating MSBuild's up-to-date check so CoreCompile actually runs.
+        let iopPsi =
+            ProcessStartInfo(
+                FileName = "dotnet",
+                Arguments =
+                    $"msbuild \"{fsprojPath}\" /p:BUILDING_USING_DOTNET=true /p:Configuration={config.Configuration}{tfmArg} /nologo /v:q /getProperty:IntermediateOutputPath",
+                RedirectStandardOutput = true,
+                RedirectStandardError = true,
+                UseShellExecute = false,
+                WorkingDirectory = projDir
+            )
+        use iopProc = Process.Start(iopPsi)
+        let! iopOut = iopProc.StandardOutput.ReadToEndAsync() |> Async.AwaitTask
+        do! iopProc.WaitForExitAsync() |> Async.AwaitTask
+
+        let iopOut = iopOut.Trim()
+        // Handle both plain path and JSON output from /getProperty
+        let intermediateDir =
+            if iopOut.StartsWith("{") then
+                try
+                    let doc = JsonDocument.Parse(iopOut)
+                    doc.RootElement.GetProperty("Properties").GetProperty("IntermediateOutputPath").GetString()
+                with _ -> ""
+            else iopOut
+        let intermediateDir =
+            if Path.IsPathRooted(intermediateDir) then intermediateDir
+            elif intermediateDir.Length > 0 then Path.Combine(projDir, intermediateDir)
+            else ""
+        if intermediateDir.Length > 0 then
+            let intermediateDll = Path.Combine(intermediateDir, projName + ".dll")
+            if File.Exists(intermediateDll) then
+                try File.Delete(intermediateDll) with _ -> ()
+
+        // /t:CoreCompile + SkipCompilerExecution + ProvideCommandLineArgs populates FscCommandLineArgs.
+        // BuildProjectReferences=false avoids rebuilding dependencies.
         let psi =
             ProcessStartInfo(
                 FileName = "dotnet",
                 Arguments =
-                    $"msbuild \"{fsprojPath}\" /t:Build /p:DesignTimeBuild=true /p:SkipCompilerExecution=true /p:ProvideCommandLineArgs=true /p:CopyBuildOutputToOutputDirectory=false /p:CopyOutputSymbolsToOutputDirectory=false /p:BUILDING_USING_DOTNET=true /p:Configuration={config.Configuration}{tfmArg} /nologo /v:q /getItem:FscCommandLineArgs",
+                    $"msbuild \"{fsprojPath}\" /t:CoreCompile /p:SkipCompilerExecution=true /p:ProvideCommandLineArgs=true /p:CopyBuildOutputToOutputDirectory=false /p:CopyOutputSymbolsToOutputDirectory=false /p:BUILDING_USING_DOTNET=true /p:BuildProjectReferences=false /p:Configuration={config.Configuration}{tfmArg} /nologo /v:q \"/getItem:FscCommandLineArgs;ReferencePath\"",
                 RedirectStandardOutput = true,
                 RedirectStandardError = true,
                 UseShellExecute = false,
@@ -51,7 +87,6 @@ let run (fsprojPath: string) (config: DtbConfig) =
             return Error $"DTB failed (exit {proc.ExitCode}): {stderr}"
         else
             try
-                // MSBuild may emit warnings before the JSON; find the JSON start
                 let jsonStart = stdout.IndexOf('{')
 
                 if jsonStart < 0 then
@@ -65,7 +100,22 @@ let run (fsprojPath: string) (config: DtbConfig) =
                         |> Seq.map (fun e -> e.GetProperty("Identity").GetString())
                         |> Seq.toArray
 
-                    return Ok { CompilerArgs = args }
+                    let refs =
+                        match items.TryGetProperty("ReferencePath") with
+                        | true, refItems ->
+                            refItems.EnumerateArray()
+                            |> Seq.map (fun e ->
+                                let path = e.GetProperty("Identity").GetString()
+                                "-r:" + path)
+                            |> Seq.toArray
+                        | false, _ -> [||]
+
+                    let combined = Array.append args refs
+
+                    if args.Length = 0 then
+                        return Error "DTB returned empty FscCommandLineArgs (CoreCompile was skipped)"
+                    else
+                        return Ok { CompilerArgs = combined; IntermediateOutputPath = intermediateDir }
             with ex ->
                 return Error $"Failed to parse DTB output: {ex.Message}"
     }

.github/skills/fsharp-diagnostics/server/Directory.Build.props

@@ -5,5 +5,8 @@
     <ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
     <BaseOutputPath>$(MSBuildThisFileDirectory)../../../../.tools/fsharp-diag/bin/</BaseOutputPath>
     <BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)../../../../.tools/fsharp-diag/obj/</BaseIntermediateOutputPath>
+    <SignAssembly>true</SignAssembly>
+    <PublicSign>true</PublicSign>
+    <AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)../../../../buildtools/keys/MSFT.snk</AssemblyOriginatorKeyFile>
   </PropertyGroup>
 </Project>

.github/skills/fsharp-diagnostics/server/FSharpDiagServer.fsproj

@@ -3,16 +3,41 @@
   <PropertyGroup>
     <OutputType>Exe</OutputType>
     <TargetFramework>net10.0</TargetFramework>
+    <!-- Derive repo root: this fsproj is at .github/skills/fsharp-diagnostics/server/ -->
+    <_DiagServerRepoRoot>$([MSBuild]::NormalizeDirectory('$(MSBuildThisFileDirectory)', '..', '..', '..', '..'))</_DiagServerRepoRoot>
+    <!-- Use Release-built FCS (built from current source, includes latest changes) -->
+    <_ReleaseFcsPath>$(_DiagServerRepoRoot)artifacts/bin/FSharp.Compiler.Service/Release/net10.0/FSharp.Compiler.Service.dll</_ReleaseFcsPath>
+    <!-- Fallback: Proto-built FCS -->
+    <_ProtoFcsPath>$(_DiagServerRepoRoot)artifacts/Bootstrap/fsc/FSharp.Compiler.Service.dll</_ProtoFcsPath>
   </PropertyGroup>
 
-  <ItemGroup>
+  <!-- Prefer Release-built FCS (includes latest source changes) -->
+  <ItemGroup Condition="Exists('$(_ReleaseFcsPath)')">
+    <Reference Include="FSharp.Compiler.Service">
+      <HintPath>$(_ReleaseFcsPath)</HintPath>
+    </Reference>
+  </ItemGroup>
+
+  <!-- Fallback to Proto when no Release build exists -->
+  <ItemGroup Condition="!Exists('$(_ReleaseFcsPath)') AND Exists('$(_ProtoFcsPath)')">
+    <Reference Include="FSharp.Compiler.Service">
+      <HintPath>$(_ProtoFcsPath)</HintPath>
+    </Reference>
+  </ItemGroup>
+
+  <!-- Fallback to NuGet when neither exists -->
+  <ItemGroup Condition="!Exists('$(_ReleaseFcsPath)') AND !Exists('$(_ProtoFcsPath)')">
     <PackageReference Include="FSharp.Compiler.Service" Version="43.*" />
+  </ItemGroup>
+
+  <ItemGroup>
     <PackageReference Update="FSharp.Core" Version="10.*" />
   </ItemGroup>
 
   <ItemGroup>
     <Compile Include="DesignTimeBuild.fs" />
     <Compile Include="ProjectManager.fs" />
+    <Compile Include="ProjectRouting.fs" />
     <Compile Include="DiagnosticsFormatter.fs" />
     <Compile Include="Server.fs" />
     <Compile Include="Program.fs" />