[WIP] Investigation: Object expressions in struct types generating invalid IL with byref fields

[Copilot]

Investigation: Object expressions in struct types generate invalid IL with byref fields

Problem

When object expressions inside struct member methods capture struct constructor parameters or fields, the compiler generates illegal byref<Struct> fields in closure classes, causing TypeLoadException at runtime.

Current Status

INVESTIGATION COMPLETE - TRANSFORMATION CODE REVERTED

After extensive investigation testing 11 hypotheses (documented in HYPOTHESIS.md), confirmed:

1. ✅ The bug is real and reproducible - all 3 tests fail with the expected errors
2. ✅ Tests are correctly written - they detect both compile-time and runtime failures
3. ❌ Initial approach was flawed - env.eFamilyType cannot distinguish static vs instance contexts

Test Results (Baseline - No Fix Applied)

All 3 tests FAILING as expected - This confirms the bug exists and tests detect it:

Test 1: Simple case

• Error: TypeLoadException: A ByRef or ByRef-like type cannot be used as the type for an instance field
• Demonstrates runtime failure when IL contains illegal byref fields

Test 2: Multiple fields

• Error: Error 406: The byref-typed variable '_' is used in an invalid way
• Location: Line 8, column 17-25 (the CreateObj() method)
• Demonstrates compile-time detection

Test 3: Interface override

• Error: Error 406: The byref-typed variable '_' is used in an invalid way
• Location: Line 9, column 17-28 (the CreateFoo() method)
• Demonstrates compile-time detection

Investigation Summary (11 Hypotheses - See HYPOTHESIS.md)

Hypotheses 1-6: Various filter and context detection attempts
Hypothesis 7: Discovered env.eFamilyType overwritten by EnterFamilyRegion
Hypothesis 8: Simplified filter for constructor parameters
Hypothesis 9: Removed interface-only guard
Hypothesis 10: CRITICAL - env.eFamilyType cannot distinguish static vs instance contexts
Hypothesis 11: CONFIRMED - Tests properly fail without fix, validating test design

Why the Initial Approach Failed

env.eFamilyType represents the TYPE being type-checked (the object expression's base type), NOT the execution context. Both static and instance members have env.eFamilyType = Some(structTcref) when inside a struct type definition, making it impossible to distinguish between:

• Instance methods (where transformation SHOULD apply)
• Static methods (where transformation should NOT apply)

This caused false positives like the StructBox.Comparer regression in seqcore.fs.

Path Forward

A correct solution needs to:

1. Detect instance member context (not just struct type context)
2. Check for this value presence in scope, OR
3. Pass explicit instance/static context through TcObjectExpr, OR
4. Use baseValOpt more intelligently to detect struct instance captures

Investigation Artifacts

• HYPOTHESIS.md: Complete investigation documentation (11 hypotheses)
• StructObjectExpression.fs: Test cases (preserved for future fix validation)
• Transformation code: Removed (131 lines) - clean slate for correct approach

Conclusion

Investigation complete. The original bug is confirmed and well-tested. A different architectural approach is required for a correct fix.

Relates to #19068

Original prompt

Fix: Object expressions in struct types generate invalid IL with byref fields

Problem

When an object expression is created inside a struct member method and references values from the struct's constructor parameters or fields, the F# compiler generates invalid IL code. The generated closure class contains a byref<Struct> field, which is illegal according to CLI rules. This results in a TypeLoadException at runtime.

Repro Code (Currently Crashes)

type Class(test : obj) = class end

[<Struct>]
type Struct(test : obj) =
  member _.Test() = {
    new Class(test) with
    member _.ToString() = ""
  }

// Runtime crash: TypeLoadException
let s = Struct(42)
let obj = s.Test()

Current Broken Behavior

The compiler generates IL equivalent to:

class Test@5 {
    ref Struct _;  // ← ILLEGAL! Classes cannot have byref fields
    
    Test@5(ref Struct s) {
        Unsafe.As<Struct, ref Struct>(ref this._) = ref s;
        base(s.test);
    }
}

This violates .NET CLI rules and crashes at runtime.

Root Cause

The compiler's object expression handling in src/Compiler/Checking/Expressions/CheckExpressions.fs treats the enclosing struct instance (baseValOpt) as a captured variable. Instead of extracting the specific values needed from the struct, it tries to capture a reference to the entire struct, which becomes an illegal byref field in the generated closure class.

Solution: Extract and Capture Values Instead of Struct Reference

At type-checking time (before closure conversion), detect when an object expression inside a struct would capture the struct instance, and instead:

1. Analyze what specific struct fields/parameters are actually used by the object expression methods
2. Extract those values into local variables before creating the object expression
3. Rewrite the object expression methods to use the local variables instead of accessing through this/baseValOpt
4. Eliminate baseValOpt since we no longer need the struct reference

This preserves F# struct copy-by-value semantics and generates legal IL.

Expected Behavior After Fix

The compiler should generate IL equivalent to:

class Test@5 {
    object test$captured;  // ← LEGAL! Just an object field
    
    Test@5(object test) {
        this.test$captured = test;
        base(test);
    }
}

Implementation Location

Primary File: src/Compiler/Checking/Expressions/CheckExpressions.fs

Target Function: The function that type-checks object expressions and constructs Expr.Obj (likely named TcObjectExpr or similar, typically around lines 3000-4500)

Implementation Steps

Step 1: Add Helper to Collect Captured Struct Members

Add a function that analyzes object expression methods to find which struct members are captured:

/// Analyzes object expression methods to determine what struct members are captured
let collectCapturedStructMembers (g: TcGlobals) (structVal: Val) (overrides: TObjExprMethod list) (iimpls: (TType * TObjExprMethod list) list) =
    // Get all override method bodies
    let allMethods = overrides @ (iimpls |> List.collect snd)
    let methodBodies = allMethods |> List.map (fun (TObjExprMethod(_, _, _, _, body, _)) -> body)
    
    // Find free variables in all method bodies
    let allFreeVars = 
        methodBodies 
        |> List.collect (fun body -> 
            let frees = freeInExpr (CollectLocalsWithStackGuard()) body
            frees.FreeLocals |> Zset.toList)
        |> List.distinct
    
    // Filter to variables that are fields/parameters of the struct
    allFreeVars
    |> List.filter (fun freeVar ->
        // Check if this variable originates from the struct's declaring entity
        match freeVar.DeclaringEntity with
        | Parent tcref when tyconRefEq g tcref structVal.DeclaringEntity -> true
        | _ -> false)

Step 2: Add Helper to Create Capture Bindings

Add a function that creates local bindings for captured struct members:

/// Creates local bindings to capture specific struct members by value
let createCaptureBindings (g: TcGlobals) (structVal: Val) (capturedMembers: Val list) m =
    capturedMembers
    |> List.map (fun memberVal ->
        // Create a fresh local variable
        let localId = mkSynId m (memberVal.LogicalName + "$captured")
        let localVal = mkLocal localId memberVal.Type
        
        // Create expression to read the member value from struct
        let readExpr = exprForValRef m (mkLocalValRef memberVal)
        
        // Return the local variable and its binding
        localVal, mkInvisibleBind localVal readExpr)
    |> List.unzip

Step 3: Add Helper to Rewrite Method Bodies

Add a function that rewrites object expression methods to use captured locals:

/// Rewrites object expression method bodies to use captured locals instead of struct members
let rewriteMethodsToUseCapturedLocals (capturedMembers: Val list)...

</details>

*This pull request was created as a result of the following prompt from Copilot chat.*
> 
> # Fix: Object expressions in struct types generate invalid IL with byref fields
> 
> ## Problem
> 
> When an object expression is created inside a struct member method and references values from the struct's constructor parameters or fields, the F# compiler generates invalid IL code. The generated closure class contains a `byref<Struct>` field, which is illegal according to CLI rules. This results in a `TypeLoadException` at runtime.
> 
> ### Repro Code (Currently Crashes)
> ```fsharp
> type Class(test : obj) = class end
> 
> [<Struct>]
> type Struct(test : obj) =
>   member _.Test() = {
>     new Class(test) with
>     member _.ToString() = ""
>   }
> 
> // Runtime crash: TypeLoadException
> let s = Struct(42)
> let obj = s.Test()
> ```
> 
> ### Current Broken Behavior
> 
> The compiler generates IL equivalent to:
> ```csharp
> class Test@5 {
>     ref Struct _;  // ← ILLEGAL! Classes cannot have byref fields
>     
>     Test@5(ref Struct s) {
>         Unsafe.As<Struct, ref Struct>(ref this._) = ref s;
>         base(s.test);
>     }
> }
> ```
> 
> This violates .NET CLI rules and crashes at runtime.
> 
> ## Root Cause
> 
> The compiler's object expression handling in `src/Compiler/Checking/Expressions/CheckExpressions.fs` treats the enclosing struct instance (`baseValOpt`) as a captured variable. Instead of extracting the specific values needed from the struct, it tries to capture a reference to the entire struct, which becomes an illegal byref field in the generated closure class.
> 
> ## Solution: Extract and Capture Values Instead of Struct Reference
> 
> At type-checking time (before closure conversion), detect when an object expression inside a struct would capture the struct instance, and instead:
> 
> 1. **Analyze** what specific struct fields/parameters are actually used by the object expression methods
> 2. **Extract** those values into local variables before creating the object expression
> 3. **Rewrite** the object expression methods to use the local variables instead of accessing through `this`/`baseValOpt`
> 4. **Eliminate** `baseValOpt` since we no longer need the struct reference
> 
> This preserves F# struct copy-by-value semantics and generates legal IL.
> 
> ### Expected Behavior After Fix
> 
> The compiler should generate IL equivalent to:
> ```csharp
> class Test@5 {
>     object test$captured;  // ← LEGAL! Just an object field
>     
>     Test@5(object test) {
>         this.test$captured = test;
>         base(test);
>     }
> }
> ```
> 
> ## Implementation Location
> 
> **Primary File**: `src/Compiler/Checking/Expressions/CheckExpressions.fs`
> 
> **Target Function**: The function that type-checks object expressions and constructs `Expr.Obj` (likely named `TcObjectExpr` or similar, typically around lines 3000-4500)
> 
> ## Implementation Steps
> 
> ### Step 1: Add Helper to Collect Captured Struct Members
> 
> Add a function that analyzes object expression methods to find which struct members are captured:
> 
> ```fsharp
> /// Analyzes object expression methods to determine what struct members are captured
> let collectCapturedStructMembers (g: TcGlobals) (structVal: Val) (overrides: TObjExprMethod list) (iimpls: (TType * TObjExprMethod list) list) =
>     // Get all override method bodies
>     let allMethods = overrides @ (iimpls |> List.collect snd)
>     let methodBodies = allMethods |> List.map (fun (TObjExprMethod(_, _, _, _, body, _)) -> body)
>     
>     // Find free variables in all method bodies
>     let allFreeVars = 
>         methodBodies 
>         |> List.collect (fun body -> 
>             let frees = freeInExpr (CollectLocalsWithStackGuard()) body
>             frees.FreeLocals |> Zset.toList)
>         |> List.distinct
>     
>     // Filter to variables that are fields/parameters of the struct
>     allFreeVars
>     |> List.filter (fun freeVar ->
>         // Check if this variable originates from the struct's declaring entity
>         match freeVar.DeclaringEntity with
>         | Parent tcref when tyconRefEq g tcref structVal.DeclaringEntity -> true
>         | _ -> false)
> ```
> 
> ### Step 2: Add Helper to Create Capture Bindings
> 
> Add a function that creates local bindings for captured struct members:
> 
> ```fsharp
> /// Creates local bindings to capture specific struct members by value
> let createCaptureBindings (g: TcGlobals) (structVal: Val) (capturedMembers: Val list) m =
>     capturedMembers
>     |> List.map (fun memberVal ->
>         // Create a fresh local variable
>         let localId = mkSynId m (memberVal.LogicalName + "$captured")
>         let localVal = mkLocal localId memberVal.Type
>         
>         // Create expression to read the member value from struct
>         let readExpr = exprForValRef m (mkLocalValRef memberVal)
>         
>         // Return the local variable and its binding
>         localVal, mkInvisibleBind localVal readExpr)
>     |> List.unzip
> ```
> 
> ### Step 3: Add Helper to Rewrite Method Bodies
> 
> Add a function that rewrites object expression methods to use captured locals:
> 
> ```fsharp
> /// Rewrites object expression method bodies to use captured locals instead of struct members
> let rewriteMethodsToUseCapturedLocals (capturedMembers: Val list) (captureLocals: Val list) (methods: TObjExprMethod list) =
>     // Build substitution map: struct member -> local variable
>     let substMap = 
>         (capturedMembers, captureLocals)
>         ||> List.zip
>         |> List.fold (fun acc (oldVal, newVal) ->
>             ValMap.add oldVal (mkLocalValRef newVal) acc) ValMap.empty
>     
>     // Rewrite each method
>     methods |> List.map (fun (TObjExprMethod(slotsig, attribs, tps, vs, body, methodRange)) ->
>         let newBody = remarkExpr methodRange (remapValRefsInExpr substMap body)
>         TObjExprMethod(slotsig, attribs, tps, vs, newBody, methodRange))
> ```
> 
> ### Step 4: Modify Object Expression Type-Checking
> 
> In the main `TcObjectExpr` function (or wherever `Expr.Obj` is constructed), add logic to handle struct captures:
> 
> ```fsharp
> // Find where baseValOpt, overrides, iimpls are created, then add:
> 
> // Check if we need to extract captures from a struct
> let baseValOpt, prependBindings, overridesRewritten, iimplsRewritten = 
>     match baseValOpt with
>     | Some baseVal when isStructTy cenv.g baseVal.Type ->
>         // Analyze what members are captured
>         let capturedMembers = 
>             collectCapturedStructMembers cenv.g baseVal overrides iimpls
>         
>         if List.isEmpty capturedMembers then
>             // No captures - object expression doesn't use anything from struct
>             None, [], overrides, iimpls
>         else
>             // Create local bindings for each captured member
>             let captureLocals, captureBindings = 
>                 createCaptureBindings cenv.g baseVal capturedMembers m
>             
>             // Rewrite override methods to use the local variables
>             let overridesRewritten = 
>                 rewriteMethodsToUseCapturedLocals capturedMembers captureLocals overrides
>             
>             let iimplsRewritten = 
>                 iimpls |> List.map (fun (ty, methods) -> 
>                     ty, rewriteMethodsToUseCapturedLocals capturedMembers captureLocals methods)
>             
>             // Return: no baseVal, but we have bindings to prepend
>             None, captureBindings, overridesRewritten, iimplsRewritten
>     
>     | _ -> 
>         // Not a struct or no baseValOpt - use existing logic
>         baseValOpt, [], overrides, iimpls
> 
> // Create the object expression with rewritten methods
> let objExpr = 
>     mkObjExpr(overallTy, baseValOpt, basecall, overridesRewritten, iimplsRewritten, m)
> 
> // Wrap with capture bindings if any
> let finalExpr =
>     if List.isEmpty prependBindings then
>         objExpr
>     else
>         mkLetsBind m prependBindings objExpr
> ```
> 
> ## Test Cases to Add
> 
> ### Test 1: Basic Capture (Original Repro) - Runtime Test
> File: `tests/FSharp.Compiler.ComponentTests/Conformance/Expressions/ObjectExpressions/ObjectExpressionsInStructs.fs`
> 
> ```fsharp
> [<Fact>]
> let ``Object expression in struct capturing constructor parameter should work`` () =
>     Fsx """
> type Class(test : obj) = class end
> 
> [<Struct>]
> type Struct(test : obj) =
>   member _.Test() = {
>     new Class(test) with
>     member _.ToString() = ""
>   }
> 
> let s = Struct(42)
> let obj = s.Test()
> printfn "%s" (obj.ToString())
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
> ```
> 
> ### Test 2: Multiple Captures - Runtime Test
> ```fsharp
> [<Fact>]
> let ``Object expression in struct capturing multiple fields`` () =
>     Fsx """
> [<Struct>]
> type Struct(a: int, b: string) =
>   member _.Test() = {
>     new Object() with
>     member _.ToString() = sprintf "%d: %s" a b
>   }
> 
> let s = Struct(42, "hello")
> let obj = s.Test()
> if obj.ToString() <> "42: hello" then failwith "Failed"
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
> ```
> 
> ### Test 3: No Captures - Regression Test
> ```fsharp
> [<Fact>]
> let ``Object expression in struct with no captures should not regress`` () =
>     Fsx """
> [<Struct>]
> type Struct(test : obj) =
>   member _.Test() = {
>     new Object() with
>     member _.ToString() = "constant"
>   }
> 
> let s = Struct(42)
> let obj = s.Test()
> if obj.ToString() <> "constant" then failwith "Failed"
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
> ```
> 
> ### Test 4: Copy Semantics - Runtime Test
> ```fsharp
> [<Fact>]
> let ``Object expression captures struct values at creation time (copy semantics)`` () =
>     Fsx """
> [<Struct>]
> type MutableStruct() =
>     [<DefaultValue>] val mutable x : int
>     member this.CreateObj() = {
>         new Object() with
>         member _.Get() = this.x
>     }
> 
> let mutable s = MutableStruct()
> s.x <- 10
> let obj = s.CreateObj()
> s.x <- 99
> // Should see the value at capture time (10), not after mutation (99)
> if (obj :?> obj).GetType().GetMethod("Get").Invoke(obj, [||]) :?> int <> 10 then 
>     failwith "Expected 10"
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
> ```
> 
> ### Test 5: EmittedIL Verification Test
> File: `tests/FSharp.Compiler.ComponentTests/EmittedIL/Structure/ObjectExpressionsInStructs.fs`
> 
> ```fsharp
> [<Theory; Directory(__SOURCE_DIRECTORY__, Includes=[|"ObjectExprInStruct.fs"|])>]
> let ``Verify no byref fields in closure for struct object expression`` compilation =
>     compilation
>     |> asFs
>     |> withOptions ["--optimize-"]
>     |> compile
>     |> shouldSucceed
>     |> verifyIL [
>         // Verify the closure class doesn't have byref fields
>         ".class private auto ansi sealed nested assembly beforefieldinit"
>         ".field public object test\\$captured"  // Should be object, not byref
>     ]
> ```
> 
> With test file `ObjectExprInStruct.fs`:
> ```fsharp
> type Class(test : obj) = class end
> 
> [<Struct>]
> type Struct(test : obj) =
>   member _.Test() = {
>     new Class(test) with
>     member _.ToString() = ""
>   }
> ```
> 
> ### Test 6: Interface Implementation
> ```fsharp
> [<Fact>]
> let ``Object expression in struct implementing interface`` () =
>     Fsx """
> [<Struct>]
> type Struct(value: int) =
>   member _.CreateDisposable() = {
>     new System.IDisposable with
>     member _.Dispose() = printfn "Disposed: %d" value
>   }
> 
> let s = Struct(42)
> let disp = s.CreateDisposable()
> disp.Dispose()
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
>     |> withStdOutContains "Disposed: 42"
> ```
> 
> ### Test 7: Nested Struct Access
> ```fsharp
> [<Fact>]
> let ``Object expression in struct with nested struct field`` () =
>     Fsx """
> [<Struct>]
> type Inner(x: int) =
>     member _.X = x
> 
> [<Struct>]
> type Outer(inner: Inner) =
>     member _.Test() = {
>         new Object() with
>         member _.ToString() = inner.X.ToString()
>     }
> 
> let s = Outer(Inner(42))
> let obj = s.Test()
> if obj.ToString() <> "42" then failwith "Failed"
>     """
>     |> compileExeAndRun
>     |> shouldSucceed
> ```
> 
> ## Success Criteria
> 
> 1. ✅ Original repro code compiles without errors
> 2. ✅ Generated IL contains no byref fields in closure classes
> 3. ✅ Code runs without `TypeLoadException`
> 4. ✅ Struct copy-by-value semantics are preserved
> 5. ✅ No regressions in existing compiler tests
> 6. ✅ All new test cases pass
> 
> ## Additional Context
> 
> - Related Issue: https://github.com/dotnet/fsharp/issues/19068
> - Risk: Low - all affected code currently crashes, no working code can regress
> - Priority: High - runtime crash preventing valid F# patterns
> 

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.