Fix thread-safety: snapshot TieLineCollection before enumeration in GetRouteToSource

[Copilot]

Concurrent NVX tie line registration modifies TieLineCollection.Default (List<TieLine>) while GetRouteToSource lazily enumerates it via LINQ .Where(), causing InvalidOperationException: Collection was modified; enumeration operation may not execute during room combiner scenario activation.

Change

• Extensions.cs — GetRouteToSource (private recursive overload): Take an eager snapshot of TieLineCollection.Default at the top of each call via .ToList(). All downstream Where() / FirstOrDefault() queries operate on the stable copy.

// Before — lazy query holds a live reference; concurrent Add() throws
destinationTieLines = TieLineCollection.Default.Where(t => ...);

// After — snapshot isolates enumeration from concurrent writes
var tieLines = TieLineCollection.Default.ToList();
destinationTieLines = tieLines.Where(t => ...);

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• crl.entrust.net
  • Triggering command: /usr/bin/dotnet dotnet build src/PepperDash.Essentials.Core/PepperDash.Essentials.Core.csproj (dns block)
  • Triggering command: /usr/bin/dotnet dotnet restore --no-dependencies /home/REDACTED/work/Essentials/Essentials/PepperDash.Essentials.4Series.sln --packages /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/packages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal /p:TargetFrameworkRootPath=/tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/emptyFakeDotnetRoot /p:NetCoreTargetingPackRoot=/tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/emptyFakeDotnetRoot /p:AllowMissingPrunePackageData=true (dns block)
  • Triggering command: /usr/bin/dotnet dotnet restore --no-dependencies /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/8E3C38744AF97AEA71D98A0334D7EC97/missingpackages_workingdir --packages /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/missingpackages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal --configfile /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/nugetconfig/nuget.config --force (dns block)
• ocsp.entrust.net
  • Triggering command: /usr/bin/dotnet dotnet build src/PepperDash.Essentials.Core/PepperDash.Essentials.Core.csproj (dns block)
  • Triggering command: /usr/bin/dotnet dotnet restore --no-dependencies /home/REDACTED/work/Essentials/Essentials/PepperDash.Essentials.4Series.sln --packages /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/packages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal /p:TargetFrameworkRootPath=/tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/emptyFakeDotnetRoot /p:NetCoreTargetingPackRoot=/tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/emptyFakeDotnetRoot /p:AllowMissingPrunePackageData=true (dns block)
  • Triggering command: /usr/bin/dotnet dotnet restore --no-dependencies /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/8E3C38744AF97AEA71D98A0334D7EC97/missingpackages_workingdir --packages /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/missingpackages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal --configfile /tmp/codeql-scratch-34409fe8f58e31ec/dbs/csharp/working/nugetconfig/nuget.config --force (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[BUG]-Thread-safety issue in GetRouteToSource - Collection modified during enumeration</issue_title>
<issue_description>## Description

When multiple route requests or NVX tie line registrations occur concurrently (e.g., during room combiner scenario activation), a System.InvalidOperationException is thrown because the routing graph's internal collection is modified while being enumerated.

Environment

• Essentials Version: 2.24.2
• Platform: Crestron 4-Series
• Trigger: Room combiner scenario with multiple rooms activating simultaneously while NVX endpoints register tie lines

Stack Trace

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.Collections.Generic.List1+Enumerator[T].MoveNextRare()    at System.Collections.Generic.List1+Enumerator[T].MoveNext()
at System.Linq.Enumerable+WhereListIterator1[TSource].MoveNext()    at PepperDash.Essentials.Core.Extensions.GetRouteToSource(        IRoutingInputs destination,         IRoutingOutputs source,         RoutingOutputPort outputPortToUse,         List1 alreadyCheckedDevices,
eRoutingSignalType signalType,
Int32 cycle,
RouteDescriptor routeTable,
RoutingInputPort destinationPort,
RoutingOutputPort sourcePort)
at PepperDash.Essentials.Core.Extensions.GetRouteToSource(...)
at PepperDash.Essentials.Core.Extensions.RunRouteRequest(RouteRequest request)

Log Context

Timestamps show concurrent operations causing the collision:

[14:03:26.189][EROR] Exception Running Route Request Route codec-b:auto to display-b1:hdmiIn1
[14:03:26.222][VERB] Adding rx Tie line: NvxRouter-PrimaryStream:display-d1-nvx-rx-StreamOutput --> display-d1-nvx-rx:Stream AudioVideo
[14:03:26.224][VERB] Generating rx tieLine for codec-a-camera1-nvx-rx

Steps to Reproduce

1. Configure a room combiner with 3+ rooms (including NVX-based routing)
2. Configure scenario activationActions that call Activate() on multiple rooms
3. Trigger a partition state change that activates the scenario
4. Observe exception when concurrent route requests overlap with NVX tie line registration

Expected Behavior

Route requests should complete without exception regardless of concurrent operations.

Suggested Fix

The collection being enumerated in GetRouteToSource (likely TieLines or OutputPorts) should either:

1. Use a thread-safe collection (ConcurrentBag<T> or ConcurrentDictionary<K,V>)

2. Create a snapshot before enumeration:

   // Instead of:
   foreach (var item in collection.Where(x => condition))
   
   // Use:
   foreach (var item in collection.ToList().Where(x => condition))

3. Add locking around routing operations in RunRouteRequest and tie line registration
   </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [BUG]-Thread-safety issue in GetRouteToSource - Collection modified during enumeration #1400

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[Copilot]

Pull request overview

This PR aims to prevent InvalidOperationException caused by concurrent modifications to TieLineCollection.Default while GetRouteToSource is enumerating tie lines during route discovery.

Changes:

• Snapshot TieLineCollection.Default into a local list via ToList() inside the recursive GetRouteToSource implementation.
• Update downstream LINQ queries to operate on the snapshot rather than the live global collection.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

TieLineCollection.Default.ToList() still enumerates the underlying List<TieLine> and can throw InvalidOperationException if another thread modifies the collection during the snapshot. To fully address this, reads and writes of TieLineCollection.Default need synchronization (shared lock, or a thread-safe/copy-on-write collection), ideally via a safe snapshot method on TieLineCollection. Also, because this method is recursive, taking a new ToList() snapshot on every recursion can add significant allocation/CPU overhead; consider taking one snapshot in the public entry point and passing it down through the recursive calls.

Suggested change

	var tieLines = TieLineCollection.Default.ToList();
	List<TieLine> tieLines;
	lock (TieLineCollection.Default)
	{
	tieLines = TieLineCollection.Default.ToList();
	}