Optimize graph traversal by using a more appropriate algorithm

The previous implementation could visit nodes multiple times when a
graph had diamond like patterns. The new implementation uses [Kahn's
algorithm](https://en.wikipedia.org/wiki/Topological_sorting#Kahn's_algorithm)
which ensures that we only ever visit nodes once.

I tested this on firefox-main with `./mach taskgraph full -J -p
taskcluster/test/params/mc-onpush.yml`. Before this change, with a
profiler attached, it would take ~3mn55. After this change, we're down
to ~2mn30. Without profiling we go from 22s to 17s.

I added a `links_and_reverse_links_dict` helper here since computing
them both at the same time is faster than iterating over the graph
twice. In my testing the function was called 11 times, 10 of which were
on the exact same graph. Because that function is very expensive to
call, I chose to cache it.

The caching itself resulted in ~5-6s of time save when profiled on the
full graph for firefox and ~1s when not profiled.

Hyperfine output, full firefox graph, no profiler attached:

```
Benchmark 1: baseline
  Time (mean ± σ):     22.565 s ±  0.260 s    [User: 53.205 s, System: 5.034 s]
  Range (min … max):   22.255 s … 23.043 s    10 runs

Benchmark 2: with-patch
  Time (mean ± σ):     19.032 s ±  0.110 s    [User: 49.556 s, System: 4.829 s]
  Range (min … max):   18.856 s … 19.195 s    10 runs

Benchmark 3: with-patch-cache
  Time (mean ± σ):     17.800 s ±  0.250 s    [User: 48.605 s, System: 4.893 s]
  Range (min … max):   17.281 s … 18.159 s    10 runs

Summary
  with-patch-cache ran
    1.07 ± 0.02 times faster than with-patch
    1.27 ± 0.02 times faster than baseline
```

Author: Eijebong

src/taskgraph/graph.py

@@ -4,6 +4,7 @@
 
 
 import collections
+import functools
 from dataclasses import dataclass
 
 
@@ -75,20 +76,25 @@ def transitive_closure(self, nodes, reverse=False):
         return Graph(new_nodes, new_edges)
 
     def _visit(self, reverse):
-        queue = collections.deque(sorted(self.nodes))
-        links_by_node = self.reverse_links_dict() if reverse else self.links_dict()
-        seen = set()
+        forward_links, reverse_links = self.links_and_reverse_links_dict()
+
+        dependencies = reverse_links if reverse else forward_links
+        dependents = forward_links if reverse else reverse_links
+
+        indegree = {node: len(dependencies[node]) for node in self.nodes}
+
+        queue = collections.deque(
+            node for node, degree in indegree.items() if degree == 0
+        )
+
         while queue:
             node = queue.popleft()
-            if node in seen:
-                continue
-            links = links_by_node[node]
-            if all((n in seen) for n in links):
-                seen.add(node)
-                yield node
-            else:
-                queue.extend(n for n in links if n not in seen)
-                queue.append(node)
+            yield node
+
+            for dependent in dependents[node]:
+                indegree[dependent] -= 1
+                if indegree[dependent] == 0:
+                    queue.append(dependent)
 
     def visit_postorder(self):
         """
@@ -107,6 +113,21 @@ def visit_preorder(self):
         """
         return self._visit(True)
 
+    @functools.cache
+    def links_and_reverse_links_dict(self):
+        """
+        Return both links and reverse_links dictionaries.
+        Returns a (forward_links, reverse_links) tuple where forward_links maps
+        each node to the set of nodes it links to, and reverse_links maps each
+        node to the set of nodes linking to it.
+        """
+        forward = collections.defaultdict(set)
+        reverse = collections.defaultdict(set)
+        for left, right, _ in self.edges:
+            forward[left].add(right)
+            reverse[right].add(left)
+        return (forward, reverse)
+
     def links_dict(self):
         """
         Return a dictionary mapping each node to a set of the nodes it links to