Skip to content

Commit 8174fb5

Browse files
authored
Merge pull request #705 from asger-semmle/loop-index-concurrent-modification
Approved by mc-semmle, xiemaisi
2 parents 6c76826 + f24313a commit 8174fb5

File tree

10 files changed

+383
-0
lines changed

10 files changed

+383
-0
lines changed

change-notes/1.20/analysis-javascript.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
| Incomplete regular expression for hostnames (`js/incomplete-hostname-regexp`) | correctness, security, external/cwe/cwe-020 | Highlights hostname sanitizers that are likely to be incomplete, indicating a violation of [CWE-020](https://cwe.mitre.org/data/definitions/20.html). Results are shown on LGTM by default.|
1919
| Incomplete URL substring sanitization | correctness, security, external/cwe/cwe-020 | Highlights URL sanitizers that are likely to be incomplete, indicating a violation of [CWE-020](https://cwe.mitre.org/data/definitions/20.html). Results shown on LGTM by default. |
2020
| Incorrect suffix check (`js/incorrect-suffix-check`) | correctness, security, external/cwe/cwe-020 | Highlights error-prone suffix checks based on `indexOf`, indicating a potential violation of [CWE-20](https://cwe.mitre.org/data/definitions/20.html). Results are shown on LGTM by default. |
21+
| Loop iteration skipped due to shifting (`js/loop-iteration-skipped-due-to-shifting`) | correctness | Highlights code that removes an element from an array while iterating over it, causing the loop to skip over some elements. Results are shown on LGTM by default. |
2122
| Useless comparison test (`js/useless-comparison-test`) | correctness | Highlights code that is unreachable due to a numeric comparison that is always true or always false. Results are shown on LGTM by default. |
2223

2324
## Changes to existing queries

javascript/config/suites/javascript/correctness-more

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
+ semmlecode-javascript-queries/LanguageFeatures/IllegalInvocation.ql: /Correctness/Language Features
1212
+ semmlecode-javascript-queries/LanguageFeatures/InconsistentNew.ql: /Correctness/Language Features
1313
+ semmlecode-javascript-queries/LanguageFeatures/SpuriousArguments.ql: /Correctness/Language Features
14+
+ semmlecode-javascript-queries/Statements/LoopIterationSkippedDueToShifting.ql: /Correctness/Statements
1415
+ semmlecode-javascript-queries/Statements/MisleadingIndentationAfterControlStmt.ql: /Correctness/Statements
1516
+ semmlecode-javascript-queries/Statements/ReturnOutsideFunction.ql: /Correctness/Statements
1617
+ semmlecode-javascript-queries/Statements/SuspiciousUnusedLoopIterationVariable.ql: /Correctness/Statements
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
<!DOCTYPE qhelp PUBLIC
2+
"-//Semmle//qhelp//EN"
3+
"qhelp.dtd">
4+
<qhelp>
5+
<overview>
6+
<p>
7+
Items can be removed from an array using the <code>splice</code> method, but when doing so,
8+
all subsequent items will be shifted to a lower index. If this is done while iterating over
9+
the array, the shifting may cause the loop to skip over the element immediately after the
10+
removed element.
11+
</p>
12+
13+
</overview>
14+
<recommendation>
15+
16+
<p>
17+
Determine what the loop is supposed to do:
18+
</p>
19+
20+
<ul>
21+
<li>
22+
If the intention is to remove <em>every occurrence</em> of a certain value, decrement the loop counter after removing an element, to counterbalance
23+
the shift.
24+
</li>
25+
<li>
26+
If the loop is only intended to remove <em>a single value</em> from the array, consider adding a <code>break</code> after the <code>splice</code> call.
27+
</li>
28+
<li>
29+
If the loop is deliberately skipping over elements, consider moving the index increment into the body of the loop,
30+
so it is clear that the loop is not a trivial array iteration loop.
31+
</li>
32+
</ul>
33+
34+
</recommendation>
35+
<example>
36+
37+
<p>
38+
In this example, a function is intended to remove "<code>..</code>" parts from a path:
39+
</p>
40+
41+
<sample src="examples/LoopIterationSkippedDueToShifting.js" />
42+
43+
<p>
44+
However, whenever the input contain two "<code>..</code>" parts right after one another, only the first will be removed.
45+
For example, the string "<code>../../secret.txt</code>" will be mapped to "<code>../secret.txt</code>". After removing
46+
the element at index 0, the loop counter is incremented to 1, but the second "<code>..</code>" string has now been shifted down to
47+
index 0 and will therefore be skipped.
48+
</p>
49+
50+
<p>
51+
One way to avoid this is to decrement the loop counter after removing an element from the array:
52+
</p>
53+
54+
<sample src="examples/LoopIterationSkippedDueToShiftingGood.js" />
55+
56+
<p>
57+
Alternatively, use the <code>filter</code> method:
58+
</p>
59+
60+
<sample src="examples/LoopIterationSkippedDueToShiftingGoodFilter.js" />
61+
62+
</example>
63+
<references>
64+
65+
<li>MDN: <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/splice">Array.prototype.splice()</a>.</li>
66+
<li>MDN: <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter">Array.prototype.filter()</a>.</li>
67+
68+
</references>
69+
</qhelp>
Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
/**
2+
* @name Loop iteration skipped due to shifting
3+
* @description Removing elements from an array while iterating over it can cause the loop to skip over some elements,
4+
* unless the loop index is decremented accordingly.
5+
* @kind problem
6+
* @problem.severity warning
7+
* @id js/loop-iteration-skipped-due-to-shifting
8+
* @tags correctness
9+
* @precision high
10+
*/
11+
import javascript
12+
13+
/**
14+
* An operation that inserts or removes elements from an array while shifting all elements
15+
* occuring after the insertion/removal point.
16+
*
17+
* Does not include `push` and `pop` since these never shift any elements.
18+
*/
19+
class ArrayShiftingCall extends DataFlow::MethodCallNode {
20+
string name;
21+
22+
ArrayShiftingCall() {
23+
name = getMethodName() and
24+
(name = "splice" or name = "shift" or name = "unshift")
25+
}
26+
27+
DataFlow::SourceNode getArray() {
28+
result = getReceiver().getALocalSource()
29+
}
30+
}
31+
32+
/**
33+
* A call to `splice` on an array.
34+
*/
35+
class SpliceCall extends ArrayShiftingCall {
36+
SpliceCall() {
37+
name = "splice"
38+
}
39+
40+
/**
41+
* Gets the index from which elements are removed and possibly new elemenst are inserted.
42+
*/
43+
DataFlow::Node getIndex() {
44+
result = getArgument(0)
45+
}
46+
47+
/**
48+
* Gets the number of removed elements.
49+
*/
50+
int getNumRemovedElements() {
51+
result = getArgument(1).asExpr().getIntValue() and
52+
result >= 0
53+
}
54+
55+
/**
56+
* Gets the number of inserted elements.
57+
*/
58+
int getNumInsertedElements() {
59+
result = getNumArgument() - 2 and
60+
result >= 0
61+
}
62+
}
63+
64+
/**
65+
* A `for` loop iterating over the indices of an array, in increasing order.
66+
*/
67+
class ArrayIterationLoop extends ForStmt {
68+
DataFlow::SourceNode array;
69+
LocalVariable indexVariable;
70+
71+
ArrayIterationLoop() {
72+
exists (RelationalComparison compare | compare = getTest() |
73+
compare.getLesserOperand() = indexVariable.getAnAccess() and
74+
compare.getGreaterOperand() = array.getAPropertyRead("length").asExpr()) and
75+
getUpdate().(IncExpr).getOperand() = indexVariable.getAnAccess()
76+
}
77+
78+
/**
79+
* Gets the variable holding the loop variable and current array index.
80+
*/
81+
LocalVariable getIndexVariable() {
82+
result = indexVariable
83+
}
84+
85+
/**
86+
* Gets the loop entry point.
87+
*/
88+
ReachableBasicBlock getLoopEntry() {
89+
result = getTest().getFirstControlFlowNode().getBasicBlock()
90+
}
91+
92+
/**
93+
* Gets a call that potentially shifts the elements of the given array.
94+
*/
95+
ArrayShiftingCall getAnArrayShiftingCall() {
96+
result.getArray() = array
97+
}
98+
99+
/**
100+
* Gets a call to `splice` that removes elements from the looped-over array at the current index
101+
*
102+
* The `splice` call is not guaranteed to be inside the loop body.
103+
*/
104+
SpliceCall getACandidateSpliceCall() {
105+
result = getAnArrayShiftingCall() and
106+
result.getIndex().asExpr() = getIndexVariable().getAnAccess() and
107+
result.getNumRemovedElements() > result.getNumInsertedElements()
108+
}
109+
110+
/**
111+
* Holds if `cfg` modifies the index variable or shifts array elements, disturbing the
112+
* relationship between the array and the index variable.
113+
*/
114+
predicate hasIndexingManipulation(ControlFlowNode cfg) {
115+
cfg.(VarDef).getAVariable() = getIndexVariable() or
116+
cfg = getAnArrayShiftingCall().asExpr()
117+
}
118+
119+
/**
120+
* Holds if there is a `loop entry -> cfg` path that does not involve index manipulation or a successful index equality check.
121+
*/
122+
predicate hasPathTo(ControlFlowNode cfg) {
123+
exists(getACandidateSpliceCall()) and // restrict size of predicate
124+
cfg = getLoopEntry().getFirstNode()
125+
or
126+
hasPathTo(cfg.getAPredecessor()) and
127+
getLoopEntry().dominates(cfg.getBasicBlock()) and
128+
not hasIndexingManipulation(cfg) and
129+
130+
// Ignore splice calls guarded by an index equality check.
131+
// This indicates that the index of an element is the basis for removal, not its value,
132+
// which means it may be okay to skip over elements.
133+
not exists (ConditionGuardNode guard, EqualityTest test | cfg = guard |
134+
test = guard.getTest() and
135+
test.getAnOperand() = getIndexVariable().getAnAccess() and
136+
guard.getOutcome() = test.getPolarity()) and
137+
138+
// Block flow after inspecting an array element other than that at the current index.
139+
// For example, if the splice happens after inspecting `array[i + 1]`, then the next
140+
// element has already been "looked at" and so it doesn't matter if we skip it.
141+
not exists (IndexExpr index | cfg = index |
142+
array.flowsToExpr(index.getBase()) and
143+
not index.getIndex() = getIndexVariable().getAnAccess())
144+
}
145+
146+
/**
147+
* Holds if there is a `loop entry -> splice -> cfg` path that does not involve index manipulation,
148+
* other than the `splice` call.
149+
*/
150+
predicate hasPathThrough(SpliceCall splice, ControlFlowNode cfg) {
151+
splice = getACandidateSpliceCall() and
152+
cfg = splice.asExpr() and
153+
hasPathTo(cfg.getAPredecessor())
154+
or
155+
hasPathThrough(splice, cfg.getAPredecessor()) and
156+
getLoopEntry().dominates(cfg.getBasicBlock()) and
157+
not hasIndexingManipulation(cfg)
158+
}
159+
}
160+
161+
from ArrayIterationLoop loop, SpliceCall splice
162+
where loop.hasPathThrough(splice, loop.getUpdate().getFirstControlFlowNode())
163+
select splice, "Removing an array item without adjusting the loop index '" + loop.getIndexVariable().getName() + "' causes the subsequent array item to be skipped."
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
function removePathTraversal(path) {
2+
let parts = path.split('/');
3+
for (let i = 0; i < parts.length; ++i) {
4+
if (parts[i] === '..') {
5+
parts.splice(i, 1);
6+
}
7+
}
8+
return path.join('/');
9+
}
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function removePathTraversal(path) {
2+
let parts = path.split('/');
3+
for (let i = 0; i < parts.length; ++i) {
4+
if (parts[i] === '..') {
5+
parts.splice(i, 1);
6+
--i; // adjust for array shift
7+
}
8+
}
9+
return path.join('/');
10+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
function removePathTraversal(path) {
2+
return path.split('/').filter(part => part !== '..').join('/');
3+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
| tst.js:4:27:4:44 | parts.splice(i, 1) | Removing an array item without adjusting the loop index 'i' causes the subsequent array item to be skipped. |
2+
| tst.js:13:29:13:46 | parts.splice(i, 1) | Removing an array item without adjusting the loop index 'i' causes the subsequent array item to be skipped. |
3+
| tst.js:24:9:24:26 | parts.splice(i, 1) | Removing an array item without adjusting the loop index 'i' causes the subsequent array item to be skipped. |
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Statements/LoopIterationSkippedDueToShifting.ql

0 commit comments

Comments
 (0)