Skip to content

Commit f6fe8d5

Browse files
author
Felicity Chapman
committed
Text updates for consistency and clarity
1 parent 595e6fc commit f6fe8d5

File tree

1 file changed

+27
-35
lines changed

1 file changed

+27
-35
lines changed
Lines changed: 27 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# Improvements to Python analysis
22

3-
43
## General improvements
54

6-
> Changes that affect alerts in many files or from many queries
7-
> For example, changes to file classification
5+
A new predicate `Stmt.getAnEntryNode()` has been added to make it easier to write reachability queries involving statements.
86

97
### Representation of the control flow graph
108

119
The representation of the control flow graph (CFG) has been modified to better reflect the semantics of Python.
1210

11+
#### CFG nodes removed
12+
1313
The following statement types no longer have a CFG node for the statement itself, as their sub-expressions already contain all the
1414
semantically significant information:
1515

@@ -20,42 +20,36 @@ semantically significant information:
2020

2121
For example, the CFG for `if cond: foo else bar` now starts with the CFG node for `cond`.
2222

23-
For the following statement types, the CFG node for the statement now follows the CFG nodes of its sub-expressions to better reflect the semantics:
23+
#### CFG nodes reordered
24+
25+
For the following statement types, the CFG node for the statement now follows the CFG nodes of its sub-expressions to follow Python semantics:
2426

2527
* `Print`
2628
* `TemplateWrite`
2729
* `ImportStar`
2830

29-
For example the CFG for `print foo` (in Python 2) has changed from `print -> foo` to `foo -> print`, better reflecting the runtime behavior.
30-
31+
For example the CFG for `print foo` (in Python 2) has changed from `print -> foo` to `foo -> print`, to reflect the runtime behavior.
3132

3233
The CFG for the `with` statement has been re-ordered to more closely reflect the semantics.
3334
For the `with` statement:
3435
```python
3536
with cm as var:
3637
body
3738
```
38-
The order of the CFG changes from:
3939

40-
<with>
41-
cm
42-
var
43-
body
44-
45-
to:
46-
47-
cm
48-
<with>
49-
var
50-
body
51-
52-
A new predicate `Stmt.getAnEntryNode()` has been added to make it easier to write reachability queries involving statements.
40+
| Old CFG node order | New CFG node order |
41+
|--------------------|--------------------|
42+
| `<with>` | `cm` |
43+
| `cm` | `<with>` |
44+
| `var` | `var` |
45+
| `body` | `body` |
5346

5447

5548
## New queries
5649

5750
| **Query** | **Tags** | **Purpose** |
5851
|-----------------------------|-----------|--------------------------------------------------------------------|
52+
| Assert statement tests the truth value of a literal constant (`py/assert-literal-constant`) | reliability, correctness | Checks whether an assert statement is testing the truth of a literal constant value. Results are hidden on LGTM by default. |
5953
| Flask app is run in debug mode (`py/flask-debug`) | security, external/cwe/cwe-215, external/cwe/cwe-489 | Finds instances where a Flask application is run in debug mode. Results are shown on LGTM by default. |
6054
| Information exposure through an exception (`py/stack-trace-exposure`) | security, external/cwe/cwe-209, external/cwe/cwe-497 | Finds instances where information about an exception may be leaked to an external user. Results are shown on LGTM by default. |
6155
| Jinja2 templating with autoescape=False (`py/jinja2/autoescape-false`) | security, external/cwe/cwe-079 | Finds instantiations of `jinja2.Environment` with `autoescape=False` which may allow XSS attacks. Results are hidden on LGTM by default. |
@@ -65,35 +59,33 @@ A new predicate `Stmt.getAnEntryNode()` has been added to make it easier to writ
6559
## Changes to existing queries
6660

6761
All taint-tracking queries now support visualization of paths in QL for Eclipse.
68-
Most security alerts are now visible on LGTM by default.
62+
Most security alerts are now visible on LGTM by default. This means that you may see results that were previously hidden for the following queries:
63+
64+
* Code injection (`py/code-injection`)
65+
* Reflected server-side cross-site scripting (`py/reflective-xss`)
66+
* SQL query built from user-controlled sources (`py/sql-injection`)
67+
* Uncontrolled data used in path expression (`py/path-injection`)
68+
* Uncontrolled command line (`py/command-line-injection`)
6969

7070
| **Query** | **Expected impact** | **Change** |
7171
|----------------------------|------------------------|------------------------------------------------------------------|
72-
| Assert statement tests the truth value of a literal constant (`py/assert-literal-constant`) | reliability, correctness | Checks whether an assert statement is testing the truth of a literal constant value. Not shown by default. |
73-
| Code injection (`py/code-injection`) | Supports path visualization and is now visible on LGTM by default | No change to expected results |
74-
| Command injection (`py/command-line-injection`) | Additional sinks in the `os`, and `popen` modules | Possibility of new results |
75-
| Deserializing untrusted input (`py/unsafe-deserialization`) | Supports path visualization | No change to expected results |
76-
| Encoding error (`py/encoding-error`) | Better alert location | Alert is now shown at the position of the first offending character, rather than at the top of the file. |
72+
| Command injection (`py/command-line-injection`) | More results | Additional sinks in the `os`, and `popen` modules may find more results in some projects. |
73+
| Encoding error (`py/encoding-error`) | Better alert location | Alerts are now shown at the start of the encoding error, rather than at the top of the file. |
7774
| Missing call to \_\_init\_\_ during object initialization (`py/missing-call-to-init`) | Fewer false positive results | Results where it is likely that the full call chain has not been analyzed are no longer reported. |
78-
| Reflected server-side cross-site scripting (`py/reflective-xss`) | Supports path visualization and is now visible on LGTM by default | No change to expected results |
79-
| SQL query built from user-controlled sources (`py/sql-injection`) | Supports path visualization and is now visible on LGTM by default | No change to expected results |
80-
| Uncontrolled data used in path expression (`py/path-injection`) | Supports path visualization and is now visible on LGTM by default | No change to expected results |
81-
| Uncontrolled command line (`py/command-line-injection`) | Supports path visualization and is now visible on LGTM by default | No change to expected results |
82-
| URL redirection from remote source (`py/url-redirection`) | Fewer false positive results and now supports path visualization | Taint is no longer tracked from the right hand side of binary expressions. In other words `SAFE + TAINTED` is now treated as safe. |
75+
| URL redirection from remote source (`py/url-redirection`) | Fewer false positive results | Taint is no longer tracked from the right-hand side of binary expressions. In other words `SAFE + TAINTED` is now treated as safe. |
8376

8477

8578
## Changes to code extraction
8679

8780
* Improved scalability: Scaling is near linear to at least 20 CPU cores.
88-
* Five levels of logging can be selected: `ERROR`, `WARN`, `INFO`, `DEBUG` and `TRACE`. `WARN` is the stand-alone default, but `INFO` will be used when run by LGTM.
81+
* Five levels of logging can be selected: `ERROR`, `WARN`, `INFO`, `DEBUG` and `TRACE`. LGTM uses `INFO` level logging. QL tools use `WARN` level logging by default.
8982
* The `-v` flag can be specified multiple times to increase logging level by one per `-v`.
9083
* The `-q` flag has been added and can be specified multiple times to reduce the logging level by one per `-q`.
9184
* Log lines are now in the `[SEVERITY] message` style and never overlap.
92-
* Extractor now outputs the location of the first offending character when an EncodingError is encountered.
85+
* The extractor now outputs the location of the first character that triggers an EncodingError.
9386

9487
## Changes to QL libraries
9588

96-
* Taint tracking analysis now understands HTTP requests in the `twisted` library.
97-
89+
* Taint-tracking analysis now understands HTTP requests in the `twisted` library.
9890
* The analysis now handles `isinstance` and `issubclass` tests involving the basic abstract base classes better. For example, the test `issubclass(list, collections.Sequence)` is now understood to be `True`
9991
* Taint tracking automatically tracks tainted mappings and collections, without you having to add additional taint kinds. This means that custom taints are tracked from `x` to `y` in the following flow: `l = [x]; y =l[0]`.

0 commit comments

Comments
 (0)