Merge branch 'master' into python-objectapi-to-valueapi-useofapply

Author: BekaValentine

.codeqlmanifest.json

@@ -2,5 +2,4 @@
                "*/ql/test/qlpack.yml",
                "*/upgrades/qlpack.yml",
                "misc/legacy-support/*/qlpack.yml",
-               "misc/suite-helpers/qlpack.yml",
-               "codeql/.codeqlmanifest.json" ] }
+               "misc/suite-helpers/qlpack.yml" ] }

.gitignore

@@ -1,6 +1,7 @@
 # editor and OS artifacts
 *~
 .DS_STORE
+*.swp
 
 # query compilation caches
 .cache

change-notes/1.24/analysis-cpp.md

@@ -27,10 +27,10 @@ The following changes in version 1.24 affect C/C++ analysis in all applications.
 
 ## Changes to libraries
 
-* The data-flow library has been improved when flow through functions needs to be
-  combined with both taint tracking and flow through fields allowing more flow
-  to be tracked. This affects and improves some security queries, which may
-  report additional results.
+* The data-flow library has been improved, which affects and improves some security queries. The improvements are:
+  - Track flow through functions that combine taint tracking with flow through fields.
+  - Track flow through clone-like functions, that is, functions that read contents of a field from a
+    parameter and stores the value in the field of a returned object.
 * Created the `semmle.code.cpp.models.interfaces.Allocation` library to model allocation such as `new` expressions and calls to `malloc`. This in intended to replace the functionality in `semmle.code.cpp.commons.Alloc` with a more consistent and useful interface.
 * Created the `semmle.code.cpp.models.interfaces.Deallocation` library to model deallocation such as `delete` expressions and calls to `free`. This in intended to replace the functionality in `semmle.code.cpp.commons.Alloc` with a more consistent and useful interface.
 * The new class `StackVariable` should be used in place of `LocalScopeVariable`

change-notes/1.24/analysis-csharp.md

@@ -33,10 +33,10 @@ The following changes in version 1.24 affect C# analysis in all applications.
 
 ## Changes to libraries
 
-* The data-flow library has been improved when flow through methods needs to be
-  combined with both taint tracking and flow through fields allowing more flow
-  to be tracked. This affects and improves most security queries, which may
-  report additional results.
+* The data-flow library has been improved, which affects and improves most security queries. The improvements are:
+  - Track flow through methods that combine taint tracking with flow through fields.
+  - Track flow through clone-like methods, that is, methods that read contents of a field from a
+    parameter and stores the value in the field of a returned object.
 * The taint tracking library now tracks flow through (implicit or explicit) conversion operator calls.
 * [Code contracts](https://docs.microsoft.com/en-us/dotnet/framework/debug-trace-profile/code-contracts) are now recognized, and are treated like any other assertion methods.
 * Expression nullability flow state is given by the predicates `Expr.hasNotNullFlowState()` and `Expr.hasMaybeNullFlowState()`.

change-notes/1.24/analysis-java.md

@@ -26,10 +26,10 @@ The following changes in version 1.24 affect Java analysis in all applications.
 
 ## Changes to libraries
 
-* The data-flow library has been improved when flow through methods needs to be
-  combined with both taint tracking and flow through fields allowing more flow
-  to be tracked. This affects and improves most security queries, which may
-  report additional results.
+* The data-flow library has been improved, which affects and improves most security queries. The improvements are:
+  - Track flow through methods that combine taint tracking with flow through fields.
+  - Track flow through clone-like methods, that is, methods that read contents of a field from a
+    parameter and stores the value in the field of a returned object.
 * Identification of test classes has been improved. Previously, one of the
   match conditions would classify any class with a name containing the string
   "Test" as a test class, but now this matching has been replaced with one that

change-notes/1.24/analysis-javascript.md

@@ -9,22 +9,26 @@
 
 * Imports that rely on path-mappings from a `tsconfig.json` file can now be resolved.
 
+* Export declarations of the form `export * as ns from "x"` are now analyzed more precisely.
+
 * The analysis of sanitizer guards has improved, leading to fewer false-positive results from the security queries.
 
 * Support for the following frameworks and libraries has been improved:
-  - [react](https://www.npmjs.com/package/react)
-  - [typeahead.js](https://www.npmjs.com/package/typeahead.js)
-  - [Handlebars](https://www.npmjs.com/package/handlebars)
   - [Electron](https://electronjs.org/)
+  - [Handlebars](https://www.npmjs.com/package/handlebars)
+  - [Koa](https://www.npmjs.com/package/koa)
   - [Node.js](https://nodejs.org/)
   - [Socket.IO](https://socket.io/)
-  - [ws](https://github.com/websockets/ws)
   - [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
-  - [Koa](https://www.npmjs.com/package/koa)
-  - [lazy-cache](https://www.npmjs.com/package/lazy-cache)
+  - [chrome-remote-interface](https://www.npmjs.com/package/chrome-remote-interface)
   - [for-in](https://www.npmjs.com/package/for-in)
   - [for-own](https://www.npmjs.com/package/for-own)
+  - [http2](https://nodejs.org/api/http2.html)
+  - [lazy-cache](https://www.npmjs.com/package/lazy-cache)
+  - [react](https://www.npmjs.com/package/react)
   - [send](https://www.npmjs.com/package/send)
+  - [typeahead.js](https://www.npmjs.com/package/typeahead.js)
+  - [ws](https://github.com/websockets/ws)
 
 ## New queries
 
@@ -47,6 +51,7 @@
 | Use of call stack introspection in strict mode (`js/strict-mode-call-stack-introspection`) | Fewer false positive results | The query no longer flags expression statements. |
 | Missing CSRF middleware (`js/missing-token-validation`) | Fewer false positive results | The query reports fewer duplicates and only flags handlers that explicitly access cookie data. |
 | Uncontrolled data used in path expression (`js/path-injection`) | More results | This query now recognizes additional ways dangerous paths can be constructed. |
+| Uncontrolled command line (`js/command-line-injection`) | More results | This query now recognizes additional ways of constructing arguments to `cmd.exe` and `/bin/sh`. |
 
 ## Changes to libraries
 

change-notes/1.24/analysis-python.md

@@ -0,0 +1,37 @@
+# Improvements to Python analysis
+
+The following changes in version 1.24 affect Python analysis in all applications.
+
+## General improvements
+
+## New queries
+
+| **Query**                   | **Tags**  | **Purpose**                                                        |
+|-----------------------------|-----------|--------------------------------------------------------------------|
+
+## Changes to existing queries
+
+| **Query**                  | **Expected impact**    | **Change**                                                       |
+|----------------------------|------------------------|------------------------------------------------------------------|
+
+### Web framework support
+
+The QL-library support for the web frameworks Bottle, CherryPy, Falcon, Pyramid, TurboGears, Tornado, and Twisted have
+been fixed so they provide a proper HttpRequestTaintSource, instead of a TaintSource. This will enable results for the following queries:
+
+- py/path-injection
+- py/command-line-injection
+- py/reflective-xss
+- py/sql-injection
+- py/code-injection
+- py/unsafe-deserialization
+- py/url-redirection
+
+The QL-library support for the web framework Twisted have been fixed so they provide a proper
+HttpResponseTaintSink, instead of a TaintSink. This will enable results for the following
+queries:
+
+- py/reflective-xss
+- py/stack-trace-exposure
+
+## Changes to libraries