gh-136073: Fix expandvars() in Tools/freeze/checkextensions.py

[serhiy-storchaka]

• Fix potential infinite recursion and/or memory consumption.
• Fix a bug when reference can cross boundaries of substitutions, e.g. expandvars('$a)', {'a': '$(b', 'b': 'c'}).
• Fix potential quadratic complexity.

• Issue: Possible resource exhaustation of expandvars in Tools/freeze/checkextensions.py #136073

[kexinoh]

I just realized that the newly discovered code still has some issues, such as when constructing the following example as input:

def generate_bomb_vars(depth):
    vars_dict = {'v0': 'X'}
    for i in range(1, depth + 1):
        vars_dict[f'v{i}'] = f'$(v{i-1})$(v{i-1})'
    target_str = f'$(v{depth})'
    return target_str, vars_dict

[serhiy-storchaka]

The new version is optimized, it returns the result much faster for depth=20 and depth=30. But note that for depth=30 the result takes 1 GiB, and for depth=60 it requires the amount of memory larger than the mainstream computers support. The code will produce the result as fast as it can, but it cannot do anything if the result cannot fit in a memory.

This is similar to a billion laughs attack. The only way to treat it is to impose an artificial application specific limit on the result size. If this was a code facing an external world, it would be an issue. But this is an internal function in an internal tool. It is only used with data generated by other parts of the tool.