Global OOB read in `utf8proc_map` via `CHARBOUND`+`COMPOSE`

[hgarrereyn]

Hi, there is a potential bug in utf8proc_map reachable with CHARBOUND+COMPOSE.

This bug was reproduced on 90daf9f.

Description

My fuzzer found this crash caused by using CHARBOUND+COMPOSE in an otherwise seemingly valid way.

I believe it is the COMPOSE variant of my previous report #310 which has since been patched.

POC

The following testcase demonstrates the bug:

testcase.cpp

#include <cstdlib>
#include <cstdint>
extern "C" {
#include "/fuzz/install/include/utf8proc.h"
}
int main() {
  // Input: "A\xCC\x81" (A + COMBINING ACUTE ACCENT)
  const utf8proc_uint8_t data[4] = { 'A', 0xCC, 0x81, 0 };
  utf8proc_uint8_t *out = NULL;
  
  const utf8proc_option_t opt = (utf8proc_option_t)(UTF8PROC_CHARBOUND | UTF8PROC_COMPOSE);
  
  const utf8proc_ssize_t slen = 3;
  utf8proc_ssize_t r = utf8proc_map(data, slen, &out, opt);

  if (r >= 0 && out) free(out);
  return 0;
}

stdout

stderr

=================================================================
==1==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55da63cc919e at pc 0x55da63c589ff bp 0x7fff4092cbf0 sp 0x7fff4092cbe8
READ of size 2 at 0x55da63cc919e thread T0
    #0 0x55da63c589fe in utf8proc_normalize_utf32 (/fuzz/test+0x10a9fe) (BuildId: 66106500c4b83547372f5ec73e1c7e6f14b887ce)
    #1 0x55da63c58abc in utf8proc_reencode (/fuzz/test+0x10aabc) (BuildId: 66106500c4b83547372f5ec73e1c7e6f14b887ce)
    #2 0x55da63c592d0 in utf8proc_map_custom (/fuzz/test+0x10b2d0) (BuildId: 66106500c4b83547372f5ec73e1c7e6f14b887ce)
    #3 0x55da63c55522 in main /fuzz/testcase.cpp:14:24
    #4 0x7f95552fdd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #5 0x7f95552fde3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #6 0x55da63b7a2e4 in _start (/fuzz/test+0x2c2e4) (BuildId: 66106500c4b83547372f5ec73e1c7e6f14b887ce)

0x55da63cc919e is located 2 bytes before global variable 'utf8proc_stage1table' defined in '/fuzz/src/utf8proc.c' (0x55da63cc91a0) of size 8704
0x55da63cc919e is located 23166 bytes after global variable 'utf8proc_stage2table' defined in '/fuzz/src/utf8proc.c' (0x55da63cacd20) of size 92672
SUMMARY: AddressSanitizer: global-buffer-overflow (/fuzz/test+0x10a9fe) (BuildId: 66106500c4b83547372f5ec73e1c7e6f14b887ce) in utf8proc_normalize_utf32
Shadow bytes around the buggy address:
  0x55da63cc8f00: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x55da63cc8f80: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x55da63cc9000: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x55da63cc9080: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x55da63cc9100: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
=>0x55da63cc9180: f9 f9 f9[f9]00 00 00 00 00 00 00 00 00 00 00 00
  0x55da63cc9200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x55da63cc9280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x55da63cc9300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x55da63cc9380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x55da63cc9400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1==ABORTING

Steps to Reproduce

The crash was triaged with the following Dockerfile:

Dockerfile

# Ubuntu 22.04 with some packages pre-installed
FROM hgarrereyn/stitch_repro_base@sha256:3ae94cdb7bf2660f4941dc523fe48cd2555049f6fb7d17577f5efd32a40fdd2c

RUN git clone https://github.com/JuliaStrings/utf8proc /fuzz/src && \
    cd /fuzz/src && \
    git checkout 90daf9f396cfec91668758eb9cc54bd5248a6b89 && \
    git submodule update --init --remote --recursive

ENV LD_LIBRARY_PATH=/fuzz/install/lib
ENV ASAN_OPTIONS=hard_rss_limit_mb=1024:detect_leaks=0

RUN echo '#!/bin/bash\nexec clang-17 -fsanitize=address -O0 "$@"' > /usr/local/bin/clang_wrapper && \
    chmod +x /usr/local/bin/clang_wrapper && \
    echo '#!/bin/bash\nexec clang++-17 -fsanitize=address -O0 "$@"' > /usr/local/bin/clang_wrapper++ && \
    chmod +x /usr/local/bin/clang_wrapper++

ENV DEBIAN_FRONTEND=noninteractive

# Install minimal build dependencies (cmake and ninja)
RUN apt-get update && apt-get install -y --no-install-recommends \
    cmake ninja-build \
 && rm -rf /var/lib/apt/lists/*

# Use the fuzzing compiler wrappers
ENV CC=clang_wrapper \
    CXX=clang_wrapper++

WORKDIR /fuzz

# Configure, build, and install
RUN cmake -S /fuzz/src -B /fuzz/build \
      -G Ninja \
      -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_SHARED_LIBS=OFF \
      -DUTF8PROC_INSTALL=ON \
      -DCMAKE_INSTALL_PREFIX=/fuzz/install \
  && cmake --build /fuzz/build --target install -- -v

Build Command

clang++-17 -fsanitize=address -g -O0 -o /fuzz/test /fuzz/testcase.cpp -I/fuzz/install/include -L/fuzz/install/lib -lutf8proc -DUTF8PROC_STATIC && /fuzz/test

Reproduce

1. Copy Dockerfile and testcase.cpp into a local folder.
2. Build the repro image:

docker build . -t repro --platform=linux/amd64

3. Compile and run the testcase in the image:

docker run \
    -it --rm \
    --platform linux/amd64 \
    --mount type=bind,source="$(pwd)/testcase.cpp",target=/fuzz/testcase.cpp \
    repro \
    bash -c "clang++-17 -fsanitize=address -g -O0 -o /fuzz/test /fuzz/testcase.cpp -I/fuzz/install/include -L/fuzz/install/lib -lutf8proc -DUTF8PROC_STATIC && /fuzz/test"

---

Additional Info

This testcase was discovered by STITCH, an autonomous fuzzing system. All reports are reviewed manually (by a human) before submission.

[stevengj]

Might be due to this line:

utf8proc/utf8proc.c

Line 665 in a36778d

const utf8proc_property_t *current_property = unsafe_get_property(current_char);

where we call unsafe_get_property without checking whether current_char == -1 (the sentinel value for CHARBOUND)

(The CHARBOUND feature isn't tested as much as it should be, probably — in Julia, we don't use it, since we use the newer utf8proc_grapheme_break_stateful function instead to non-destructively detect grapheme breaks.)

[eschnett]

Is this related to #310? They look very similar. Maybe we need to audit the code for calls to unsafe_get_property?

D'oh, it is, see above.

[eschnett]

Idea: Create a "debug build" which adds run-time index checks? We know where array indexing or buffer access in invalid, we just intentionally do not check to make things fast.