`x86_64-unknown-linux-musl` + profiling = segmentation fault

[howardjohn]

We see that any binaries built with Jemalloc with profiling enabled, on the x86_64-unknown-linux-musl target, immediate segfaults:

$ cargo run --target x86_64-unknown-linux-musl
   Compiling new v0.1.0 (/tmp/tmp.iluiyelSHz/new)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/x86_64-unknown-linux-musl/debug/new`
zsh: segmentation fault  cargo run --target x86_64-unknown-linux-musl

Reproducer:

[package]
name = "new"
version = "0.1.0"
edition = "2021"

[dependencies]
tikv-jemallocator = { version = "0.6.0", features = ["profiling", "unprefixed_malloc_on_supported_platforms"] }

#[global_allocator]
static ALLOC: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;

#[allow(non_upper_case_globals)]
#[unsafe(export_name = "malloc_conf")]
pub static malloc_conf: &[u8] = b"prof:true\0";

fn main() {
    println!("Hello, world!");
}

This was tested with rustc 1.83 and 1.90; both saw the same. Other fields in malloc_conf do not trigger the segfault, which leads me to believe it is specific to profiling and not general malloc_conf usage.

[ti-chi-bot]

Welcome @howardjohn! It looks like this is your first issue to tikv/jemallocator 🎉

[BusyJay]

Have you try it in release mode?

[howardjohn]

Yes, the same occurs in release mode:

$ cargo run --target x86_64-unknown-linux-musl --profile release
    Finished `release` profile [optimized] target(s) in 0.00s
     Running `target/x86_64-unknown-linux-musl/release/new`
zsh: segmentation fault  cargo run --target x86_64-unknown-linux-musl --profile release

[BusyJay]

After trying your reproduce project, I can see the segment fault is at

0x00007f9a21eaa83e in prof_backtrace_impl (vec=0x7f9a2180c0b8, len=0x7ffde3cd3d68, max_len=128) at src/prof_sys.c:138
138             BT_FRAME(10)
(gdb) bt
#0  0x00007f3ecb6aa83e in prof_backtrace_impl (vec=0x7f3ecb00c0b8, len=0x7ffebc800208, max_len=128) at src/prof_sys.c:138
#1  0x00007f3ecb6b8d3c in _rjem_je_prof_backtrace (tsd=0x7f3ecb930018, bt=0x7ffebc800200) at src/prof_sys.c:284
#2  0x00007f3ecb64309a in _rjem_je_prof_tctx_create (tsd=0x7f3ecb930018) at src/prof.c:195
#3  0x00007f3ecb429b79 in prof_alloc_prep (sample_event=true, prof_active=true, tsd=0x7f3ecb930018) at include/jemalloc/internal/prof_inlines.h:141
#4  imalloc_body (tsd=0x7f3ecb930018, dopts=0x7ffebc806fe0, sopts=0x7ffebc806f80) at src/jemalloc.c:2550
#5  imalloc (dopts=0x7ffebc806fe0, sopts=0x7ffebc806f80) at src/jemalloc.c:2687
#6  _rjem_je_malloc_default (size=1024) at src/jemalloc.c:2722
#7  0x00007f3ecb45c7c6 in imalloc_fastpath (fallback_alloc=0x7f3ecb425a5b <_rjem_je_malloc_default>, size=1024) at include/jemalloc/internal/jemalloc_internal_inlines_c.h:310
#8  malloc (size=1024) at src/jemalloc.c:2746
#9  0x00007f3ecb411a5d in tikv_jemallocator::{impl#0}::alloc (self=0x7f3ecb793c04, layout=...) at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tikv-jemallocator-0.6.0/src/lib.rs:105
#10 0x00007f3ecb411e79 in new::_::__rust_alloc (size=1024, align=1) at src/main.rs:3

It's likely due to compilation of the project. When using frame pointer based unwinding, you need to make sure the whole project is built with force-frame-pointers set to yes.

You can also check how TiKV project enables frame pointer: https://github.com/tikv/tikv/blob/master/scripts/run-cargo.sh.

Normally not forcing frame pointer will only cause broken backtrace, I guess there is something wrong between the integration of gcc and musl that cause it segment fault instead.

Maybe we can also support unwinding using libunwind, which is a lot more reliable. You can send a PR if you like.