Missing jemalloc dependency

[akoshchiy]

Seems that duckdb sources is not bundled with jemalloc. duckdb/duckdb#13075
It could be the reason of some memory issues, e.g. duckdb/duckdb#9712

[brianwyka]

@lnkuiper , do you think this could be the issue?

[lnkuiper]

@akoshchiy, indeed, the JDBC client has not been bundled with jemalloc since we moved it out of the main repository to this repository. I don't see why it can't be bundled with jemalloc, so we should look into this.

@brianwyka, I'm not sure. I have been able to reproduce a similar memory retention issue, but only on MacOS regardless of whether jemalloc was bundled. On Linux, I was not able to reproduce this issue.

[brianwyka]

@lnkuiper , do you have instructions on how to bundle it? Perhaps I can contribute if its not on the roadmap or radar.

[lnkuiper]

You can enable jemalloc by adding it to the list of extensions in vendor.py and calling python3 scripts/vendor.py --duckdb=/path/to/duckdb/repo, and then make release in duckdb-jdbc repo.

For other clients, we only bundle jemalloc on x86 linux, and we'd like to keep it the same for JDBC, so it needs to check this. You can check how we do this in duckdb repo at tools/pythonpkg/setup.py.

[brianwyka]

@lnkuiper , finally got around to this! ☝️

[guizmaii]

Will this be fixed in DuckDB 1.2.0? 👀

[brianwyka]

@lnkuiper , I've confirmed that the jemalloc extensions is now available from DuckDB Java bundled in version 1.2.0. However, it is not loaded, or installed. According to https://duckdb.org/docs/extensions/jemalloc.html, it can not be loaded at runtime. IS there additional work required to have it loaded by default or am I missing something?

[staticlibs]

@brianwyka

Just FYI, I can confirm that jemalloc is loaded by default and is actually used in the latest dev version on Linux.

[staticlibs]

jemalloc was added in #133 and #171, closing this as completed.