MDEV-22186 : Add innodb_buffer_pool_in_core_file to control buffer pool in core by mariadb-TafzeelShams · Pull Request #4651 · MariaDB/server

mariadb-TafzeelShams · 2026-02-14T12:31:45Z

The Jira issue number for this PR is: MDEV-22186

Description

MDEV-22186: Add innodb_buffer_pool_in_core_file to control buffer pool in cores

Problem:
There is no control to include or exclude the InnoDB buffer pool from core files,
which can lead to unnecessarily large core dumps or prevent capturing useful
memory state.

Solution:
Introduce a dynamic global system variable innodb_buffer_pool_in_core_file.
When set to OFF (or via --skip-innodb-buffer-pool-in-core-file),
the server attempts to exclude the buffer pool from core dumps using
madvise(MADV_DONTDUMP) where supported. If exclusion cannot be guaranteed,
@@core_file is automatically disabled and a warning is emitted to respect the
user’s intention.

Inspired from mysql commit@891460995137598a6e0ae3684ba1cc6ccd0c3ca3

How can this PR be tested?

Added
mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool.test
mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool_dynamic.test
mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool_with_resizing.test

Basing the PR against the correct MariaDB version

This is a new feature and the PR is based against the latest MariaDB development branch
This is a bug fix and the PR is based against the earliest branch in which the bug can be reproduced

Backward compatibility

The default value of innodb_buffer_pool_in_core_file preserves existing behavior:

Debug builds (or builds without MADV_DONTDUMP support) will dump the buffer pool.
Non-debug builds on systems with MADV_DONTDUMP support will not dump the buffer pool.

This ensures the new variable behaves the same way as previous versions by default.

…l in cores Problem: There is no control to include or exclude the InnoDB buffer pool from core files, which can lead to unnecessarily large core dumps or prevent capturing useful memory state. Solution: Introduce a dynamic global system variable innodb_buffer_pool_in_core_file. When set to OFF (or via --skip-innodb-buffer-pool-in-core-file), the server attempts to exclude the buffer pool from core dumps using madvise(MADV_DONTDUMP) where supported. If exclusion cannot be guaranteed, @@core_file is automatically disabled and a warning is emitted to respect the user’s intention. - innodb_buffer_pool_in_core_file Determines whether the buffer pool should be included in core files when @@core_file=ON. No effect if @@core_file=OFF. Default: OFF on non-debug builds with MADV_DONTDUMP support, ON otherwise. - innobase_should_madvise_buf_pool() Evaluates @@core_file and @@innodb_buffer_pool_in_core_file to determine if MADV_DONTDUMP should be applied. - innobase_disable_core_dump() Clears TEST_CORE_ON_SIGNAL, disabling @@core_file when buffer pool exclusion cannot be guaranteed. - buf_pool_t::madvise_dont_dump() Applies MADV_DONTDUMP to buffer pool memory. Emits warning and returns false if unsupported or fails. - buf_pool_t::madvise_dump() Applies MADV_DODUMP to re-include buffer pool in core dumps. Returns false if unsupported or fails. - buf_pool_t::madvise_update_dump() Reevaluates and updates madvise state. If exclusion fails, invokes innobase_disable_core_dump(). - buf_pool_t::buf_pool_should_madvise_dont_dump Tracks current dump state, protected by buf_pool_t::mutex. - buf_pool_t::create() Initializes buf_pool_should_madvise_dont_dump and conditionally applies MADV_DONTDUMP after allocation; disables core dumps on failure. - buf_pool_t::close() Calls madvise_dump() before releasing memory if MADV_DONTDUMP was applied. - buf_pool_t::resize() Invokes madvise_update_dump(true) to reapply correct madvise state after resizing. - innodb_srv_buffer_pool_in_core_file_update() Update hook that updates srv_buffer_pool_in_core_file and calls buf_pool.madvise_update_dump(). - srv_buffer_pool_in_core_file Global variable backing @@innodb_buffer_pool_in_core_file. Inspired from mysql commit@891460995137598a6e0ae3684ba1cc6ccd0c3ca3

dr-m

The parameter name had better be innodb_buffers_in_core_file or similar or innodb_trim_core_file, because the change that was originally introduced in #364 affects also other large buffers than the buffer pool.

dr-m · 2026-02-17T07:59:18Z

mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool.test

+# madvise MADV_DONTDUMP is non-posix extension available in Linux 3.4
+--let $minimum_required_linux_version = 3.4
+--source include/linux-version.inc


In b600f30 #364, @grooverdan introduced the madvise() calls, and there never was an issue reported with any GNU/Linux distribution in all these years. I see that Linux kernel 3.4 was released in 2012. MariaDB Server 10.4 switched the code base to C++11 (ISO/IEC 14882:2011), which bumped the compiler requirement to GCC 4.8.5. This was available in Red Hat Enterprise Linux 7, which has meanwhile reached its End of Maintenance support on 2024-06-30. According to https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux the kernel major version number should be reported as 3.10 by that dated system, which had originally been introduced in 2014.

I checked the environment details on https://buildbot.mariadb.org/#/builders/222 but unfortunately it’s running in a container on a much newer kernel (6.x).

Please remove this check as well as the entire file.

dr-m · 2026-02-17T08:01:52Z

mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool.test

+--echo # Shutdown server
+--source include/shutdown_mysqld.inc
+
+--echo # Restart server with --log-error
+--exec echo "restart:--log-error=$MYSQLTEST_VARDIR/log/core_dump.err" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+--enable_reconnect
+--source include/wait_until_connected_again.inc


Why do we need this restart? Other tests use the following pattern just fine:

let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err; let SEARCH_PATTERN= InnoDB: foo; -- source include/search_pattern_in_file.inc

dr-m · 2026-02-17T08:03:38Z

mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool.test

+--enable_reconnect
+--source include/wait_until_connected_again.inc
+
+SHOW VARIABLES LIKE '%core%';


Why such a broad pattern that could break when someone adds a variable that happens to match it, such as optimizer_plan_score_for_something_funny?

dr-m · 2026-02-17T08:05:58Z

mysql-test/suite/innodb/t/mysqld_core_dump_without_buffer_pool_dynamic.test

+# Embedded mode doesn't support restart
+--source include/not_embedded.inc
+
+--echo # Shutdown server
+--source include/shutdown_mysqld.inc
+
+--echo # Restart server with --log-error
+--exec echo "restart:--log-error=$MYSQLTEST_VARDIR/log/core_dump.err" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+--enable_reconnect
+--source include/wait_until_connected_again.inc


There does not seem to be any reason for any restart in this test.

dr-m · 2026-02-17T08:11:57Z

mysql-test/include/mysqld_core_dump.inc

+# Kill mysqld to dump a core
+system("kill", "-s", "SIGABRT", "$pid");
+print "# Perl: Sent a SIGABRT to mysqld to dump a core.\n";
+
+$core_dir = $ENV{'MYSQLTEST_VARDIR'} . '/mysqld.1/data/';
+
+$found_core = 0;
+$core_size = 0;
+$core_size_good = 0;
+
+# Check the files in the core file directory
+$wait_sec = 60;
+while ($wait_sec > 0) {
+  opendir(my $dir, $core_dir) or die "Failed to open dir $core_dir: $!\n";


I think that there is a much less intrusive ways to check what would included in a core dump. man gcore informs me that /proc/$pid/coredump_filter can be consulted. There is also /proc/$pid/maps. I think that you should experiment with those and identify the impact of the madvise() calls. There is no need to significantly increase the load on https://buildbot.mariadb.org workers by generating potentially huge core dumps. It can lead to significant interference with other concurrently running workload. Keep in mind that the data directory is usually stored in /dev/shm, and we don’t want to trigger any swapping.

mariadb-TafzeelShams added the MariaDB Corporation label Feb 14, 2026

mariadb-TafzeelShams requested a review from Thirunarayanan February 14, 2026 12:51

mariadb-TafzeelShams changed the title ~~Add innodb_buffer_pool_in_core_file to control buffer pool in core~~ MDEV-22186 : Add innodb_buffer_pool_in_core_file to control buffer pool in core Feb 14, 2026

mariadb-TafzeelShams force-pushed the 10.11-MDEV-22186 branch 8 times, most recently from 0034625 to 9b91138 Compare February 17, 2026 05:09

mariadb-TafzeelShams force-pushed the 10.11-MDEV-22186 branch from 9b91138 to b881bdc Compare February 17, 2026 07:37

dr-m requested changes Feb 17, 2026

View reviewed changes

dr-m requested a review from grooverdan February 17, 2026 08:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comments

MDEV-22186 : Add innodb_buffer_pool_in_core_file to control buffer pool in core#4651

MDEV-22186 : Add innodb_buffer_pool_in_core_file to control buffer pool in core#4651
mariadb-TafzeelShams wants to merge 1 commit into10.11from
10.11-MDEV-22186

mariadb-TafzeelShams commented Feb 14, 2026 •

edited

Loading

Uh oh!

dr-m left a comment

Uh oh!

dr-m Feb 17, 2026

Uh oh!

dr-m Feb 17, 2026

Uh oh!

dr-m Feb 17, 2026

Uh oh!

dr-m Feb 17, 2026

Uh oh!

dr-m Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Uh oh!

Comments

Conversation

mariadb-TafzeelShams commented Feb 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How can this PR be tested?

Basing the PR against the correct MariaDB version

Backward compatibility

Uh oh!

dr-m left a comment

Choose a reason for hiding this comment

Uh oh!

dr-m Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

dr-m Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

dr-m Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

dr-m Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

dr-m Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

mariadb-TafzeelShams commented Feb 14, 2026 •

edited

Loading