Skip to content

blk-mq: recover from stale cached request in blk_mq_submit_bio#767

Open
blktests-ci[bot] wants to merge 2 commits into
linus-master_basefrom
series/1085314=>linus-master
Open

blk-mq: recover from stale cached request in blk_mq_submit_bio#767
blktests-ci[bot] wants to merge 2 commits into
linus-master_basefrom
series/1085314=>linus-master

Conversation

@blktests-ci
Copy link
Copy Markdown

@blktests-ci blktests-ci Bot commented Apr 24, 2026

Pull request for series with
subject: blk-mq: recover from stale cached request in blk_mq_submit_bio
version: 1
url: https://patchwork.kernel.org/project/linux-block/list/?series=1085314

@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented Apr 24, 2026

Upstream branch: dd6c438
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented Apr 29, 2026

Upstream branch: dca922e
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from d783494 to 44dd998 Compare April 29, 2026 02:21
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from 482ce5b to 5a9f7c7 Compare April 30, 2026 07:29
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented Apr 30, 2026

Upstream branch: e75a43c
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 44dd998 to 5a29d22 Compare April 30, 2026 07:30
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from 5a9f7c7 to 25a041f Compare May 3, 2026 02:07
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 3, 2026

Upstream branch: 66edb90
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 5a29d22 to 46b536b Compare May 3, 2026 02:41
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from 25a041f to 6f75bd1 Compare May 4, 2026 10:57
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 4, 2026

Upstream branch: 6d35786
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 46b536b to 8eeb4e3 Compare May 4, 2026 11:32
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from 6f75bd1 to 1f0d33a Compare May 5, 2026 15:39
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 5, 2026

Upstream branch: 6d35786
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 8eeb4e3 to 5dcdd05 Compare May 5, 2026 16:04
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from 1f0d33a to b1870f6 Compare May 10, 2026 15:59
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 10, 2026

Upstream branch: aa54b1d
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 5dcdd05 to 2392f2e Compare May 10, 2026 16:27
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from b1870f6 to ca57796 Compare May 15, 2026 07:55
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 15, 2026

Upstream branch: 70eda68
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 2392f2e to 76aa410 Compare May 15, 2026 08:36
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from ca57796 to c1feb59 Compare May 21, 2026 02:54
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 21, 2026

Upstream branch: 8bc67e4
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from 76aa410 to b7c8dc7 Compare May 21, 2026 03:39
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from c1feb59 to ea833a1 Compare May 22, 2026 01:53
When submitting a bio to blk-mq, if the task should sleep after peeking
a cached request, but before it pops it, the plug is flushed and calls
blk_mq_free_plug_rqs, freeing the cached_rqs.

The code had already warned of this possibility, and specifically popped
the request before other known blocking calls, but it didn't handle a
blocking GFP_NOIO alloc. Allocating the split bio or the integrity
payload are two such cases that can block under high memory pressure.
The blk-mq submit_bio function continues with the peeked request that
was just freed and re-initialized, so the driver receives that request
with a NULL'ed mq_hctx, and inevitably panics.

Relevant kernel messages if you should encounter this condition, where
the "WARNING" is the harbinger of the panic about to happen:

 ------------[ cut here ]------------
 WARNING: CPU: 4 PID: 80820 at block/blk-mq.c:3071 blk_mq_submit_bio+0x2cf/0x5b0
...
 BUG: kernel NULL pointer dereference, address: 0000000000000100
 #PF: supervisor read access in kernel mode
 #PF: error_code(0x0000) - not-present page
 PGD 6b367b067 P4D 6b367b067 PUD 6bb5eb067 PMD 0
 Oops: Oops: 0000 [#1] SMP
 CPU: 4 UID: 36666 PID: 80820 Comm: IOUringThread0 Kdump: loaded Tainted: G S      W           6.16.1-0_fbk3_0_gd6c130b80483 #1 NONE
 Tainted: [S]=CPU_OUT_OF_SPEC, [W]=WARN
 Hardware name: Quanta Twin Lakes MP/Twin Lakes Passive MP, BIOS F09_3A23 12/08/2020
 RIP: 0010:nvme_queue_rqs+0x93/0x180
 Code: 44 24 48 00 00 00 00 4d 85 f6 74 19 49 8b 44 24 10 4c 3b b0 00 01 00 00 74 0b 4c 89 f7 4c 89 fe e8 c2 62 cd ff 49 8b 44 24 10 <4c> 8b b0 00 01 00 00 49 f7 46 78 01 00 00 00 74 1a 49 8b 3e 8b 8f
 RSP: 0018:ffffc9004a8f34a8 EFLAGS: 00010246
 RAX: 0000000000000000 RBX: ffff888c12b92920 RCX: 0000000000000020
 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888c12b92920
 RBP: 0000000000000000 R08: 000000000000000b R09: ffffffffffffffff
 R10: 0000000000092800 R11: 000000000002e000 R12: ffff88828e325c00
 R13: 0000000000000000 R14: 0000000000000000 R15: ffffc9004a8f34a8
 FS:  00007f7cf30e6640(0000) GS:ffff8890fa908000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000100 CR3: 00000006bb387006 CR4: 00000000007726f0
 PKRU: 55555554
 Call Trace:
  <TASK>
  blk_mq_dispatch_queue_requests+0x46/0x120
  blk_mq_flush_plug_list+0x38/0x130
  blk_add_rq_to_plug+0xa2/0x160
  blk_mq_submit_bio+0x3ab/0x5b0
  __submit_bio+0x3a/0x260
  submit_bio_noacct_nocheck+0xc6/0x2b0
  btrfs_submit_bbio+0x14d/0x520
  ? btrfs_get_extent+0x43f/0x640
  submit_extent_folio+0x31f/0x340
  btrfs_do_readpage+0x2d7/0xac0
  btrfs_readahead+0x142/0x200
  ? clear_state_bit+0x520/0x520
  read_pages+0x57/0x200
  ? folio_alloc_noprof+0x10c/0x310
  page_cache_ra_unbounded+0x28c/0x480
  ? asm_sysvec_call_function+0x16/0x20
  ? blk_cgroup_congested+0xa/0x50
  ? page_cache_sync_ra+0x41/0x2d0
  filemap_get_pages+0x347/0xd50
  filemap_read+0xd3/0x500
  ? 0xffffffff81000000
  __io_read+0x111/0x440
  io_read+0x23/0x90
  __io_issue_sqe+0x40/0x120
  io_issue_sqe+0x3f/0x3a0
  io_submit_sqes+0x2bd/0x790
  __se_sys_io_uring_enter+0x100/0xc10
  ? eventfd_read+0x100/0x1f0
  ? futex_wake+0x1b9/0x260
  ? syscall_trace_enter+0x34/0x1d0
  do_syscall_64+0x6a/0x250
  entry_SYSCALL_64_after_hwframe+0x4b/0x53
 RIP: 0033:0x95d621e
 Code: 8b 12 f6 c2 01 75 2a 84 c9 74 22 44 0f b6 d1 41 09 c2 8b bf cc 00 00 00 41 b9 08 00 00 00 b8 aa 01 00 00 31 d2 45 31 c0 0f 05 <48> 89 c6 89 f0 5d c3 83 c8 02 eb d5 cc cc cc cc cc cc 55 48 89 e5
 RSP: 002b:00007f7cf30dc630 EFLAGS: 00000246 ORIG_RAX: 00000000000001aa
 RAX: ffffffffffffffda RBX: 00007f7d18907000 RCX: 00000000095d621e
 RDX: 0000000000000000 RSI: 0000000000000006 RDI: 00000000000004cf
 RBP: 00007f7cf30dc630 R08: 0000000000000000 R09: 0000000000000008
 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
 R13: 0000000000000001 R14: 00007f7d18907320 R15: 0000000000000009
  </TASK>

Fixes: b0077e2 ("blk-mq: make sure active queue usage is held for bio_integrity_prep()")
Fixes: 7b4f36c ("block: ensure we hold a queue reference when using queue limits")
Signed-off-by: Keith Busch <kbusch@kernel.org>
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 22, 2026

Upstream branch: 6779b50
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

@blktests-ci blktests-ci Bot force-pushed the series/1085314=>linus-master branch from b7c8dc7 to a1b27e1 Compare May 22, 2026 02:36
@blktests-ci blktests-ci Bot force-pushed the linus-master_base branch from ea833a1 to 7af85d1 Compare May 23, 2026 06:11
@blktests-ci
Copy link
Copy Markdown
Author

blktests-ci Bot commented May 23, 2026

Upstream branch: 79bd2dd
series: https://patchwork.kernel.org/project/linux-block/list/?series=1085314
version: 1

Pull request is NOT updated. Failed to apply https://patchwork.kernel.org/project/linux-block/list/?series=1085314
error message:

Cmd('git') failed due to: exit code(128)
  cmdline: git am --3way
  stdout: 'Applying: blk-mq: recover from stale cached request in blk_mq_submit_bio
Using index info to reconstruct a base tree...
M	block/blk-mq.c
Falling back to patching base and 3-way merge...
Auto-merging block/blk-mq.c
CONFLICT (content): Merge conflict in block/blk-mq.c
Patch failed at 0001 blk-mq: recover from stale cached request in blk_mq_submit_bio
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".'
  stderr: 'error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch'

conflict:

diff --cc block/blk-mq.c
index 28c2d931e75e,2ca914505d93..000000000000
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@@ -3097,6 -3096,25 +3097,28 @@@ static struct request *blk_mq_get_cache
  	return rq;
  }
  
++<<<<<<< HEAD
++=======
+ static bool blk_mq_use_cached_rq(struct request *rq, struct blk_plug *plug,
+ 		struct bio *bio)
+ {
+ 	/*
+ 	 * We will have flushed the plug and hence killed the cached_rq list as
+ 	 * well if anything had blocked. Pop this entry before we throttle if
+ 	 * the entry is still valid.
+ 	 */
+ 	if (rq_list_pop(&plug->cached_rqs) != rq)
+ 		return false;
+ 
+ 	rq_qos_throttle(rq->q, bio);
+ 
+ 	blk_mq_rq_time_init(rq, blk_time_get_ns());
+ 	rq->cmd_flags = bio->bi_opf;
+ 	INIT_LIST_HEAD(&rq->queuelist);
+ 	return true;
+ }
+ 
++>>>>>>> blk-mq: recover from stale cached request in blk_mq_submit_bio
  static bool bio_unaligned(const struct bio *bio, struct request_queue *q)
  {
  	unsigned int bs_mask = queue_logical_block_size(q) - 1;
@@@ -3194,11 -3212,13 +3216,21 @@@ void blk_mq_submit_bio(struct bio *bio
  
  new_request:
  	if (rq) {
++<<<<<<< HEAD
 +		rq_qos_throttle(rq->q, bio);
 +		blk_mq_rq_time_init(rq, blk_time_get_ns());
 +		rq->cmd_flags = bio->bi_opf;
 +		INIT_LIST_HEAD(&rq->queuelist);
 +	} else {
++=======
+ 		if (!blk_mq_use_cached_rq(rq, plug, bio)) {
+ 			rq = NULL;
+ 			if (unlikely(bio_queue_enter(bio)))
+ 				return;
+ 		}
+ 	}
+ 	if (!rq) {
++>>>>>>> blk-mq: recover from stale cached request in blk_mq_submit_bio
  		rq = blk_mq_get_new_requests(q, plug, bio);
  		if (unlikely(!rq)) {
  			if (bio->bi_opf & REQ_NOWAIT)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant