2022-01-27 17:13:08

by QiuLaibin

[permalink] [raw]
Subject: [PATCH -next] blk-mq: Fix wrong wakeup batch configuration which will cause hang

Commit 180dccb0dba4f ("blk-mq: fix tag_get wait task can't be
awakened") will recalculating wake_batch when inc or dec active_queues
to avoid wake_batch is > hctx_max_depth. At the same time, in order to
not affect performance as much as possible, the minimum wakeup batch is
set to 4. But when the QD is small (such as QD=1), if inc or dec
active_queues will increase wakeup batch, which will lead to hang.

Fix this problem with the following strategies:
QD : >= 32 | < 32
---------------------------------
wakeup batch: 8~4 | 3~1

Fixes: 180dccb0dba4f ("blk-mq: fix tag_get wait task can't be awakened")
Link: https://lore.kernel.org/linux-block/[email protected]/T/#t
Reported-by: Alex Xu (Hello71) <[email protected]>
Signed-off-by: Laibin Qiu <[email protected]>
---
lib/sbitmap.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index 6220fa67fb7e..09d293c30fd2 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -488,9 +488,13 @@ void sbitmap_queue_recalculate_wake_batch(struct sbitmap_queue *sbq,
unsigned int users)
{
unsigned int wake_batch;
+ unsigned int min_batch;
+ unsigned int depth = (sbq->sb.depth + users - 1) / users;

- wake_batch = clamp_val((sbq->sb.depth + users - 1) /
- users, 4, SBQ_WAKE_BATCH);
+ min_batch = sbq->sb.depth >= (4 * SBQ_WAIT_QUEUES) ? 4 : 1;
+
+ wake_batch = clamp_val(depth / SBQ_WAIT_QUEUES,
+ min_batch, SBQ_WAKE_BATCH);
__sbitmap_queue_update_wake_batch(sbq, wake_batch);
}
EXPORT_SYMBOL_GPL(sbitmap_queue_recalculate_wake_batch);
--
2.22.0


2022-01-28 04:56:25

by Alex Xu (Hello71)

[permalink] [raw]
Subject: Re: [PATCH -next] blk-mq: Fix wrong wakeup batch configuration which will cause hang

Excerpts from Laibin Qiu's message of January 27, 2022 5:00 am:
> Commit 180dccb0dba4f ("blk-mq: fix tag_get wait task can't be
> awakened") will recalculating wake_batch when inc or dec active_queues
> to avoid wake_batch is > hctx_max_depth. At the same time, in order to
> not affect performance as much as possible, the minimum wakeup batch is
> set to 4. But when the QD is small (such as QD=1), if inc or dec
> active_queues will increase wakeup batch, which will lead to hang.
>
> Fix this problem with the following strategies:
> QD : >= 32 | < 32
> ---------------------------------
> wakeup batch: 8~4 | 3~1
>
> Fixes: 180dccb0dba4f ("blk-mq: fix tag_get wait task can't be awakened")
> Link: https://lore.kernel.org/linux-block/[email protected]/T/#t
> Reported-by: Alex Xu (Hello71) <[email protected]>
> Signed-off-by: Laibin Qiu <[email protected]>
> ---
> lib/sbitmap.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbitmap.c b/lib/sbitmap.c
> index 6220fa67fb7e..09d293c30fd2 100644
> --- a/lib/sbitmap.c
> +++ b/lib/sbitmap.c
> @@ -488,9 +488,13 @@ void sbitmap_queue_recalculate_wake_batch(struct sbitmap_queue *sbq,
> unsigned int users)
> {
> unsigned int wake_batch;
> + unsigned int min_batch;
> + unsigned int depth = (sbq->sb.depth + users - 1) / users;
>
> - wake_batch = clamp_val((sbq->sb.depth + users - 1) /
> - users, 4, SBQ_WAKE_BATCH);
> + min_batch = sbq->sb.depth >= (4 * SBQ_WAIT_QUEUES) ? 4 : 1;
> +
> + wake_batch = clamp_val(depth / SBQ_WAIT_QUEUES,
> + min_batch, SBQ_WAKE_BATCH);
> __sbitmap_queue_update_wake_batch(sbq, wake_batch);
> }
> EXPORT_SYMBOL_GPL(sbitmap_queue_recalculate_wake_batch);
> --
> 2.22.0
>
>

Tested-by: Alex Xu (Hello71) <[email protected]>

2022-01-28 11:23:16

by Jens Axboe

[permalink] [raw]
Subject: Re: [PATCH -next] blk-mq: Fix wrong wakeup batch configuration which will cause hang

On Thu, 27 Jan 2022 18:00:47 +0800, Laibin Qiu wrote:
> Commit 180dccb0dba4f ("blk-mq: fix tag_get wait task can't be
> awakened") will recalculating wake_batch when inc or dec active_queues
> to avoid wake_batch is > hctx_max_depth. At the same time, in order to
> not affect performance as much as possible, the minimum wakeup batch is
> set to 4. But when the QD is small (such as QD=1), if inc or dec
> active_queues will increase wakeup batch, which will lead to hang.
>
> [...]

Applied, thanks!

[1/1] blk-mq: Fix wrong wakeup batch configuration which will cause hang
commit: 10825410b956dc1ed8c5fbc8bbedaffdadde7f20

Best regards,
--
Jens Axboe