Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
Date:   Wed, 25 Oct 2023 09:05:03 +0800
From:   Ming Lei <ming.lei@redhat.com>
To:     Bart Van Assche <bvanassche@acm.org>
Cc:     Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org,
        Tejun Heo <tj@kernel.org>, linux-kernel@vger.kernel.org,
        Juri Lelli <juri.lelli@redhat.com>,
        Andrew Theurer <atheurer@redhat.com>,
        Joe Mario <jmario@redhat.com>,
        Sebastian Jug <sejug@redhat.com>,
        Frederic Weisbecker <frederic@kernel.org>, ming.lei@redhat.com
Subject: Re: [PATCH V2] blk-mq: don't schedule block kworker on isolated CPUs
Message-ID: <ZThpvzrLGQFqpsYb@fedora>
References: <20231013124758.1492796-1-ming.lei@redhat.com>
 <293c16d2-f556-4d42-ab06-2e0acf1cc601@acm.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <293c16d2-f556-4d42-ab06-2e0acf1cc601@acm.org>
Precedence: bulk

On Tue, Oct 24, 2023 at 09:53:49AM -0700, Bart Van Assche wrote:
> 
> On 10/13/23 05:47, Ming Lei wrote:
> > @@ -3926,6 +3941,15 @@ static void blk_mq_map_swqueue(struct request_queue *q)
> >   		 */
> >   		sbitmap_resize(&hctx->ctx_map, hctx->nr_ctx);
> > +		/*
> > +		 * rule out isolated CPUs from hctx->cpumask for avoiding to
> > +		 * run wq worker on isolated CPU
> > +		 */
> > +		for_each_cpu(cpu, hctx->cpumask) {
> > +			if (cpu_is_isolated(cpu))
> > +				cpumask_clear_cpu(cpu, hctx->cpumask);
> > +		}
> 
> What will happen if this code makes hctx->cpumask empty? Code like
> blk_mq_first_mapped_cpu() and blk_mq_hctx_next_cpu() assumes that
> hctx->cpumask is not empty. There may be other code that assumes that
> hctx->cpumask is not empty.

hctx->cpumask is only used for selecting next cpu to schedule run
queue now, so it is fine for hctx->cpumask to be empty.

But the patch has one hole, following the delta fix:


diff --git a/block/blk-mq.c b/block/blk-mq.c
index 0917f8eabab9..1d9a7ded27af 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2159,7 +2159,7 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx)
 	bool tried = false;
 	int next_cpu = hctx->next_cpu;
 
-	if (hctx->queue->nr_hw_queues == 1)
+	if (hctx->queue->nr_hw_queues == 1 || next_cpu >= nr_cpu_ids)
 		return WORK_CPU_UNBOUND;
 
 	if (--hctx->next_cpu_batch <= 0) {


Thanks,
Ming