Received-SPF: pass (google.com: domain of linux-kernel+bounces-35975-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Tue, 23 Jan 2024 12:14:53 -0700
From: Keith Busch <kbusch@kernel.org>
To: Yi Sun <yi.sun@unisoc.com>
Cc: axboe@kernel.dk, mst@redhat.com, jasowang@redhat.com,
	xuanzhuo@linux.alibaba.com, pbonzini@redhat.com,
	stefanha@redhat.com, virtualization@lists.linux.dev,
	linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
	zhiguo.niu@unisoc.com, hongyu.jin@unisoc.com, sunyibuaa@gmail.com
Subject: Re: [PATCH 1/2] blk-mq: introduce
 blk_mq_tagset_wait_request_completed()
Message-ID: <ZbAQLRLNSx-IRRwM@kbusch-mbp.dhcp.thefacebook.com>
References: <20240122110722.690223-1-yi.sun@unisoc.com>
 <20240122110722.690223-2-yi.sun@unisoc.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240122110722.690223-2-yi.sun@unisoc.com>

On Mon, Jan 22, 2024 at 07:07:21PM +0800, Yi Sun wrote:
> In some cases, it is necessary to wait for all requests to become complete
> status before performing other operations. Otherwise, these requests will never
> be processed successfully.
> 
> For example, when the virtio device is in hibernation, the virtqueues
> will be deleted. It must be ensured that virtqueue is not in use before being deleted.
> Otherwise the requests in the virtqueue will be lost. This function can ensure
> that all requests have been taken out of the virtqueues.
> 
> Prepare for fixing this kind of issue by introducing
> blk_mq_tagset_wait_request_completed().

Does blk_mq_freeze_queue() not work for your use case? I think that
should work unless you have some driver specific requests entered that
don't ever get released.
 
> +static bool blk_mq_tagset_count_inflight_rqs(struct request *rq, void *data)
> +{
> +	unsigned int *count = data;
> +
> +	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq))
> +		(*count)++;
> +	return true;
> +}
> +
> +/**
> + * blk_mq_tagset_wait_request_completed - Wait for all inflight requests
> + * to become completed.
> + *
> + * Note: This function has to be run after all IO queues are shutdown.
> + */
> +void blk_mq_tagset_wait_request_completed(struct blk_mq_tag_set *tagset)
> +{
> +	while (true) {
> +		unsigned int count = 0;
> +
> +		blk_mq_tagset_busy_iter(tagset,
> +				blk_mq_tagset_count_inflight_rqs, &count);

If the tagset is shared, then one active user can prevent this from ever
completing. It sounds like your use case cares about just one specific
request_queue, not all of them.

> +		if (!count)
> +			break;
> +		msleep(20);
> +	}
> +}
> +EXPORT_SYMBOL(blk_mq_tagset_wait_request_completed);