Received-SPF: pass (google.com: domain of linux-kernel+bounces-36851-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240122110722.690223-1-yi.sun@unisoc.com> <20240122110722.690223-2-yi.sun@unisoc.com>
 <ZbAQLRLNSx-IRRwM@kbusch-mbp.dhcp.thefacebook.com>
In-Reply-To: <ZbAQLRLNSx-IRRwM@kbusch-mbp.dhcp.thefacebook.com>
From: yi sun <sunyibuaa@gmail.com>
Date: Wed, 24 Jan 2024 19:22:21 +0800
Message-ID: <CALpufv1XRYX7bv-NAUMmRjWr0jhDC1xrd+d48XejJZPjZZLEYQ@mail.gmail.com>
Subject: Re: [PATCH 1/2] blk-mq: introduce blk_mq_tagset_wait_request_completed()
To: Keith Busch <kbusch@kernel.org>
Cc: Yi Sun <yi.sun@unisoc.com>, axboe@kernel.dk, mst@redhat.com, jasowang@redhat.com, 
	xuanzhuo@linux.alibaba.com, pbonzini@redhat.com, stefanha@redhat.com, 
	virtualization@lists.linux.dev, linux-block@vger.kernel.org, 
	linux-kernel@vger.kernel.org, zhiguo.niu@unisoc.com, hongyu.jin@unisoc.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

In my case, I want all hw queues owned by this device to be clean.
Because in the virtio device, each hw queue corresponds to a virtqueue,
and all virtqueues will be deleted when vdev suspends.

The blk_mq_tagset_wait_request_completed() function can ensure that
the device has processed all in_flight requests , and these requests have
become the complete state. I don=E2=80=99t understand the blk_mq_freeze_que=
ue()
function very well. Can the function ensure that all requests have become
complete status? How should I use the function to achieve the same effect?

ps: requests become in_flight status in virtio_queue_rq() and become comple=
te
status in virtblk_done().

Yi

On Wed, Jan 24, 2024 at 3:14=E2=80=AFAM Keith Busch <kbusch@kernel.org> wro=
te:
>
> On Mon, Jan 22, 2024 at 07:07:21PM +0800, Yi Sun wrote:
> > In some cases, it is necessary to wait for all requests to become compl=
ete
> > status before performing other operations. Otherwise, these requests wi=
ll never
> > be processed successfully.
> >
> > For example, when the virtio device is in hibernation, the virtqueues
> > will be deleted. It must be ensured that virtqueue is not in use before=
 being deleted.
> > Otherwise the requests in the virtqueue will be lost. This function can=
 ensure
> > that all requests have been taken out of the virtqueues.
> >
> > Prepare for fixing this kind of issue by introducing
> > blk_mq_tagset_wait_request_completed().
>
> Does blk_mq_freeze_queue() not work for your use case? I think that
> should work unless you have some driver specific requests entered that
> don't ever get released.
>
> > +static bool blk_mq_tagset_count_inflight_rqs(struct request *rq, void =
*data)
> > +{
> > +     unsigned int *count =3D data;
> > +
> > +     if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq))
> > +             (*count)++;
> > +     return true;
> > +}
> > +
> > +/**
> > + * blk_mq_tagset_wait_request_completed - Wait for all inflight reques=
ts
> > + * to become completed.
> > + *
> > + * Note: This function has to be run after all IO queues are shutdown.
> > + */
> > +void blk_mq_tagset_wait_request_completed(struct blk_mq_tag_set *tagse=
t)
> > +{
> > +     while (true) {
> > +             unsigned int count =3D 0;
> > +
> > +             blk_mq_tagset_busy_iter(tagset,
> > +                             blk_mq_tagset_count_inflight_rqs, &count)=
;
>
> If the tagset is shared, then one active user can prevent this from ever
> completing. It sounds like your use case cares about just one specific
> request_queue, not all of them.
>
> > +             if (!count)
> > +                     break;
> > +             msleep(20);
> > +     }
> > +}
> > +EXPORT_SYMBOL(blk_mq_tagset_wait_request_completed);