Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 2 Feb 2022 06:24:05 -0500
From:   "Michael S. Tsirkin" <mst@redhat.com>
To:     Stefan Hajnoczi <stefanha@redhat.com>
Cc:     Stefano Garzarella <sgarzare@redhat.com>,
        virtualization@lists.linux-foundation.org, netdev@vger.kernel.org,
        Jason Wang <jasowang@redhat.com>, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Subject: Re: [PATCH v3] vhost: cache avail index in vhost_enable_notify()
Message-ID: <20220202062340-mutt-send-email-mst@kernel.org>
References: <20220128094129.40809-1-sgarzare@redhat.com>
 <Yfpnlv2GudpPFwok@stefanha-x1.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Yfpnlv2GudpPFwok@stefanha-x1.localdomain>
Precedence: bulk

On Wed, Feb 02, 2022 at 11:14:30AM +0000, Stefan Hajnoczi wrote:
> On Fri, Jan 28, 2022 at 10:41:29AM +0100, Stefano Garzarella wrote:
> > In vhost_enable_notify() we enable the notifications and we read
> > the avail index to check if new buffers have become available in
> > the meantime.
> > 
> > We do not update the cached avail index value, so when the device
> > will call vhost_get_vq_desc(), it will find the old value in the
> > cache and it will read the avail index again.
> > 
> > It would be better to refresh the cache every time we read avail
> > index, so let's change vhost_enable_notify() caching the value in
> > `avail_idx` and compare it with `last_avail_idx` to check if there
> > are new buffers available.
> > 
> > We don't expect a significant performance boost because
> > the above path is not very common, indeed vhost_enable_notify()
> > is often called with unlikely(), expecting that avail index has
> > not been updated.
> > 
> > We ran virtio-test/vhost-test and noticed minimal improvement as
> > expected. To stress the patch more, we modified vhost_test.ko to
> > call vhost_enable_notify()/vhost_disable_notify() on every cycle
> > when calling vhost_get_vq_desc(); in this case we observed a more
> > evident improvement, with a reduction of the test execution time
> > of about 3.7%.
> > 
> > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> > ---
> > v3
> > - reworded commit description [Stefan]
> > ---
> >  drivers/vhost/vhost.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> > index 59edb5a1ffe2..07363dff559e 100644
> > --- a/drivers/vhost/vhost.c
> > +++ b/drivers/vhost/vhost.c
> > @@ -2543,8 +2543,9 @@ bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
> >  		       &vq->avail->idx, r);
> >  		return false;
> >  	}
> > +	vq->avail_idx = vhost16_to_cpu(vq, avail_idx);
> >  
> > -	return vhost16_to_cpu(vq, avail_idx) != vq->avail_idx;
> > +	return vq->avail_idx != vq->last_avail_idx;
> >  }
> >  EXPORT_SYMBOL_GPL(vhost_enable_notify);
> 
> This changes behavior (fixes a bug?): previously the function returned
> false when called with avail buffers still pending (vq->last_avail_idx <
> vq->avail_idx). Now it returns true because we compare against
> vq->last_avail_idx and I think that's reasonable.
> 
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

I don't see the behaviour change... could you explain the
scanario in more detail pls?

Thanks!