Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp203205imu; Thu, 8 Nov 2018 17:53:45 -0800 (PST) X-Google-Smtp-Source: AJdET5cCePepXmivie+Xlukv66O54IIFUHZ9HghKh1j2Zv685bEh+OwBtmrM3oOLHg1ulL81nNj0 X-Received: by 2002:a63:a41:: with SMTP id z1mr5809523pgk.117.1541728424953; Thu, 08 Nov 2018 17:53:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541728424; cv=none; d=google.com; s=arc-20160816; b=Y4zFOEz4d6q/WzA3pqY1X9pk/gFaylgn7Y56UKj1iJOIgLaPdMO9GA5AlzuD3VWZ44 ivMHiKv+vj7WQdglVvK0f+yvVQP5n7mktAGk0IDSIKKrNYb4kVom6osdHXGF/Q70x8s3 2H0U2Ji0uWOKtNdZs/DwS9d/FLlv7+MvJW86tkr3wKUgNIvs2lu/ewHTfWUDMV8eHUnG foMT2Z46caveGBDvoKBFSYmqUiyGhFV7zHN8+SaHoCsk+B4U1XwIQLEDLDWf0sAJ1pty 81GLKJS2nNP2uQwPiCWCxTOHLskV7qmK9ornSJw30iZs53L8aDmMzJdlyPb23Nkp5AGm ZYbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date; bh=oC9WQ9R+Ku5TKztev3mBvF+zcLQ7zv7Rf6ErD7ROUBA=; b=QChj5XjRN2XOV1KKMjLmL9G426AH/UtOXeGjrK5Fj6HOyRMqabDZ7s0iYcwFK8iaJa SObX858Fo369z1VW7CQ8nFZJ1xM4MN57Fn58ff8WXS0bzpsNc83I3McoW2KvXsH1m1UW NRzrki2KFLed9sQG5XbKa6kE9O6cFx9BnKR39xUE3DBW0BH1cDwfw3EjtIwv+t3HRW0I 3jHY8HB0e8eHW6kwhG36yJqCicsZsvl7LFF7/f0HUo//IeIMkZke8UejTq7ft6EXsy+j s3xamWBVZxNLSKb1zVlR46C0kwQS73dZZD4FNxD5yXx4tZIyzpNrxCwfn8z1b0khI3d1 TMJA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a5-v6si1234669plh.157.2018.11.08.17.53.28; Thu, 08 Nov 2018 17:53:44 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727803AbeKILaJ (ORCPT + 99 others); Fri, 9 Nov 2018 06:30:09 -0500 Received: from mga02.intel.com ([134.134.136.20]:32618 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727157AbeKILaJ (ORCPT ); Fri, 9 Nov 2018 06:30:09 -0500 X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Nov 2018 17:51:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,481,1534834800"; d="scan'208";a="98801745" Received: from btwcube1.sh.intel.com (HELO debian) ([10.67.104.173]) by orsmga003.jf.intel.com with ESMTP; 08 Nov 2018 17:51:45 -0800 Date: Fri, 9 Nov 2018 09:50:11 +0800 From: Tiwei Bie To: "Michael S. Tsirkin" Cc: Jason Wang , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, virtio-dev@lists.oasis-open.org, wexu@redhat.com, jfreimann@redhat.com Subject: Re: [PATCH net-next v2 3/5] virtio_ring: add packed ring support Message-ID: <20181109015011.GA24544@debian> References: <20180711022711.7090-1-tiwei.bie@intel.com> <20180711022711.7090-4-tiwei.bie@intel.com> <20181107123933-mutt-send-email-mst@kernel.org> <20181108013759.GA20591@debian> <2d46a41e-bc00-276a-e19a-105c9dffc75a@redhat.com> <20181108115148.GA15701@debian> <20181108103155-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20181108103155-mutt-send-email-mst@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 08, 2018 at 10:56:02AM -0500, Michael S. Tsirkin wrote: > On Thu, Nov 08, 2018 at 07:51:48PM +0800, Tiwei Bie wrote: > > On Thu, Nov 08, 2018 at 04:18:25PM +0800, Jason Wang wrote: > > > > > > On 2018/11/8 上午9:38, Tiwei Bie wrote: > > > > > > + > > > > > > + if (vq->vq.num_free < descs_used) { > > > > > > + pr_debug("Can't add buf len %i - avail = %i\n", > > > > > > + descs_used, vq->vq.num_free); > > > > > > + /* FIXME: for historical reasons, we force a notify here if > > > > > > + * there are outgoing parts to the buffer. Presumably the > > > > > > + * host should service the ring ASAP. */ > > > > > I don't think we have a reason to do this for packed ring. > > > > > No historical baggage there, right? > > > > Based on the original commit log, it seems that the notify here > > > > is just an "optimization". But I don't quite understand what does > > > > the "the heuristics which KVM uses" refer to. If it's safe to drop > > > > this in packed ring, I'd like to do it. > > > > > > > > > According to the commit log, it seems like a workaround of lguest networking > > > backend. > > > > Do you know why removing this notify in Tx will break "the > > heuristics which KVM uses"? Or what does "the heuristics > > which KVM uses" refer to? > > Yes. QEMU has a mode where it disables notifications and processes TX > ring periodically from a timer. It's off by default but used to be on > by default a long time ago. If ring becomes full this causes traffic > stalls. As a work-around Rusty put in this hack to kick on ring full > even with notifications disabled. It's easy enough to make sure QEMU > does not combine devices with packed ring support with the timer hack. > And I am guessing it's safe enough to also block that option completely > e.g. when virtio 1.0 is enabled. I see. Thanks! > > > > > > I agree to drop it, we should not have such burden. > > > > > > But we should notice that, with this removed, the compare between packed vs > > > split is kind of unfair. Consider the removal of lguest support recently, > > > maybe we can drop this for split ring as well? > > > > > > Thanks > > > > > > > > > > > > > > commit 44653eae1407f79dff6f52fcf594ae84cb165ec4 > > > > Author: Rusty Russell > > > > Date: Fri Jul 25 12:06:04 2008 -0500 > > > > > > > > virtio: don't always force a notification when ring is full > > > > We force notification when the ring is full, even if the host has > > > > indicated it doesn't want to know. This seemed like a good idea at > > > > the time: if we fill the transmit ring, we should tell the host > > > > immediately. > > > > Unfortunately this logic also applies to the receiving ring, which is > > > > refilled constantly. We should introduce real notification thesholds > > > > to replace this logic. Meanwhile, removing the logic altogether breaks > > > > the heuristics which KVM uses, so we use a hack: only notify if there are > > > > outgoing parts of the new buffer. > > > > Here are the number of exits with lguest's crappy network implementation: > > > > Before: > > > > network xmit 7859051 recv 236420 > > > > After: > > > > network xmit 7858610 recv 118136 > > > > Signed-off-by: Rusty Russell > > > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > > > index 72bf8bc09014..21d9a62767af 100644 > > > > --- a/drivers/virtio/virtio_ring.c > > > > +++ b/drivers/virtio/virtio_ring.c > > > > @@ -87,8 +87,11 @@ static int vring_add_buf(struct virtqueue *_vq, > > > > if (vq->num_free < out + in) { > > > > pr_debug("Can't add buf len %i - avail = %i\n", > > > > out + in, vq->num_free); > > > > - /* We notify*even if* VRING_USED_F_NO_NOTIFY is set here. */ > > > > - vq->notify(&vq->vq); > > > > + /* FIXME: for historical reasons, we force a notify here if > > > > + * there are outgoing parts to the buffer. Presumably the > > > > + * host should service the ring ASAP. */ > > > > + if (out) > > > > + vq->notify(&vq->vq); > > > > END_USE(vq); > > > > return -ENOSPC; > > > > } > > > > > > > >