Received: by 2002:ac0:a5a7:0:0:0:0:0 with SMTP id m36-v6csp641imm; Tue, 10 Jul 2018 19:29:23 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfI62eajq2zStIFs5VNMU9Zhjmml+esMqF+KCeAe+l3LgmZjzQykubZ/q6Plf2/+6MwdkEO X-Received: by 2002:a17:902:7c0a:: with SMTP id x10-v6mr2860448pll.77.1531276163556; Tue, 10 Jul 2018 19:29:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531276163; cv=none; d=google.com; s=arc-20160816; b=w5TUJHtcHTNv4ALKc8yxTtLtjXoAChlA16Nl6JNN2Lg1CAU88T8yXXzBEmMX4lZxkW oOL2KvYYnJ/lp+v8pETp0drD6isueFkuCwAiHx2vAAXXPWwXPrsGLqqwKrEfLObWbAGx rTHLeKs+zO393kf8OtmgV+kAO+8FvH7/f0dwYuuzrPvKPx46nCGBl9LK52g21fxOZXLB /TtPk9KRy092wDsljFlA6cs0zYU4vp+HqdymfNra0XDsd/HU5s8jUC34udzMRUDYcTkD wq6UWqoJOL2rdkrgr/lJQ+/vTvgHA8g8wtlQ0LYl2MldaWfiBsMPBGqAwlFCwHDW+7dH 0wTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=0zZL31+YyzBkiXvfEtt16aX8KFYqgEaRQVND2Bq5RHY=; b=AN3MXGPR/ABHMLs6nPoH2lf+SpjAvIF8+syRBGdMhibx3bu0yR5EZDN1DMAkvgDf+e uNX2BOXq7eIAd5aJYjMDmWfloSXwmENvu31YFFWKLCoUZ3fg7Dem5upaqmIMHMc+PsH/ LwlyDmwai6p1J/xmnqEgQXL+iLT2DBGKJ7LAIiJ2YpFrOmvK3WdraKFI/UxqqChxSv0x Q3BGOyLwc7eUpZwuqxM5711TmqZixc8pn+MLf4qRl+YRWQmNW2VE8pMlRlAZLvD+I2+C eMMmh8cMMIA4w9dz+DfGqYr24tX9/qP/RjGntZk/b3WE4dUJq+O9l4+aXze7jrDKZecw /D8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h35-v6si16627276pgl.176.2018.07.10.19.29.08; Tue, 10 Jul 2018 19:29:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732755AbeGKC3l (ORCPT + 99 others); Tue, 10 Jul 2018 22:29:41 -0400 Received: from mga11.intel.com ([192.55.52.93]:24665 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732524AbeGKC3j (ORCPT ); Tue, 10 Jul 2018 22:29:39 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Jul 2018 19:27:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,336,1526367600"; d="scan'208";a="71744499" Received: from debian.sh.intel.com ([10.67.104.228]) by orsmga001.jf.intel.com with ESMTP; 10 Jul 2018 19:27:40 -0700 From: Tiwei Bie To: mst@redhat.com, jasowang@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, virtio-dev@lists.oasis-open.org Cc: wexu@redhat.com, jfreimann@redhat.com, tiwei.bie@intel.com Subject: [PATCH net-next v2 4/5] virtio_ring: add event idx support in packed ring Date: Wed, 11 Jul 2018 10:27:10 +0800 Message-Id: <20180711022711.7090-5-tiwei.bie@intel.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180711022711.7090-1-tiwei.bie@intel.com> References: <20180711022711.7090-1-tiwei.bie@intel.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This commit introduces the EVENT_IDX support in packed ring. Signed-off-by: Tiwei Bie --- drivers/virtio/virtio_ring.c | 73 ++++++++++++++++++++++++++++++++---- 1 file changed, 65 insertions(+), 8 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index f317b485ba54..f79a1e17f7d1 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1050,7 +1050,7 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, static bool virtqueue_kick_prepare_packed(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); - u16 flags; + u16 new, old, off_wrap, flags, wrap_counter, event_idx; bool needs_kick; u32 snapshot; @@ -1059,9 +1059,19 @@ static bool virtqueue_kick_prepare_packed(struct virtqueue *_vq) * suppressions. */ virtio_mb(vq->weak_barriers); + old = vq->next_avail_idx - vq->num_added; + new = vq->next_avail_idx; + vq->num_added = 0; + snapshot = READ_ONCE(*(u32 *)vq->vring_packed.device); + off_wrap = virtio16_to_cpu(_vq->vdev, (__virtio16)(snapshot & 0xffff)); flags = virtio16_to_cpu(_vq->vdev, (__virtio16)(snapshot >> 16)) & 0x3; + wrap_counter = off_wrap >> 15; + event_idx = off_wrap & ~(1 << 15); + if (wrap_counter != vq->avail_wrap_counter) + event_idx -= vq->vring_packed.num; + #ifdef DEBUG if (vq->last_add_time_valid) { WARN_ON(ktime_to_ms(ktime_sub(ktime_get(), @@ -1070,7 +1080,10 @@ static bool virtqueue_kick_prepare_packed(struct virtqueue *_vq) vq->last_add_time_valid = false; #endif - needs_kick = (flags != VRING_EVENT_F_DISABLE); + if (flags == VRING_EVENT_F_DESC) + needs_kick = vring_need_event(event_idx, new, old); + else + needs_kick = (flags != VRING_EVENT_F_DISABLE); END_USE(vq); return needs_kick; } @@ -1185,6 +1198,15 @@ static void *virtqueue_get_buf_ctx_packed(struct virtqueue *_vq, ret = vq->desc_state_packed[id].data; detach_buf_packed(vq, id, ctx); + /* If we expect an interrupt for the next entry, tell host + * by writing event index and flush out the write before + * the read in the next get_buf call. */ + if (vq->event_flags_shadow == VRING_EVENT_F_DESC) + virtio_store_mb(vq->weak_barriers, + &vq->vring_packed.driver->off_wrap, + cpu_to_virtio16(_vq->vdev, vq->last_used_idx | + ((u16)vq->used_wrap_counter << 15))); + #ifdef DEBUG vq->last_add_time_valid = false; #endif @@ -1213,8 +1235,18 @@ static unsigned virtqueue_enable_cb_prepare_packed(struct virtqueue *_vq) /* We optimistically turn back on interrupts, then check if there was * more to do. */ + if (vq->event) { + vq->vring_packed.driver->off_wrap = cpu_to_virtio16(_vq->vdev, + vq->last_used_idx | + ((u16)vq->used_wrap_counter << 15)); + /* We need to update event offset and event wrap + * counter first before updating event flags. */ + virtio_wmb(vq->weak_barriers); + } + if (vq->event_flags_shadow == VRING_EVENT_F_DISABLE) { - vq->event_flags_shadow = VRING_EVENT_F_ENABLE; + vq->event_flags_shadow = vq->event ? VRING_EVENT_F_DESC : + VRING_EVENT_F_ENABLE; vq->vring_packed.driver->flags = cpu_to_virtio16(_vq->vdev, vq->event_flags_shadow); } @@ -1238,22 +1270,47 @@ static bool virtqueue_poll_packed(struct virtqueue *_vq, unsigned off_wrap) static bool virtqueue_enable_cb_delayed_packed(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); + u16 bufs, used_idx, wrap_counter; START_USE(vq); /* We optimistically turn back on interrupts, then check if there was * more to do. */ + if (vq->event) { + /* TODO: tune this threshold */ + bufs = (vq->vring_packed.num - _vq->num_free) * 3 / 4; + wrap_counter = vq->used_wrap_counter; + + used_idx = vq->last_used_idx + bufs; + if (used_idx >= vq->vring_packed.num) { + used_idx -= vq->vring_packed.num; + wrap_counter ^= 1; + } + + vq->vring_packed.driver->off_wrap = cpu_to_virtio16(_vq->vdev, + used_idx | (wrap_counter << 15)); + + /* We need to update event offset and event wrap + * counter first before updating event flags. */ + virtio_wmb(vq->weak_barriers); + } else { + used_idx = vq->last_used_idx; + wrap_counter = vq->used_wrap_counter; + } + if (vq->event_flags_shadow == VRING_EVENT_F_DISABLE) { - vq->event_flags_shadow = VRING_EVENT_F_ENABLE; + vq->event_flags_shadow = vq->event ? VRING_EVENT_F_DESC : + VRING_EVENT_F_ENABLE; vq->vring_packed.driver->flags = cpu_to_virtio16(_vq->vdev, vq->event_flags_shadow); - /* We need to enable interrupts first before re-checking - * for more used buffers. */ - virtio_mb(vq->weak_barriers); } - if (more_used_packed(vq)) { + /* We need to update event suppression structure first + * before re-checking for more used buffers. */ + virtio_mb(vq->weak_barriers); + + if (is_used_desc_packed(vq, used_idx, wrap_counter)) { END_USE(vq); return false; } -- 2.18.0