Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp1380741imu; Fri, 4 Jan 2019 20:36:58 -0800 (PST) X-Google-Smtp-Source: ALg8bN6epmKGs4jMeMbrvazrZ/5QRr684A7h4zCrge2nMcvKyU92A224wpDPSQDP/CtGwWgfhOih X-Received: by 2002:a17:902:e10a:: with SMTP id cc10mr54112865plb.165.1546663018574; Fri, 04 Jan 2019 20:36:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1546663018; cv=none; d=google.com; s=arc-20160816; b=gfyGvPgL3AsKF16ochOIEAARLtDbq5FvuOX3NEodjw4V30/IH2okkeYspisVWwMIpz xH9CJJMH4sYWl44+Rt0l1xfj3vKXZleVCSCDSvhpluaXDil1oF6T4wz7d+hwfSoxJt+n gC+en2AkLWHMIVEaHFyV3M/zf8EHwp4I2tvqpyDAvNQqliFPYyEE/r4Kd45MY0Ahg5+A jl/tNBMyCzZfwIqagrJobv7zf7+4qiiI/KLphvpaSW+0RxuJ46LtPhQJZTG2/iBIDa0F yVRunyzIc2U2vfplvG0kXDgrCZTQba37Ds+sx1Uw9S4UYUtHexV0rsDC3UlUp5ypNNaS Uj0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=UdhWk2jCXjOswIH7xe6cmxrqB+fK1vn+RvO2k+Im3T8=; b=b5IMQwEVdpE1pDkBVOUKosQ15aLtc2adDrXReiz/vk0AR2S31wlVD8IBSsysz4KzWz O9m8iJuTGnyx1NzaHSq4JCo+mXiIhSnYehFJp8Z1Ak9vZ5n9GdTuJ3FzjMLajd1oU7/F B+H6LlwCzn3JXG9GpmOZjuy4PRQUUHnLPaKuiOBLwBGYeBFIj3+EXHZR9qtPueulOziN usmr6PeE6lggM6vbskHXHWkLWCP5n/DchlELXxgrPfr3P32IuQ4qqD6CFAD91hBfCRq8 CE4X9IeFqaqUsJQK4LCXwrDYlAAilnTDUbKbdnvqMlyspcLWVU6MYdhc6QZypUjPSurQ bEkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=EE3mT0qO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w61si55338617plb.309.2019.01.04.20.36.43; Fri, 04 Jan 2019 20:36:58 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=EE3mT0qO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726249AbfAEEfX (ORCPT + 99 others); Fri, 4 Jan 2019 23:35:23 -0500 Received: from mail-it1-f195.google.com ([209.85.166.195]:34236 "EHLO mail-it1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726057AbfAEEfW (ORCPT ); Fri, 4 Jan 2019 23:35:22 -0500 Received: by mail-it1-f195.google.com with SMTP id x124so2800035itd.1 for ; Fri, 04 Jan 2019 20:35:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=UdhWk2jCXjOswIH7xe6cmxrqB+fK1vn+RvO2k+Im3T8=; b=EE3mT0qOVF69TlTxvXx5NZ4Yejf5LYE3o4WRi6ChBcQkXAd53tjj9jLm7/bcrWOZmo iGlkHequWHVUtbT3CgPHL15fw/JOa/U8ZaOxhfE8JZGlfgve9sWFhGFBWCAKP2gTkh4N vmQzkdlxPGbr5mbgn3GxGQ/CMVG1SjXfS20JySoYlnfp3/3Aww8YrJp8jIbttgvENxCp mhHoNWCpiscob/9KNZqMHaBZEm93AXHKjFGeGGJRPFiwWTgwTxcFO0hYtMAi3KbnxKlb ZcErxEYsTIfqLwfDAzv+YKsjaxNJE1+55mXM5UMX/paA0Sw0TAPZT3stEoBfTkZojlAg qzlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=UdhWk2jCXjOswIH7xe6cmxrqB+fK1vn+RvO2k+Im3T8=; b=LLr9Y4dlH8nnBD5HyE9lTS3W3aJiAHFrFNKzwthkCE2YTn7eFe3bDyx++48dHC83xa RhnL8nqAMfVc86796VbUzdoFqkM843Iw1yQo00Ax8WjNAO0fRQqxkaYC9NAIp2wL+nng //V5/fu1/CNe5oyTka+LqVq0OsguvIAaCtreUp4FWQa/0tZwsx9oWVyMdSYB3oQCK26c ccqSyzodBSzMbMKbMCWZrGfHSQbj6L9rtm8NcSVbw1QB4solF4nU8VH3LhEqlEg2NCmS HJWbWpGKxLOL8xo0F8yKxQOVv0EMwiGapFpbS1zL5XzTF2fv/HE283vW1zDHPd21eVpw CwaQ== X-Gm-Message-State: AA+aEWbMpt7LSTwaq1/LDJhxEUsEQ5Kbd/M8A0h1dD9RmYSDMfm1WPEL NMOGbjDdOo3Hbu58fbjurPk= X-Received: by 2002:a02:94eb:: with SMTP id x98mr35432150jah.88.1546662921163; Fri, 04 Jan 2019 20:35:21 -0800 (PST) Received: from ubu-Virtual-Machine (66-188-57-61.dhcp.bycy.mi.charter.com. [66.188.57.61]) by smtp.gmail.com with ESMTPSA id g21sm1471086itg.44.2019.01.04.20.35.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Jan 2019 20:35:20 -0800 (PST) Date: Fri, 4 Jan 2019 23:35:18 -0500 From: Kimberly Brown To: Michael Kelley , Long Li , Sasha Levin , Dexuan Cui Cc: "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , devel@linuxdriverproject.org, linux-kernel@vger.kernel.org Subject: [PATCH v2] Drivers: hv: vmbus: Expose counters for interrupts and full conditions Message-ID: <20190105043518.GA4072@ubu-Virtual-Machine> References: <0181213190901.GA3843@ubu-Virtual-Machine> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0181213190901.GA3843@ubu-Virtual-Machine> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Counter values for per-channel interrupts and ring buffer full conditions are useful for investigating performance. Expose counters in sysfs for 2 types of guest to host interrupts: 1) Interrupts caused by the channel's outbound ring buffer transitioning from empty to not empty 2) Interrupts caused by the channel's inbound ring buffer transitioning from full to not full while a packet is waiting for enough buffer space to become available Expose 2 counters in sysfs for the number of times that write operations encountered a full outbound ring buffer: 1) The total number of write operations that encountered a full condition 2) The number of write operations that were the first to encounter a full condition I tested this patch by confirming that the sysfs files were created and observing the counter values. The values seemed to increase by a reasonable amount when the Hyper-v related drivers were in use. Signed-off-by: Kimberly Brown --- Changes in v2: - Added mailing lists to the cc list - Removed the host to guest interrupt counters proposed in v1 because they were not accurate - Added full condition counters for the channel's outbound ring buffer Documentation/ABI/stable/sysfs-bus-vmbus | 33 ++++++++++++++++++++++++ drivers/hv/ring_buffer.c | 14 +++++++++- drivers/hv/vmbus_drv.c | 32 +++++++++++++++++++++++ include/linux/hyperv.h | 32 +++++++++++++++++++++++ 4 files changed, 110 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus b/Documentation/ABI/stable/sysfs-bus-vmbus index 3fed8fdb873d..31dc89989032 100644 --- a/Documentation/ABI/stable/sysfs-bus-vmbus +++ b/Documentation/ABI/stable/sysfs-bus-vmbus @@ -146,3 +146,36 @@ KernelVersion: 4.16 Contact: Stephen Hemminger Description: Binary file created by uio_hv_generic for ring buffer Users: Userspace drivers + +What: /sys/bus/vmbus/devices//channels//intr_in_full +Date: January 2019 +KernelVersion: 4.21 +Contact: Michael Kelley +Description: Number of guest to host interrupts caused by the inbound ring + buffer transitioning from full to not full while a packet is + waiting for buffer space to become available +Users: Debugging tools + +What: /sys/bus/vmbus/devices//channels//intr_out_empty +Date: January 2019 +KernelVersion: 4.21 +Contact: Michael Kelley +Description: Number of guest to host interrupts caused by the outbound ring + buffer transitioning from empty to not empty +Users: Debugging tools + +What: /sys/bus/vmbus/devices//channels//out_full_first +Date: January 2019 +KernelVersion: 4.21 +Contact: Michael Kelley +Description: Number of write operations that were the first to encounter an + outbound ring buffer full condition +Users: Debugging tools + +What: /sys/bus/vmbus/devices//channels//out_full_total +Date: January 2019 +KernelVersion: 4.21 +Contact: Michael Kelley +Description: Total number of write operations that encountered an outbound + ring buffer full condition +Users: Debugging tools diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c index 64d0c85d5161..be2cbd0bea7d 100644 --- a/drivers/hv/ring_buffer.c +++ b/drivers/hv/ring_buffer.c @@ -74,8 +74,10 @@ static void hv_signal_on_write(u32 old_write, struct vmbus_channel *channel) * This is the only case we need to signal when the * ring transitions from being empty to non-empty. */ - if (old_write == READ_ONCE(rbi->ring_buffer->read_index)) + if (old_write == READ_ONCE(rbi->ring_buffer->read_index)) { + ++channel->intr_out_empty; vmbus_setevent(channel); + } } /* Get the next write location for the specified ring buffer. */ @@ -273,10 +275,19 @@ int hv_ringbuffer_write(struct vmbus_channel *channel, * is empty since the read index == write index. */ if (bytes_avail_towrite <= totalbytes_towrite) { + ++channel->out_full_total; + + if (!channel->out_full_flag) { + ++channel->out_full_first; + channel->out_full_flag = true; + } + spin_unlock_irqrestore(&outring_info->ring_lock, flags); return -EAGAIN; } + channel->out_full_flag = false; + /* Write to the ring buffer */ next_write_location = hv_get_next_write_location(outring_info); @@ -531,6 +542,7 @@ void hv_pkt_iter_close(struct vmbus_channel *channel) if (curr_write_sz <= pending_sz) return; + ++channel->intr_in_full; vmbus_setevent(channel); } EXPORT_SYMBOL_GPL(hv_pkt_iter_close); diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 283d184280af..460b7f75251c 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -1445,6 +1445,34 @@ static ssize_t channel_events_show(const struct vmbus_channel *channel, char *bu } static VMBUS_CHAN_ATTR(events, S_IRUGO, channel_events_show, NULL); +static ssize_t channel_intr_in_full_show(const struct vmbus_channel *channel, + char *buf) +{ + return sprintf(buf, "%llu\n", channel->intr_in_full); +} +static VMBUS_CHAN_ATTR(intr_in_full, 0444, channel_intr_in_full_show, NULL); + +static ssize_t channel_intr_out_empty_show(const struct vmbus_channel *channel, + char *buf) +{ + return sprintf(buf, "%llu\n", channel->intr_out_empty); +} +static VMBUS_CHAN_ATTR(intr_out_empty, 0444, channel_intr_out_empty_show, NULL); + +static ssize_t channel_out_full_first_show(const struct vmbus_channel *channel, + char *buf) +{ + return sprintf(buf, "%llu\n", channel->out_full_first); +} +static VMBUS_CHAN_ATTR(out_full_first, 0444, channel_out_full_first_show, NULL); + +static ssize_t channel_out_full_total_show(const struct vmbus_channel *channel, + char *buf) +{ + return sprintf(buf, "%llu\n", channel->out_full_total); +} +static VMBUS_CHAN_ATTR(out_full_total, 0444, channel_out_full_total_show, NULL); + static ssize_t subchannel_monitor_id_show(const struct vmbus_channel *channel, char *buf) { @@ -1470,6 +1498,10 @@ static struct attribute *vmbus_chan_attrs[] = { &chan_attr_latency.attr, &chan_attr_interrupts.attr, &chan_attr_events.attr, + &chan_attr_intr_in_full.attr, + &chan_attr_intr_out_empty.attr, + &chan_attr_out_full_first.attr, + &chan_attr_out_full_total.attr, &chan_attr_monitor_id.attr, &chan_attr_subchannel_id.attr, NULL diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index b3e24368930a..673f58c4bb1d 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -751,6 +751,27 @@ struct vmbus_channel { u64 interrupts; /* Host to Guest interrupts */ u64 sig_events; /* Guest to Host events */ + /* Interrupt counts for 2 types of Guest to Host interrupts */ + u64 intr_in_full; /* in ring buffer, full to not full */ + u64 intr_out_empty; /* out ring buffer, empty to not empty */ + + /* + * The total number of write operations that encountered a full + * outbound ring buffer. + */ + u64 out_full_total; + /* + * The number of write operations that were the first to encounter a + * full outbound ring buffer. + */ + u64 out_full_first; + /* + * Indicates that a full outbound ring buffer was encountered. The flag + * is set to true when a full outbound ring buffer is encountered and + * set to false when a write to the outbound ring buffer is completed. + */ + bool out_full_flag; + /* Channel callback's invoked in softirq context */ struct tasklet_struct callback_event; void (*onchannel_callback)(void *context); @@ -938,6 +959,17 @@ static inline void *get_per_channel_state(struct vmbus_channel *c) static inline void set_channel_pending_send_size(struct vmbus_channel *c, u32 size) { + if (size) { + ++c->out_full_total; + + if (!c->out_full_flag) { + ++c->out_full_first; + c->out_full_flag = true; + } + } else { + c->out_full_flag = false; + } + c->outbound.ring_buffer->pending_send_sz = size; } -- 2.17.1