Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1101BC43217 for ; Mon, 29 Nov 2021 16:07:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244750AbhK2QK0 (ORCPT ); Mon, 29 Nov 2021 11:10:26 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:49094 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232047AbhK2QIZ (ORCPT ); Mon, 29 Nov 2021 11:08:25 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6B26CB811F5; Mon, 29 Nov 2021 16:05:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F363C53FAD; Mon, 29 Nov 2021 16:05:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1638201905; bh=4N87j1xF0qZOK6vkTN+k9+VLDwxHQq/Q1A1VR38B4AQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=hYd722QiYpC36Bf6Q9uJFPlKPCw4bp52LuJo88+FRlqHBDPZiJV2xxZjo5MUc2moR hNcG5kc9SZg08L6NdNQ0reOmJIK1mIvsFygISYhUiJ6MF9siH9EUqUOj7eLQsFWlvc 73QsXWTh7LmGU+fDc5lAIFUsUpf+NXCnpB4qQASFhFOqBauVjGuzAs991gYajYQElr ng6WG+dmtohCxSRwl0esJqyoAD4Ye6lAZMKHbWz1JP10FvCcHktWdX+dRAK2TARvCP ek5C6JqhSLqcO9P9xiKcnXSQiLy6RbxYR9QImV2JZIjuBM/KXbDM66Sto0EIB70cHi 6KEDTLA4Ptsfg== Date: Mon, 29 Nov 2021 08:05:02 -0800 From: Jakub Kicinski To: Petr Machata Cc: Ido Schimmel , Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdl?= =?UTF-8?B?bnNlbg==?= , Alexander Lobakin , Daniel Borkmann , "David S. Miller" , Jesse Brandeburg , Michal Swiatkowski , Maciej Fijalkowski , Jonathan Corbet , "Shay Agroskin" , Arthur Kiyanovski , "David Arinzon" , Noam Dagan , "Saeed Bishara" , Ioana Ciornei , "Claudiu Manoil" , Tony Nguyen , Thomas Petazzoni , Marcin Wojtas , Russell King , Saeed Mahameed , Leon Romanovsky , Alexei Starovoitov , Jesper Dangaard Brouer , John Fastabend , Edward Cree , "Martin Habets" , "Michael S. Tsirkin" , Jason Wang , Andrii Nakryiko , "Martin KaFai Lau" , Song Liu , Yonghong Song , KP Singh , Lorenzo Bianconi , Yajun Deng , Sergey Ryazanov , David Ahern , Andrei Vagin , Johannes Berg , "Vladimir Oltean" , Cong Wang , , , , , , , Subject: Re: [PATCH v2 net-next 21/26] ice: add XDP and XSK generic per-channel statistics Message-ID: <20211129080502.53f7d316@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> In-Reply-To: <874k7vq7tl.fsf@nvidia.com> References: <20211123163955.154512-22-alexandr.lobakin@intel.com> <77407c26-4e32-232c-58e0-2d601d781f84@iogearbox.net> <87bl28bga6.fsf@toke.dk> <20211125170708.127323-1-alexandr.lobakin@intel.com> <20211125094440.6c402d63@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211125204007.133064-1-alexandr.lobakin@intel.com> <87sfvj9k13.fsf@toke.dk> <20211126100611.514df099@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <87ee72ah56.fsf@toke.dk> <20211126111431.4a2ed007@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211129064755.539099c0@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <874k7vq7tl.fsf@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 29 Nov 2021 16:51:02 +0100 Petr Machata wrote: > Jakub Kicinski writes: > > On Sun, 28 Nov 2021 19:54:53 +0200 Ido Schimmel wrote: > >> I agree. I think IFLA_STATS64 in RTM_NEWLINK should contain statistics > >> of all the packets seen by the netdev. The breakdown into software / > >> hardware / XDP should be reported via RTM_NEWSTATS. > > > > Hm, in the offload case "seen by the netdev" may be unclear. For > > the offload case I believe our recommendation was phrased more like > > "all packets which would be seen by the netdev if there was no > > routing/tc offload", right? > > Yes. The idea is to expose to Linux stats about traffic at conceptually > corresponding objects in the HW. Great. > >> Currently, for soft devices such as VLANs, bridges and GRE, user space > >> only sees statistics of packets forwarded by software, which is quite > >> useless when forwarding is offloaded from the kernel to hardware. > >> > >> Petr is working on exposing hardware statistics for such devices via > >> rtnetlink. Unlike XDP (?), we need to be able to let user space enable / > >> disable hardware statistics as we have a limited number of hardware > >> counters and they can also reduce the bandwidth when enabled. We are > >> thinking of adding a new RTM_SETSTATS for that: > >> > >> # ip stats set dev swp1 hw_stats on > > > > Does it belong on the switch port? Not the netdev we want to track? > > Yes, it does, and is designed that way. That was just muscle memory > typing that "swp1" above :) > > You would do e.g. "ip stats set dev swp1.200 hw_stats on" or, "dev br1", > or something like that. I see :) > >> For query, something like (under discussion): > >> > >> # ip stats show dev swp1 // all groups > >> # ip stats show dev swp1 group link > >> # ip stats show dev swp1 group offload // all sub-groups > >> # ip stats show dev swp1 group offload sub-group cpu > >> # ip stats show dev swp1 group offload sub-group hw > >> > >> Like other iproute2 commands, these follow the nesting of the > >> RTM_{NEW,GET}STATS uAPI. > > > > But we do have IFLA_STATS_LINK_OFFLOAD_XSTATS, isn't it effectively > > the same use case? > > IFLA_STATS_LINK_OFFLOAD_XSTATS is a nest. Currently it carries just > CPU_HIT stats. The idea is to carry HW stats as well in that group. Hm, the expectation was that the HW stats == total - SW. I believe that still holds true for you, even if HW stats are not "complete" (e.g. user enabled them after device was already forwarding for a while). Is the concern about backward compat or such?