Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Tue, 8 Mar 2022 16:28:02 -0800
From:   Jakub Kicinski <kuba@kernel.org>
To:     Eric Dumazet <edumazet@google.com>
Cc:     David Ahern <dsahern@kernel.org>,
        Jeffrey Ji <jeffreyjilinux@gmail.com>,
        "David S . Miller" <davem@davemloft.net>,
        Brian Vazquez <brianvv@google.com>,
        Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
        Antoine Tenart <atenart@kernel.org>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        Alexander Lobakin <alexandr.lobakin@intel.com>,
        Kumar Kartikeya Dwivedi <memxor@gmail.com>,
        Arnd Bergmann <arnd@arndb.de>, netdev <netdev@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        jeffreyji <jeffreyji@google.com>
Subject: Re: [PATCH v3 net-next] net-core: add rx_otherhost_dropped counter
Message-ID: <20220308162802.0ecec1bf@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <CANn89iKvP-8VpOrf_ppVVgsd4kQtAEFWkBVxKW4BP+rtu_Egrw@mail.gmail.com>
References: <20220308212531.752215-1-jeffreyjilinux@gmail.com>
        <d1b25466-6f83-591e-39a6-8fdbd56846fb@kernel.org>
        <CANn89iKvP-8VpOrf_ppVVgsd4kQtAEFWkBVxKW4BP+rtu_Egrw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Tue, 8 Mar 2022 15:18:25 -0800 Eric Dumazet wrote:
> > that's an expensive packet counter for a common path (e.g., hosting
> > environments).  
> 
> This was the reason for the initial patch, using SNMP stat, being per cpu.
> 
> Adding per-device per-cpu data for this counter will increase cost of
> netdevice dismantle phase,
> and increase time for ndo_get_stats64(), especially on hosts with 256
> or 512 cpus.

Two ways to solve this:
 - make dev->pcpu_refcnt point to a structure which holds both
   refcnt and whatever stats
 - combine these stats into lstats, assuming the netdevs we care
   about spawning / destroying fast are sw devices anyway;
   struct rtnl_link_ops can indicate to the core if the driver
   wants lstats (or just put how many bytes it wants), otherwise 
   we'd only allocate enough mem for core's stats

Option three - both.