Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 3 Feb 2022 12:10:27 -0800
From:   Jakub Kicinski <kuba@kernel.org>
To:     Vladimir Oltean <olteanv@gmail.com>
Cc:     Ansuel Smith <ansuelsmth@gmail.com>,
        Florian Fainelli <f.fainelli@gmail.com>,
        Andrew Lunn <andrew@lunn.ch>,
        Vivien Didelot <vivien.didelot@gmail.com>,
        "David S. Miller" <davem@davemloft.net>,
        linux-kernel@vger.kernel.org, netdev@vger.kernel.org
Subject: Re: [RFC PATCH v7 00/16] Add support for qca8k mdio rw in Ethernet
 packet
Message-ID: <20220203121027.7a6ea0f8@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <20220203182128.z6xflse7fezccvhx@skbuf>
References: <20220123013337.20945-1-ansuelsmth@gmail.com>
        <YfaZrsewBMhqr0Db@Ansuel-xps.localdomain>
        <a8244311-175d-79d3-d61b-c7bb99ffdfb7@gmail.com>
        <YfwX8YDDygBEsyo3@Ansuel-xps.localdomain>
        <20220203182128.z6xflse7fezccvhx@skbuf>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Thu, 3 Feb 2022 20:21:28 +0200 Vladimir Oltean wrote:
> To my knowledge, when you call dev_queue_xmit(), the skb is no longer
> yours, end of story, it doesn't matter whether you increase the refcount
> on it or not. The DSA master may choose to do whatever it wishes with
> that buffer after its TX completion interrupt fires: it may not call
> napi_consume_skb() but directly recycle that buffer in its pool of RX
> buffers, as part of some weird buffer recycling scheme. So you'll think
> that the buffer is yours, but it isn't, because the driver hasn't
> returned it to the allocator, and your writes for the next packet may be
> concurrent with some RX DMA transactions. I don't have a mainline
> example to give you, but I've seen the pattern, and I don't think it's
> illegal (although of course, I stand to be corrected if necessary).

Are we talking about holding onto the Tx skb here or also recycling 
the Rx one? Sorry for another out of context comment in advance..

AFAIK in theory shared skbs are supposed to be untouched or unshared
explicitly by the driver on Tx. pktgen takes advantage of it.
We have IFF_TX_SKB_SHARING. 

In practice everyone gets opted into SKB_SHARING because ether_setup()
sets the flag. A lot of drivers are not aware of the requirement and
will assume full ownership (and for example use skb->cb[]) :/

I don't think there is any Tx completion -> Rx pool recycling scheme
inside the drivers (if that's what you described).