Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 17 May 2022 09:06:15 +0200
From:   Maxime Chevallier <maxime.chevallier@bootlin.com>
To:     Florian Fainelli <f.fainelli@gmail.com>
Cc:     davem@davemloft.net, Rob Herring <robh+dt@kernel.org>,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        devicetree@vger.kernel.org, thomas.petazzoni@bootlin.com,
        Andrew Lunn <andrew@lunn.ch>,
        Heiner Kallweit <hkallweit1@gmail.com>,
        Russell King <linux@armlinux.org.uk>,
        linux-arm-kernel@lists.infradead.org,
        Vladimir Oltean <vladimir.oltean@nxp.com>,
        Luka Perkov <luka.perkov@sartura.hr>,
        Robert Marko <robert.marko@sartura.hr>
Subject: Re: [PATCH net-next v2 2/5] net: dsa: add out-of-band tagging
 protocol
Message-ID: <20220517090615.34b82d28@pc-20.home>
In-Reply-To: <89c52305-71da-843e-b6c5-77648fb2f4d3@gmail.com>
References: <20220514150656.122108-1-maxime.chevallier@bootlin.com>
        <20220514150656.122108-3-maxime.chevallier@bootlin.com>
        <89c52305-71da-843e-b6c5-77648fb2f4d3@gmail.com>
Organization: Bootlin
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Florian,

On Sat, 14 May 2022 09:33:44 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> Hi Maxime,
> 
> On 5/14/2022 8:06 AM, Maxime Chevallier wrote:
> > This tagging protocol is designed for the situation where the link
> > between the MAC and the Switch is designed such that the Destination
> > Port, which is usually embedded in some part of the Ethernet
> > Header, is sent out-of-band, and isn't present at all in the
> > Ethernet frame.
> > 
> > This can happen when the MAC and Switch are tightly integrated on an
> > SoC, as is the case with the Qualcomm IPQ4019 for example, where
> > the DSA tag is inserted directly into the DMA descriptors. In that
> > case, the MAC driver is responsible for sending the tag to the
> > switch using the out-of-band medium. To do so, the MAC driver needs
> > to have the information of the destination port for that skb.
> > 
> > This out-of-band tagging protocol is using the very beggining of
> > the skb headroom to store the tag. The drawback of this approch is
> > that the headroom isn't initialized upon allocating it, therefore
> > we have a chance that the garbage data that lies there at
> > allocation time actually ressembles a valid oob tag. This is only
> > problematic if we are sending/receiving traffic on the master port,
> > which isn't a valid DSA use-case from the beggining. When dealing
> > from traffic to/from a slave port, then the oob tag will be
> > initialized properly by the tagger or the mac driver through the
> > use of the dsa_oob_tag_push() call.  
> 
> What I like about your approach is that you have aligned the way an
> out of band switch tag is communicated to the networking stack the
> same way that an "in-band" switch tag would be communicated. I think
> this is a good way forward to provide the out of band tag and I don't
> think it creates a performance problem because the Ethernet frame is
> hot in the cache (dma_unmap_single()) and we already have an
> "expensive" read of the DMA descriptor in coherent memory anyway.
> 
> You could possibly optimize the data flow a bit to limit the amount
> of sk_buff data movement by asking your Ethernet controller to DMA
> into the data buffer N bytes into the beginning of the data buffer.
> That way, if you have reserved say, 2 bytes at the front data buffer
> you can deposit the QCA tag there and you do not need to push,
> process the tag, then pop it, just process and pop. Consider using
> the 2byte stuffing that the Ethernet controller might be adding to
> the beginning of the Ethernet frame to align the IP header on a
> 4-byte boundary to provide the tag in there?
> 
> If we want to have a generic out of band tagger like you propose, it 
> seems to me that we will need to invent a synthetic DSA tagging
> format which is the largest common denominator of the out of band
> tags that we want to support. We could imagine being more compact in
> the representation for instance by using an u8 for storing a bitmask
> of ports (works for both RX and TX then) and another u8 for various
> packet forwarding reasons.

Thanks, that was my initial idea indeed. Having a generic tagger that
can be re-used would be great IMO. I'll modify the format as you
propose, and also give a try to you approach of DMA'ing 2 bytes forward
so that the tag location is already allocated, that's a nice idea.

> Then we would request the various Ethernet MAC drivers to marshall
> their proprietary tag into the DSA synthetic one on receive, and
> unmarshall it on transmit.
> 
> Another approach IMHO which maybe helps the maintainability of the
> code moving forward as well as ensuring that all Ethernet switch
> tagging code lives in one place, is to teach each tagger driver how
> to optimize their data paths to minimize the amount of data movements
> and checksum re-calculations, this is what I had in mind a few years
> ago:
> 
> https://lore.kernel.org/lkml/1438322920.20182.144.camel@edumazet-glaptop2.roam.corp.google.com/T/
> 
> This might scale a little less well, and maybe this makes too many 
> assumptions as to where and how the checksums are calculated on the 
> packet contents, but at least, you don't have logic processing the
> same type of switch tag scattered between the Ethernet MAC drivers
> (beyond copying/pushing) and DSA switch taggers.

That would definitely fit well with this tagger, I didn't know about
that series !

Thanks for the review,

Maxime

> I would like to hear other's opinion on this.