Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Vladimir Oltean <vladimir.oltean@nxp.com>
To:     Ferenc Fejes <ferenc.fejes@ericsson.com>
CC:     Xiaoliang Yang <xiaoliang.yang_1@nxp.com>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "vinicius.gomes@intel.com" <vinicius.gomes@intel.com>,
        "michael.chan@broadcom.com" <michael.chan@broadcom.com>,
        "saeedm@nvidia.com" <saeedm@nvidia.com>,
        "jiri@nvidia.com" <jiri@nvidia.com>,
        "idosch@nvidia.com" <idosch@nvidia.com>,
        "alexandre.belloni@bootlin.com" <alexandre.belloni@bootlin.com>,
        "allan.nielsen@microchip.com" <allan.nielsen@microchip.com>,
        "joergen.andreasen@microchip.com" <joergen.andreasen@microchip.com>,
        "jhs@mojatatu.com" <jhs@mojatatu.com>
Subject: Re: [RFC, net-next] net: qos: introduce a frer action to implement
 802.1CB
Thread-Topic: [RFC, net-next] net: qos: introduce a frer action to implement
 802.1CB
Thread-Index: AQHXtFznyGH2JvuT50mNvd/a5fV2Ja0TF8IAgAAHuQA=
Date:   Fri, 6 May 2022 12:23:35 +0000
Message-ID: <20220506122334.i7eqt2ngbfwqlrwn@skbuf>
References: <20210928114451.24956-1-xiaoliang.yang_1@nxp.com>
 <df67ceaa-4240-d084-7ba1-d703f0c38f33@ericsson.com>
In-Reply-To: <df67ceaa-4240-d084-7ba1-d703f0c38f33@ericsson.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CMxxxknFAE4Rl48pt3njgbvtBkY74YWZbERHnfl78dGHcyaAU15RsagEDoA7?=
 =?us-ascii?Q?u6PL4duOdh7u1UtvzLLERZzZQ8gO9WFlO/kPkaFB/YYf/0opUkLFC0Smp4ca?=
 =?us-ascii?Q?xZF6047UfbV4sZrSlK8z0ZSfWFsEwSKOz5l5fzTSHQDIjlMor3bKpTNQ2qug?=
 =?us-ascii?Q?l9IHa/9rZ2OieCO7e4dZJHffLo5sKRDY89PTdq5umoNMbEliU5H2eoc/33WA?=
 =?us-ascii?Q?l3h8i+O6TasCYp1IF6LGw/nuY1ecchgNWSpahOaSWuOO+WVO3KeIMFTYiFE1?=
 =?us-ascii?Q?BerZ3heqS7m70g0GIiUwGuLQW/dV1jwwUQPnTXrndrQoMRjcbXHNjIPWeW2v?=
 =?us-ascii?Q?GkqoTH5VGpLISJ8IKraCcrRKp8juoZvqq8gr1JsvG2oZN4X6wZsIdvXgV1mF?=
 =?us-ascii?Q?OObCBwhKX+5o4nQVRSCRRzpzJl3CrDwZgBupl3mKjwOeVNwBm2xXOtsnv0sS?=
 =?us-ascii?Q?7Hu6wV4CQIi5XkMy0zzSQUoo4u9iupVDwHb5k8Exsm9EAfxtbmJ4LrSw4ku/?=
 =?us-ascii?Q?6vvtH3s2+ZlpwyQ6/YzDzoXX/YWLD+bkss5Vs4spZBMQnqQ3yu+JWo7k8HCu?=
 =?us-ascii?Q?E2BDxyVnr7FdA2xvhU+edGCdB6IFBQbDoLC1DYq4jg/gMDsDt9ifLKKGeQCK?=
 =?us-ascii?Q?mkkHkSpU3Evn6aMoqQwEJHIhOk6qm+FtTJ1KoxEbQF6bI9D6qTFA65EuMLOu?=
 =?us-ascii?Q?9ofxmtzG9M39sURGu1G1OVKSJ/fDYskQpZWrfWyM/wWRNfym0O29UubJPnq5?=
 =?us-ascii?Q?U5MjxA34SFviR/+IEFXUUAuDMWxceV1JmboiU99eL4bBc2LbY7jtAUB1povy?=
 =?us-ascii?Q?PMfH79sioH2GD6/sYFfrArvoO3iuPZdO0peugTUfHvQyED0lv6Mkev8RhEHX?=
 =?us-ascii?Q?RC7zIkMXfqzuhK/GCMOtz4R1CmBMLTtYuJ84alM0XTdX4RrdsB1LEPF96XkZ?=
 =?us-ascii?Q?ck9FTsn8snIEizmTKYTM5x6RTyE7M6ZVEAMVRDQkUk3B/jFb8p6YUvs1SDq0?=
 =?us-ascii?Q?qhiUwM47e6WfrdZKu1iy3wDs7C00oZOUbELLvdIyLmYrkCwfUt54fW+GcA7n?=
 =?us-ascii?Q?v2/CEFtVi4a3t1OpnsXReJ5wUXYXLZJuAtpsnpBKuTduDA6OXnq/OSBfR4Xl?=
 =?us-ascii?Q?UvEZkaplxmo3VTmXZ+ru9TPvLxCAEbw8CMFz0GfsaaJtxyR6oVKKNrnMDIob?=
 =?us-ascii?Q?K5wcGfZW61Rr6OCUHbq2Xj78g3+6rYiGvvh4sQ0/JmCCLnLcvpUxpU3oBZWb?=
 =?us-ascii?Q?wQ5eOKND3KqAwERuwmtfa/2hbVRb/KtK/sJGzq25seTw/I1NXRT7DjttK1X5?=
 =?us-ascii?Q?Ch2ix/0d1xfzm0HuxPJ/APO+6GJe7pnr5SvuvczLFgJNDF6fslyUdMbhNc4/?=
 =?us-ascii?Q?LZJaX0rmEqcJ8m2NzuGcSg7uUUGLM7piPTOb8PlFdYqHjFfb3/6s1jWYBSFA?=
 =?us-ascii?Q?PaFCssG1CwX1XgoJvb6B7QG3ESlOfzJTRXYjpPyz/QMKvKmcbNEI+63rp9xW?=
 =?us-ascii?Q?d6cmW+Z/WdsEEyEP7fGfSrZDV6V4eEZqEWKDVidblc+xYGdQIvg986Ht83Z5?=
 =?us-ascii?Q?9JIOdwrZ2U5dNTs5qGqdIyOYOpIHMkpM8RBR1l3kzo7wfA6YcPLRSu4MAgpe?=
 =?us-ascii?Q?CUzKezWLWPh7GWBBMix04PbzuJSrOxSozxQhB6zvPigh0QeLVOBlpCLwcx93?=
 =?us-ascii?Q?KwSfT/T6MCABjpzk12m6adO+v0kPM7ThaSDL2uA6W6lf/9nmbr0+GvkkMnaM?=
 =?us-ascii?Q?dTC6qiE4q5uahouQ9i5UQqjxnnFjKUY=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <B69E6B15CB656B478A6DB09B2CA54853@eurprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5136.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 60415b9a-0968-470c-fe8b-08da2f5b3dbf
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 May 2022 12:23:35.0712
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Mtp4VPlDE52YeJgc+xMtpNCxiEt8aJ+1imv6B48S29wak82Horx8yzjTSbL2I55diBeIrxLskUDvARjaJQiCmw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2926
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Ferenc,

(I adjusted the CC list)

On Fri, May 06, 2022 at 11:55:56AM +0000, Ferenc Fejes wrote:
> On 2021. 09. 28. 13:44, Xiaoliang Yang wrote:
> > This patch introduce a frer action to implement frame replication and
> > elimination for reliability, which is defined in IEEE P802.1CB.
>
> HiXiaoliang!
>
> thanks for your efforts to introduce afreraction to implement frame
> replication and elimination for reliability, which is defined in IEEE
> P802.1CB-2017. I would like to relay a small comment from our team,
> regarding to the FRER, not particularly to the code.
>
> Support of RTAG format is very straightforward.
>
> Since 2017, several maintenance items were opened regarding IEEE
> P802.1CB-2017 to fix some errors in the standard. Discussions results
> will be published soon e.g., in IEEE P802.1CBdb
> (https://1.ieee802.org/tsn/802-1cbdb/).
>
> One of the maintenance items impacts the vector recovery algorithm itself=
.
>
> Details on the problem and the solution are here:
>
> -https://www.802-1.org/items/370
>
> -https://www.ieee802.org/1/files/public/docs2020/maint-varga-257-FRER-rec=
overy-window-0320-v01.pdf
> <https://www.ieee802.org/1/files/public/docs2020/maint-varga-257-FRER-rec=
overy-window-0320-v01.pdf>
>
> It is a small but important fix. There is an incorrect reference to the
> size of the recovery window, when a received packet is checked to be
> out-of-range or not. Without this fix the vector recovery algorithm do
> not work properly in some scenarios.
>
> Please consider to update your patch to reflect the maintenance efforts
> of IEEE to correct .1CB-2017 related issues.
>
> > There are two modes for frer action: generate and push the tag, recover
> > and pop the tag. frer tag has three types: RTAG, HSR, and PRP. This
> > patch only supports RTAG now.
> >
> > User can push the tag on egress port of the talker device, recover and
> > pop the tag on ingress port of the listener device. When it's a relay
> > system, push the tag on ingress port, or set individual recover on
> > ingress port. Set the sequence recover on egress port.
> >
> > Use action "mirred" to do split function, and use "vlan-modify" to do
> > active stream identification function on relay system.
> >
> All of our research in the topic based on a in-house userspace FRER
> implementation but we are looking forward to test your work in the future=
.
>
> Thanks,
>
> Ferenc

Glad to see someone familiar with 802.1CB. I have a few questions and
concerns if you don't mind.

I think we are seeing a bit of a stall on the topic of FRER modeling in
the Linux networking stack, in no small part due to the fact that we are
working with pre-standard hardware.

The limitation with Xiaoliang's proposal here (to model FRER stream
replication and recovery as a tc action) is that I don't think it works
well for traffic termination - it only covers properly the use case of a
switch. More precisely, there isn't a single convergent termination
point for either locally originating traffic, or locally received
traffic (i.e. you, as user, don't know on which interface of several
available to open a socket).

In our hardware, this limitation isn't really visible because of the way
in which the Ethernet switch is connected inside the NXP LS1028A.
It is something like this:

  +---------------------------------------+
  |                                       |
  |           +------+ +------+           |
  |           | eno2 | | eno3 |           |
  |           +------+ +------+           |
  |              |         |              |
  |           +------+ +------+           |
  |           | swp4 | | swp5 |           |
  |           +------+ +------+           |
  |  +------+ +------+ +------+ +------+  |
  |  | swp0 | | swp1 | | swp2 | | swp3 |  |
  +--+------+-+------+-+------+-+------+--+

In the above picture, the switch ports swp0-swp3 have eno3 as a DSA
master (connected to the internal swp5, a CPU port). The other internal
port, swp5, is configured as a DSA user port, so it has a net device.
Analogously, while eno3 is a DSA master and receives DSA-tagged traffic
(so it is useless for direct IP termination), eno2 receives DSA untagged
traffic and is therefore an IP termination endpoint into a switched
network.

What we do in this case is put tc-frer rules for stream replication and
recovery on swp4 itself, and we use eno2 as the convergence point for
locally terminated streams.

However, naturally, a hardware design that does not look like this can't
terminate traffic like this.

My idea was that it might be better if FRER was its own virtual network
interface (like a bridge), with multiple slave interfaces. The FRER net
device could keep its own database of streams and actions (completely
outside of tc) which would be managed similar to "bridge fdb add ...".
This way, the frer0 netdevice would be the local termination endpoint,
logically speaking.

What I don't know for sure is if a FRER netdevice is supposed to forward
frames which aren't in its list of streams (and if so, by which rules).
Because if a FRER netdevice is supposed to behave like a regular bridge
for non-streams, the implication is that the FRER logic should then be
integrated into the Linux bridge.

Also, this new FRER software model complicates the offloading on NXP
LS1028A, but let's leave that aside, since it shouldn't really be the
decisive factor on what should the software model look like.

Do you have any comments on this topic?=