Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 12 Apr 2023 17:15:03 +0300
From:   Ido Schimmel <idosch@nvidia.com>
To:     Vladimir Oltean <vladimir.oltean@nxp.com>
Cc:     netdev@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Paolo Abeni <pabeni@redhat.com>,
        "Hans J. Schultz" <netdev@kapio-technology.com>,
        Roopa Prabhu <roopa@nvidia.com>,
        Nikolay Aleksandrov <razor@blackwall.org>,
        Ivan Vecera <ivecera@redhat.com>,
        Jiri Pirko <jiri@resnulli.us>,
        Arkadi Sharshevsky <arkadis@mellanox.com>,
        Ido Schimmel <idosch@mellanox.com>,
        bridge@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH net] net: bridge: switchdev: don't notify FDB entries
 with "master dynamic"
Message-ID: <ZDa856x2rhzNrrXa@shredder>
References: <20230410204951.1359485-1-vladimir.oltean@nxp.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230410204951.1359485-1-vladimir.oltean@nxp.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pwaCYnD7Fu9oJoRPz+s3dAZgHUTm0DnMWiRhTEc7faFdW/rWMsdoSaU4Gkqo?=
 =?us-ascii?Q?OxqIPOVgcYxP6dNBrosUbvZ7FLGguAmHuTl70mlOuMMbqznXJjp55aAm8pP1?=
 =?us-ascii?Q?gLTUG1Ru/xrMr5nzrGtKRo/dTX8rN/+KADH45NIxTeeB1ou7ux4tmAoTF/tg?=
 =?us-ascii?Q?otEXkZ57/EG3J1QAi5tq1e6rJTZYT8TDsuu0ShrVSM9VDlBtBAO/LmzY2EDF?=
 =?us-ascii?Q?zzM+eHFLjzQOaKCbEmH4Nyo4rokx3eNqQoKg+X5tNVe4v/EQOfb6G+AD2GNN?=
 =?us-ascii?Q?joCFOLrtyoEOODCYtw0psZ+IbxIopfjzCVZ3VkRPPDn9oY8nb0VQyeB4y81L?=
 =?us-ascii?Q?COWGRzZjUTSF+OdbFFAbti3zHmRC5LfwgeOT9JX8VRYCnrReAZ0g+BD3uXWc?=
 =?us-ascii?Q?R5tMEfoMF2t/V40zYO6h0CogbMl9LcbLxplGRATJvUUExOea9hO9vxnply4R?=
 =?us-ascii?Q?vH9SvxwUqiqR0iiESbB8SrHjMwFsJrBG7kzhzdzlyXQHx4KI5QI8KRCQRmqP?=
 =?us-ascii?Q?ASSi+MyGd6Jxs+x14rmusym2+eEBMImG7BQGzFPKb3ykAjRoWhCaIFFC2i2v?=
 =?us-ascii?Q?znO9HkkaShYz0r3jPN3vvzBPaxhkOYdFWesHt+JocP6XcxhdnFDgR2IQLeWe?=
 =?us-ascii?Q?OWl9ECnViSNxxad1ymIl2viHhq23meIYVt7mfgNvxaRsCwW0utu2VjHdP+VG?=
 =?us-ascii?Q?uP+mZvEN+gpTsXq9hY2xk/8FLDzBsxW7mHp6QCIGE/sTrumOw5xjaGm0jqsb?=
 =?us-ascii?Q?dst42yQSOatcYuMc+GkYwIwqfoL5b2bzG56tvUMCwqcuf4U4HrG9ngU7/4Uk?=
 =?us-ascii?Q?KJg/FaPz2yeCqdwIuK+HECquTz+O920AluZLW0JBIAaPEDJvh7n6RVVoHM1n?=
 =?us-ascii?Q?zWz2JSEzWiDOEqwtc8RKccCv2fjqw7tKpGbGPgWfEXj1u2LuAgr7xaX2pRAC?=
 =?us-ascii?Q?IZIdrNKHuMXQCLtk/28OqxcdcnXlq5VXt8fByMFJNAPWdwLa4NhY//1alXuK?=
 =?us-ascii?Q?uc+h1m8JjtuGT0U0tkoWO/kZyuQ4hgTgb/5832fYIPePwD8uR7jkiKlQkk3p?=
 =?us-ascii?Q?Cxj3QSeUyO+qjHdIzga6t8DEv18mgAjt3AP9UD3ovFvVdt0uKC8XQ+cQ7Bjj?=
 =?us-ascii?Q?JsEdL/zc8fkYF6eel0oPUbbAMg4+SDHZY/AE7ddchN8OAe9rXMQpr/e0lhyO?=
 =?us-ascii?Q?1y03bs92jeDtLyFu9My16Ga9ZRQLZtCUE5dYJb0g1+Iw70mdzGwQw/zxDhKy?=
 =?us-ascii?Q?zyEQDl2XNHKA2Cpt49EooPfuuNJeoKSqXVZ4nchhi6/rPDUOx2ZIf5V3aQgd?=
 =?us-ascii?Q?aGKSd+EHkbYfa8zUzLC4KHKWTN4mdm8kfD8N9ZP8SSFq2PPGJKpoQuoNaw4Q?=
 =?us-ascii?Q?8chT8AN1nVjWnEdnEYkRkRlVYizp4jmIONrq8SKPLNrZx1POcbuoOmriiuWq?=
 =?us-ascii?Q?dxnoPLfTe+voVrjAIZC2aguODrFdchB0m6sfWhFAIE2Gm68LKQWkADlwb/ob?=
 =?us-ascii?Q?jKqE1OW5+nis2bZZ7psDpE8G97bxxJE0cPhYKHnh8qXUEH1mT4woPQbySNyF?=
 =?us-ascii?Q?C9zm8yfjJlEzP/9XpTOAwMeXYkKi4AIO4XvJ9oqt?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 95c0586d-1295-4bda-9656-08db3b605300
X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6179.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2023 14:15:10.0828
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DfhBiy51x6B5EvOwok6YPTfHcBnAOpZ0aPhjH853HJBZiIBG4S/aV+kIb0UN7JCqgBcCeeFNHq++Nga4xx2uag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4997
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Apr 10, 2023 at 11:49:51PM +0300, Vladimir Oltean wrote:
> There is a structural problem in switchdev, where the flag bits in
> struct switchdev_notifier_fdb_info (added_by_user, is_local etc) only
> represent a simplified / denatured view of what's in struct
> net_bridge_fdb_entry :: flags (BR_FDB_ADDED_BY_USER, BR_FDB_LOCAL etc).
> Each time we want to pass more information about struct
> net_bridge_fdb_entry :: flags to struct switchdev_notifier_fdb_info
> (here, BR_FDB_STATIC), we find that FDB entries were already notified to
> switchdev with no regard to this flag, and thus, switchdev drivers had
> no indication whether the notified entries were static or not.
> 
> For example, this command:
> 
> ip link add br0 type bridge && ip link set swp0 master br0
> bridge fdb add dev swp0 00:01:02:03:04:05 master dynamic
> 
> causes a struct net_bridge_fdb_entry to be passed to
> br_switchdev_fdb_notify() which has a single flag set:
> BR_FDB_ADDED_BY_USER.
> 
> This is further passed to the switchdev notifier chain, where interested
> drivers have no choice but to assume this is a static FDB entry.
> So currently, all drivers offload it to hardware as such.
> 
> bridge fdb get 00:01:02:03:04:05 dev swp0 master
> 00:01:02:03:04:05 dev swp0 offload master br0
> 
> The software FDB entry expires after the $ageing_time and the bridge
> notifies its deletion as well, so it eventually disappears from hardware
> too.
> 
> This is a problem, because it is actually desirable to start offloading
> "master dynamic" FDB entries correctly, and this is how the current
> incorrect behavior was discovered.
> 
> To see why the current behavior of "here's a static FDB entry when you
> asked for a dynamic one" is incorrect, it is possible to imagine a
> scenario like below, where this decision could lead to packet loss:
> 
> Step 1: management prepares FDB entries like this:
> 
> bridge fdb add dev swp0 ${MAC_A} master dynamic
> bridge fdb add dev swp2 ${MAC_B} master dynamic
> 
>         br0
>       /  |  \
>      /   |   \
>   swp0  swp1  swp2
>    |           |
>    A           B
> 
> Step 2: station A migrates to swp1 (assume that swp0's link doesn't flap
> during that time so that the port isn't flushed, for example station A
> was behind an intermediary switch):
> 
>         br0
>       /  |  \
>      /   |   \
>   swp0  swp1  swp2
>    |     |     |
>          A     B
> 
> Whenever A wants to ping B, its packets will be autonomously forwarded
> by the switch (because ${MAC_B} is known). So the software will never
> see packets from ${MAC_A} as source address, and will never know it
> needs to invalidate the dynamic FDB entry towards swp0. As for the
> hardware FDB entry, that's static, it doesn't move when the station
> roams.
> 
> So when B wants to reply to A's pings, the switch will forward those
> replies to swp0 until the software bridge ages out its dynamic entry,
> and that can cause connectivity loss for up to 5 minutes after roaming.
> 
> With a correctly offloaded dynamic FDB entry, the switch would update
> its entry for ${MAC_A} to be towards swp1 as soon as it sees packets
> from it (no need for CPU intervention).
> 
> Looking at tools/testing/selftests/net/forwarding/, there is no valid
> use of the "bridge fdb add ... master dynamic" command there, so I am
> fairly confident that no one used to rely on this behavior.

Yes, but there are tests that use "extern_learn". If you post a v2 that
takes "BR_FDB_ADDED_BY_EXT_LEARN" into account, then I can ask Petr to
run it through our regression and report back (not sure we will make it
to this week's PR though).

Thanks

> 
> With the change in place, these FDB entries are no longer offloaded:
> 
> bridge fdb get 00:01:02:03:04:05 dev swp0 master
> 00:01:02:03:04:05 dev swp0 master br0
> 
> and this also constitutes a better way (assuming a backport to stable
> kernels) for user space to determine whether the switchdev driver did
> actually act upon the dynamic FDB entry or not.
> 
> Fixes: 6b26b51b1d13 ("net: bridge: Add support for notifying devices about FDB add/del")
> Link: https://lore.kernel.org/netdev/20230327115206.jk5q5l753aoelwus@skbuf/
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  net/bridge/br_switchdev.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/net/bridge/br_switchdev.c b/net/bridge/br_switchdev.c
> index de18e9c1d7a7..0ec3d5e5e77d 100644
> --- a/net/bridge/br_switchdev.c
> +++ b/net/bridge/br_switchdev.c
> @@ -148,6 +148,10 @@ br_switchdev_fdb_notify(struct net_bridge *br,
>  	if (test_bit(BR_FDB_LOCKED, &fdb->flags))
>  		return;
>  
> +	if (test_bit(BR_FDB_ADDED_BY_USER, &fdb->flags) &&
> +	    !test_bit(BR_FDB_STATIC, &fdb->flags))
> +		return;
> +
>  	br_switchdev_fdb_populate(br, &item, fdb, NULL);
>  
>  	switch (type) {
> -- 
> 2.34.1
>