Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <260c2921-44e1-466b-0bfb-05006e6ad4c1@intel.com>
Date:   Tue, 11 Apr 2023 08:30:51 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.9.1
Subject: Re: [PATCH net] net: bridge: switchdev: don't notify FDB entries with
 "master dynamic"
Content-Language: en-US
To:     Vladimir Oltean <vladimir.oltean@nxp.com>, <netdev@vger.kernel.org>
CC:     Jakub Kicinski <kuba@kernel.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Paolo Abeni <pabeni@redhat.com>,
        "Ido Schimmel" <idosch@nvidia.com>,
        "Hans J. Schultz" <netdev@kapio-technology.com>,
        Roopa Prabhu <roopa@nvidia.com>,
        "Nikolay Aleksandrov" <razor@blackwall.org>,
        Ivan Vecera <ivecera@redhat.com>,
        "Jiri Pirko" <jiri@resnulli.us>,
        Arkadi Sharshevsky <arkadis@mellanox.com>,
        "Ido Schimmel" <idosch@mellanox.com>,
        <bridge@lists.linux-foundation.org>, <linux-kernel@vger.kernel.org>
References: <20230410204951.1359485-1-vladimir.oltean@nxp.com>
From:   Jesse Brandeburg <jesse.brandeburg@intel.com>
In-Reply-To: <20230410204951.1359485-1-vladimir.oltean@nxp.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZjdCUTBobzRyTDV6WDl3L2Nrekc0Zlh0NE9ZblYyaEJENStpOXVTOG9lci8x?=
 =?utf-8?B?OVNGRUNFbGcwcTVmY0tzc1R2VWJka093bVhJazl6TFR5cnY2QnM5TTFnWlhI?=
 =?utf-8?B?UGE5YXhJTmEwUm92akFrcmlTc0ZFVmpPNXhRNkpIYkU1ZzdHenlTdnMxQU1i?=
 =?utf-8?B?bVFuMGk4cG1TVFUrUDkxMEV6aURVc2lGOHUyVUtvc2J5bkhYMUJNeDhNdnE0?=
 =?utf-8?B?NDZLeUY5cldjd0xtODhybklPMDdmQ2dJNHp5Y25ScW9EZWJTaHEwN1V3K2lW?=
 =?utf-8?B?VFJmcXJRTCtkOFdVVEdjL0VBN2c1QUNLWGIwWHhoME5VMG1ORzNxSEkvVXUw?=
 =?utf-8?B?b0JOM3FCWlpmd1FFbDJnY2s2SEx3ZHpJd0IzclRWTGU5TkRIcjI1ZzVOV25O?=
 =?utf-8?B?bHI0K2pCRzZjd0txN1U3bGVXMmtrbUpoWUNyMHZiRkkwZ2Q4UzlCZ3ExbEZw?=
 =?utf-8?B?YkY2TzhTUTh0bUxXNDR2K0ZTUy9nQVNlMk9STm1lbEFhY0R6ZVhwWTRSN3Ew?=
 =?utf-8?B?dW1ETnUxejNLK2Fka043R3RSRlNQOTZ1NU5oT3Rldm9pQS9ZbGNrVGhvYzJC?=
 =?utf-8?B?UjBKenBvUFJ5RjRxQ0lLKzVBSkJSeFY1a2ZnVCt0SUVRY3B3NjBlUnJpMHdH?=
 =?utf-8?B?RGRFM2tJSWtFTVM0UStFeitPd00wRFNQV3ZyYXM5andCWGpOK2tOZDdqZkR3?=
 =?utf-8?B?ejFwUjVMWStIQ3dqNXFqMnVRNVVEZDAvTUx5MHRFcEl3Ukg3QXhzK1UvTTd2?=
 =?utf-8?B?NHMyK2pWREFjaUdrZDZrTHl4VU44T2FLRHREZTFINGFTYVNMTGxkTERzcGJV?=
 =?utf-8?B?NzhyZC9GeVBOOVhYQUNiTmIycE1OVzFHMis2dWZwZ0MrcHZBMGVTV2w0ZWND?=
 =?utf-8?B?NUxxM3lrV1IvanZISVE4R3FMTldXaFFvK0VMQ0J3TEg4UUVndXY2OEdUQ3hJ?=
 =?utf-8?B?aEFieldJNVZ0eTBSQlJOT21SRXdYbmVvdHQvVUwzWjJHdXFXUll4ZERBSFZF?=
 =?utf-8?B?MW9qaWZ6YjZPRFhrRHVLdjdJQnZlR2hiZy9DNzdLcVdrRGJucFBjcmhvbko1?=
 =?utf-8?B?b3VPbE9jcHIxZ0VCbnh0NmRVNHpheUxmODZCNE5sazhNVjB0UjdTM3lpSlJr?=
 =?utf-8?B?ZXFXVjd6ME5CK0xWL25NU0dqOXU5ZE0xcUxLYTFhblFjc2Q0L2J1NFJEUjVW?=
 =?utf-8?B?K1JsYXpUbE93UlRPdTZrK2tGcHU5ajdOeVhGQUNCMVJtNXlPUHlQU2svME81?=
 =?utf-8?B?VEluN1IzMWk1MTlTaEZ0bWMwdHk5bWFKckF0UWtYUm1nWUdRMS9NRTBoNnNM?=
 =?utf-8?B?V0w2dmhvYks2dVlDOFNHOW9KWklPYUVyT2N4eU9QQkJXZ2g1ZE5pQmxHK1R5?=
 =?utf-8?B?eXRJWUJXYTIxU1ZWcTgyN3E2Y3ZTZFgzL1Iya0JMdjZvUXhGeS81YXpPNzZL?=
 =?utf-8?B?VEpPSlpucVl0ZlJ3OTZQWHdyL21waVVGYlZzQ3hnbkR0UHgxNHBtb0x0Z1FP?=
 =?utf-8?B?NlBnVmVCQ0NFUUp1TFdkbHRrb203N0RtektDY1lHYWQvQXB6bWtjbFdTRWxo?=
 =?utf-8?B?T0l0cFFFM0tDMlI2S2dHQmYyZjFBT1hncG1nblIrMHNjV0hicEtlbHJLSUlL?=
 =?utf-8?B?Y0N5cmYzdzFsMkhqNjZtVjhwMlkrM3AvRUZ3SzhIZ3d2NlFLeUtkS0xZNDZB?=
 =?utf-8?B?WEtZZVFVaVNrdnFueVpXdzN0SkhmQzlxUTBacGkrc095VjVibTc4bGR0SlFT?=
 =?utf-8?B?NjJIemhSMUoxakR1SHpLaCtGVCtZbFgrSDk0QmZWMVhXNjFqZWJuajc0Wm1F?=
 =?utf-8?B?N3pJZm82UHVjRWxMVS9XSEpXb0RMNHFVUWZtbkNpNWd4aTVYbGVCdytzdUpo?=
 =?utf-8?B?SHhuNmlrSGdYdTNXZkMyRFMvV2Rjc2JrcGtQQmtnZlVQaVFIVXlPd3pGd2ZR?=
 =?utf-8?B?TC8rUTVtOTlRMkRUaGU3OXlJWGdZNUVCY293UWh0VGRpSGVsbk9aOHJuUjd0?=
 =?utf-8?B?SVlwZmZHOSsyT01WRFN1Q1NvUklIWkJGYmRZR3RsWWMwYWZJcGZEVE5GOUJa?=
 =?utf-8?B?d0tJUVV6RnhpbzlFd1RSZ2JGd3NoOFF3MGtDR2xMYkt4YmN5SWFUY1ZqbU1G?=
 =?utf-8?B?dlpEeVNvSyt1dW9hLzVTbDRlNkxudW1ZYytXSjJoWlJjbUphOXM1ampsR2Fk?=
 =?utf-8?B?c2c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c32f94b4-3bb3-4c7e-215c-08db3aa1bd79
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4914.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 15:30:54.8254
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EF/Sh2jRQgaAjL2yH//yO08KlDtKi8iaPbsJKeV8d1WCGNhIdPqXVoZFilE9AKLuzapcG0tOS9BdenrHuxyfY63uKfQjrm4lpAMSn6hwGI8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6204
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 4/10/2023 1:49 PM, Vladimir Oltean wrote:
> There is a structural problem in switchdev, where the flag bits in
> struct switchdev_notifier_fdb_info (added_by_user, is_local etc) only
> represent a simplified / denatured view of what's in struct
> net_bridge_fdb_entry :: flags (BR_FDB_ADDED_BY_USER, BR_FDB_LOCAL etc).
> Each time we want to pass more information about struct
> net_bridge_fdb_entry :: flags to struct switchdev_notifier_fdb_info
> (here, BR_FDB_STATIC), we find that FDB entries were already notified to
> switchdev with no regard to this flag, and thus, switchdev drivers had
> no indication whether the notified entries were static or not.
> 
> For example, this command:
> 
> ip link add br0 type bridge && ip link set swp0 master br0
> bridge fdb add dev swp0 00:01:02:03:04:05 master dynamic
> 
> causes a struct net_bridge_fdb_entry to be passed to
> br_switchdev_fdb_notify() which has a single flag set:
> BR_FDB_ADDED_BY_USER.
> 
> This is further passed to the switchdev notifier chain, where interested
> drivers have no choice but to assume this is a static FDB entry.
> So currently, all drivers offload it to hardware as such.
> 
> bridge fdb get 00:01:02:03:04:05 dev swp0 master
> 00:01:02:03:04:05 dev swp0 offload master br0
> 
> The software FDB entry expires after the $ageing_time and the bridge
> notifies its deletion as well, so it eventually disappears from hardware
> too.
> 
> This is a problem, because it is actually desirable to start offloading
> "master dynamic" FDB entries correctly, and this is how the current
> incorrect behavior was discovered.
> 
> To see why the current behavior of "here's a static FDB entry when you
> asked for a dynamic one" is incorrect, it is possible to imagine a
> scenario like below, where this decision could lead to packet loss:
> 
> Step 1: management prepares FDB entries like this:
> 
> bridge fdb add dev swp0 ${MAC_A} master dynamic
> bridge fdb add dev swp2 ${MAC_B} master dynamic
> 
>         br0
>       /  |  \
>      /   |   \
>   swp0  swp1  swp2
>    |           |
>    A           B
> 
> Step 2: station A migrates to swp1 (assume that swp0's link doesn't flap
> during that time so that the port isn't flushed, for example station A
> was behind an intermediary switch):
> 
>         br0
>       /  |  \
>      /   |   \
>   swp0  swp1  swp2
>    |     |     |
>          A     B
> 
> Whenever A wants to ping B, its packets will be autonomously forwarded
> by the switch (because ${MAC_B} is known). So the software will never
> see packets from ${MAC_A} as source address, and will never know it
> needs to invalidate the dynamic FDB entry towards swp0. As for the
> hardware FDB entry, that's static, it doesn't move when the station
> roams.
> 
> So when B wants to reply to A's pings, the switch will forward those
> replies to swp0 until the software bridge ages out its dynamic entry,
> and that can cause connectivity loss for up to 5 minutes after roaming.
> 
> With a correctly offloaded dynamic FDB entry, the switch would update
> its entry for ${MAC_A} to be towards swp1 as soon as it sees packets
> from it (no need for CPU intervention).
> 
> Looking at tools/testing/selftests/net/forwarding/, there is no valid
> use of the "bridge fdb add ... master dynamic" command there, so I am
> fairly confident that no one used to rely on this behavior.
> 
> With the change in place, these FDB entries are no longer offloaded:
> 
> bridge fdb get 00:01:02:03:04:05 dev swp0 master
> 00:01:02:03:04:05 dev swp0 master br0
> 
> and this also constitutes a better way (assuming a backport to stable
> kernels) for user space to determine whether the switchdev driver did
> actually act upon the dynamic FDB entry or not.
> 
> Fixes: 6b26b51b1d13 ("net: bridge: Add support for notifying devices about FDB add/del")
> Link: https://lore.kernel.org/netdev/20230327115206.jk5q5l753aoelwus@skbuf/
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Looks fine to me, but I'd like to see other switchdev experts reply.

Reviewed-by: Jesse Brandeburg <jesse.brandeburg@intel.com>