Received: by 2002:a05:6a10:8395:0:0:0:0 with SMTP id n21csp599175pxh; Tue, 9 Nov 2021 15:58:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJwmnEDPNbH8ePYV3smYF9LkfdU8IvJ5pzF9pJ2oEbfDTq+20js6eTBPO62QzWhovUMmCsDN X-Received: by 2002:a92:c846:: with SMTP id b6mr8390414ilq.255.1636502290825; Tue, 09 Nov 2021 15:58:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636502290; cv=none; d=google.com; s=arc-20160816; b=mgHB/68ti+2xuJPHYGymP4FLUN549MQAq4I8pX8RMt9kHcSczGYzYATdGbk68vlQ90 UIJd11825N8dgIGz+5sUea/pl40nq+JiAHezeG08A3WI5eTX5CHb7Xhcj58jmhSeVakX M1zHIrLIvCC0K1DZq0AOhdVQEnBIEgtmhOOOPP6BlzNp3OD/D5Bx7s5VFr27Nf6EXbZL ZfsrZjtN+x88qUw29UurtxShWR3fSVz5DwDVgYgNO5ZPHxy2gwOhotZGV5JQPnHh/hiU sObZx/Xckd+N52LFwaUuaVZk85VB6/NLpG9SgaRH/VDTh/6DOmcAEb/SCHBvunk6yEXX E43w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=fl6pKaMX3IYvsiA7vQ8B3YqZc/js19tN9dzwIH0x9hA=; b=vTZb1wqhyUizrA1t/lBQ2RUkBr81chOO+89IcVw4IFGg5TjpNzzKcVaFNLC1dmlKKd lyrQp86BMe1yGaRG1vleD82YR4zYNOZlTgubuQwwYBicMChbEIR8rR9w+hkeWBUNq6yr iLt5GYukNxYu383+wR++kYW23bRBqyvEcYaTU1LTgyLadUPuZkkIjdJDIfN9SaoEWSDG vMO3FgPZ6eeYMbXV8IFUQwW/v2aTx/1fqN1Gm40QYRJKYrX5NIrk+mp1epKnvxUuslY5 y8QYApTwiRdcAIzn0lJwRukiid2IGVl78G0XkioEnsYDuYWcekhrAQxPCx/dB1zf5Rvu 8LPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=HkNXFTvi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id n17si35143580ilt.86.2021.11.09.15.57.58; Tue, 09 Nov 2021 15:58:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=HkNXFTvi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237138AbhKIOwL (ORCPT + 97 others); Tue, 9 Nov 2021 09:52:11 -0500 Received: from mail.kernel.org ([198.145.29.99]:59320 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232967AbhKIOwK (ORCPT ); Tue, 9 Nov 2021 09:52:10 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6FC5F600CC; Tue, 9 Nov 2021 14:49:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1636469364; bh=ZUUw0hG0gNScdGrI9rGDPhca7P96fHksKizZuV5o9ZU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=HkNXFTviO47paw92Jh/pOYSVeWFD1HI30AiZJkVExd2HwxvPtsTenarWoyVq2eaha MbY35npBQfZiYjNyB2U3aSYfeSmeZAxssJFkDlr0ugUg2K+hGtSeScErilxhrf+EHS s6Wjp1Md0ER7wQQN/uZCY8hm6Ld2rKTmFnr+2qAOm/AUnQMRwu5KI/Aq3+WzEi/wok 7A4kAoKDLeCu0J6xBDlIs9XaIkNvdYkkJOLjEJrajxBvQr89Wm7vXqbgr9DmCOIWTw G++3NXpfxB0AJHWJgaKoGNl6GS5ql6z4ZdnvBC90Md8lqyZ3A+XR3receeMfcGCHaF GHY+G25Lsf7BQ== Date: Tue, 9 Nov 2021 06:49:21 -0800 From: Jakub Kicinski To: Leon Romanovsky Cc: Ido Schimmel , Jiri Pirko , "David S . Miller" , Jiri Pirko , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, edwin.peer@broadcom.com Subject: Re: [PATCH net-next] devlink: Require devlink lock during device reload Message-ID: <20211109064921.6bd65f3c@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> In-Reply-To: References: <20211108080918.2214996c@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211108101646.0a4e5ca4@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211108104608.378c106e@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211108153126.1f3a8fe8@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> <20211109061729.32f20616@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 9 Nov 2021 16:30:16 +0200 Leon Romanovsky wrote: > On Tue, Nov 09, 2021 at 06:17:29AM -0800, Jakub Kicinski wrote: > > On Tue, 9 Nov 2021 16:12:33 +0200 Leon Romanovsky wrote: > > > RDMA subsystem supports two net namespace aware scenarios. > > > > > > We need global netdev_notifier for shared mode. This is legacy mode where > > > we listen to all namespaces. We must support this mode otherwise we break > > > whole RDMA world. > > > > > > See commit below: > > > de641d74fb00 ("Revert "RDMA/mlx5: Fix devlink deadlock on net namespace deletion"") > > > > But why re-reg? To take advantage of clean event replay? > > > > IIUC the problem is that the un-reg is called from the reload_down path. > > This is how devlink reload was explained to me by Jiri. It suppose to > unload and load whole driver again. In our case, it unloads mlx5_core, > which destroys all ULPs (VDPA, RDMA, e.t.c) and these ULPs are not aware > of devlink at all. Especially they are not aware of the reason of > devlink reload. Anything registering the global notifier outside of init_net needs scrutiny. If the devlink instance was moved into namespace A then paying attention to anything outside namespace A weakens the namespace abstraction. init_net is legacy/special and it can't disappear so the problem at hand won't happen (as it is triggered on namespace cleanup). Maybe you can install the global notifier only if instance is in init_net? That's my thinking with the scant information at hand. > This is why I asked you. It is not related to devlink locking directly, > but indirectly I didn't want to work on anything related to locking > without making sure that devlink.c is correct.