Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Laba, SlawomirX" <slawomirx.laba@intel.com>
To:     ivecera <ivecera@redhat.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>
CC:     "Keller, Jacob E" <jacob.e.keller@intel.com>,
        "Piotrowski, Patryk" <patryk.piotrowski@intel.com>,
        Vitaly Grinberg <vgrinber@redhat.com>,
        "Brandeburg, Jesse" <jesse.brandeburg@intel.com>,
        "Nguyen, Anthony L" <anthony.l.nguyen@intel.com>,
        "David S. Miller" <davem@davemloft.net>,
        "Eric Dumazet" <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
        "moderated list:INTEL ETHERNET DRIVERS" 
        <intel-wired-lan@lists.osuosl.org>,
        open list <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH net] iavf: Detach device during reset task
Thread-Topic: [PATCH net] iavf: Detach device during reset task
Thread-Index: AQHYsyNwOjkGMXAyxk6WhUnxS4n8Aq27MxGw
Date:   Mon, 22 Aug 2022 17:42:44 +0000
Message-ID: <DM6PR11MB311314DD438AE5A5AA4F5ED687719@DM6PR11MB3113.namprd11.prod.outlook.com>
References: <20220818165558.997984-1-ivecera@redhat.com>
In-Reply-To: <20220818165558.997984-1-ivecera@redhat.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.6.500.17
dlp-product: dlpe-windows
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?SoD2MV7J+yDeaB8nAzSmgmOtJ9YUGG5kLdbrqP9JK8XmYQE/QyRj4RoGLAyL?=
 =?us-ascii?Q?dSemDjU4bnMWBjLzXAJicLHO+iYgipQYm+XayZs/WkYIoTESpi2S8+EckQIS?=
 =?us-ascii?Q?vDwMMpeKHtHSvfqMo3jt5zAHs/2lXS47EgTitHvWEP2UaXbiTBWytwgIvr3t?=
 =?us-ascii?Q?rFnrXD+zHyfNBtDBNG46Uf1n/NrGg0R62bzxME02bbNVmno88bXItrUD8nR1?=
 =?us-ascii?Q?jHiKQCkKeSY2vQ1sv5DXazKiIBXBvmqeiR+JymauN45G9vtjcf7hZXm+VVlc?=
 =?us-ascii?Q?BfZcLxQY/mSd8okejwcmzqPHUJi/JH1d1s3WjRzSKWamfps6cVagAb0jFw1H?=
 =?us-ascii?Q?tatzgAUPFC7y6ELSYs+zRAENc2P41AVft5xFh+lIIQgChaRp5QXvCRf0Qwhh?=
 =?us-ascii?Q?kDWa6DadvWm6vzncG/tLtaWerQZan7x9xCRhXEPosw49Qnhk8p1wG8HBF+mU?=
 =?us-ascii?Q?WjPij4mF+DfrU6BgBo2MP/4XDS4z6vpstVBgAR6Z+qTjmk0a5onSG0QgQByZ?=
 =?us-ascii?Q?4sew3KiG/GoRNnHlsFNyIu3P36zysVyylo2cJgr5bUJKOzU1rU9Tn3g9FrjR?=
 =?us-ascii?Q?qY0eYmHe8ignPLVgVMjVUQgz67fgLZf9YH1JGyCxNAvT0SBwpVxyWQGU8Zxj?=
 =?us-ascii?Q?N7B+PqwwAY4xg4VQg8iXLUOzA4I2dMZONZcirHpiIf23Wfy9V9fLaoyttK99?=
 =?us-ascii?Q?LFjA5MLev420W92rhoFntD/x/tTLj2sTn7DVp7HJWPOi9ick0wrgXZW5fieQ?=
 =?us-ascii?Q?i+/ONeXdHNPYND50rSZqDimCdia/XK+lL7MzRwai3PqUD0Puo188NQ2FqOf9?=
 =?us-ascii?Q?w7cwvELxLEJmi+N7UdSkS0wtd3VDLjc1w9FjPFPhjSJy2rHEqkAXlnmpeX3t?=
 =?us-ascii?Q?qGIbEd89z0b9RSsNq9ONfzBUt2uTn/DVNKGcv+6jluz1ZWsMOSP8CJsf3X1L?=
 =?us-ascii?Q?9XR1avY7PUTiT/Ttg64enHmLNe9B5h9AEVdtCRvpPgxIWQ2lrgNiWCp0QD19?=
 =?us-ascii?Q?qUYFp4wD+5VfxyDBbQQOZf7/HW+tdonDFUtos0eE5eMpzUSgnoau+uRpfsuU?=
 =?us-ascii?Q?cc6iPLtgdEb4pgI9RZ3ipJqMjwAOCM+WDb0N8PpL9DI9M/KPiJgBQ33R+6VD?=
 =?us-ascii?Q?kPWVk0NywR5Vlvx+9ytP69ZpSZpAqLixZkxd4U37WjAcYei0cNKe5HlROF2A?=
 =?us-ascii?Q?91/RIPt7RlbH0jQuC6LOopKKcn3KnJvesQd0KQz6NOYXcpePQp7l3VNsTdaE?=
 =?us-ascii?Q?vCdmqALkynmKmgaUgAX0dokhkp9pdtAxoX8YPI5rYdnWoY/shdeqRLKmBw/u?=
 =?us-ascii?Q?s7SW9yD80taZhfSqQCa/RnffHFFaLJ82BAKAxoYeBsivX22qsFM4WiJVCVJU?=
 =?us-ascii?Q?RJP57YbXvCHY0z6LVaFe6oihTNWGqb+k8yzeFdPlNCJwSLTzqL6JfZJGKwkO?=
 =?us-ascii?Q?4x57pyXEcX8dW/xyDPe/IO6kwO4bEEjrYsIc88qAfulTmFyCgFHLlA1tDYVV?=
 =?us-ascii?Q?zPhGfGP7fzv/Z6FDHVAsoPGvSNYDi55BggQfNH3sr8GDPlewuBbAcpRB8NC+?=
 =?us-ascii?Q?ZRnYPEKt6QKSanwxJANSMF+Hf4nwDOT1TpGb/G1Y?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3113.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5311416-8d49-4e2f-71ab-08da8465b88f
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Aug 2022 17:42:44.9005
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yI6K8dyYN3lU1B4phSD7MamQ5rK41K9oRv7yv/hoZTPYYbD+8soF9bp1lvzcVct1RQzmfPyyRwbzZQTybHbrLDmFyYPa8TN4QNSIEFJg0JY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4831
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> -----Original Message-----
> From: ivecera <ivecera@redhat.com>
> Sent: Thursday, August 18, 2022 6:56 PM
> To: netdev@vger.kernel.org
> Cc: Keller, Jacob E <jacob.e.keller@intel.com>; Piotrowski, Patryk
> <patryk.piotrowski@intel.com>; Vitaly Grinberg <vgrinber@redhat.com>;
> Brandeburg, Jesse <jesse.brandeburg@intel.com>; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; David S. Miller <davem@davemloft.net>; Eric
> Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo
> Abeni <pabeni@redhat.com>; Jeff Kirsher <jeffrey.t.kirsher@intel.com>;
> moderated list:INTEL ETHERNET DRIVERS <intel-wired-lan@lists.osuosl.org>;
> open list <linux-kernel@vger.kernel.org>
> Subject: [PATCH net] iavf: Detach device during reset task
>=20
> iavf_reset_task() takes crit_lock at the beginning and holds it during wh=
ole call.
> The function subsequently calls
> iavf_init_interrupt_scheme() that grabs RTNL. Problem occurs when userspa=
ce
> initiates during the reset task any ndo callback that runs under RTNL lik=
e
> iavf_open() because some of that functions tries to take crit_lock. This =
leads to
> classic A-B B-A deadlock scenario.
>=20
> To resolve this situation the device should be detached in
> iavf_reset_task() prior taking crit_lock to avoid subsequent ndos running=
 under
> RTNL and reattach the device at the end.
>=20
> Fixes: 62fe2a865e6d ("i40evf: add missing rtnl_lock() around
> i40evf_set_interrupt_capability")
> Cc: Jacob Keller <jacob.e.keller@intel.com>
> Cc: Patryk Piotrowski <patryk.piotrowski@intel.com>
> Tested-by: Vitaly Grinberg <vgrinber@redhat.com>
> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
> ---
>  drivers/net/ethernet/intel/iavf/iavf_main.c | 22 +++++++++++++++------
>  1 file changed, 16 insertions(+), 6 deletions(-)
>=20
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c
> b/drivers/net/ethernet/intel/iavf/iavf_main.c
> index f39440ad5c50..ee8f911b57ea 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_main.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
> @@ -2877,6 +2877,13 @@ static void iavf_reset_task(struct work_struct
> *work)
>  	int i =3D 0, err;
>  	bool running;
>=20
> +	/*
> +	 * Detach interface to avoid subsequent NDO callbacks
> +	 */

nit:
The comment should start this way: /* Detach ...

> +	rtnl_lock();
> +	netif_device_detach(netdev);
> +	rtnl_unlock();
> +
>  	/* When device is being removed it doesn't make sense to run the reset
>  	 * task, just return in such a case.
>  	 */
> @@ -2884,7 +2891,7 @@ static void iavf_reset_task(struct work_struct *wor=
k)
>  		if (adapter->state !=3D __IAVF_REMOVE)
>  			queue_work(iavf_wq, &adapter->reset_task);
>=20
> -		return;
> +		goto reset_finish;

Correct me if I'm wrong.
In case when you fail to grab a crit_lock you'd jump to the reset_finish la=
bel and unlock the locks you didn't lock.

>  	}
>=20
>  	while (!mutex_trylock(&adapter->client_lock))
> @@ -2954,7 +2961,6 @@ static void iavf_reset_task(struct work_struct *wor=
k)
>=20
>  	if (running) {
>  		netif_carrier_off(netdev);
> -		netif_tx_stop_all_queues(netdev);
>  		adapter->link_up =3D false;
>  		iavf_napi_disable_all(adapter);
>  	}
> @@ -3081,10 +3087,8 @@ static void iavf_reset_task(struct work_struct
> *work)
>=20
>  	adapter->flags &=3D ~IAVF_FLAG_REINIT_ITR_NEEDED;
>=20
> -	mutex_unlock(&adapter->client_lock);
> -	mutex_unlock(&adapter->crit_lock);
> +	goto reset_finish;
>=20
> -	return;
>  reset_err:
>  	if (running) {
>  		set_bit(__IAVF_VSI_DOWN, adapter->vsi.state); @@ -3092,9
> +3096,15 @@ static void iavf_reset_task(struct work_struct *work)
>  	}
>  	iavf_disable_vf(adapter);
>=20
> +	dev_err(&adapter->pdev->dev, "failed to allocate resources during
> +reinit\n");
> +
> +reset_finish:
>  	mutex_unlock(&adapter->client_lock);
>  	mutex_unlock(&adapter->crit_lock);
> -	dev_err(&adapter->pdev->dev, "failed to allocate resources during
> reinit\n");
> +
> +	rtnl_lock();
> +	netif_device_attach(netdev);
> +	rtnl_unlock();
>  }
>=20
>  /**
> --
> 2.35.1