Received-SPF: pass (google.com: domain of linux-kernel+bounces-196605-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <d71d21ef-7e72-4962-acbe-a4cd82e6e1ca@intel.com>
Date: Fri, 31 May 2024 11:38:23 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net,v2] ice: avoid IRQ collision to fix init failure on
 ACPI S3 resume
To: Ricky Wu <en-wei.wu@canonical.com>, <jesse.brandeburg@intel.com>
CC: <anthony.l.nguyen@intel.com>, <davem@davemloft.net>,
	<edumazet@google.com>, <kuba@kernel.org>, <pabeni@redhat.com>,
	<intel-wired-lan@lists.osuosl.org>, <netdev@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <rickywu0421@gmail.com>,
	<michal.swiatkowski@linux.intel.com>, <pmenzel@molgen.mpg.de>, Cyrus Lien
	<cyrus.lien@canonical.com>
References: <20240530142131.26741-1-en-wei.wu@canonical.com>
Content-Language: en-US
From: Wojciech Drewek <wojciech.drewek@intel.com>
In-Reply-To: <20240530142131.26741-1-en-wei.wu@canonical.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZlNIWTNMRlM5b2VuSmpUZmFKeDUxVm9JclFWSlJEbk9Kb3NWQWoySGxtdlMx?=
 =?utf-8?B?bVNralFkSFpPSWZwRFllcUl0MzBtdnRnUmhIY1p1eWQzcEJ6K0Uzbi9WN096?=
 =?utf-8?B?UFl6L1ovQy8rRGd1RlA0SEF2WFlZWjhjcEd2TUlkSW03VWRlaGh1azFrUUNj?=
 =?utf-8?B?Q0xsaFoxeHhwSUN0ZEt1bUtRNnp2OEo1SERuQk1oKytMVThlaWlYSng5WUQ1?=
 =?utf-8?B?akhTYzkvVkMwQ3R5WmoyRGhlT2g4eWpjakl6bmN0ZjkzL1hzVFZXcWJEZ3FZ?=
 =?utf-8?B?VGR6Y2FSRFRZcEp3TUk2Z2pQYTk3WlVlZWtjZ3pBVFhKS2hwWkZOaXJ0MGYw?=
 =?utf-8?B?MkFBNUc1M0N0Vk1Nam9qdHd4bFc3eDRmRmkvQnZzR3E2L1FwTnJheHd4MlJM?=
 =?utf-8?B?c1FINTNZbmVBTkZiSVhvakFjeVdtOWc0bGpzdkxnN2RXWU1uWTc1eEpUUys0?=
 =?utf-8?B?dzdkZjNCMU1EV1VMLytGNkhQTm9XT2MrWER3Q2EvcHJJQzZBV0FvVWhONWpX?=
 =?utf-8?B?N0Q1UnppeEFNR1d1OUVvak1EV0RtRFNvL3Bvb3UyR092c1NqU0drdnBHN0Mx?=
 =?utf-8?B?UVc3L3RidXEza0tKUDRnSS9SRSs4UWdIWkZoSFhydWpGTEVocUZSMC95SHM2?=
 =?utf-8?B?WTZKUGpUTTgzaFVhVWxBa2FOM09DZU9RQTZBRnIvUjliLzB0cGt5NndmL1ZG?=
 =?utf-8?B?QVFWbHo4TnVoVmdMNEZwRjFXTUdrVFowS0VMa29hTCtReU1MT3AzM0JsSjgr?=
 =?utf-8?B?d055NFNVN2tuenVsNlBGV3JKeEZvRWR1YStuMEFLOWNPUDdxWDFtaE03WUVo?=
 =?utf-8?B?ZklvMjVDYklOekhRMlNRWU9HVXE3TlZTTXdPbnBnMWsyR0Z4QmhNKy90cHNz?=
 =?utf-8?B?ZC9VZTBWM09kTHFoRTRMbGxPd1M3RmY1eW9Xbzc1S1Y1NUl5amZWeVNZaXVt?=
 =?utf-8?B?cGFUb05NL3NSTjQ2aCtrOWxQVTM0ZWtRbVlEemlPYmdKUVh1RkliZ3N3WkYx?=
 =?utf-8?B?TDU5cTRiU2JGWkNxWWtVNWNRQ3czTGViMHovUUt3RTFDbUFnK3RzZytjY3RK?=
 =?utf-8?B?cldhOXZOMWhFZXd2SERzTWROM2svRloyZGFjYW8yWnlDbFZaR1kzNCs2Rzdk?=
 =?utf-8?B?UlV3VUtPaE9mc0ZQS2FGRVVYWkhQLzlNT25zYVhJbktPK2tEcXRJRDYrOER6?=
 =?utf-8?B?SC96TzMrQ3d3bzhrUnpZWXZyUzJmTlJpMjhZK28xUUZYTzNuTU11a2RhTUNt?=
 =?utf-8?B?S05uVkY3ZlpQRXBjcTVUNE96NGx5U0ZDUlRGdDRCcEFTMzQxaEtsS1lHYXUz?=
 =?utf-8?B?VmVXbUY2S0ZlTXpMWGM2OHZVYkdxWVdVV3NlVkNhbHpyNEdsVkYvNXUxMFBH?=
 =?utf-8?B?cUdIbWptU01rQXl5MDY2ZmRaYW9DZjhteERCUWM4cHNPWURYL3FLeFI3N3lK?=
 =?utf-8?B?QkoveVdjWmVOZW5wOUJGMFp4Z0M3ZEFxVStBbFcweU9OZ3JPUE9YOHdYOFUr?=
 =?utf-8?B?RTlnR1dVMUU4VFhsdEdKVjBOdTU4Q3duMmhrbEN3NUFhR0QzQU0yTFEwU0Fk?=
 =?utf-8?B?TVU0eEZQTTFGdkFBVEZ5cWtxbk1JVnk2YVNmOXJJWGFKMDVlU0J1RXNlLytS?=
 =?utf-8?B?NEpwUXRHcURkOGFOWHFvZGhGdHgyYWRKN2NWd2tIc25NdVVaM3c2YU0vdWdp?=
 =?utf-8?B?cXVla3RsWGN2eVNGS2dQa1dZUkRmejVsbloyTGYrakdud2hVYWZFNDlHTFNF?=
 =?utf-8?B?YUhGSUt6ZUFJb1ZHYzJmU3dSNXp1TWFtMmlqYkpLTEU1eURMVVZOR0lmWVJr?=
 =?utf-8?B?TlZQbkYwdDRQNW9jcDF1bm9pTTZacDA4OEdsY1lhZWpKdlZ1K3UrMkhnYk15?=
 =?utf-8?B?ZWQ0cUt6Tmd1Ym41SWVPd0w5dXQ0NmNDTzVaRURKTDlDQmJPbVZRZjRqN2Y4?=
 =?utf-8?B?SEQ2dnlOekZkYytOM2pON0NpOGpGWG03bU8xTFNhci9xTnpORXRiaHViZ1Rv?=
 =?utf-8?B?VitXZUVtbWV1K2ljTXE3b1MyZlByRlU2aWsrSEhoN2FxQ2N0Z3VNaWZuazJh?=
 =?utf-8?B?M3ZLUGdNNWR4ZDMvTWVTTDBSbmZqNXRqMkxxVWdqcElUNkRQeVppZi9zRmhS?=
 =?utf-8?B?YnJ4ejd0dEhSVGtWUGVic0Q0djhEc0dnY01kNHNON3hOSTBzU25VMVdMaDJw?=
 =?utf-8?B?R1E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 85ba2ae6-2bd6-4612-c259-08dc81556f8a
X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5776.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 09:38:32.5883
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 80tcK9H1UJdka2kgrHwaE0SCBq2YNlbvRdskh30k4RGEoAYnGPQswf7SKtCezszakImr5SOdC0hKSJ/C1gaqrlZ2bojWu1rUgeWbdpl7APg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5969
X-OriginatorOrg: intel.com


On 30.05.2024 16:21, Ricky Wu wrote:
> A bug in https://bugzilla.kernel.org/show_bug.cgi?id=218906 describes
> that irdma would break and report hardware initialization failed after
> suspend/resume with Intel E810 NIC (tested on 6.9.0-rc5).
> 
> The problem is caused due to the collision between the irq numbers
> requested in irdma and the irq numbers requested in other drivers
> after suspend/resume.
> 
> The irq numbers used by irdma are derived from ice's ice_pf->msix_entries
> which stores mappings between MSI-X index and Linux interrupt number.
> It's supposed to be cleaned up when suspend and rebuilt in resume but
> it's not, causing irdma using the old irq numbers stored in the old
> ice_pf->msix_entries to request_irq() when resume. And eventually
> collide with other drivers.
> 
> This patch fixes this problem. On suspend, we call ice_deinit_rdma() to
> clean up the ice_pf->msix_entries (and free the MSI-X vectors used by
> irdma if we've dynamically allocated them). On resume, we call
> ice_init_rdma() to rebuild the ice_pf->msix_entries (and allocate the
> MSI-X vectors if we would like to dynamically allocate them).
> 
> Fixes: f9f5301e7e2d ("ice: Register auxiliary device to provide RDMA")
> Tested-by: Cyrus Lien <cyrus.lien@canonical.com>
> Signed-off-by: Ricky Wu <en-wei.wu@canonical.com>
> ---

Reviewed-by: Wojciech Drewek <wojciech.drewek@intel.com>

> Changes in v2:
> - Change title
> - Add Fixes and Tested-by tags
> - Fix typo
> ---
>  drivers/net/ethernet/intel/ice/ice_main.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
> index f60c022f7960..ec3cbadaa162 100644
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -5544,7 +5544,7 @@ static int ice_suspend(struct device *dev)
>  	 */
>  	disabled = ice_service_task_stop(pf);
>  
> -	ice_unplug_aux_dev(pf);
> +	ice_deinit_rdma(pf);
>  
>  	/* Already suspended?, then there is nothing to do */
>  	if (test_and_set_bit(ICE_SUSPENDED, pf->state)) {
> @@ -5624,6 +5624,10 @@ static int ice_resume(struct device *dev)
>  	if (ret)
>  		dev_err(dev, "Cannot restore interrupt scheme: %d\n", ret);
>  
> +	ret = ice_init_rdma(pf);
> +	if (ret)
> +		dev_err(dev, "Reinitialize RDMA during resume failed: %d\n", ret);
> +
>  	clear_bit(ICE_DOWN, pf->state);
>  	/* Now perform PF reset and rebuild */
>  	reset_type = ICE_RESET_PFR;