Received-SPF: pass (google.com: domain of linux-kernel+bounces-42468-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <bfbf7847-6a47-4a1a-a04a-1151b7732e84@intel.com>
Date: Mon, 29 Jan 2024 17:33:21 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 5/5] iommu/vt-d: improve ITE fault handling if target
 device isn't present
Content-Language: en-US
To: Ethan Zhao <haifeng.zhao@linux.intel.com>, <baolu.lu@linux.intel.com>,
	<bhelgaas@google.com>, <robin.murphy@arm.com>, <jgg@ziepe.ca>
CC: <kevin.tian@intel.com>, <dwmw2@infradead.org>, <will@kernel.org>,
	<lukas@wunner.de>, <iommu@lists.linux.dev>, <linux-kernel@vger.kernel.org>,
	<linux-pci@vger.kernel.org>
References: <20240129034924.817005-1-haifeng.zhao@linux.intel.com>
 <20240129034924.817005-6-haifeng.zhao@linux.intel.com>
From: Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <20240129034924.817005-6-haifeng.zhao@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q0VKM09tQXRUclVMcnkzaDZWZmVDaGFXazFZdUQ0amlvcGdLVHJwVjZtcUJG?=
 =?utf-8?B?QmNrQm1nS2RvbXBmY0VKWDZEOWV2TEhNM0s5T0xmTDdBa1NHcWk0ODNNYklu?=
 =?utf-8?B?M3NxVHBYNTRoTFo2RlFYVlAxdTRidmg0N1Bpalo3VkJYejBIeTQyTUpscExn?=
 =?utf-8?B?Nk5VYUFEVGV2TDQrbkVEWDg0TUk3NUZXMXpFRU9LSTZUaVVZV1VhenBXM2Vo?=
 =?utf-8?B?MWxvOVA4dmk3RzNLc2VGNzV1WmNCNk1Tc1NNZnpxSnRWVE51MThPbnd0VXdo?=
 =?utf-8?B?ZlFTRFVCbFIweWhoN3dkWm5LNlc3Q2J6TVVrQ2ZUZkRKR2FNWDNZVjNsRUNx?=
 =?utf-8?B?MnlBWXd6SjJGb0JodnpjOEpaMEpEMm54WjRXRUdqdVJRMExGMUo2akoxTjFY?=
 =?utf-8?B?UlArRnIzK1RKazRKUmRsUEpYeFFuQWVCYi9Md1cvRmlGblFjdTNoMmJQSnBP?=
 =?utf-8?B?UWpEQUxGc1puOWVUVm1FUENCcXdOSzhRZWFOSFVXSlA4bW9uM2N0UTFrMFFj?=
 =?utf-8?B?cUh3elEraDMvYnBLNHNOdTB4ZURFSldqMU1waStWS0F5UHZpRGlqOTdQMms3?=
 =?utf-8?B?QzYySk1KdE9LQ2Q0QmZnQzBma1psU0Q1RmR4ak9oWVo2bkZuK0l0aDdHWncv?=
 =?utf-8?B?cmphK0twVFJzVnQxNE93UVdQMFNYVCtSRkRtVXFlaGtqckpkM1NHMDJvZVFF?=
 =?utf-8?B?bmJJbTN4K2lBNERaTW9lTnhRenRDdkN2NXNyaTJia285dEZYdzlNZW42Y3lx?=
 =?utf-8?B?WTdDQ1BNUHRxNzhTZE5RamdTc1JJNzhZdlgxeUhUUG9KUm5DR1kwand2cXJU?=
 =?utf-8?B?dzZhL2hFZFZPdm9GY09pdnhneTNCOXAvTTRmSUlSZUVhek5YU3NHRk1pL3Rq?=
 =?utf-8?B?Yk9QR1RrRHlzTjB5NnUrbExlcjQ2UjgwTjRnWER3OW82eHYrT1F1RnRUS3Zz?=
 =?utf-8?B?S2NKTmg3T05VTFlkMXVhd0gzdmE0alArR2kyMzgyQ3RqUjlYdUpoRXY5ajda?=
 =?utf-8?B?MS9Cb0I4eGswY3FpYW04REZ6eWNXVUl6bDVzUnUvcHhrWjFXSmVabTI4THpp?=
 =?utf-8?B?RkJTTTBuSHRLMmI5cWRKbjhZWnMxcmhmaGhJMVRYbmljVjdtcVJKM3hHelVK?=
 =?utf-8?B?RzBPbU5LR2RVYW5nMEhOL0QwcDJKeCt1UCtEUmV3VmxZTW1FT3VlK1RMOXgz?=
 =?utf-8?B?QVg2TytISWd5bWVtK21pbmdlbExWWVBoYnFnVzBWSGdzeTgyRzZTYitPV1R4?=
 =?utf-8?B?YmlENUFuRlk5a3FZOTZGMjFLUE5SU1JZMi80OWJxWFRrOHBFbVpDNTF3L2l0?=
 =?utf-8?B?ak40WkNSSytUK3ZlQzFLZTg5VUkyOWhHSndrT2pEVVpBZjJXdXdHSGVYZzhm?=
 =?utf-8?B?MWY0RkYvTVd4UTY4bWFUemgzMWVvVVN6MVpmSzFMMlZta3JnbVd6YWZZZi9p?=
 =?utf-8?B?enl1SWRKT3FTL3NyRFRUeFpvV3YrcFV3VUF3SFhNR1ZHdmJETXVpOThtcis2?=
 =?utf-8?B?TkQzQkxxb05aSnBJeUtjVkhSbWRSV21ocUhTdDNJclhuZEpLc3U0RmUzM2VI?=
 =?utf-8?B?ZlFpdzRSR0YvUlM4TFo1VzFPU1dKVU9iS3AzR0FFRFNpTWdmcVRLR2d3N1E4?=
 =?utf-8?B?eE5lMVIzU29ZZStxajdBMytJWHRJRy9YRlJwRm5ra0NLSFh5ZVlWaDBjWUt3?=
 =?utf-8?B?T0pUdS90Wk1XMUdPdXhwWXphbHgxazBzUXRQVDlKaTBFZ0dPSW5VOCtRRXZo?=
 =?utf-8?B?YkNDNk00RCtYN1JLOUNEWjk2Q0VScXhWSVVyaitSSjhhSzI3TzNxV0xJNW5j?=
 =?utf-8?B?ZDB5QkRDYjgraDBMQmFWZnhDYmVrcVhVWlMwR04zNFhoK2VtU2JuM0FMNmsv?=
 =?utf-8?B?eGdhMjJVVEdqYXhaZEoxcGgyb3lBdkk2bHZKQ2hUTmhCbWFBWC96aHV4M3lZ?=
 =?utf-8?B?SDRuVjZ3VU1zQmRyMDYyUUE3SWh6RDRmeUxsSzM3YWY1c2Faa05UYmdRTXhv?=
 =?utf-8?B?MFNrWTU1UUthWVAvM1NVeHJCMFhaYTVGSGFURlp6WjRZdVhhTzAvam9HL2s3?=
 =?utf-8?B?dEFFTHdUeXdQT0dmaDJ0UnlORnFmVEQ3eW1XekphK2lKTWxCWTZ2c25URk5v?=
 =?utf-8?Q?9bCYlsH2K78DkTbMmVDSUDC5/?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 35a1ebf0-a2ce-4366-053d-08dc20ace478
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2024 09:30:12.2168
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Sb1izzkLoEncT0QKXkt9uI3RlY4hcc2LlL3XX9cDgL5rHSNAmvH83BWo9v6Oz/JDO53snG17jvdlndx29Oi5+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6264
X-OriginatorOrg: intel.com

On 2024/1/29 11:49, Ethan Zhao wrote:
> Because surprise removal could happen anytime, e.g. user could request safe
> removal to EP(endpoint device) via sysfs and brings its link down to do
> surprise removal cocurrently. such aggressive cases would cause ATS
> invalidation request issued to non-existence target device, then deadly
> loop to retry that request after ITE fault triggered in interrupt context.
> this patch aims to optimize the ITE handling by checking the target device
> presence state to avoid retrying the timeout request blindly, thus avoid
> hard lockup or system hang.
>

should include a Fix tag here?

> Signed-off-by: Ethan Zhao <haifeng.zhao@linux.intel.com>
> ---
>   drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
>   1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
> index 814134e9aa5a..2e214b43725c 100644
> --- a/drivers/iommu/intel/dmar.c
> +++ b/drivers/iommu/intel/dmar.c
> @@ -1272,6 +1272,7 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index,
>   {
>   	u32 fault;
>   	int head, tail;
> +	u64 iqe_err, ite_sid;
>   	struct q_inval *qi = iommu->qi;
>   	int shift = qi_shift(iommu);
>   
> @@ -1316,6 +1317,13 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index,
>   		tail = readl(iommu->reg + DMAR_IQT_REG);
>   		tail = ((tail >> shift) - 1 + QI_LENGTH) % QI_LENGTH;
>   
> +		/*
> +		 * SID field is valid only when the ITE field is Set in FSTS_REG
> +		 * see Intel VT-d spec r4.1, section 11.4.9.9
> +		 */
> +		iqe_err = dmar_readq(iommu->reg + DMAR_IQER_REG);
> +		ite_sid = DMAR_IQER_REG_ITESID(iqe_err);
> +
>   		writel(DMA_FSTS_ITE, iommu->reg + DMAR_FSTS_REG);
>   		pr_info("Invalidation Time-out Error (ITE) cleared\n");
>   
> @@ -1325,6 +1333,16 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index,
>   			head = (head - 2 + QI_LENGTH) % QI_LENGTH;
>   		} while (head != tail);
>   
> +		/*
> +		 * If got ITE, we need to check if the sid of ITE is the same as
> +		 * current ATS invalidation target device, if yes, don't try this
> +		 * request anymore if the target device isn't present.
> +		 * 0 value of ite_sid means old VT-d device, no ite_sid value.
> +		 */
> +		if (pdev && ite_sid && !pci_device_is_present(pdev) &&
> +			ite_sid == pci_dev_id(pci_physfn(pdev)))
> +			return -ETIMEDOUT;
> +
>   		if (qi->desc_status[wait_index] == QI_ABORT)
>   			return -EAGAIN;
>   	}

-- 
Regards,
Yi Liu