Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        "Will Deacon" <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        "Jason Gunthorpe" <jgg@ziepe.ca>,
        Jean-Philippe Brucker <jean-philippe@linaro.org>,
        Nicolin Chen <nicolinc@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v5 12/12] iommu: Improve iopf_queue_flush_dev()
Thread-Topic: [PATCH v5 12/12] iommu: Improve iopf_queue_flush_dev()
Thread-Index: AQHZ5un40D2tpmzdOUyPzVi0H1ZCHLArLI+Q
Date:   Mon, 25 Sep 2023 07:00:43 +0000
Message-ID: <BN9PR11MB5276E30109C63D06675042758CFCA@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20230914085638.17307-1-baolu.lu@linux.intel.com>
 <20230914085638.17307-13-baolu.lu@linux.intel.com>
In-Reply-To: <20230914085638.17307-13-baolu.lu@linux.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?BU0Jr+GJ3HsyMZPgk+kPRh5qcG0XdWgvfra++giPBgsizSmAsytWUg9HkHU1?=
 =?us-ascii?Q?p96ugXfUdGBX2KEOULvAI9hFlMtMRxw5LXSUqKQns/2gnvK/n4Knxbv7BhCq?=
 =?us-ascii?Q?Nrr0E13QYxiQIRNoJFFKfmCRU38G+JQAoYnWk3uGs3/Q4aZJhe4vcN/f9cDJ?=
 =?us-ascii?Q?yOiPf5X/AOnjLDua9WtRUF0cCbCw1qfaBS46ax6BImnKDLpw5Q10/DpOi73V?=
 =?us-ascii?Q?c+hj2+c264wHn9i0cSToeH9ptwn0ygIAh/93chUkNzhxgzeDYRFDkdAjUo+u?=
 =?us-ascii?Q?2jfOdkAHB1btl7H38pD/8btL+aPFfuOHD1MQjYqbbMjNshA03+8O9PNFgJfL?=
 =?us-ascii?Q?bpWpTu3vGtK4uy945evd9nOlbYXiKzZZ8T9MBsyCXHNHMVjaYlkaNoJ3lckL?=
 =?us-ascii?Q?0FI13TQw+FDEhEAsQD0he7QOcvr7AXl48STVtSwAnvS+RL+27vAFQ1g3nGAU?=
 =?us-ascii?Q?OElx601tlZGX/DD2wd86UHg9fhro9t69EZh322BgjXD09iIlDVZa4SDi978P?=
 =?us-ascii?Q?IABXhoVHsUdpK86yKx8YRHBpySZ+sr22epWFtfS3+C8Cc5bqA+zNZy1KWXLb?=
 =?us-ascii?Q?7lceqwO9Qkz52MWKiei3vyzTHKOPfVdq34eQfX5LIeMRKS4uLG4I0UFBDQAw?=
 =?us-ascii?Q?RQZI318lTtvkf/audHix247pBD62E/mINZILc2Nl7qAd61jU88jGO5UWaXqJ?=
 =?us-ascii?Q?xGhIHsn1JSOj1FWbUvSmM5sJa//jMnM9X+LJlkgyADM7906tPaGzPfkBCN4u?=
 =?us-ascii?Q?GL6LURFEcJaazXT4Zcy8XsL8rk7iZ7JPlHq5cbfO+bKel/Mpxb7z4lY/52w/?=
 =?us-ascii?Q?BGDWQ02rkBrm9TG924UwoCQ7nlgz8OGZefBkArf4PTdIww93ZvrZ2k/CBxgN?=
 =?us-ascii?Q?YOr9dnrPGwR2ayGnVEKNdHVOkUG33vvNjddO5dHs9tA6ILuPf8lBd85srRJy?=
 =?us-ascii?Q?1u11ICbgSmVnmm2MgbOsOwRpNQ6lDUccI6SY/cY1OH/CmDrSr2VTkty4lpoN?=
 =?us-ascii?Q?LMz1pahUNwwE2f5M6+moPr5Ixj33+fOdMEpgiIjBgMPBF4VoCdVK7rYQ3JPd?=
 =?us-ascii?Q?RKN95WFp4qOavCrHr0O6N/22mF8KnSdHxAKcaARQDe94vMzVoq8qrYY9Ml6L?=
 =?us-ascii?Q?trUtdRjOHe+p9lc99kgBz3AlweIqrc4RfYIEbcDWpqe/sD/fulH7TFJzxjrs?=
 =?us-ascii?Q?e1KtLxdFRkCdtHwA5yfO7ZJXGSFkZHp80MDIETpXW32xssqQILHxiyUr8Wf3?=
 =?us-ascii?Q?xEri5kTeKkZcZXfSuno6SzATUCHfrKozlECXKg97Z/FrRl9sOFMhJeh8MleC?=
 =?us-ascii?Q?yeaBI14bLDKWiaJgmp1IYT54B03ousW05Z7NzTntBQYHVmXP2prjsmsT2A2S?=
 =?us-ascii?Q?OCZ53wiEs8+4r2Sfv2S4W9r9DwEvZdjAkKFtf+644GmuhNDsft3i1un+mNLY?=
 =?us-ascii?Q?8YEIP6DDHIQxtplWjr/cN2JIqpbSJOKzz1RqAuoExUAY255Rm13mGsKgvMRO?=
 =?us-ascii?Q?5k0soEBNxhIs+8IRMBvRjIvVGw/DWNZ3BaHdzsPr+oQG2Fg5m2JgzkD/jiys?=
 =?us-ascii?Q?u97WbnTfjUpq8/pS1Z8YfE7B3J2kYXitH9hxPTxw?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 925e6696-64bb-4638-f2a1-08dbbd952299
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Sep 2023 07:00:43.1182
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: WjUykPxaCnuYyS1Qq8L/Vx2X1aQMfOAZiX/5bbVkiiBRebmOWcm8KA4uFY390NefkE3F9MBH5sMtB8cgCqbwSA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7934
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 25 Sep 2023 00:01:08 -0700 (PDT)

> From: Lu Baolu <baolu.lu@linux.intel.com>
> Sent: Thursday, September 14, 2023 4:57 PM
> @@ -300,6 +299,7 @@ EXPORT_SYMBOL_GPL(iommu_page_response);
>  /**
>   * iopf_queue_flush_dev - Ensure that all queued faults have been
> processed
>   * @dev: the endpoint whose faults need to be flushed.
> + * @pasid: the PASID of the endpoint.
>   *
>   * The IOMMU driver calls this before releasing a PASID, to ensure that =
all
>   * pending faults for this PASID have been handled, and won't hit the
> address

the comment should be updated too.

> @@ -309,17 +309,53 @@ EXPORT_SYMBOL_GPL(iommu_page_response);
>   *
>   * Return: 0 on success and <0 on error.
>   */
> -int iopf_queue_flush_dev(struct device *dev)
> +int iopf_queue_flush_dev(struct device *dev, ioasid_t pasid)

iopf_queue_flush_dev_pasid()?

>  {
>  	struct iommu_fault_param *iopf_param =3D
> iopf_get_dev_fault_param(dev);
> +	const struct iommu_ops *ops =3D dev_iommu_ops(dev);
> +	struct iommu_page_response resp;
> +	struct iopf_fault *iopf, *next;
> +	int ret =3D 0;
>=20
>  	if (!iopf_param)
>  		return -ENODEV;
>=20
>  	flush_workqueue(iopf_param->queue->wq);
> +
> +	mutex_lock(&iopf_param->lock);
> +	list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) {
> +		if (!(iopf->fault.prm.flags &
> IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) ||
> +		    iopf->fault.prm.pasid !=3D pasid)
> +			break;
> +
> +		list_del(&iopf->list);
> +		kfree(iopf);
> +	}
> +
> +	list_for_each_entry_safe(iopf, next, &iopf_param->faults, list) {
> +		if (!(iopf->fault.prm.flags &
> IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) ||
> +		    iopf->fault.prm.pasid !=3D pasid)
> +			continue;
> +
> +		memset(&resp, 0, sizeof(struct iommu_page_response));
> +		resp.pasid =3D iopf->fault.prm.pasid;
> +		resp.grpid =3D iopf->fault.prm.grpid;
> +		resp.code =3D IOMMU_PAGE_RESP_INVALID;
> +
> +		if (iopf->fault.prm.flags &
> IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID)
> +			resp.flags =3D IOMMU_PAGE_RESP_PASID_VALID;
> +
> +		ret =3D ops->page_response(dev, iopf, &resp);
> +		if (ret)
> +			break;
> +
> +		list_del(&iopf->list);
> +		kfree(iopf);
> +	}
> +	mutex_unlock(&iopf_param->lock);
>  	iopf_put_dev_fault_param(iopf_param);
>=20
> -	return 0;
> +	return ret;
>  }

Is it more accurate to call this function as iopf_queue_drop_dev_pasid()?
The added logic essentially implies that the caller doesn't care about
responses and all the in-fly states are either flushed (request) or
abandoned (response).

A normal flush() helper usually means just the flush action. If there is
a need to wait for responses after flush then we could add a
flush_dev_pasid_wait_timeout() later when there is a demand...