Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   "Zhang, Tina" <tina.zhang@intel.com>
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        David Stevens <stevensd@chromium.org>,
        Lu Baolu <baolu.lu@linux.intel.com>
CC:     "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v2] iommu/vt-d: calculate mask for non-aligned flushes
Thread-Topic: [PATCH v2] iommu/vt-d: calculate mask for non-aligned flushes
Thread-Index: AQHYPbc0IgBxJPf1AE2VaLrNBkFNvazPo4aAgAAO31A=
Date:   Fri, 25 Mar 2022 07:15:12 +0000
Message-ID: <MW5PR11MB58814148CD05C8284C3BCB23891A9@MW5PR11MB5881.namprd11.prod.outlook.com>
References: <20220322063555.1422042-1-stevensd@google.com>
 <BN9PR11MB52761E41CAD28CFCB935688C8C1A9@BN9PR11MB5276.namprd11.prod.outlook.com>
In-Reply-To: <BN9PR11MB52761E41CAD28CFCB935688C8C1A9@BN9PR11MB5276.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
dlp-reaction: no-action
dlp-version: 11.6.401.20
dlp-product: dlpe-windows
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?h84z1JjujjsCqSVX6R0yPsDTpkyIYrI9A+NGwQHEEM8e2FTfKS64LKp1KQiu?=
 =?us-ascii?Q?Bcehsj8/nmQ19UN3PJve+liWoZ5BrmgjNCBEbg3c1m1rW9RLjMzBSrg9UBkV?=
 =?us-ascii?Q?08p0idVwJOEIakPeLM5n9yGlOYQ1eHW25TTXs8cFjMmvHeUqq/IxYHQSZina?=
 =?us-ascii?Q?/b0+UOE9GC6zXPfJALFUHjqgco5XeJiJ9IQnhY0FmCwSadhAECKk4VGSBQC0?=
 =?us-ascii?Q?gDGrZiIgIk4Ee+u2K0l2WSp66DukoA6pXgncXBNxf8cj0ZgEdHsz6biY2KM1?=
 =?us-ascii?Q?//jciN9GCQ6hHfF3PM1wjoqehSBbJidYdc0s+UlzhVoNdI7qLo7QqO2wncW2?=
 =?us-ascii?Q?bZZVTYt6ilVM4x/UNgvrbyAtskkhYMceqN1+23IVAiV6zqEu/8cyC94os2xg?=
 =?us-ascii?Q?qEdidHHihNycLHK/VDx3Qij7rjViqROTXUGl59LKer0uMvpudEWTSKHIbs5o?=
 =?us-ascii?Q?n6eG86nPhkTEaIootBoZs+l8dfXdml3921ZTJXCefvv1OK7YXWsB2IcDNpJq?=
 =?us-ascii?Q?nT3MAykeLKLqZZVfEP5GAWz6/q9k3HyLFVkElhTZ8F+ZdCQxhKlPVCBtcHD7?=
 =?us-ascii?Q?BbMtFKmmvRAUWIxpB7ItbsgAogzDM0y8CnJu/SfrboRs0DfGBOG20unRiziB?=
 =?us-ascii?Q?/uu7Jp1WQvoWEU77fbjTT4YAjYSUoM1M7YkjYaPhpBBb0JeJWyD5zAwHlUY0?=
 =?us-ascii?Q?aWMr9jheTO3PEs4j6AugOAeRTE/+ppnKAuFbFOi7LVZRSV4VmHjHtIiCMcmp?=
 =?us-ascii?Q?lpJRHAThvI7N1DsHt0LYOmLHehBKkZ9ee2hMJcEvL/DPdOL3V+0rBIe0MAYP?=
 =?us-ascii?Q?zp0Wx2ESZ3Ps7+lDFe1f5Tp8dDJ9sRLWZDCKujsoJup+VsD/7kE/h/XMN45v?=
 =?us-ascii?Q?kYig+JZALJjOM+qlLjnaDt0bT1N6n+mugzODzGMCk+8Dt8JKyu2v1j+5wOlz?=
 =?us-ascii?Q?ZZZ03dqmZxr9/igQ6ia2CihfjFgRZoCO7vrO7aA2NsgIRgAi+8z7o3KMTS6q?=
 =?us-ascii?Q?34IW0MThnXi0766ddSGw3TcekXSBXb+bAbihHY5n1E8/aBZGQyO5NLBzKA9r?=
 =?us-ascii?Q?1U+Ma9gzGvPcvfBZH9PtKtdUmivlC6gQKj+SPkDKd7Q6xPD/BT/RnRM52P9l?=
 =?us-ascii?Q?RCgV43wQcBRHP7l+V1hZtBdgPV/5Kjj+tEXIPPghBSmb5Cb8SnoA1P6Peu9e?=
 =?us-ascii?Q?eQT/sFDTrW2/BN6MwYXP5LIY9//PFZreBJapy5AGpA9sbRZ94S4GaKr1onPN?=
 =?us-ascii?Q?uiQ9mgp5VCCv/n4s18Fep37zXWTfjnyKSKzhWvoUpDdsqU8XC5tXnS2SXIJ1?=
 =?us-ascii?Q?hsLB9nXvQtmJv7OzqTrqnRP/ECiGlhGFw+YC76NJSl6sthvA5+TOPHGj7Hk4?=
 =?us-ascii?Q?/ILW9GN9S6iLSvxU8by6fujHJGH/yWdUlK0YWvSAqdtFewq4nX/NElOHYioS?=
 =?us-ascii?Q?mXE3dvhyk+xpBdIE6BEjmq+qf2+mZuB3?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW5PR11MB5881.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0abc14ab-1cbf-479c-cb41-08da0e2f3432
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Mar 2022 07:15:12.7873
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: B/k2OIKHvre+/KWVAIHV26cAmq8gWBTWMVirPd1522ZHOgx6Z6TBacsB8U6nNFEds82tlUDJ+8DCtm9dZHS3mQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5305
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: iommu <iommu-bounces@lists.linux-foundation.org> On Behalf Of
> Tian, Kevin
> Sent: Friday, March 25, 2022 2:14 PM
> To: David Stevens <stevensd@chromium.org>; Lu Baolu
> <baolu.lu@linux.intel.com>
> Cc: iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org
> Subject: RE: [PATCH v2] iommu/vt-d: calculate mask for non-aligned flushe=
s
>=20
> > From: David Stevens
> > Sent: Tuesday, March 22, 2022 2:36 PM
> >
> > From: David Stevens <stevensd@chromium.org>
> >
> > Calculate the appropriate mask for non-size-aligned page selective
> > invalidation. Since psi uses the mask value to mask out the lower
> > order bits of the target address, properly flushing the iotlb requires
> > using a mask value such that [pfn, pfn+pages) all lie within the
> > flushed size-aligned region.  This is not normally an issue because
> > iova.c always allocates iovas that are aligned to their size. However,
> > iovas which come from other sources (e.g. userspace via VFIO) may not
> > be aligned.
> >
> > Signed-off-by: David Stevens <stevensd@chromium.org>
> > ---
> > v1 -> v2:
> >  - Calculate an appropriate mask for non-size-aligned iovas instead
> >    of falling back to domain selective flush.
> >
> >  drivers/iommu/intel/iommu.c | 27 ++++++++++++++++++++++++---
> >  1 file changed, 24 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 5b196cfe9ed2..ab2273300346 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -1717,7 +1717,8 @@ static void iommu_flush_iotlb_psi(struct
> > intel_iommu *iommu,
> >  				  unsigned long pfn, unsigned int pages,
> >  				  int ih, int map)
> >  {
> > -	unsigned int mask =3D ilog2(__roundup_pow_of_two(pages));
> > +	unsigned int aligned_pages =3D __roundup_pow_of_two(pages);
> > +	unsigned int mask =3D ilog2(aligned_pages);
> >  	uint64_t addr =3D (uint64_t)pfn << VTD_PAGE_SHIFT;
> >  	u16 did =3D domain->iommu_did[iommu->seq_id];
> >
> > @@ -1729,10 +1730,30 @@ static void iommu_flush_iotlb_psi(struct
> > intel_iommu *iommu,
> >  	if (domain_use_first_level(domain)) {
> >  		domain_flush_piotlb(iommu, domain, addr, pages, ih);
> >  	} else {
> > +		unsigned long bitmask =3D aligned_pages - 1;
> > +
> > +		/*
> > +		 * PSI masks the low order bits of the base address. If the
> > +		 * address isn't aligned to the mask, then compute a mask
> > value
> > +		 * needed to ensure the target range is flushed.
> > +		 */
> > +		if (unlikely(bitmask & pfn)) {
> > +			unsigned long end_pfn =3D pfn + pages - 1, shared_bits;
> > +
> > +			/*
> > +			 * Since end_pfn <=3D pfn + bitmask, the only way bits
> > +			 * higher than bitmask can differ in pfn and end_pfn
> > is
> > +			 * by carrying. This means after masking out bitmask,
> > +			 * high bits starting with the first set bit in
> > +			 * shared_bits are all equal in both pfn and end_pfn.
> > +			 */
> > +			shared_bits =3D ~(pfn ^ end_pfn) & ~bitmask;
> > +			mask =3D shared_bits ? __ffs(shared_bits) :
> > BITS_PER_LONG;
> > +		}
>=20
> While it works I wonder whether below is simpler regarding to readability=
:
>=20
> 	} else {
> +		/*
> +		 * PSI masks the low order bits of the base address. If the
> +		 * address isn't aligned to the mask and [pfn, pfn+pages)
> +		 * don't all lie within the flushed size-aligned region,
> +		 * simply increment the mask by one to cover the trailing
> pages.
> +		 */
> +		if (unlikely((pfn & (aligned_pages - 1)) &&
> +			     (pfn + pages - 1 >=3D ALIGN(pfn, aligned_pages))))
> +			mask++;

According to the vt-d spec, increasing mask means more bits of the pfn woul=
d be masked out. So simply increasing the mask number might not be correct.=
=20
This second version does give more consideration on that.

BR,
Tina
>=20
> Thanks
> Kevin
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu