Received-SPF: pass (google.com: domain of linux-kernel+bounces-174096-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Thu, 9 May 2024 11:36:19 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>
CC: "Tian, Kevin" <kevin.tian@intel.com>, "kvm@vger.kernel.org"
	<kvm@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "x86@kernel.org" <x86@kernel.org>,
	"jgg@nvidia.com" <jgg@nvidia.com>, "iommu@lists.linux.dev"
	<iommu@lists.linux.dev>, "pbonzini@redhat.com" <pbonzini@redhat.com>,
	"seanjc@google.com" <seanjc@google.com>, "dave.hansen@linux.intel.com"
	<dave.hansen@linux.intel.com>, "luto@kernel.org" <luto@kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>, "tglx@linutronix.de"
	<tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de"
	<bp@alien8.de>, "hpa@zytor.com" <hpa@zytor.com>, "corbet@lwn.net"
	<corbet@lwn.net>, "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org"
	<will@kernel.org>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>, "Liu, Yi L"
	<yi.l.liu@intel.com>
Subject: Re: [PATCH 1/5] x86/pat: Let pat_pfn_immune_to_uc_mtrr() check MTRR
 for untracked PAT range
Message-ID: <ZjxEswEtGpOEbApc@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20240507061802.20184-1-yan.y.zhao@intel.com>
 <20240507061924.20251-1-yan.y.zhao@intel.com>
 <BN9PR11MB5276DA8F389AAE7237C7F48E8CE42@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZjnwiKcmdpDAjMQ5@yzhao56-desk.sh.intel.com>
 <20240508161424.5bf4bdfc.alex.williamson@redhat.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240508161424.5bf4bdfc.alex.williamson@redhat.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ahxV1fCWHqWy+GM7sMeqXMATfGHxpD5JtNDZtxu1yKpsDwGHbNkyd+z5DCr5?=
 =?us-ascii?Q?h+p1CcRVri2vhjl8au9JklWpNg+EpIZfZo3RQmax7VvoEphZoDQQ2VyORDwL?=
 =?us-ascii?Q?UK8fWKh3kS41YBZ/6EIwzuveVyfxg6x2ixcEsZwlpaPWkjxBTgu28hlZGvg3?=
 =?us-ascii?Q?zcvk70J4XoR2v8yLPJSmokPJX4KjyLtZR4Fl9dRKFQ/ig4OfWlLybN898TkV?=
 =?us-ascii?Q?7g6YzpZ33/o8uxcGXRvfnj5Cx5SAHjGPEBv84kd2TgakrNhFc2LQTTTPPrRa?=
 =?us-ascii?Q?SOQhvL74+IWt61FkhKiAW+1pD7pwbJ6M4hbZAsIAYOd+NpZzyOd1Z8NCHXzn?=
 =?us-ascii?Q?A9p+M/mB6rhlFb58sVgfZ5hPUMEbSZbqxUT/QVyyhaWCkn7QIJNjRWTTfczy?=
 =?us-ascii?Q?DI596uXf7pPnaSAx9FhIBz/XJ9v/ug12Pj1izYT7OgQzbW1rY4oNyXc2YLzx?=
 =?us-ascii?Q?ZRn3Dw03NrTZesuYRfNGKKJSjI65oDkoHFWdFEoTqA8A4NI2yF3btV10v5AU?=
 =?us-ascii?Q?WMcqxmsGE4NWuseTnIImeOi6Uj4DOg4SO42bbDNDVE1zZ5C9CEIuL93SsDUq?=
 =?us-ascii?Q?B9je84m8Vl2VhUagtKQg1VoCJWGM8CpH051S5O6cSQH68yt3/RZFxxU0QuyW?=
 =?us-ascii?Q?Dsh7NaEug3b3u0ngrBEK1ZAA0w/VE/1+/KkWNCpEM3aQIgjoMqHUKf4vDsdR?=
 =?us-ascii?Q?n26WodXV8+yVWrIg2scfPdZQL+N6AUDrd4NbtWhbzai/XLkxJx5m4ws1J9zF?=
 =?us-ascii?Q?5iU9V7R4F42P25MrZ+ce8kxfIpvBq/DfdreLJIw4e2pVybZi5Pxao+p/pauL?=
 =?us-ascii?Q?4fdAx3gRFO4LKtp32EiXjFA0as/pplhL5TxJ0vGfrgdVGMtgWMTm9i5WN2pd?=
 =?us-ascii?Q?0fwyBPY0Uga/jNynPa1p1otQbYuct58w3/WG7wx5190ZoKVu1jEG4ul5N+t+?=
 =?us-ascii?Q?ZXvsx6oubxtAMaAQg4nJenNf3RuU6lxkyBLZXG1G1pst90WG1g8FmOtH5v+M?=
 =?us-ascii?Q?AvC7onFno7R9UDwArxe1KIHrFnxgu/l9JFxOXWCYonPKHjQ4XrxbZTzeXOhD?=
 =?us-ascii?Q?ny7xi/NNzh2Hc+Bs8KFlmINpudOYcatRZQXb9OC+q4zIRcHmzll1/Q6B028G?=
 =?us-ascii?Q?B32+s5kZzwrSQU3JscuGjT0XyOu2qI5XnTmvUXaP9EVVh69QHSHoSJtQ0GbE?=
 =?us-ascii?Q?0dcyxS5bnCchXci5Ak0ydYDSiOyjz0WIjlVzsbX1hbic68x3NnTsg0Rnjcsu?=
 =?us-ascii?Q?ZAjP7PZVXiPuJknk9xdTfw4l0pAZipw2RUk3Ob7ocG5lreTG+09DweP0ysxE?=
 =?us-ascii?Q?jcCfoh7OQrg0xXKBNBoTPzyQTGkfcc/Dv84FFvb8S6ZPhtChewvByS//5aw2?=
 =?us-ascii?Q?2MHTUDszOyZj4ZgcKGlpaDnheIAW/G1+i48NRnjPLJmMOJR7XpzQuUFt/iIY?=
 =?us-ascii?Q?sXCETmk58xR1xXyr9kYmjuqGIzFqct0qqitJ6O+m8QPHOKrDLZA7tVUKs5Lx?=
 =?us-ascii?Q?Wh62+qJoHfhQ+ciOl1aEKi8C5HGxeFaLQ1RMHUtBTAUubImBYMpR5/JrYxni?=
 =?us-ascii?Q?I2mcR+KbFBPdU9HY8z/9ch23PxZbhZg53wzmTMAT?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cf94d140-802d-41d5-6e12-08dc6fd94af6
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 03:37:03.8222
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JFblnNAijo/itDOWAap9A12KNTm5DrrGUbBzY67t10SiT7iBlRdMxgnErbofkMTH3bCDMJkNd3nuLyKb8Hmieg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR11MB6505
X-OriginatorOrg: intel.com

On Wed, May 08, 2024 at 04:14:24PM -0600, Alex Williamson wrote:
> On Tue, 7 May 2024 17:12:40 +0800
> Yan Zhao <yan.y.zhao@intel.com> wrote:
> 
> > On Tue, May 07, 2024 at 04:26:37PM +0800, Tian, Kevin wrote:
> > > > From: Zhao, Yan Y <yan.y.zhao@intel.com>
> > > > Sent: Tuesday, May 7, 2024 2:19 PM
> > > > 
> > > > However, lookup_memtype() defaults to returning WB for PFNs within the
> > > > untracked PAT range, regardless of their actual MTRR type. This behavior
> > > > could lead KVM to misclassify the PFN as non-MMIO, permitting cacheable
> > > > guest access. Such access might result in MCE on certain platforms, (e.g.
> > > > clflush on VGA range (0xA0000-0xBFFFF) triggers MCE on some platforms).  
> > > 
> > > the VGA range is not exposed to any guest today. So is it just trying to
> > > fix a theoretical problem?  
> > 
> > Yes. Not sure if VGA range is allowed to be exposed to guest in future, given
> > we have VFIO variant drivers.
> 
> include/uapi/linux/vfio.h:
>         /*
>          * Expose VGA regions defined for PCI base class 03, subclass 00.
>          * This includes I/O port ranges 0x3b0 to 0x3bb and 0x3c0 to 0x3df
>          * as well as the MMIO range 0xa0000 to 0xbffff.  Each implemented
>          * range is found at it's identity mapped offset from the region
>          * offset, for example 0x3b0 is region_info.offset + 0x3b0.  Areas
>          * between described ranges are unimplemented.
>          */
>         VFIO_PCI_VGA_REGION_INDEX,
> 
> We don't currently support mmap for this region though, so I think we
> still don't technically require this, but I guess an mmap through KVM
> is theoretically possible.  Thanks,

Thanks, Alex, for pointing it out.
KVM does not mmap this region currently, and I guess KVM will not do the mmap
by itself in future too.

I added this check for VGA range is because I want to call
pat_pfn_immune_to_uc_mtrr() in arch_clean_nonsnoop_dma() in patch 3 to exclude VGA
ranges from CLFLUSH,  as arch_clean_nonsnoop_dma() is under arch/x86 and not
virtualization specific.

Also, as Jason once said that "Nothinig about vfio actually guarantees that"
"there's no ISA range" (VGA range), I think KVM might see this range after
hva_to_pfn_remapped() translation, and adding this check may be helpful to KVM,
too.

Thanks
Yan

> 
> > > > @@ -705,7 +705,17 @@ static enum page_cache_mode
> > > > lookup_memtype(u64 paddr)
> > > >   */
> > > >  bool pat_pfn_immune_to_uc_mtrr(unsigned long pfn)
> > > >  {
> > > > -	enum page_cache_mode cm = lookup_memtype(PFN_PHYS(pfn));
> > > > +	u64 paddr = PFN_PHYS(pfn);
> > > > +	enum page_cache_mode cm;
> > > > +
> > > > +	/*
> > > > +	 * Check MTRR type for untracked pat range since lookup_memtype()
> > > > always
> > > > +	 * returns WB for this range.
> > > > +	 */
> > > > +	if (x86_platform.is_untracked_pat_range(paddr, paddr + PAGE_SIZE))
> > > > +		cm = pat_x_mtrr_type(paddr, paddr + PAGE_SIZE,
> > > > _PAGE_CACHE_MODE_WB);  
> > > 
> > > doing so violates the name of this function. The PAT of the untracked
> > > range is still WB and not immune to UC MTRR.  
> > Right.
> > Do you think we can rename this function to something like
> > pfn_of_uncachable_effective_memory_type() and make it work under !pat_enabled()
> > too?
> > 
> > >   
> > > > +	else
> > > > +		cm = lookup_memtype(paddr);
> > > > 
> > > >  	return cm == _PAGE_CACHE_MODE_UC ||
> > > >  	       cm == _PAGE_CACHE_MODE_UC_MINUS ||  
> > >   
> > 
>