Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 20 Jun 2023 11:17:10 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     Chao Gao <chao.gao@intel.com>
CC:     <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <pbonzini@redhat.com>, <seanjc@google.com>, <kai.huang@intel.com>,
        <robert.hoo.linux@gmail.com>
Subject: Re: [PATCH v3 07/11] KVM: VMX: drop IPAT in memtype when CD=1 for
 KVM_X86_QUIRK_CD_NW_CLEARED
Message-ID: <ZJEaNlDe1jVkYyqz@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20230616023101.7019-1-yan.y.zhao@intel.com>
 <20230616023815.7439-1-yan.y.zhao@intel.com>
 <ZJESMaG5Thb5LWtt@chao-email>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZJESMaG5Thb5LWtt@chao-email>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LtDby6+GhF0RyRVtYUYIo0UJyYNwzl5P9dUnuopZqJDfuC96Mx+C8zlqabaG?=
 =?us-ascii?Q?K52Xe6cgn3RHE2ZufaPH6dpWnKnLbFpM7BO0B0LUtgO+Se9Wq65zlFr2N8Lg?=
 =?us-ascii?Q?aA5LEerUI0y5oq9He973Ndlec5FM2VJm1JOj65HtM/XaOiTBZ34Km5oApM+u?=
 =?us-ascii?Q?YRD2r1qWQ3WyIX30zxyIrOGOcGZUjg7BdOXKKYfz8nR/6Y7kYNWspkF/YMrs?=
 =?us-ascii?Q?5VX9ui1uk2InHOs6J9vw3M/MkjPh4Zh+rDUkFBQtco+sFw9F2AgkRPE9Yso+?=
 =?us-ascii?Q?k0IKc4+FkRTslSiMClBFr4rrw5XhT9UOhP1QnFSlhWmfHAhU82Gobg3LbDty?=
 =?us-ascii?Q?IZ5MoTfXTb9lz/aWlFPD4Sdfh76sCvOkEQwsilpxIEqso4t0wCw+N7gu2bDq?=
 =?us-ascii?Q?KXDZ/ttwbA68B4Eapr1LD/sbb10lWrBtpinOOzu71xFzVcX+XA8rDjRbK0l3?=
 =?us-ascii?Q?uP6vAptmbg6c73lSETCTKGDNjyqFr3tyJUMhUMWr+PZYiMMfVDa2i/FVhnh9?=
 =?us-ascii?Q?U2GFPCY2lT5OsjMHqzdtMY16+IFt5dk3tBDIrjrYqaYh8U2rEyGs9sRvUU6q?=
 =?us-ascii?Q?NKojBNk+JEoMoYAk+ibIiB8CmOYw+3qqqIueKIs4gJbP2QLWtUSXHuRgXzw7?=
 =?us-ascii?Q?aPQsZ00NBKU2fMj6ulFEB8hMQTWVxa7CwLClzjFlf1ndCQkohf/2aQiW6DYF?=
 =?us-ascii?Q?Isvtbc2guK8W9N/7kJcwgL50yDi9mNxfKr4/xy7u4eyLfcI0L6+h/Lu/9G5G?=
 =?us-ascii?Q?iVUAbLTa32UmOdPrPzWC8Tfjxvr+0+aRdLTZ8C+wp+Xns+GZY9xn/sQzwcw1?=
 =?us-ascii?Q?5WB04+GELv1Vch0iiwn7m0nm0g/ZJzOVKrHxCTwWPpUl048HhoTkN+fsyaxY?=
 =?us-ascii?Q?6usjGM8ZMCjv7NkBg6k7QWzoafAis2dG8jxR8D/23Mxrr4KS31FyF8IZ9gzD?=
 =?us-ascii?Q?sbLGaC2QBZcFZQbl8KFy07eobhFJ1iw8ZP952VbF8bHOdW84NYAT1lBgc6B+?=
 =?us-ascii?Q?fN+1v6uRR49oSa0MDe1mG4fWweQHhINl7rOaBYHUAXXj1xgtzw1n22x+bQkj?=
 =?us-ascii?Q?ig+aZlzxL471yA1arIxaE03OR8cEm4VyRcCcQwtTGaPDKZrn1DL+hToSrlcv?=
 =?us-ascii?Q?zMPwRo1rdlAujDcT/bDsJ015Ae1q8L06C6ocQwcmZXS+3AlvfCoL9gVCt1c0?=
 =?us-ascii?Q?NGcdSstloLjsQ/CIBrO98KtLKRO7DAJcPj0bkBaru4IUG8h5RfzYwQupB1V8?=
 =?us-ascii?Q?cn7gnwtq3+hYGezIoetcKbAGtsRBuVaMWKFNuGiqy/wtNzK30D6pO3hA46T7?=
 =?us-ascii?Q?eKLUywsevR25LDRgQJ10TG9O5pFkX68mH2muDlBZ/8cdmfxHwXb5yIYtq+Im?=
 =?us-ascii?Q?fS5ZnZRzHZqv0TUlWP/lOMNfsaa0PA+IPCKN0dsq04NwF7xuJE/68fMOi5F/?=
 =?us-ascii?Q?SSeDjlCn0E/1fxJMxOrYCwdsTTcDq3HfwJVfWa1emisEttLTnaHMox99SaeZ?=
 =?us-ascii?Q?DR3l0a2ekc7mhQwLzrHeJWFSR35B4KOTMNSj1h57LL2WbTmw4PI+VKC2/VXy?=
 =?us-ascii?Q?/SiSSAX+zHCnGr7dxJJpVcvSs8LmyGB48lqp/T/0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3647c32e-6021-4ea3-6331-08db71405db3
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2023 03:42:26.8736
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JzeVckbxVXEcZVN6dAu8rXzHa/oLvcUUW1ohgKDK0kwlmUCqdsDZKdnUAGm2/yRomP8YEcSIFcx3py2HVuc/Lw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7409
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Jun 20, 2023 at 10:42:57AM +0800, Chao Gao wrote:
> >
> >diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> >index 0ecf4be2c6af..c1e93678cea4 100644
> >--- a/arch/x86/kvm/vmx/vmx.c
> >+++ b/arch/x86/kvm/vmx/vmx.c
> >@@ -7548,8 +7548,6 @@ static int vmx_vm_init(struct kvm *kvm)
> > 
> > static u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
> > {
> >-	u8 cache;
> >-
> > 	/* We wanted to honor guest CD/MTRR/PAT, but doing so could result in
> > 	 * memory aliases with conflicting memory types and sometimes MCEs.
> > 	 * We have to be careful as to what are honored and when.
> >@@ -7576,11 +7574,10 @@ static u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
> > 
> > 	if (kvm_read_cr0_bits(vcpu, X86_CR0_CD)) {
> > 		if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_CD_NW_CLEARED))
> >-			cache = MTRR_TYPE_WRBACK;
> >+			return MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT;
> 
> Shouldn't KVM honor guest MTRRs as well? i.e., as if CR0.CD isn't set.

I proposed to return guest mtrr type when CR0.CD=1.
https://lore.kernel.org/all/ZHDZpHYQhPtkNnQe@google.com/

Sean rejected it because before guest MTRR is first time enabled, we
have to return WB with the quirk, otherwise the VM bootup will be super
slow.
Or maybe we can return WB when guest MTRR is not enabled and guest MTRR
type when guest MTRR is anabled to woraround it. e.g.

if (kvm_read_cr0_bits(vcpu, X86_CR0_CD)) {
	if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_CD_NW_CLEARED))
	{
		cache = !mtrr_is_enabled(mtrr_state) ? MTRR_TYPE_WRBACK :
		kvm_mtrr_get_guest_memory_type(vcpu, gfn);
		return cache << VMX_EPT_MT_EPTE_SHIFT;
	}
	...
}

But Sean does not like piling too much on top of the quirk and I think it is right
to keep the quirk simple.

"In short, I am steadfastly against any change that piles more arbitrary behavior
functional behavior on top of the quirk, especially when that behavior relies on
heuristics to try and guess what the guest is doing."

> 
> > 		else
> >-			cache = MTRR_TYPE_UNCACHABLE;
> >-
> >-		return (cache << VMX_EPT_MT_EPTE_SHIFT) | VMX_EPT_IPAT_BIT;
> >+			return (MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT) |
> >+				VMX_EPT_IPAT_BIT;
> > 	}
> > 
> > 	return kvm_mtrr_get_guest_memory_type(vcpu, gfn) << VMX_EPT_MT_EPTE_SHIFT;
> >-- 
> >2.17.1
> >