Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 26 Jun 2023 08:08:20 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     Xiaoyao Li <xiaoyao.li@intel.com>
CC:     Chao Gao <chao.gao@intel.com>, <kvm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <pbonzini@redhat.com>,
        <seanjc@google.com>, <kai.huang@intel.com>,
        <robert.hoo.linux@gmail.com>
Subject: Re: [PATCH v3 07/11] KVM: VMX: drop IPAT in memtype when CD=1 for
 KVM_X86_QUIRK_CD_NW_CLEARED
Message-ID: <ZJjW9Kstbfa+LmAV@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20230616023101.7019-1-yan.y.zhao@intel.com>
 <20230616023815.7439-1-yan.y.zhao@intel.com>
 <ZJESMaG5Thb5LWtt@chao-email>
 <ZJEQNTvfwOSsSzrf@yzhao56-desk.sh.intel.com>
 <45be3429-f352-920a-d1af-7f3a07930f5d@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <45be3429-f352-920a-d1af-7f3a07930f5d@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?72afpJis1qv9VB/eVAwFed0DY4MG+suLT1L1zeU/0gQOCN6RhBOVceHNUI9j?=
 =?us-ascii?Q?bt+PaN8AuQ2cQY0j1Y+3yod6u2bbLQEY+ErYLGZVzbv6MIRMzRdG8JgkrEJa?=
 =?us-ascii?Q?pofKAEQvH/aFnddCTZnwZxEh432fME5kh1uLLuL4smgmijq+e/3awZ8O8XU+?=
 =?us-ascii?Q?BnQbhY0zqgZK1SxNwypqMw3rvRMFAFvM98JAR+rvbNS0PmO3ST3Fm6APOjKT?=
 =?us-ascii?Q?vo5g0c0A3inky4TeZJI1FCvl5Wc2POYLdpFONZobGGOocab9QsJwciYqWZiQ?=
 =?us-ascii?Q?x+F/WRTNC9wlf0ZMjY1/2rc0iLfJGs91gNue5W8KETWP9a41kmv77jquiwpx?=
 =?us-ascii?Q?dwB4MY+Walw1SVH5fgjJ6rLtDPLFok0LkFadE5GMGQZDlUOg25P6GhfeJ9lp?=
 =?us-ascii?Q?c1eJj6BwKPgzEldKdx37RPlfuqK0N5FHX4R3LFvFdLAF235XYQlI3GlctBvC?=
 =?us-ascii?Q?YbMQVyg3eIXzmLH3nQV53UA2nQoxta6cyAGt8zcAJVGklwPR1XzPSL7CfU61?=
 =?us-ascii?Q?YMsZM/S6jZ2grKXLoOhKqBsj/vt5+wu9+iUTHiqoN8IJNIosGDkZfdaWyPRZ?=
 =?us-ascii?Q?lcZwIs4oY9xBai9kyFrW6iuDWdLt/eNkHDFDCstrLhVzRRBSi/evnAGBI28J?=
 =?us-ascii?Q?NN4RiSlI1XJJdD2AzJ5NH0UxECpSH67hq/qViToVXhFyrP5D6qSo4FDdgJ7L?=
 =?us-ascii?Q?4EnTf7PCkQf1WvCr3zRgdf7P/wUkDdg3t71HvYcrkXKlY9L78MMjSl3qofiO?=
 =?us-ascii?Q?82dzbHl+bnGWp65tRS++w2LFAHwYhAlsmNViWMq7AzwYbs6RjNCfe2OjW+4f?=
 =?us-ascii?Q?QoUf0eABEeeB/kJX7QdXx4sK/jKwcr8qnXC4EqtQ6Upw3ktHDk2xki5mfw0Q?=
 =?us-ascii?Q?r8ZJY+Qu/JufvizHagyxzPCzPdhnsH8kRmb0hsBHi3qRmFlr/LsECO8vi06l?=
 =?us-ascii?Q?3M/fT/f3CuCybZcHserrCyuZ7xlhpuPKYs3yamB4zie2htmU1XbvxWUnvUGV?=
 =?us-ascii?Q?xDvB3M2HL/FbdUc26sm9XdJPAZ41Jd/+ucFGwXAykR7+fmVJRc+ciAANENtr?=
 =?us-ascii?Q?oUqmhoXx4Vvl3jNSrkxlCezwwtfl3AGqQfBYsMaisrM47rOhvBrg89Y100fZ?=
 =?us-ascii?Q?VmU/eMLbgj8adF/VLCqdHqKrgogRtg0X7BDsTN5iMjSPDk9rXUPIdgY66Z+b?=
 =?us-ascii?Q?pRtnpwJ8uaXjIr3w/EvDIf1TeHN+5+cuO4AFWBisJpIlQ13Jkqpsi6x8EeXX?=
 =?us-ascii?Q?S6rh1hv9B3hoPYxg25aWiGUDh4pZgOXBZOXGqpDyo5NO3taALrqhJsZJ8DOJ?=
 =?us-ascii?Q?3jCgqcwK7vlSHH9wj4ewZBBUm/BHyG8io74459jiZ4blSvEwyL9XOpv+hNBr?=
 =?us-ascii?Q?7U6k4NGiF8B7VoX8NN0M84W9NHlavE4n7w4OySq+nh588KnhnJBjhvBMhlj0?=
 =?us-ascii?Q?WPnC4orDZlKTP87dSSPqdc4U0EpZKTS2vB5sIXvYH2a8tnAQbXAKT5HQnxgg?=
 =?us-ascii?Q?gTYDjRoZR1GpAlxdd0MD9N2BNWaxAiIzky3hDU+W5Aeu7fEOtBo6nEYjvuR8?=
 =?us-ascii?Q?lpWRuFDHcDYYW/QgF2MPjuXb5vtOMDUe2ujHNMJ5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 83fd7e15-3291-41db-86ef-08db75dcfede
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2023 00:33:43.4298
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EjvyqF0PEb2i/zBNzJgCksGavPuwiC76axSQ49grrczrvJbI0DZDIANPUx+cBkzte7RggfKM1b+EQr3NodCdKQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4693
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, Jun 25, 2023 at 03:14:37PM +0800, Xiaoyao Li wrote:
> On 6/20/2023 10:34 AM, Yan Zhao wrote:
> > On Tue, Jun 20, 2023 at 10:42:57AM +0800, Chao Gao wrote:
> > > On Fri, Jun 16, 2023 at 10:38:15AM +0800, Yan Zhao wrote:
> > > > For KVM_X86_QUIRK_CD_NW_CLEARED, remove the ignore PAT bit in EPT memory
> > > > types when cache is disabled and non-coherent DMA are present.
> > > > 
> > > > With the quirk KVM_X86_QUIRK_CD_NW_CLEARED, WB + IPAT are returned as the
> > > > EPT memory type when guest cache is disabled before this patch.
> > > > Removing the IPAT bit in this patch will allow effective memory type to
> > > > honor PAT values as well, which will make the effective memory type
> > > Given guest sets CR0.CD, what's the point of honoring (guest) PAT? e.g.,
> > > which guests can benefit from this change?
> > This patch is actually a preparation for later patch 10 to implement
> > fine-grained zap.
> > If when CR0.CD=1 the EPT type is WB + IPAT, and
> > when CR0.CD=0 + mtrr enabled, EPT type is WB or UC or ..., which are
> > without IPAT, then we have to always zap all EPT entries.
> > 
> > Given removing the IPAT bit when CR0.CD=1 only makes the quirk
> > KVM_X86_QUIRK_CD_NW_CLEARED more strict (meaning it could be WC/UC... if
> > the guest PAT overwrites it), it's still acceptable.
> 
> Per my understanding, the reason why KVM had KVM_X86_QUIRK_CD_NW_CLEARED is
> to ensure the memory type is WB to achieve better boot performance for old
> OVMF.
It works well for OVMF c9e5618f84b0cb54a9ac2d7604f7b7e7859b45a7,
which is Apr 14 2015.


> you need to justify the original purpose is not broken by this patch.

Hmm, to dig into the history, the reason for this quirk is explained below:

commit fb279950ba02e3210a16b11ecfa8871f3ee0ca49
Author: Xiao Guangrong <guangrong.xiao@intel.com>
Date:   Thu Jul 16 03:25:56 2015 +0800

    KVM: vmx: obey KVM_QUIRK_CD_NW_CLEARED

    OVMF depends on WB to boot fast, because it only clears caches after
    it has set up MTRRs---which is too late.

    Let's do writeback if CR0.CD is set to make it happy, similar to what
    SVM is already doing.


which means WB is only a must for fast boot before OVMF has set up MTRRs.
At that period, PAT is default to WB.

After OVMF setting up MTRR, according to the definition of no-fill cache
mode, "Strict memory ordering is not enforced unless the MTRRs are
disabled and/or all memory is referenced as uncached", it's valid to
honor PAT in no-fill cache mode.
Besides, if the guest explicitly claim UC via PAT, why should KVM return
WB?
In other words, if it's still slow caused by a UC value in guest PAT,
it's desired to be fixed in guest instead of a workaround in KVM.