Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 25 Nov 2022 09:04:44 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     "Huang, Kai" <kai.huang@intel.com>
CC:     "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Christopherson,, Sean" <seanjc@google.com>,
        "Shahar, Sagi" <sagis@google.com>,
        "isaku.yamahata@gmail.com" <isaku.yamahata@gmail.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Aktas, Erdem" <erdemaktas@google.com>,
        "sean.j.christopherson@intel.com" <sean.j.christopherson@intel.com>,
        "dmatlack@google.com" <dmatlack@google.com>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>
Subject: Re: [PATCH v10 035/108] KVM: x86/mmu: Track shadow MMIO value on a
 per-VM basis
Message-ID: <Y4AUrLJtk7pK082i@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <cover.1667110240.git.isaku.yamahata@intel.com>
 <f1c9996fa4f4bc71b3feee8407d247aeabb8968e.1667110240.git.isaku.yamahata@intel.com>
 <Y3wvhWyIKNzczFov@yzhao56-desk.sh.intel.com>
 <887a77acc9bf96f7c7bea519ab7ebdd27fb67985.camel@intel.com>
 <Y4AIYGGEU3BdrQgV@yzhao56-desk.sh.intel.com>
 <8e14a7732a2d873846d07c4ec467fb7c48b2307f.camel@intel.com>
 <Y4AOMo00B0vlQfIU@yzhao56-desk.sh.intel.com>
 <1ad20facd81dd346e00ff686ae4a0550123de851.camel@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <1ad20facd81dd346e00ff686ae4a0550123de851.camel@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?oTWC0LOO2PQcYkqFY06QLAvfhyy5qAqmUS25nCqgsSXJHqXGJV5Y67aLuFW7?=
 =?us-ascii?Q?97YvPTT2/7Dq5Nm9jd3mVflQOLsV3MFvXlmr4NaauoaKFKfzJLrajanPlO9h?=
 =?us-ascii?Q?u7w/rb1at4E/Akn6wTSXYVwQt9HykxB0JLVkUC/PO72YGf5pcsdHTyAsID88?=
 =?us-ascii?Q?drkc2/kfrciq7mbWnMluwDPJRPjlaQCJwBB8kt1Hb9LS0XiQHUIpXdtdvvoh?=
 =?us-ascii?Q?vjWcmvFQazCNQPjA6PyaM+pwdKLG9a2hk5kvioKt6kv7Ef9yfXLtr/5PmVhL?=
 =?us-ascii?Q?rSnwKAamaTgrLEGhxaKpxYBvhLVoe8JBwb14emuFwPPRYVZRbFyPPxRp1Odd?=
 =?us-ascii?Q?b2Cr+zgDwxolZS7Y47WhowwOjGYWUu67VWWdSPYfuo5FmTdjgxrVIDzsTIBP?=
 =?us-ascii?Q?HPsIRdkafxlp4wo29225Pu1tVD6JW/uDeejC+Rlccmx6jFIJ1ASDmSJXh1AR?=
 =?us-ascii?Q?fXabjsjSrjIfmDD418hsB3VxVVjVk3DUpxuq5QOsSZ+jDtABkE/C6yoor6f/?=
 =?us-ascii?Q?9jgOlVqTJkRE7kF+qo+5ZsD01Vtcc1RHM/Y1pha0IEc5RYDJvJrysGg4CPNZ?=
 =?us-ascii?Q?Jcu4PA3K0QG1QzcuvAmJaSV69K/a+Sc47LxSsEMKllnFoJ9Ll+F3Counhdtd?=
 =?us-ascii?Q?IZYZwfRrO8WCj+OnFWBgnp77s0VpaTTCs5vIUQmdEzwF48W37GLKAFEpLRDa?=
 =?us-ascii?Q?NjEJblJzooO6Mt+AqnvQ/agqPnYsGSea+Tk2wYZAotAZXXgjGKlkzNZofc0h?=
 =?us-ascii?Q?FNBwYhTqtINQg85LKYbA6sNkvukobH7fPj1LCjnhCEgbv/iW8uv7fyZkP5wS?=
 =?us-ascii?Q?Yee1arMeZ68rOm5bt8P7Yxhxb/OxayWqyIU0jf8e0vwA34QiQ3sDWxXg9FZ6?=
 =?us-ascii?Q?SYUWB6RCOTlxR2do60mUPwsrdd1PAYG5osu4auroj9eRRno7My0MZLB4iYYp?=
 =?us-ascii?Q?+EK3xtaunq9Crsx5mF9/29gnxs8giK6+xHWJU4OmGToprojWIZBJHHFPsUuf?=
 =?us-ascii?Q?BM8GytEo1L+5wJYlbCIpPt6G931DKrlX60MMECkKUUlm1oEXZ7Yo9BV7rJKb?=
 =?us-ascii?Q?t/7LLvaD0pWc61kSzaAzmqwkwjGsKODoFB5MoxPTLhVcomfbIWKj565GWCH0?=
 =?us-ascii?Q?iTqDtbuGB1t1D65UfXXewheH8DlGsM5LySfz7ZjI38KCK2IpIiCMmvFP8Tay?=
 =?us-ascii?Q?P9i+FtMeCLm53qFFC50Ejk2TuyUas7peROV2VQiuyOFEY6JhFf9oESThIMQr?=
 =?us-ascii?Q?TlnKe28WejdKIohB+JdriAXj5zXMUzwYJzhmoHvG5nmeDUYMN2xLugyFioUh?=
 =?us-ascii?Q?jlIXWwz7r1FBQwlrN+PFNYQ/X2yLFghAHaxIaUt8t8Zhq+HxPP9ZA4looKnd?=
 =?us-ascii?Q?/28t+I5wc5JyYz1/NaXxnqCA0qw9K5rur9QYQenCHRqOc26FbQ1km8yDuDru?=
 =?us-ascii?Q?TbA0kXTxQNq5gkxPX6fzT+IJi+9mRZq7VYIJIb9qfeKt7mWOMijZfDnjieyp?=
 =?us-ascii?Q?RFu4yUuTmANZr7GgtcsqNsn1c06EJCgY5r7a/fvptPsWv2yMJJ4zmqPJUreG?=
 =?us-ascii?Q?gVBHXC535CiWKhxx+AcIG/lR/ZB3g1WPgoNDhiAe?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3419cc3a-23e5-401a-24b4-08dace84408e
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2022 01:27:44.1658
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PRgTQq3jy3s8W6iMyUm8F+3puorHMWFV1DfkLnxBijIASaroRsccnQtyXw8y1tkvU7mZcw46/KMdlMIMLrtbYA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6907
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Nov 25, 2022 at 09:07:09AM +0800, Huang, Kai wrote:
> On Fri, 2022-11-25 at 08:37 +0800, Yan Zhao wrote:
> > On Fri, Nov 25, 2022 at 08:45:01AM +0800, Huang, Kai wrote:
> > > On Fri, 2022-11-25 at 08:12 +0800, Yan Zhao wrote:
> > > > On Fri, Nov 25, 2022 at 08:13:48AM +0800, Huang, Kai wrote:
> > > > > On Tue, 2022-11-22 at 10:10 +0800, Yan Zhao wrote:
> > > > > > Also make enable_mmio_caching to be a per-VM value?
> > > > > > As if the shadow_mmio_value is 0, mmio_caching needs to be disabled.
> > > > > 
> > > > > If I recall correctly, Sean said we can disable TDX guests if mmio_caching is
> > > > > disabled (we also will need to change to allow enable_mmio_caching to still be
> > > > > true when mmio_value is 0).
> > > > > 
> > > > > SEV_ES has similar logic:
> > > > > 
> > > > > void __init sev_hardware_setup(void)
> > > > > {
> > > > > 
> > > > > 	...
> > > > > 
> > > > >         /*
> > > > >          * SEV-ES requires MMIO caching as KVM doesn't have access to the guest
> > > > >          * instruction stream, i.e. can't emulate in response to a #NPF and
> > > > >          * instead relies on #NPF(RSVD) being reflected into the guest as #VC
> > > > >          * (the guest can then do a #VMGEXIT to request MMIO emulation).
> > > > >          */
> > > > >         if (!enable_mmio_caching)
> > > > >                 goto out;
> > > > > 
> > > > 
> > > > Would enabling mmio caching in per-VM basis be better?
> > > > 
> > > 
> > > We need Paolo/Sean to decide.
> > > 
> > > The thing is TDX guests always require mmio_caching being enabled.  For VMX
> > > guests, normally we will always enable mmio_caching too.  So I think per-VM
> > > basis mmio_caching is not that useful.
> > With per-VM basis enabling, I think we can get rid of the kvm_gfn_shared_mask(kvm)
> > in below code and also in handle_abnormal_pfn()
> > 
> > static inline bool is_mmio_spte(struct kvm *kvm, u64 spte)
> > {
> >         return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&
> >                likely(enable_mmio_caching || kvm_gfn_shared_mask(kvm));
> > }
> > 
> 
> It needs to go anyway regardless per-VM mmio_caching or not, as explained we
> need to change to allow enable_mmio_caching to be true even mmio_value is 0.

Or it's better to check enable_mmio_caching is true in
kvm_mmu_set_mmio_spte_value() as below. 

void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value)
{
        WARN_ON(!enable_mmio_caching);
        kvm->arch.shadow_mmio_value = mmio_value;
}