Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 9 May 2023 11:11:17 +0800
From:   Chao Gao <chao.gao@intel.com>
To:     Yan Zhao <yan.y.zhao@intel.com>
CC:     <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <pbonzini@redhat.com>, <seanjc@google.com>
Subject: Re: [PATCH] KVM: x86: Zap all TDP leaf entries according noncoherent
 DMA count
Message-ID: <ZFm51cHIqCD4uhUh@chao-email>
References: <20230508034700.7686-1-yan.y.zhao@intel.com>
 <ZFiinKOqGDvyVAkT@chao-email>
 <ZFmkleUwgv/38UrI@yzhao56-desk.sh.intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZFmkleUwgv/38UrI@yzhao56-desk.sh.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BpTpfSK0y7IJl9oh0lnWOtmk7KNtuYMNhY/Ap+L9Pauo+5k+HLK52I+4ufbn?=
 =?us-ascii?Q?iB3uDt/19Zd7zHQyjAm0wCXmVyKxGpg8uKhS8qt9Ema2BTmvC63aRX3abCGu?=
 =?us-ascii?Q?tvUuHm3hbaEIA3tjk34qAB66HRPznjhPRV5qz3ReHGTcgUDrRT6XFPJfZi6K?=
 =?us-ascii?Q?BLAOhhcsxXglMQFTWHqq2T/BMJ8VqCovQzmorghYAJQMfqsTbeFjkx827R1/?=
 =?us-ascii?Q?FWG3AbuyZ41oRgTa+DGSdTPJUpyuLxMbaPFZkkP1IgDmC9Fm9/WZJ2w+UJpl?=
 =?us-ascii?Q?V+mFjnSNRFugzmQ8ZP0n3uYqVPjZ0rPGC3pVoMbf/Lo+Cdf+vJ9AcJ1yO0fw?=
 =?us-ascii?Q?OOlaCvMa/s+hf+KvYkf1d8fjTFucksyraM5rrOlfJaSZtGnd+g0OBNN4+A1i?=
 =?us-ascii?Q?HJMTJ7/kJyZdLNn2zRtl5Kjq4PZp9ym7CMPBRK+gMNnwCZ3C1xuFm92m7REM?=
 =?us-ascii?Q?gnbkhJnzpBieXPlV9Ik6SPMwUwDXVNJN33TEDcD2tn3s0RKCWSK9VE7jBUnw?=
 =?us-ascii?Q?VLjEC0CosVaB0ayR9APNkFOfUb55JQVkfn/i6+b3RXfengmWYBim0+ajS+Dj?=
 =?us-ascii?Q?CIwAmJoHwyfiw5TgzBmHEKd5Gy8fmj8s0xspgt3/wYY8P4cQW22+eXgpe/5X?=
 =?us-ascii?Q?4hy1Ep1cKdV3iqjlrO4qu9+H/lj8jq6erPYqwmCyVRima3vEbPouGNea3vcS?=
 =?us-ascii?Q?XeMb+KxY1ic+AM9CpwYqAp5me9f+gKivTadRSMoZkdyylnI6IGkZetfBf04H?=
 =?us-ascii?Q?DgbL79W5AtoF5SgVRE0592P5dRXDfzJ17zhvBD+BMS2TxchPAMAtG4cJ3j/4?=
 =?us-ascii?Q?ASLbYazcG9RjpmGL8sfRcdvaaEicKTshLdKd9wWJAypv8gxZb4RBqYh0bqDc?=
 =?us-ascii?Q?Ls9tO3gZek0fqKPf/LSmWOc8cmesORUaaHmPK9a/lWKOgZ5lV2h+1F87EgXc?=
 =?us-ascii?Q?SSOmCs2EUfg93S5wRi4r6KdzF0EPkhfRS0Sg5+OiIjpmBV2ZbpoNS3Ruet5x?=
 =?us-ascii?Q?xRldPvaqSyevawGmbe6ub+TOUkz0oPgo8NaDVJvdnNUJrdS+dEA9xgkvWtlq?=
 =?us-ascii?Q?uMK/dZBbhsd+kzZJX8VWb0zb59ySqxdpRXWbSy/HAmuVC/ndkIQVIdbL+NCs?=
 =?us-ascii?Q?AaDc9ZZX0VWP9zI8mtrP3BVpw9v0ZqlQEucb8ciKdRDZ76h6H9Ghr1n9xqeH?=
 =?us-ascii?Q?RxveXEX63CwwwA+kybYb/lKVfGzQAGRr31pXAzNpIGuzklH2FYd9Mg4P9QaG?=
 =?us-ascii?Q?nxiAAmzS9nhx6PhWBtMd4R8g+AGS6zkuCkBfsuiG4xtjdDBUcvjm3GAVvAx7?=
 =?us-ascii?Q?KqIpQplNdDOvHkfKzoh5EZqVyMxV2H3kOvSTJmsrPzgjIkXBt1JRUNppKSEl?=
 =?us-ascii?Q?M1DooK9dCrGYhqJpH+ox7NefEKCdomoLqMlVR7q8145rikg8HhfD06wq76Ns?=
 =?us-ascii?Q?Vqd/rINsvVY5zoqjif0qijTkAj3xTn9FUT5JcGAkc2+/8RDov2IClElzdbTQ?=
 =?us-ascii?Q?6aSwp2m6caqG483QfGCaF5zg3AS0p9gOUiReBWF3tWTN3hp7Fz1cCngRalCo?=
 =?us-ascii?Q?ChWiKY6uN++FTDi3w6/DPUVQQfNDxt5DHemswV52?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 76103fbe-c385-49e5-c479-08db503b13a8
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB6780.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2023 03:11:26.8272
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wjQf6rcI13t0UruO55wV8AvDR9MxMiOvcdVYtQ2yo5P9ifyhdzYhaer0wFiK8TNpo+c/xCi9bLYNaBS/Fy+AGg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR11MB8510
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, May 09, 2023 at 09:40:37AM +0800, Yan Zhao wrote:
>On Mon, May 08, 2023 at 03:19:56PM +0800, Chao Gao wrote:
>> >However, this issue may appear when kvm_mmu_zap_all_fast() is not called
>> >before KVM slot removal, e.g. as for TDX, only leaf entries for the
>> >memslot to be removed is zapped.
>> >
>> >static void kvm_mmu_invalidate_zap_pages_in_memslot(struct kvm *kvm,
>> >                        struct kvm_memory_slot *slot,
>> >                        struct kvm_page_track_notifier_node *node)
>> >{
>> >        if (kvm_gfn_shared_mask(kvm))
>> >                /*
>> >                 * Secure-EPT requires to release PTs from the leaf.  The
>> >                 * optimization to zap root PT first with child PT doesn't
>> >                 * work.
>> >                 */
>> >                kvm_mmu_zap_memslot(kvm, slot);
>> >        else
>> >                kvm_mmu_zap_all_fast(kvm);
>> >}
>> 
>> TDX code isn't merged. So, I think you'd better not use TDX as an argument.
>> 
>Ok. But I just want to explain that kvm_mmu_zap_all_fast() is not
>desired in some cases during slot DELETE. TDX is a good example here.

Ok. Maybe just remove the above code snippet. The code snippet is confusing
because people may think it is what current KVM has, but actually it turns out
to be a modified version in pending TDX patches, which are subject to change.

>
>> >
>> >And even without TDX's case, in some extreme conditions if MMIO regions
>> >are not disabled during device attaching, e.g. if guest does not cause
>> >the MMIO region disabling in QEMU.
>> >Then TDP zap will not be called and wrong EPT memory type might be
>> >retained.
>> >
>> >So, do the TDP zapping of all leaf entries when present/non-present state
>> >of noncoherent DMA devices changes to ensure stale entries cleaned away.
>> >And as this is not a frequent operation, the extra zap should be fine.
>> >
>> >Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
>> >---
>> > arch/x86/kvm/x86.c | 6 ++++--
>> > 1 file changed, 4 insertions(+), 2 deletions(-)
>> >
>> >diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> >index e7f78fe79b32..99a825722d95 100644
>> >--- a/arch/x86/kvm/x86.c
>> >+++ b/arch/x86/kvm/x86.c
>> >@@ -13145,13 +13145,15 @@ EXPORT_SYMBOL_GPL(kvm_arch_has_assigned_device);
>> > 
>> > void kvm_arch_register_noncoherent_dma(struct kvm *kvm)
>> > {
>> >-	atomic_inc(&kvm->arch.noncoherent_dma_count);
>> >+	if (atomic_inc_return(&kvm->arch.noncoherent_dma_count) == 1)
>> 
>> >+		kvm_zap_gfn_range(kvm, gpa_to_gfn(0), gpa_to_gfn(~0ULL));
>> 
>> The issue is specific to EPT. shouldn't this be conditional on tdp_enabled, like
>> update_mtrr()?
>> 
>Yes. good point.
>Maybe also include checking of shadow_memtype_mask.

looks good.