Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [PATCH] KVM: nVMX: Always enable TSC scaling for L2 when it was
 enabled for L1
To:     Vitaly Kuznetsov <vkuznets@redhat.com>
Cc:     Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Maxim Levitsky <mlevitsk@redhat.com>,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>
References: <20220712135009.952805-1-vkuznets@redhat.com>
From:   Dongli Zhang <dongli.zhang@oracle.com>
Message-ID: <ed923f16-86a7-1f87-f192-c935371dc48c@oracle.com>
Date:   Tue, 12 Jul 2022 13:13:28 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.6.1
In-Reply-To: <20220712135009.952805-1-vkuznets@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aFpvQlJHSmxuZGUyRCtScUw2MmFlNGw0Q29tVWFuUWJZYTRsdHpIcWVRdEw5?=
 =?utf-8?B?WmM4aExneFY1V1VsOCtQT3hULzQvbWd2RGVIRWVwUE1mbi9KNGI4NnN2MU0w?=
 =?utf-8?B?alFmS3AwKzF1enRKQWFFWEV1alh3VzVMaG13ZElWckcvaXZRVjVaaUFCZ1Iw?=
 =?utf-8?B?U3BzVkdvUlkvSU9qakJ3ZHJpWTliNHZpRTdGQkVWekR0WW9kcHBGNkF1LzhC?=
 =?utf-8?B?Y2EwQ0JOWDBhMTlnT2NDZW5uMGxGb2hBNWVIUUpnck9tVVlzSkxJeVFYVkRt?=
 =?utf-8?B?OU94WUZOQmdOb20yenkyc3Z4WC9rRStlcjRJU3JpZ1VtbzFwRGpyMGkwa0t4?=
 =?utf-8?B?cjdvL3p0ZWZaY1JsQ0FPakx3ajM2YUx6WTVrQXNsMFpnc2ttYU5pUCtaZHZ1?=
 =?utf-8?B?VlJLSFF5YWtVMXhzdW14cXIvTFkxdFpQUzJpc1cxZW5nMVVlNUxJeHZyVjBv?=
 =?utf-8?B?L3dwelh3YnVhRm5aTDRsU0JtTXpWTFVkaWdiUk1zM212VkNQUkhLM2JWSGgw?=
 =?utf-8?B?RFlCUFA0clpneWRpMmRSZHhRaTB6NGlnWXJldkh3eUNlMXljcjJnb01RZW5j?=
 =?utf-8?B?T2luczRYeldpM05KU1dNNmFmdk1yWkF6SWJCVGREWWdFZmxiNlR4di9DaVhF?=
 =?utf-8?B?SUhIMFlkb3NlTXB1MzZCZHpEMHg5U0NhWVV2OVp3YW51YnVlQmwrVjd5ZGNw?=
 =?utf-8?B?aUY2Qm5NdDZuS0Y1ZEhDc3V3bWFYZkdXdS9weFJiK0ZVKzJ4NjVBZ0RUQ1Zs?=
 =?utf-8?B?aEtIQ0NiQkxJNVd3QW15VEtGOTl6OFUyMEROOVdzbnZVTG9kTk5hNkxHWkt2?=
 =?utf-8?B?SXBuRmZNNXZvdVI0Y3ZXMXhySFk0WXFxNFJYMk9tRzNSVWphaW1MNnJMeENT?=
 =?utf-8?B?Vk9ZaDZjcHFDSVRKa1R0QmtHeUkxMzNaQ0g1UVB4RUJ4MUlqa3NSZVJRREVs?=
 =?utf-8?B?RzkvbnB1V1RaTEVTc3ZheVRPRXg1aGFUVHh4NmJ0eXBaTjBuNStURTVuNzBY?=
 =?utf-8?B?d1gwOFREeUZtNktCaU90Q0ppQ1ZsbXFNRzdKQU1IbkVlOFhLWGtqM0wzaEhZ?=
 =?utf-8?B?VjVEYnNqWkJnSEFGaXhLT0hrdFZ1TDZiWFBiM0g4Vy9YWERacXdreXJTQUR5?=
 =?utf-8?B?WmZBQUc2VWw2b0lDb05GZnBvVEk4NGFreWdDMFBoRm54K0U1V0N5dzlOcXlR?=
 =?utf-8?B?QXJycE1LbC8vWUJEOTA0MTVuMFlZRGJYYlFlajByL2VTTGl0OXlJdnNwdlRX?=
 =?utf-8?B?SUpUTmdiSXo2c2dmanQ1TldyeHFpNGxIRC9pZFdScTdoajJOVHp2VWdwb1U2?=
 =?utf-8?B?Umx4cWxiaTVoT0lJaEdoNlUwTVBUMjBUTVBPV25SZnlNMHJNaTloREFnZHFU?=
 =?utf-8?B?TDBhbTZhQTBHazBVUENBQXRySHZrS0l1Y0d4akNhVVpCdHVsUzRaRCtWYnRl?=
 =?utf-8?B?d3BBcmp3Wi9qTFBuZmRhcGo3enlhaVVwcHJrR0JGWlNPM3NwMmovdFhwQU44?=
 =?utf-8?B?VTNkVlRwTmtmR3lzUXlkTkx4bmRsTXhIY0RxcjlTVWIyN3ZhMlNOdUF5RHFq?=
 =?utf-8?B?WFNyQU9ZZUV0L3hBZ1dKa1Y4VU5ja2dPcUJ4Y2FGcE9Md0hrRm1SN1Jjek41?=
 =?utf-8?B?QTFLWERReThEVmpRNFFMU2lMTW5vN05ZeUNCVHVreFpoL0pLb0Y1RDVpN3RX?=
 =?utf-8?B?bmhrbWRpcGJMcG4wTU1WS0hja0F5UWpJVlpFbzlrUmZoU0JEbDNrWHpOUFQ3?=
 =?utf-8?B?Q01pS3prVkQwdEhhNTY4Und1UEJJMFZzZkNVNE81a083L3puVlNOSkhBSnAw?=
 =?utf-8?B?OEdyMkFPRGMxTFlsQ0V0cEdRcmcxZUtPZlNCSWgybjVNSzdRUUo4WU5qOWNK?=
 =?utf-8?B?ZXhlaDZmTGpIdk1LczlHeVcrNHpvLzQvVktBbUhrZWFLcnVlRUw1TDBGeGRn?=
 =?utf-8?B?OWR5VU1kNnZncmpFVXFYZlF5eUU3N2h2VjVaOUNBaHFXOFcxTFZaMG9peGZq?=
 =?utf-8?B?MlFwemYzenNOVTVVVEFHNG0wbnY1NGJkaGh5Y3RPbmJIZk55Zm56YlJROWRr?=
 =?utf-8?B?eW9DWGQ1aTVGMFJQdHhLMDVyN2xnRnhvSDV3WmNYaGQzRTU3SXNCWnZETFdq?=
 =?utf-8?B?Z0NGZXhBVDU3d25kclNlcDRveTVLcmN2MmNhMWR4TmRKdGNidUlnZ2VwVU9i?=
 =?utf-8?Q?4JsaEZDahNHRe4Lqp3IozuI=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1bf60b7b-46e6-44cc-96c5-08da6442c814
X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2663.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jul 2022 20:12:01.5633
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tUEiGMfZnrAfRrVWid8iF/Asu0t5G4mHfOCu/WwS6xuCVjsE3SRzyjGNt6GFZS/GjXPnncutBaiPKhKYegLnIA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR10MB3976
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883
 definitions=2022-07-12_12:2022-07-12,2022-07-12 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999
 malwarescore=0 suspectscore=0 mlxscore=0 spamscore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2206140000 definitions=main-2207120082
X-Proofpoint-ORIG-GUID: 7xX1HdofFuRoLK_WQp6vkInAgIxBvzZ5
X-Proofpoint-GUID: 7xX1HdofFuRoLK_WQp6vkInAgIxBvzZ5
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Vitaly,

On 7/12/22 6:50 AM, Vitaly Kuznetsov wrote:
> Windows 10/11 guests with Hyper-V role (WSL2) enabled are observed to
> hang upon boot or shortly after when a non-default TSC frequency was
> set for L1. The issue is observed on a host where TSC scaling is

Would you mind helping clarify if it is L1 or L2 that hangs?

The commit message "Windows 10/11 guests with Hyper-V role (WSL2)" confuses me
if it is L1 or L2 (perhaps due to my lack of knowledge on hyper-v) that hangs.

Thank you very much!

Dongli Zhang

> supported. The problem appears to be that Windows doesn't use TSC
> frequency for its guests even when the feature is advertised and KVM
> filters SECONDARY_EXEC_TSC_SCALING out when creating L2 controls from
> L1's. This leads to L2 running with the default frequency (matching
> host's) while L1 is running with an altered one.
> 
> Keep SECONDARY_EXEC_TSC_SCALING in secondary exec controls for L2 when
> it was set for L1. TSC_MULTIPLIER is already correctly computed and
> written by prepare_vmcs02().
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  arch/x86/kvm/vmx/nested.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
> index 778f82015f03..bfa366938c49 100644
> --- a/arch/x86/kvm/vmx/nested.c
> +++ b/arch/x86/kvm/vmx/nested.c
> @@ -2284,7 +2284,6 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
>  				  SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
>  				  SECONDARY_EXEC_APIC_REGISTER_VIRT |
>  				  SECONDARY_EXEC_ENABLE_VMFUNC |
> -				  SECONDARY_EXEC_TSC_SCALING |
>  				  SECONDARY_EXEC_DESC);
>  
>  		if (nested_cpu_has(vmcs12,
>