Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <40b6f9e6-90c5-ca42-13d7-5e81ff4990c6@amd.com>
Date:   Tue, 19 Jul 2022 10:24:58 +0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.10.0
Subject: Re: [PATCH] KVM: x86: Do not block APIC write for non ICR registers
Content-Language: en-US
To:     Sean Christopherson <seanjc@google.com>
Cc:     linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        pbonzini@redhat.com, mlevitsk@redhat.com, jon.grimm@amd.com,
        Zeng Guang <guang.zeng@intel.com>
References: <20220718083913.222140-1-suravee.suthikulpanit@amd.com>
 <YtWYL6mvN72kaDOi@google.com>
From:   "Suthikulpanit, Suravee" <suravee.suthikulpanit@amd.com>
In-Reply-To: <YtWYL6mvN72kaDOi@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZjZ1NFgyTHFoam5IcDRudUV1WmRLZWtkSHkzbmxnNkZGK05LQWxvRHZLTCs4?=
 =?utf-8?B?ODdNMnVDYmFrTTZMUXZhWW4zZzdveCtqSDM3bWFYa0EvNlFIeXozWmJ0Z2k3?=
 =?utf-8?B?cGZPaTdXVHZkL3U0cDBTQ3JBN3EvY213MVUrRVhESXZVZk50US9zTlhiSkU4?=
 =?utf-8?B?UTFraExLaUR6WDhTL3RhaTNITnJXb3FJU3kzWW1RcW5sV0N0NWhiZ3hLRGtG?=
 =?utf-8?B?NWRaTnpCMVV3R3c4cEJCNS9va3JXb2lBK1FOU3ViVjU5R2FONjRaRVE2SDhM?=
 =?utf-8?B?cnN1TFlEeTM3MmkxTUVqZ0dVMUlzLys2c1NQajMwRXVFTm03K1dvZEkyTi9n?=
 =?utf-8?B?TVFWSnczN0hmVjZoUUJRMWlHV2RnWGwyTnZ0MWFITlUzSHJLNmpwOG8veXV6?=
 =?utf-8?B?RVRRdWRvM2E2MTVpby9hU2cwUWc2Q28xSS9qc01NV2s2bXk5UlBsWkdZcDFt?=
 =?utf-8?B?V3NFcVBVQ0U2eTZVTEpOR3ErN0hZdTVXaStxRC80NUlRVGNKdWM4bzRxQi9F?=
 =?utf-8?B?OStIdjNlb0s5SU9ROGRydzBqN2FGTnhzTE5JVmE1WmJqNEZGWFBtaDJxRUxX?=
 =?utf-8?B?T0tGQU1EbHdOeExyYmlCNFVJYVpvdkpkY2Vva1lwTC9pZGttSEp0OVlyMUtK?=
 =?utf-8?B?c3o1TDR1RHhSd1Fja3VMSWMzeE9XK1ZSNHE1R0lsdENxRStZY2gzcFd4R1lv?=
 =?utf-8?B?NWFnWjA0bVQ0K0haQlJEam13dnhBVmhkdzduOGJhVDIxWWlSdkpNcXRhRldr?=
 =?utf-8?B?dXdoZUk3dVNDU3oxSG1nYTlGOUE2LzZ6NFA5eUdjcGZxSEREeDVtcjZxZG0r?=
 =?utf-8?B?K042Ymcwd3EvRFdhdHpmSFlmV2RQbGJjMmU1cmRVcUlsSHA4RktmQWlTOHdD?=
 =?utf-8?B?WmVSSkE1VFY5bnpzVW5EcGkwb1hTNFBVQ0k1Q2lzdkZOZG12bHF0MlF4UWNB?=
 =?utf-8?B?UkZvWHFFc2FXQUh1cm1qcEZjcW50WkQ3OWRiek1ickRNNDZ4VW9ESFZhVkox?=
 =?utf-8?B?V3pCRVhBTVhNckxjK0JXYVlHTXp5RmFsTjExem5BekY5c2NaZzRpNWNlVEJi?=
 =?utf-8?B?VXJVTWNEQktqM25RcjkzOGJUSVEwZC9PMG10elk0ajJEVHpmdGhzRC8yVmVB?=
 =?utf-8?B?V3pnbmxreEFwYUlrcE15ZFlqU3Bhb2JwMkNCT1dCTEtmeXFiYVhTNGN2cElL?=
 =?utf-8?B?MnBqRmFxdU9lWDAzTmVQNE1PVmJoRHI5U1JYMnZVcitYWk9wK3lyYXJ0ZkRh?=
 =?utf-8?B?cXdwN0MyeVZ0Zm9iL1FhOW9GRklkMWlPbUtlTU5GOHNFMkNCSS8yZ0FZR0hn?=
 =?utf-8?B?bXA1ZjcvVzVLN2tkdjdWWTdmRGwyTCtEMTJxd29hbFRBWDJQN1dnb2RZRVNG?=
 =?utf-8?B?a2w2VDFmS1pmQXdnZ3U3TGVYdW54NU1rcVVsUnUrQXF5c3dCSTI2NlFtTkVt?=
 =?utf-8?B?eVk3cWtSbGpScTVrSDhWemJSV1dJRlk5T2dxaHlGanNzQ1F6bWpFdU53dDVE?=
 =?utf-8?B?TGNlbDg1QmJhQlVqWkVQekc2NjUyS3Z6Skxob0VXZ014MmhmRFFpb01TdW5D?=
 =?utf-8?B?cUJabldTSHJIYS9tUmQ5Q0tyZVBnZlhsdEc1R3BsK2pkQllFU2Y5SG5qRUZi?=
 =?utf-8?B?eStUQ3pCV3p4T3pmbmFXb2NOT0VLcWhkQ2llb3NvTXBuZ3FNT0pVeldaSmRs?=
 =?utf-8?B?Zlo5WnRWZHRoK3U5dmF0ZGVFcm13azI3UFkwdzVQd0QrZkVpZGltMlhnamJQ?=
 =?utf-8?B?d3cyd09UblZIMzdtMitDZmhTTHY5a0NHcGJXRXN0dGhpU0tKM2N4dG5SQlNy?=
 =?utf-8?B?RDhSdnY1Ui9jcXc3WC96c3A0czBwa25FZXQwSUM4cVRiM1lxNlg0allzQity?=
 =?utf-8?B?WjVkWFJRNzAzaHJ2K2gwdmg1RVFBZHFSVVU5aGlqNjNVWFBOdFhVZEJiMVV0?=
 =?utf-8?B?VW5iVjJGeWRING1idU5VRGlKVnpZaVIrdmxVb05lbnRLY3BPanZSZm1VUEdT?=
 =?utf-8?B?MDNwUHc4L1pHc2JlSmU5OVo2U3UwZWs4ZW1ScmRWUklta1B0VS9CVHFqS2Zh?=
 =?utf-8?B?QzBaS21TUEYyZVg2emxrSHZZWjYzQkFCdFVCdFh1M0ZGSTlsOFpBTVUrQi9y?=
 =?utf-8?Q?R5YoRJSGB/esq8Y8QnWfL+gU/?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c66c18a-6ff5-43d3-9c3e-08da69364797
X-MS-Exchange-CrossTenant-AuthSource: DM8PR12MB5445.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2022 03:25:07.9895
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: p4eeQnHxK1wo7e2JOiszGbEVlyTpbTSHaPfWy+B7N6oXInDNzwgKRGrf9wMDsN82klpUkaIAK608iEdDHbb82g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4797
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Sean,

On 7/19/2022 12:28 AM, Sean Christopherson wrote:
> On Mon, Jul 18, 2022, Suravee Suthikulpanit wrote:
>> The commit 5413bcba7ed5 ("KVM: x86: Add support for vICR APIC-write
>> VM-Exits in x2APIC mode") introduces logic to prevent APIC write
>> for offset other than ICR. This breaks x2AVIC support, which requires
>> KVM to trap and emulate x2APIC MSR writes.
>>
>> Therefore, removes the warning and modify to logic to allow MSR write.
>>
>> Fixes: 5413bcba7ed5 ("KVM: x86: Add support for vICR APIC-write VM-Exits in x2APIC mode")
> 
> This tag is wrong, I believe it should be:
> 
>    Fixes: 4d1d7942e36a ("KVM: SVM: Introduce logic to (de)activate x2AVIC mode")
> 
> And that absolutely matters because this should not be backported to older
> kernels that don't support x2avic.

The commit 5413bcba7ed5 is the one that modifies the logic in the kvm_apic_write_nodecode().
I understand your point that the 5413bcba7ed is committed later than 4d1d7942e36a and being
affected by the change. However, if there is a case that only x2AVIC stuff is being backported
w/o the virtualize IPI stuff, then this fix is not needed. Hence, I would say the fix is for
the 5413bcba7ed5 as specified in the original patch.

>> .....
>>   		 */
>> -		if (WARN_ON_ONCE(offset != APIC_ICR))
>> +		if (offset == APIC_ICR) {
>> +			kvm_apic_send_ipi(apic, (u32)val, (u32)(val >> 32));
>> +			trace_kvm_apic_write(APIC_ICR, val);
>>   			return;
>> -
>> -		kvm_lapic_msr_read(apic, offset, &val);
>> -		kvm_apic_send_ipi(apic, (u32)val, (u32)(val >> 32));
>> -		trace_kvm_apic_write(APIC_ICR, val);
>> +		}
>> +		kvm_lapic_msr_write(apic, offset, val);
> 
> Because this lacks the TODO below, what about tweaking this so that there's a
> single call to kvm_lapic_msr_write()?  gcc-11 even generates more efficient code
> for this.  Alternatively, the ICR path can be an early return inside a single
> x2APIC check, but gcc generate identical code and I like making x2APIC+ICR stand
> out as being truly special.

That sounds good.

> Compile tested only.
> 
> ---
> From: Sean Christopherson <seanjc@google.com>
> Date: Mon, 18 Jul 2022 10:16:02 -0700
> Subject: [PATCH] KVM: x86: Handle trap-like x2APIC accesses for any APIC
>   register
> 
> Handle trap-like VM-Exits for all APIC registers when the guest is in
> x2APIC mode and drop the now-stale WARN that KVM encounters trap-like
> exits only for ICR.  On Intel, only writes to ICR can be trap-like when
> APICv and x2APIC are enabled, but AMD's x2AVIC can trap more registers,
> e.g. LDR and DFR.
> 
> Fixes: 4d1d7942e36a ("KVM: SVM: Introduce logic to (de)activate x2AVIC mode")
> Reported-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> Cc: Zeng Guang <guang.zeng@intel.com>
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/x86/kvm/lapic.c | 21 ++++++++++-----------
>   1 file changed, 10 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index 9d4f73c4dc02..95bb1ef37a12 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -2283,21 +2283,20 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)
>   	struct kvm_lapic *apic = vcpu->arch.apic;
>   	u64 val;
> 
> -	if (apic_x2apic_mode(apic)) {
> -		/*
> -		 * When guest APIC is in x2APIC mode and IPI virtualization
> -		 * is enabled, accessing APIC_ICR may cause trap-like VM-exit
> -		 * on Intel hardware. Other offsets are not possible.
> -		 */
> -		if (WARN_ON_ONCE(offset != APIC_ICR))
> -			return;
> -
> +	if (apic_x2apic_mode(apic))
>   		kvm_lapic_msr_read(apic, offset, &val);
> +	else
> +		val = kvm_lapic_get_reg(apic, offset);
> +
> +	/*
> +	 * ICR is a single 64-bit register when x2APIC is enabled.  For legacy
> +	 * xAPIC, ICR writes need to go down the common (slightly slower) path
> +	 * to get the upper half from ICR2.
> +	 */
> +	if (apic_x2apic_mode(apic) && offset == APIC_ICR) {
>   		kvm_apic_send_ipi(apic, (u32)val, (u32)(val >> 32));
>   		trace_kvm_apic_write(APIC_ICR, val);
>   	} else {
> -		val = kvm_lapic_get_reg(apic, offset);
> -
>   		/* TODO: optimize to just emulate side effect w/o one more write */
>   		kvm_lapic_reg_write(apic, offset, (u32)val);
>   	}
> 
> base-commit: 8031d87aa9953ddeb047a5356ebd0b240c30f233
> --

Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>

Suravee