Message-ID: <cc671a9d-7633-524b-6d88-d2a980ceecd4@amd.com>
Date:   Thu, 16 Feb 2023 03:25:17 +0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [PATCH v5 00/33] KVM: x86: AVIC and local APIC fixes+cleanups
Content-Language: en-US
To:     Sean Christopherson <seanjc@google.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Alejandro Jimenez <alejandro.j.jimenez@oracle.com>,
        Maxim Levitsky <mlevitsk@redhat.com>,
        Li RongQing <lirongqing@baidu.com>,
        Greg Edwards <gedwards@ddn.com>
References: <20230106011306.85230-1-seanjc@google.com>
From:   "Suthikulpanit, Suravee" <suravee.suthikulpanit@amd.com>
In-Reply-To: <20230106011306.85230-1-seanjc@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZHhCNUlnVDNFbjVtSlpEcGp3RjFkajZTUG9wbStUb29vNHhnTEdQSUlaekhs?=
 =?utf-8?B?RkJjczJHU0dQZ2ZaeXMyRmNTR2hLZ3JqQUpRREtKYTFxTVdGU0VSRXBxRlIv?=
 =?utf-8?B?K3N4UzBObG1hN0I1V21XSHlCMTVqNmVDZ2p4dFlYYldsd2hVMldaRDNCVG5l?=
 =?utf-8?B?aUJQakVGOCt3MjlNZk9Zb0Vmd0RkS1FabGxkMFVoUVFxZXE3ZGZGWVNwRDQ2?=
 =?utf-8?B?UGJiY0x5YXkreTcyMWNqTnlJTnRMelhCcHBmTitxUDZnU0tNUmhhdG04Y2pB?=
 =?utf-8?B?bHlUNWtiejdYdUo3RktEc0dpcVJNV0NhTUtuZ2NqbUdwc3BpRjNFRnpGcHlw?=
 =?utf-8?B?SVNMenZPOEpVaVdHOUlRUFRkRHJMRDcySUJXZ3hxSFVjSjI5WVgxeks0Vy85?=
 =?utf-8?B?SXFqdmEvME1MdGppenR0c3pqWjAyOUdhcWRhR2szRFMzOVR1UXpIRDJOMCtk?=
 =?utf-8?B?MmJuMUV6SzhxOSs4RDcyWk94MWR2NFhIUWxRbzQxbkxxTjZ1SEgxSmlMTld1?=
 =?utf-8?B?RE9qd2tOS2hCTHh2bm1Yb1Vqd2lld1NYditMencvd1VNZEFUUERnQm8wOHpN?=
 =?utf-8?B?S2FQN1ZjTlFWQ0cxbmNuOG9oMm5pckwvSXVQV3RSVFEvYW5IVFZqbERkSjhZ?=
 =?utf-8?B?WmpHSTljbGhuSzZEYUs2Q2VsU0xmeUFNK3J6VXpXcVhmNXk1TFVEWHNOaWtG?=
 =?utf-8?B?b3V4MSs3d0RHTEtoZzIxUWNaY3VnSmxwR1RMTitwU0h6Vy9kRDJzLzN3MWY1?=
 =?utf-8?B?NVNSUGxFVWFkVVI4UDQwQ1krSG1hKy9UbmpOOVJINTJyQzJFWVl6UjVDK0FE?=
 =?utf-8?B?a2xBazc2M2MzOGovUzVNc0lQYlFmS002NzNMZmV4RDVqUWZ4d05meFlqdlZE?=
 =?utf-8?B?bVBkbUlRcVRFbStjVEtQUGZieFlZWDc2RENqSCsrZ3RLVHY3Z0o0YW01QkxH?=
 =?utf-8?B?ZGdNb0hsL3k0SmVhVmU5SENhak10MHRtTW9aUUt1SHp2K1RRcGpTbFNEVHM1?=
 =?utf-8?B?WE5SOERrbXNmZWs2cHBpMEJhdmxtaXFLblI2eGZZOUxZdjJQMUtaOGo0VFdn?=
 =?utf-8?B?ZUltWHlIdG9ES29ZWEpmZFJSU1VybXRkN0p0RWx3dFEwaFA5bGxIWlhUZzZK?=
 =?utf-8?B?MExsT0ovVEd5TXRNczFwTGlabkdwQW5DdXpTUS9KbkVZLytkeTVUaTE2aTg3?=
 =?utf-8?B?bjloWTkwQTdUWFpzaTZwOXJ0d0RtNWZ6ZUFxdUJtK0lnZUcwM0s3U1l3b3FR?=
 =?utf-8?B?ZSt4OGs4SDZwbm81amR5cmE2Ry9XdU1nY0NpWDZqQm9DTVhVa1RGUkVueWcv?=
 =?utf-8?B?WGFVdGRNYVdiTWhQNkczUEtpWHJzNE56VlFIQWVzaUdCTzJlTG9rTWZmZWFY?=
 =?utf-8?B?eVlEbjRDRXNBdDJaekdqYlc0d3p3c3NhZ1d0MDV0NU9COHltaW5IN09rekF3?=
 =?utf-8?B?YUdWYmxhcmFmeXNTUnptT0JvTHZjdnhDVGNKV0VRaVF0ZmQ2ZjUzazNjOCtY?=
 =?utf-8?B?MVdMc0hhZzEyY2UrM1U5VmRXNTJteTF1UlBaSWU3UkxSTHJQeTgyS01qUGdy?=
 =?utf-8?B?ZzhRazAzazc4TjR5U2RGaDh0SXlhK05FN21uYmQ2dDRZVUJLOU9pQk0xT3hG?=
 =?utf-8?B?WndPbzNLZ2ErUDY2OUVpWDEwb3FZYm1lOUVTNTZUbzVTb3ZNdEkzR1dLRExZ?=
 =?utf-8?B?MVFUbERGSUROazFyVWszOW0zWjRoZkswcG85Qmp4NkhKMnNjbmFMZXl5VE9J?=
 =?utf-8?B?QlFNenJIcUNNbithakQ5dXg3MTZlRElXdnVJTEVwbCszdmN4Szd1MUZoOHEr?=
 =?utf-8?B?cUVIV21NMUZKcmdWeEpKL0ZpekM1Yzk0anNia3JDTm1SMk0vejVSc1ExZXdD?=
 =?utf-8?B?bEw3anlWeWlUclMrYmx6RzlGaFA5REFuSjVaaGZJK1c0SE5QdHZOSnVEYWZh?=
 =?utf-8?B?c2d0NldtYWNmUW5PWVk3T0ZScXJFM1dKOUpjNEhxM1BydUZnejBuUWxxYm5O?=
 =?utf-8?B?VUZWOEsyRFllbXpkMk16c0RsTkM0NXFaM2pHTTAwbk4wTzBEOE5aN2E5T0Yv?=
 =?utf-8?B?NHRaYUtxV001WU04aitweUpxSE1DZ2YwUHpEVTF2bGRZV2puNmdkSDJOWmsr?=
 =?utf-8?Q?fSGGgUYQH0786pkOhiqf2ugYw?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3724811e-4f03-4a11-5e66-08db0f92c70b
X-MS-Exchange-CrossTenant-AuthSource: DM8PR12MB5445.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2023 20:25:29.3369
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kUKNMpd76M+yRTKJ6aluntegx7w71/l3h8kD8/Njdk5aBx6oecI57yfOrumcxPVzhNobHq+Up8+XKoOrbwKfIw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7425
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 1/6/2023 8:12 AM, Sean Christopherson wrote:
> Paolo,
> 
> This is a wholesale (and hopefully a drop-in) replacement for the patches
> in `kvm-lapic-fix-and-cleanup`.  The changes for " KVM: x86: Inhibit APIC
> memslot if x2APIC and AVIC are enabled" in v5 are relatively minor, but it
> led to multiple conflicts in later patches, i.e. a patch-to-be-squashed
> wasn't going to work.  Let me know if you've already done a lot of
> massaging on your side, shouldn't be too difficult to generate patches to
> go on top.
> 
> The first half or so patches fix semi-urgent, real-world relevant APICv
> and AVIC bugs.
> 
> The second half fixes a variety of AVIC and optimized APIC map bugs
> where KVM doesn't play nice with various edge cases that are
> architecturally legal(ish), but are unlikely to occur in most real world
> scenarios
> 
> v5:
>    - Fix even more bugs! [Greg]
>    - Collect reviews. [Maxim]
>    - Don't use an inhibit flag for hybrid AVIC. [Maxim]
>    - Fix an LDR cluster calc goof in the AVIC code. [Maxim]
>    - Drop a redundant "ldr == 0" check. [Maxim]
>    - Add helpers for logical vs. physical optimized map calcs. [Maxim]
> 
> v4:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20221001005915.2041642-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BUOZ4e1Ov%2BwPkKTSe81P1tPUtxpEr2uddkhMn73kLhQ%3D&reserved=0
>    - Fix more bugs! [Alejandro]
>    - Delete APIC memslot to inhibit xAVIC acceleration when x2APIC is
>      enabled on AMD/SVM instead of using a "partial" inihbit. [Maxim]
> 
> v3:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220920233134.940511-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=usZ0dheeVyhBJxLP2ahA1hUlN64gwAPooWR9jnQcRyQ%3D&reserved=0
>    - Collect reviews. [Paolo]
>    - Drop "partial" x2APIC inhibit and instead delete the memslot.
>      [Maxim, Suravee]
>    - Skip logical mode updates for x2APIC, which just reuses the
>      phys_map with some clever logic. [Suravee]
>    - Add a fix for "nodecode write" traps. [Alejandro]
> 
> v2:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220903002254.2411750-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=olaieXGtEfg8K6wwNHjHRa0aRit0npXnoxTmnfR%2FhqQ%3D&reserved=0
>    - Collect reviews. [Li, Maxim]
>    - Disable only MMIO access when x2APIC is enabled (instead of disabling
>      all of AVIC). [Maxim]
>    - Inhibit AVIC when logical IDs are aliased. [Maxim]
>    - Tweak name of set_virtual_apic_mode() hook. [Maxim]
>    - Straight up revert logical ID fastpath mess. [Maxim]
>    - Reword changelog about skipping vCPU during logical setup. [Maxim]
>    - Fix LDR updates on AVIC. [Maxim?]
>    - Fix a nasty ISR caching bug.
>    - Flush TLB when activating AVIC.
> 
> v1: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220831003506.4117148-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d9my%2BQJgnq3673JLC1YFgVSNz1zlSDi7x537abArwS0%3D&reserved=0
> 
> Greg Edwards (1):
>    KVM: x86: Allow APICv APIC ID inhibit to be cleared
> 
> Sean Christopherson (31):
>    KVM: x86: Blindly get current x2APIC reg value on "nodecode write"
>      traps
>    KVM: x86: Purge "highest ISR" cache when updating APICv state
>    KVM: SVM: Flush the "current" TLB when activating AVIC
>    KVM: SVM: Process ICR on AVIC IPI delivery failure due to invalid
>      target
>    KVM: x86: Don't inhibit APICv/AVIC on xAPIC ID "change" if APIC is
>      disabled
>    KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to
>      32-bit ID
>    KVM: SVM: Don't put/load AVIC when setting virtual APIC mode
>    KVM: x86: Handle APICv updates for APIC "mode" changes via request
>    KVM: x86: Move APIC access page helper to common x86 code
>    KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled
>    KVM: SVM: Replace "avic_mode" enum with "x2avic_enabled" boolean
>    KVM: SVM: Compute dest based on sender's x2APIC status for AVIC kick
>    Revert "KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when
>      possible"
>    KVM: SVM: Document that vCPU ID == APIC ID in AVIC kick fastpatch
>    KVM: SVM: Add helper to perform final AVIC "kick" of single vCPU
>    KVM: x86: Explicitly skip optimized logical map setup if vCPU's LDR==0
>    KVM: x86: Explicitly track all possibilities for APIC map's logical
>      modes
>    KVM: x86: Skip redundant x2APIC logical mode optimized cluster setup
>    KVM: x86: Disable APIC logical map if logical ID covers multiple MDAs
>    KVM: x86: Disable APIC logical map if vCPUs are aliased in logical
>      mode
>    KVM: x86: Honor architectural behavior for aliased 8-bit APIC IDs
>    KVM: x86: Inhibit APICv/AVIC if the optimized physical map is disabled
>    KVM: SVM: Inhibit AVIC if vCPUs are aliased in logical mode
>    KVM: SVM: Always update local APIC on writes to logical dest register
>    KVM: SVM: Update svm->ldr_reg cache even if LDR is "bad"
>    KVM: SVM: Require logical ID to be power-of-2 for AVIC entry
>    KVM: SVM: Handle multiple logical targets in AVIC kick fastpath
>    KVM: SVM: Ignore writes to Remote Read Data on AVIC write traps
>    Revert "KVM: SVM: Do not throw warning when calling avic_vcpu_load on
>      a running vcpu"
>    KVM: x86: Track required APICv inhibits with variable, not callback
>    KVM: x86: Add helpers to recalc physical vs. logical optimized APIC
>      maps
> 
> Suravee Suthikulpanit (1):
>    KVM: SVM: Fix x2APIC Logical ID calculation for
>      avic_kick_target_vcpus_fast
> 
>   Documentation/virt/kvm/x86/errata.rst |  11 +
>   arch/x86/include/asm/kvm-x86-ops.h    |   1 -
>   arch/x86/include/asm/kvm_host.h       |  52 +++-
>   arch/x86/kvm/lapic.c                  | 319 +++++++++++++++++-----
>   arch/x86/kvm/lapic.h                  |   2 +
>   arch/x86/kvm/svm/avic.c               | 372 ++++++++++++--------------
>   arch/x86/kvm/svm/nested.c             |   2 +-
>   arch/x86/kvm/svm/svm.c                |   8 +-
>   arch/x86/kvm/svm/svm.h                |  27 +-
>   arch/x86/kvm/vmx/vmx.c                |  58 +---
>   arch/x86/kvm/x86.c                    |  29 +-
>   11 files changed, 524 insertions(+), 357 deletions(-)
> 
> 
> base-commit: c04ec04c0d15a51aa33660be175ed978beb8de0c

For AMD AVIC / x2AVIC:

Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Tested-by: Kishon VijayAbraham <kishon.vijayabraham@amd.com>

Thanks,
Suravee