Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <01460b72-1189-fef1-9718-816f2f658d42@amd.com>
Date:   Tue, 26 Apr 2022 09:25:03 +0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.8.1
Subject: Re: [PATCH v2 11/12] KVM: SVM: Do not inhibit APICv when x2APIC is
 present
Content-Language: en-US
To:     Maxim Levitsky <mlevitsk@redhat.com>, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Cc:     pbonzini@redhat.com, seanjc@google.com, joro@8bytes.org,
        jon.grimm@amd.com, wei.huang2@amd.com, terry.bowman@amd.com
References: <20220412115822.14351-1-suravee.suthikulpanit@amd.com>
 <20220412115822.14351-12-suravee.suthikulpanit@amd.com>
 <3fd0aabb6288a5703760da854fd6b09a485a2d69.camel@redhat.com>
From:   Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
In-Reply-To: <3fd0aabb6288a5703760da854fd6b09a485a2d69.camel@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RmcrUHVETFFmUVZ4SGd2a2Zpd2h1SWpDZDJ6cEZ6RXZvRU93UEw0d0R5M0Z2?=
 =?utf-8?B?K0RqTU5HWnZhRi92TDhaZklPYlpBT2RXb1M1Y2Fjd1A2WFM5ZVVrMTNaMVM0?=
 =?utf-8?B?elphUVZzKy8yRjhRT0RHZSt3VXFaVXNhajNpY3VkQU9JaTcyaTFVdG5MZWF2?=
 =?utf-8?B?UDVJbVo2b3V6M2NENSthakVBZXgzdGlaMG1RaTF2UlRXUm1WQ3hqcG84Nmlw?=
 =?utf-8?B?T0ZUUWNWVDZyV3VKTGVXN3IwbWZQTTV2Tk1xbC9ONStvK2x2VW1hSVlUc0Rl?=
 =?utf-8?B?ZjZBeVdSd3JDQUFMYjJqblQ5WE50dGJsR3NLc2RhK3diaGlMMkNWTU1waTNM?=
 =?utf-8?B?dWk0WFRXOVdocVhlTGRDZ1RlbGN3K0lIdzBEWGl2UzlyZ3cwMVR0T1ZNYkJl?=
 =?utf-8?B?aGJ4TmFBQnBLbFd1dGIxRFZRUmNJZkN0VTZqdGJYc2ZKK1pLUFFkRlJhR3Qv?=
 =?utf-8?B?MHlYa25VT3lwU29uZUwzQ2NCMXo0UjZOa1BiakprdllOdkxhNzl2Ym92cURy?=
 =?utf-8?B?ZWZac21SMTJ6UVZ2RHh0TWx5NGJ2WEREQ3RnT3NKdWZmbU1lT2NhYWNydlc3?=
 =?utf-8?B?NlVLNm9WNXpRM0V2VkVXOEZ3L2Iyd3RjY3F1dnVzUHk4WjBWeEhxejF2VmVQ?=
 =?utf-8?B?Sm1ZR29DZXcvVXZIemNmZXVYaXptOHlFWEpKNkJ1Rjc2d1FyQlV2b0ZEeXZj?=
 =?utf-8?B?WmNDam95aHp5YkNLYjAxWm8vRjJsZHdnZ3piVnJRVjZrdGlEd3dNYjNFTXdB?=
 =?utf-8?B?ZGVRMkJDZHJoUkZVYlBibE05S3pHZmZjREFncG81VXk2WjVMN21ZL0l0SVNB?=
 =?utf-8?B?Nk9mK0xKbXgrZzhPakZHOFplQW9HaGpvYUFNTWlkREgvMHpreFU2eklCejNL?=
 =?utf-8?B?QUNkQ3VxRnpobjdKdnFRRFBYVVIwOE4xL3pQeFFjVkgzQWdhQVBpN3RiYzF0?=
 =?utf-8?B?UXd6cFdZamxKZWswMXo2K2dRU3R6TGljcExvSFdJYURqSVBtLzZ3clRKUFZ3?=
 =?utf-8?B?aWk3dTduR3FubWlSalpuV1YwOGNDYkEveFZuTkhTWXlzdnRwQ2ZYWklaTElT?=
 =?utf-8?B?VVBIekZrb2kvY1ZGd3hLVStra0pVdUV1RkJaakZpM2NlNlphQlZjMFJISjJr?=
 =?utf-8?B?R212TUNtV0RqYlYvdGZsVHc3MDNHc1k4T1B6RTdGRWNMeVBSckpyU1FCMVV6?=
 =?utf-8?B?Q2dVQzlPb3A4c3VjVG45ajEvRjVtdFM3bnFDdldjS0FybThlOEl6SHFaZ2VY?=
 =?utf-8?B?UTZtOWZLbThpNGtrUnFiY2dJM256MldWamIzSUppUGltc3BaZ3IyS1luOUlB?=
 =?utf-8?B?NForY1daM3VoVmRUem41bWNGd1pSckdLcDZncFJKek1Vd1BhTDNxZkluZ3lp?=
 =?utf-8?B?dHlrUkMrTFdZdUo4ZCsrb0NoUnZ4R2REQnk1S1NhRWF0OGN3SWN6Tm9PYlhm?=
 =?utf-8?B?U1VNakptamN0emF1OW9UdDJBYWxaaTdKQU4rUVJucmNTOWJIc1kzaFc1Z3Zp?=
 =?utf-8?B?ZlhYTkx6RUN0SzRxSHN0dU9qZGdESFNZdjdyTXRLTDZrTXVieEd6MXlpYmdq?=
 =?utf-8?B?L0ZKdFlNVnk3ZkkycmF1ZjBELy95MVJOM1FPL1pVUGYwWmZ2bzJCaEV5eVpp?=
 =?utf-8?B?UWF6RXdVVHRrUkZoQ3owSmlUNy9ySDhGaFFRbmF1RkdZcmtybVdmdFBtZnB4?=
 =?utf-8?B?RWhOL0RVUS9aa0NEUC82TzVRWnN0MjhXVkcyREtkN1g3Qi9uVHByYTd6LzVw?=
 =?utf-8?B?aUtpVUxGYmllMVA2bTgyWUZqazNvbHQ1Q3V3cHZ2NjRZTDZMRTRUZWQvcXJG?=
 =?utf-8?B?MWJtWmVuSGV1clRRdzJWRENDSWZ0by9TaXpxazdjMGd5S0JFc0VvQk1XdW5o?=
 =?utf-8?B?QTNUbnRQMi9XTXRKN3IxRXFOVGNYT3ZHdEtkaEZ4VE9KYjIzNTNBWFU1djhK?=
 =?utf-8?B?NnhxdGJEa3BucWJzQjRVbHJZMU44MjFNOW0wTm82QTRZbFkvanpFK0tLR2Qv?=
 =?utf-8?B?bG0rZ3BJMDNReDNFVkpKNXRXZXlpUFA1VU9uT3FCajF1RHlLTWZyUDh6eVZV?=
 =?utf-8?B?SEhIa1pveExWS3cvVlA4WlJBeEszdkMwbmNhNlhOY1ZqeERlRXF0ZXpzYmNo?=
 =?utf-8?B?aVBPRktJWldRdFhpS1NIK3RPVUlJUUtlTmZaVWVCZCt0YTBHK1N5cUgzZHc3?=
 =?utf-8?B?WUVkRStEWTFMdkdpaVdXQ3RDZFRtQit6SHhnNVM2Rlp5OVJlOTNjajdFZVRi?=
 =?utf-8?B?dVpRdG5nUHBLaEdRSnhLSFcrb2Z0UEo2NlZ0anA2ZzlBaU5FVE9IWWxLZHE2?=
 =?utf-8?B?Q2NQVXIrYk1BS3h0ZE90aFNTVDIxbkhOU0hOTkc2OGxDcjMrd0d4Z3RBQ215?=
 =?utf-8?Q?fZhVRsjQDcXejHaXw9DJ5jVANE6NRPOmnT9kXk6pBDUWi?=
X-MS-Exchange-AntiSpam-MessageData-1: dpxBoE5Kmh6YOA==
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1fbcc07a-0ffa-4729-dae3-08da272bfd4a
X-MS-Exchange-CrossTenant-AuthSource: DM8PR12MB5445.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2022 02:25:11.6714
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qhlSmnYG/fdkSNFH9o2ozPpqrzKgKKLGpqnroTrAyGUSPczlSggmyazftWilkFfdDSvKu9xpI7vKfiglOG0C4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4339
X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Maim,

On 4/19/22 8:29 PM, Maxim Levitsky wrote:
> On Tue, 2022-04-12 at 06:58 -0500, Suravee Suthikulpanit wrote:
> 
> Hi!
> 
> 
> I just got an idea, while writing a kvm selftest that would use AVIC,
> and finding out that selftest code uploads the '-host' cpuid right away
> which has x2apic enabled and that inhibits AVIC, and later clearing x2apic
> in the cpuid doesn't un-inhibit it.
>   
> That can be fixed in few ways but that got me thinking:
>   
> Why do we inhibit AVIC when the guest uses x2apic, even without X2AVIC?
> I think that if we didn't it would just work, and even work faster than
> pure software x2apic.
>   
> My thinking is:
>   
> - when a vcpu itself uses its x2apic, even if its avic is not inhibited,
> the guest will write x2apic msrs which kvm intercepts and will correctly emulate a proper x2apic.
>   
> - vcpu peers will also use x2apic msrs and again it will work correctly
> (even when there are more than 256 vcpus).
>   
> - and the host + iommu will still be able to use AVIC's doorbell to send interrupts to the guest
> and that doesn't need apic ids or anything, it should work just fine.
> 
> Also AVIC should have no issues scanning IRR and injecting interrupts on VM entry,
> x2apic mode doesn't matter for that.
>   
> AVIC mmio can still be though discovered by the guest which is technically against x86 spec
> (in x2apic mode, mmio supposed to not work) but that can be fixed easily by disabing
> the AVIC memslot if any of the vCPUs are in x2apic mode, or this can be ignored since
> it should not cause any issues.
> We seem to have a quirk for that KVM_X86_QUIRK_LAPIC_MMIO_HOLE.
>   
> On top of all this, removing this inhibit will also allow to test AVIC with guest
> which does have x2apic in the CPUID but doesn't use it (e.g kvm unit test, or
> linux booted with nox2apic, which is also nice IMHO)
>   
> What do you think?

This is actually a good idea!!! Let's call it hybrid-x2AVIC :)

I am working on prototype and test out the support for this, which will be introduced in V3.

Regards,
Suravee