Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <c17e954b-0f62-e0ad-77f0-1429dcc94f6d@amd.com>
Date:   Tue, 1 Mar 2022 17:47:52 +0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.6.1
Subject: Re: [RFC PATCH 05/13] KVM: SVM: Update max number of vCPUs supported
 for x2AVIC mode
Content-Language: en-US
To:     Maxim Levitsky <mlevitsk@redhat.com>, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Cc:     pbonzini@redhat.com, seanjc@google.com, joro@8bytes.org,
        jon.grimm@amd.com, wei.huang2@amd.com, terry.bowman@amd.com
References: <20220221021922.733373-1-suravee.suthikulpanit@amd.com>
 <20220221021922.733373-6-suravee.suthikulpanit@amd.com>
 <9143d9d24d1b169668062a18a5f49bb8cf8e877b.camel@redhat.com>
From:   Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
In-Reply-To: <9143d9d24d1b169668062a18a5f49bb8cf8e877b.camel@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dGE3R1hmVlZoOUZ4U0ZVUzlOYU5jc2pzQk9uVmh5ZForenU1bXNXRDhNc0hu?=
 =?utf-8?B?T00xKzlJZkFxd3FPT0h0RnBDT1dJZFF0UGU2aWtYNUliajVuTTloL3lmYU9Z?=
 =?utf-8?B?V2dSRGxyTWEwQ25CdjZUa0tTbkJtSFB3VTdsNndGcXVaaTdNMFVnYTBVYVhz?=
 =?utf-8?B?MStBYndaWnVBUWl6ZXNvVWtEcGRSK0VvY1dIOFovZWxmMzBUZzJVOVpDTHBh?=
 =?utf-8?B?b1BuK0szMkpDczhCNGRtTW5hcHJiM2ZTR2dPZVdvdXBQRUZudGQrSzNhbTM5?=
 =?utf-8?B?WGV5eGovdkttKy9PZGZDaGcrYXlhcEFQZDQ4ZS9qSmdSSUlyYnFWd1VINEV4?=
 =?utf-8?B?RlhuUnJZWDdZWkd4dTRZcWxDcUlnc09ha2dYZHQwTTh2RmN1NFpOS09MbFk1?=
 =?utf-8?B?L0doRTNZRTRXTGdYT2pFTlU2TlVWc0hDTnF2VnJEUFB1Q1hWRmFFbUxUeGNJ?=
 =?utf-8?B?SjNDeXg3Vmc0QkJKRDNUY3ZFaXFhMXhTR2VpcDFRRERXOXA2TThTcytyYkd4?=
 =?utf-8?B?YlQwNUcvVUdWWnFvOVJRa0kzdmJxbXMvWndRMTZZZjEvYWRmYW53aHdoS2kz?=
 =?utf-8?B?ZnI1Q2hqSW03amljckZ1c0U4TE9ONGk2dFpwU1RnY2Nlc05iYUZuQVoxbmcr?=
 =?utf-8?B?RkhYQ2djS3N5WHN5TTdHNlY1eHdkSXRIaHdqSVVRWjFBc1lSdjNTWUpreVAw?=
 =?utf-8?B?YjhvY3dZR2QxRG12d0haVkkzWm5aYmUxcTFsZlppcGlnUEt6My9wb1djemJX?=
 =?utf-8?B?MUVzNjZsZ2N5TDRWVHJDT21rdUVBWE91WElzUkNLaHVQRVlFak5VRTZuTnE2?=
 =?utf-8?B?cUVMYmJrYmppejN6bXZvS04waU82cU9GcnBCaW1GVlJWVEZwUHdjek42ZEZq?=
 =?utf-8?B?dGpwN1NWWVBIWS9hcWNneTFJeVhZWWoxR3J6ZEZrYmFCalJnUUtpZjBCbDBJ?=
 =?utf-8?B?d1hXZDZvTU5YZDlpVjQwL2wvUGFmUGdoSnJ5WXc5eGc0dGFqTEwveDkrM04x?=
 =?utf-8?B?OXRxS2xiUFNyL2srNEFKYXlvenhrTmV2YVV2dGQ0dDdiUlVka2gzalcwRnI2?=
 =?utf-8?B?aElML0t6Rk5uc0IvRENvbWYzL3JHU2d5ZWJlaklaSVI0QVc1QkNOQW45ZThq?=
 =?utf-8?B?SUhPWVJiUDFuaUFQcDRlMGExRG5JbGVzYkRPWXJjZExzUlkzZXJSSm5OL3g4?=
 =?utf-8?B?R1dGYkNNM3RIQWl5eGd3OWoyZFNDa1k3ZUFtTUN6WlQ3MUtIcWFBQTR2aGla?=
 =?utf-8?B?WVluNGIvT2NUdU9idWVxcUpHdDRxelB2WkM3U0EyMVRBR3R1TXFIS1hrYllR?=
 =?utf-8?B?UWYwc1N5VlJpM1dJWnE0V1JrOEtpb01kKzF1VWV0WFBHZFhDTWVtWW41RkFD?=
 =?utf-8?B?azlsYk83bU5YU3ZZWkpkRHk5QkF0akk1bkFQblArV1p2N292SGFSaUx6Z1Jz?=
 =?utf-8?B?Rmd6dUxZWUtPd3IvWXNocjFaNzFMVDdSNWh4eXNFYXpFaTJBcW9CbkpIZ2Nq?=
 =?utf-8?B?L2ZoT1cvZzB1ZFNzUUpldjFIcUlicVVZa1czMERvNk9jSnZIYndQMEpsUWRq?=
 =?utf-8?B?dC9NSDBpMitBZ25Ja2c3YVQ1cTErOGpSNEVmUFdRWXVWeDdVYmxKbTJPcTFQ?=
 =?utf-8?B?TVZOcC9IRDBpOWdUQTZQZjNLU3hDcnd5TTJ5TGlZcnBqQkllRzNadFBud3la?=
 =?utf-8?B?ODl4Si9ZOXJlY3gzTzY5a3g4NGN3cmQ3SkloRHQreXlvVThsVGJMV1B1Z3lT?=
 =?utf-8?B?RUVZTG5rNXB0eU8xQXUzbmdySWN3VXdCeEJkQUtCdEtCcHVKZWJHNWdyMGQw?=
 =?utf-8?B?MjNUTm5RdmdiVVlzTjBFa3VuQzJWaTFSNjl4bkpqTDlkTDRSSk4rZEZhUWpm?=
 =?utf-8?B?WDZpZkxtU2pGNWxzMjMxcVowMXA5bDVzSmhFQm90N1ZxbTYxOUJEVHA3TXFn?=
 =?utf-8?B?aHA0YlkzUjROUEp0SVNqckNDRFJUNndmN3pHR29GVFBzSTdDZ010WjFpZGp2?=
 =?utf-8?B?V25FYzBvRmx2N2R2cUpMQVFvUkFzQmh5bndxOGlFSndXVTJIUEZrUGdRQkpQ?=
 =?utf-8?B?Sjd1WWtibzZZbHk1S0VrUjhudGcrRVZmOFY3NHU2bjhYTW1ndFRyMjBnNFlx?=
 =?utf-8?B?WEZoaEIveVJvaWhKK3RUOFJOQXBlUlVnbUNNN1kxekZUemcyZXB3Y0RDUmw5?=
 =?utf-8?Q?ZcOBmLytckKH8OFjg/WyPKI=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f891b75-b069-43bd-6f80-08d9fb70f744
X-MS-Exchange-CrossTenant-AuthSource: DM8PR12MB5445.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2022 10:48:05.5657
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0mZF24R/qBQbEyauR0AbFs41SzrMWgliyvvcmKoBBPGgQj5FZ7js40/FogLykrXgTTQfbPZY/ytL2rxkrFY7AA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4994
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Maxim,

On 2/25/22 12:18 AM, Maxim Levitsky wrote:
> On Sun, 2022-02-20 at 20:19 -0600, Suravee Suthikulpanit wrote:
>> xAVIC and x2AVIC modes can support diffferent number of vcpus.
>> Update existing logics to support each mode accordingly.
>>
>> Also, modify the maximum physical APIC ID for AVIC to 255 to reflect
>> the actual value supported by the architecture.
>>
>> Signed-off-by: Suravee Suthikulpanit<suravee.suthikulpanit@amd.com>
>> ---
>>   arch/x86/include/asm/svm.h | 12 +++++++++---
>>   arch/x86/kvm/svm/avic.c    |  8 +++++---
>>   2 files changed, 14 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
>> index 7a7a2297165b..681a348a9365 100644
>> --- a/arch/x86/include/asm/svm.h
>> +++ b/arch/x86/include/asm/svm.h
>> @@ -250,10 +250,16 @@ enum avic_ipi_failure_cause {
>>   
>>   
>>   /*
>> - * 0xff is broadcast, so the max index allowed for physical APIC ID
>> - * table is 0xfe.  APIC IDs above 0xff are reserved.
>> + * For AVIC, the max index allowed for physical APIC ID
>> + * table is 0xff (255).
>>    */
>> -#define AVIC_MAX_PHYSICAL_ID_COUNT	0xff
>> +#define AVIC_MAX_PHYSICAL_ID		0XFFULL
>> +
>> +/*
>> + * For x2AVIC, the max index allowed for physical APIC ID
>> + * table is 0x1ff (511).
>> + */
>> +#define X2AVIC_MAX_PHYSICAL_ID		0x1FFUL
> Yep, physid page can't hold more entries...
> 
> This brings the inventible question of what to do when a VM has more
> that 512 vCPUs...
> 
> With AVIC, since it is xapic, it would be easy - xapic supports up to
> 254 CPUs.

Actually, 255 vCPUs.

> But with x2apic, there is no such restriction on max 512 CPUs,
> thus it is legal to create a VM with x2apic and more that 512 CPUs,
> and x2AVIC won't work well in this case.
> 
> I guess AVIC_IPI_FAILURE_INVALID_TARGET, has to be extened to support those
> cases, even with loss of performance, or we need to inhibit x2AVIC.

In case of x2APIC-enabled guest w/ vCPU exceeding the max APIC ID (512) limit,
the ioctl operation for KVM_CREATE_VCPU will fail. For QEMU, this would
exit with error code. Would this be sufficient?

Regards,
Suravee