Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <264f1282c315dd66cd34cfb74f71f53fe3c84126.camel@redhat.com>
Subject: Re: [RFC PATCH 05/13] KVM: SVM: Update max number of vCPUs
 supported for x2AVIC mode
From:   Maxim Levitsky <mlevitsk@redhat.com>
To:     Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc:     pbonzini@redhat.com, seanjc@google.com, joro@8bytes.org,
        jon.grimm@amd.com, wei.huang2@amd.com, terry.bowman@amd.com
Date:   Tue, 01 Mar 2022 13:31:32 +0200
In-Reply-To: <c17e954b-0f62-e0ad-77f0-1429dcc94f6d@amd.com>
References: <20220221021922.733373-1-suravee.suthikulpanit@amd.com>
         <20220221021922.733373-6-suravee.suthikulpanit@amd.com>
         <9143d9d24d1b169668062a18a5f49bb8cf8e877b.camel@redhat.com>
         <c17e954b-0f62-e0ad-77f0-1429dcc94f6d@amd.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Tue, 2022-03-01 at 17:47 +0700, Suravee Suthikulpanit wrote:
> Hi Maxim,
> 
> On 2/25/22 12:18 AM, Maxim Levitsky wrote:
> > On Sun, 2022-02-20 at 20:19 -0600, Suravee Suthikulpanit wrote:
> > > xAVIC and x2AVIC modes can support diffferent number of vcpus.
> > > Update existing logics to support each mode accordingly.
> > > 
> > > Also, modify the maximum physical APIC ID for AVIC to 255 to reflect
> > > the actual value supported by the architecture.
> > > 
> > > Signed-off-by: Suravee Suthikulpanit<suravee.suthikulpanit@amd.com>
> > > ---
> > >   arch/x86/include/asm/svm.h | 12 +++++++++---
> > >   arch/x86/kvm/svm/avic.c    |  8 +++++---
> > >   2 files changed, 14 insertions(+), 6 deletions(-)
> > > 
> > > diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
> > > index 7a7a2297165b..681a348a9365 100644
> > > --- a/arch/x86/include/asm/svm.h
> > > +++ b/arch/x86/include/asm/svm.h
> > > @@ -250,10 +250,16 @@ enum avic_ipi_failure_cause {
> > >   
> > >   
> > >   /*
> > > - * 0xff is broadcast, so the max index allowed for physical APIC ID
> > > - * table is 0xfe.  APIC IDs above 0xff are reserved.
> > > + * For AVIC, the max index allowed for physical APIC ID
> > > + * table is 0xff (255).
> > >    */
> > > -#define AVIC_MAX_PHYSICAL_ID_COUNT	0xff
> > > +#define AVIC_MAX_PHYSICAL_ID		0XFFULL
> > > +
> > > +/*
> > > + * For x2AVIC, the max index allowed for physical APIC ID
> > > + * table is 0x1ff (511).
> > > + */
> > > +#define X2AVIC_MAX_PHYSICAL_ID		0x1FFUL
> > Yep, physid page can't hold more entries...
> > 
> > This brings the inventible question of what to do when a VM has more
> > that 512 vCPUs...
> > 
> > With AVIC, since it is xapic, it would be easy - xapic supports up to
> > 254 CPUs.
> 
> Actually, 255 vCPUs.

Sorry for off-by-one mistake - just remembered that 0xFF is reserved,
but then 255 is already 1 less that 256.

> 
> > But with x2apic, there is no such restriction on max 512 CPUs,
> > thus it is legal to create a VM with x2apic and more that 512 CPUs,
> > and x2AVIC won't work well in this case.
> > 
> > I guess AVIC_IPI_FAILURE_INVALID_TARGET, has to be extened to support those
> > cases, even with loss of performance, or we need to inhibit x2AVIC.
> 
> In case of x2APIC-enabled guest w/ vCPU exceeding the max APIC ID (512) limit,
> the ioctl operation for KVM_CREATE_VCPU will fail. For QEMU, this would
> exit with error code. Would this be sufficient?
Yes, this is the best.


Best regards,
	Maxim Levitsky


> 
> Regards,
> Suravee
> 
> 
>