2013-09-14 12:11:48

by Andrew Jones

[permalink] [raw]
Subject: [PATCH 0/3] KVM_MAX_VCPUS related changes

Andrew Jones (3):
arm: kvm: clamp NR_VCPUS to MAX_VCPUS
arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS
aarch64: kvm: introduce CONFIG_KVM_MAX_VCPUS

arch/arm/include/asm/kvm_host.h | 4 ++--
arch/arm/kvm/Kconfig | 8 ++++----
arch/arm/kvm/arm.c | 2 +-
arch/arm64/include/asm/kvm_host.h | 7 ++++++-
arch/arm64/kvm/Kconfig | 11 +++++++++++
5 files changed, 24 insertions(+), 8 deletions(-)

--
1.8.1.4


2013-09-14 12:11:49

by Andrew Jones

[permalink] [raw]
Subject: [PATCH 1/3] arm: kvm: clamp NR_VCPUS to MAX_VCPUS

Signed-off-by: Andrew Jones <[email protected]>
---
arch/arm/kvm/arm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 741f66a2edbd7..9ebf8ac3a12ff 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -201,7 +201,7 @@ int kvm_dev_ioctl_check_extension(long ext)
r = 1;
break;
case KVM_CAP_NR_VCPUS:
- r = num_online_cpus();
+ r = min(num_online_cpus(), KVM_MAX_VCPUS);
break;
case KVM_CAP_MAX_VCPUS:
r = KVM_MAX_VCPUS;
--
1.8.1.4

2013-09-14 12:12:11

by Andrew Jones

[permalink] [raw]
Subject: [PATCH 3/3] aarch64: kvm: introduce CONFIG_KVM_MAX_VCPUS

Take CONFIG_KVM_MAX_VCPUS from arm32, but set the default to 8.

Signed-off-by: Andrew Jones <[email protected]>
---
arch/arm64/include/asm/kvm_host.h | 7 ++++++-
arch/arm64/kvm/Kconfig | 11 +++++++++++
2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 0859a4ddd1e7d..d1af8c49a5ca4 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -26,7 +26,12 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>

-#define KVM_MAX_VCPUS 4
+#if defined(CONFIG_KVM_MAX_VCPUS)
+#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
+#else
+#define KVM_MAX_VCPUS 0
+#endif
+
#define KVM_USER_MEM_SLOTS 32
#define KVM_PRIVATE_MEM_SLOTS 4
#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 21e90820bd23c..c9924b02e84f7 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -35,6 +35,17 @@ config KVM_ARM_HOST
---help---
Provides host support for ARM processors.

+config KVM_MAX_VCPUS
+ int "Number maximum supported virtual CPUs per VM"
+ depends on KVM_ARM_HOST
+ default 8
+ help
+ Static number of max supported virtual CPUs per VM.
+
+ The default is set to the highest number of vcpus that
+ current hardware supports. Set to a lower number to save
+ some resources. Set to a higher number to test scalability.
+
config KVM_ARM_VGIC
bool
depends on KVM_ARM_HOST && OF
--
1.8.1.4

2013-09-14 12:12:49

by Andrew Jones

[permalink] [raw]
Subject: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

Drop the _ARM_ part of the name. We can then introduce a config option
like this to aarch64 and other arches using the same name - allowing
grep to show them all. Also update the help text to describe the option
more completely.

Signed-off-by: Andrew Jones <[email protected]>
---
arch/arm/include/asm/kvm_host.h | 4 ++--
arch/arm/kvm/Kconfig | 8 ++++----
2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 7d22517d80711..c614d3eb176c6 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -25,8 +25,8 @@
#include <asm/fpstate.h>
#include <kvm/arm_arch_timer.h>

-#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
-#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
+#if defined(CONFIG_KVM_MAX_VCPUS)
+#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
#else
#define KVM_MAX_VCPUS 0
#endif
diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index ebf5015508b52..de63bfccb3eb5 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -40,16 +40,16 @@ config KVM_ARM_HOST
---help---
Provides host support for ARM processors.

-config KVM_ARM_MAX_VCPUS
+config KVM_MAX_VCPUS
int "Number maximum supported virtual CPUs per VM"
depends on KVM_ARM_HOST
default 4
help
Static number of max supported virtual CPUs per VM.

- If you choose a high number, the vcpu structures will be quite
- large, so only choose a reasonable number that you expect to
- actually use.
+ The default is set to the highest number of vcpus that
+ current hardware supports. Set to a lower number to save
+ some resources. Set to a higher number to test scalability.

config KVM_ARM_VGIC
bool "KVM support for Virtual GIC"
--
1.8.1.4

2013-09-14 12:14:14

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH 1/3] arm: kvm: clamp NR_VCPUS to MAX_VCPUS



Am 14.09.2013 um 07:10 schrieb Andrew Jones <[email protected]>:

> Signed-off-by: Andrew Jones <[email protected]>
> ---
> arch/arm/kvm/arm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 741f66a2edbd7..9ebf8ac3a12ff 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -201,7 +201,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> r = 1;
> break;
> case KVM_CAP_NR_VCPUS:
> - r = num_online_cpus();
> + r = min(num_online_cpus(), KVM_MAX_VCPUS);

Is there any real reason to prohibit overcommit?

Alex

> break;
> case KVM_CAP_MAX_VCPUS:
> r = KVM_MAX_VCPUS;
> --
> 1.8.1.4
>
> _______________________________________________
> kvmarm mailing list
> [email protected]
> https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm

2013-09-14 12:22:24

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 1/3] arm: kvm: clamp NR_VCPUS to MAX_VCPUS

On 2013-09-14 13:14, Alexander Graf wrote:
> Am 14.09.2013 um 07:10 schrieb Andrew Jones <[email protected]>:
>
>> Signed-off-by: Andrew Jones <[email protected]>
>> ---
>> arch/arm/kvm/arm.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index 741f66a2edbd7..9ebf8ac3a12ff 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -201,7 +201,7 @@ int kvm_dev_ioctl_check_extension(long ext)
>> r = 1;
>> break;
>> case KVM_CAP_NR_VCPUS:
>> - r = num_online_cpus();
>> + r = min(num_online_cpus(), KVM_MAX_VCPUS);
>
> Is there any real reason to prohibit overcommit?

I don't think this affects overcommit. This is the "recommended" limit,
and you can still go up to KVM_MAX_CPUS.

M.
--
Fast, cheap, reliable. Pick two.

2013-09-14 12:23:54

by Andrew Jones

[permalink] [raw]
Subject: Re: [PATCH 1/3] arm: kvm: clamp NR_VCPUS to MAX_VCPUS

On Sat, Sep 14, 2013 at 07:14:02AM -0500, Alexander Graf wrote:
>
>
> Am 14.09.2013 um 07:10 schrieb Andrew Jones <[email protected]>:
>
> > Signed-off-by: Andrew Jones <[email protected]>
> > ---
> > arch/arm/kvm/arm.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> > index 741f66a2edbd7..9ebf8ac3a12ff 100644
> > --- a/arch/arm/kvm/arm.c
> > +++ b/arch/arm/kvm/arm.c
> > @@ -201,7 +201,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> > r = 1;
> > break;
> > case KVM_CAP_NR_VCPUS:
> > - r = num_online_cpus();
> > + r = min(num_online_cpus(), KVM_MAX_VCPUS);
>
> Is there any real reason to prohibit overcommit?

This doesn't prohibit it. Users can attempt to configure anything they'd
like, but only selections KVM_MAX_VCPUS and below will work.

drew

>
> Alex
>
> > break;
> > case KVM_CAP_MAX_VCPUS:
> > r = KVM_MAX_VCPUS;
> > --
> > 1.8.1.4
> >
> > _______________________________________________
> > kvmarm mailing list
> > [email protected]
> > https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm

2013-09-15 10:20:26

by Gleb Natapov

[permalink] [raw]
Subject: Re: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On Sat, Sep 14, 2013 at 02:10:55PM +0200, Andrew Jones wrote:
> Drop the _ARM_ part of the name. We can then introduce a config option
> like this to aarch64 and other arches using the same name - allowing
> grep to show them all. Also update the help text to describe the option
> more completely.
>
> Signed-off-by: Andrew Jones <[email protected]>
> ---
> arch/arm/include/asm/kvm_host.h | 4 ++--
> arch/arm/kvm/Kconfig | 8 ++++----
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 7d22517d80711..c614d3eb176c6 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -25,8 +25,8 @@
> #include <asm/fpstate.h>
> #include <kvm/arm_arch_timer.h>
>
> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
> +#if defined(CONFIG_KVM_MAX_VCPUS)
> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> #else
> #define KVM_MAX_VCPUS 0
> #endif
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index ebf5015508b52..de63bfccb3eb5 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -40,16 +40,16 @@ config KVM_ARM_HOST
> ---help---
> Provides host support for ARM processors.
>
> -config KVM_ARM_MAX_VCPUS
> +config KVM_MAX_VCPUS
> int "Number maximum supported virtual CPUs per VM"
> depends on KVM_ARM_HOST
> default 4
> help
> Static number of max supported virtual CPUs per VM.
>
> - If you choose a high number, the vcpu structures will be quite
> - large, so only choose a reasonable number that you expect to
> - actually use.
I do no see why on ARM vcpu structure size depends on KVM_ARM_MAX_VCPUS.
Can somebody point me to it.

> + The default is set to the highest number of vcpus that
> + current hardware supports. Set to a lower number to save
> + some resources. Set to a higher number to test scalability.
>
> config KVM_ARM_VGIC
> bool "KVM support for Virtual GIC"
> --
> 1.8.1.4

--
Gleb.

2013-09-16 08:09:36

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On 15/09/13 10:30, Gleb Natapov wrote:
> On Sat, Sep 14, 2013 at 02:10:55PM +0200, Andrew Jones wrote:
>> Drop the _ARM_ part of the name. We can then introduce a config option
>> like this to aarch64 and other arches using the same name - allowing
>> grep to show them all. Also update the help text to describe the option
>> more completely.
>>
>> Signed-off-by: Andrew Jones <[email protected]>
>> ---
>> arch/arm/include/asm/kvm_host.h | 4 ++--
>> arch/arm/kvm/Kconfig | 8 ++++----
>> 2 files changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
>> index 7d22517d80711..c614d3eb176c6 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -25,8 +25,8 @@
>> #include <asm/fpstate.h>
>> #include <kvm/arm_arch_timer.h>
>>
>> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
>> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
>> +#if defined(CONFIG_KVM_MAX_VCPUS)
>> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
>> #else
>> #define KVM_MAX_VCPUS 0
>> #endif
>> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
>> index ebf5015508b52..de63bfccb3eb5 100644
>> --- a/arch/arm/kvm/Kconfig
>> +++ b/arch/arm/kvm/Kconfig
>> @@ -40,16 +40,16 @@ config KVM_ARM_HOST
>> ---help---
>> Provides host support for ARM processors.
>>
>> -config KVM_ARM_MAX_VCPUS
>> +config KVM_MAX_VCPUS
>> int "Number maximum supported virtual CPUs per VM"
>> depends on KVM_ARM_HOST
>> default 4
>> help
>> Static number of max supported virtual CPUs per VM.
>>
>> - If you choose a high number, the vcpu structures will be quite
>> - large, so only choose a reasonable number that you expect to
>> - actually use.
> I do no see why on ARM vcpu structure size depends on KVM_ARM_MAX_VCPUS.
> Can somebody point me to it.

There's a number of indirections, but here you go:
arch/arm/include/asm/kvm_host.h:#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS

include/kvm/arm_vgic.h:#define VGIC_MAX_CPUS KVM_MAX_VCPUS

include/kvm/arm_vgic.h:#if (VGIC_MAX_CPUS > 8)
include/kvm/arm_vgic.h: } percpu[VGIC_MAX_CPUS];
include/kvm/arm_vgic.h: u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
include/kvm/arm_vgic.h: u8 irq_sgi_sources[VGIC_MAX_CPUS][VGIC_NR_SGIS];
include/kvm/arm_vgic.h: struct vgic_bitmap irq_spi_target[VGIC_MAX_CPUS];

Basically, the GIC emulation data structures are sized on the number of VCPUs
(as well as the number of supported interrupts).

M.
--
Jazz is not dead. It just smells funny...

2013-09-16 08:33:08

by Gleb Natapov

[permalink] [raw]
Subject: Re: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On Mon, Sep 16, 2013 at 09:09:27AM +0100, Marc Zyngier wrote:
> On 15/09/13 10:30, Gleb Natapov wrote:
> > On Sat, Sep 14, 2013 at 02:10:55PM +0200, Andrew Jones wrote:
> >> Drop the _ARM_ part of the name. We can then introduce a config option
> >> like this to aarch64 and other arches using the same name - allowing
> >> grep to show them all. Also update the help text to describe the option
> >> more completely.
> >>
> >> Signed-off-by: Andrew Jones <[email protected]>
> >> ---
> >> arch/arm/include/asm/kvm_host.h | 4 ++--
> >> arch/arm/kvm/Kconfig | 8 ++++----
> >> 2 files changed, 6 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> >> index 7d22517d80711..c614d3eb176c6 100644
> >> --- a/arch/arm/include/asm/kvm_host.h
> >> +++ b/arch/arm/include/asm/kvm_host.h
> >> @@ -25,8 +25,8 @@
> >> #include <asm/fpstate.h>
> >> #include <kvm/arm_arch_timer.h>
> >>
> >> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
> >> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
> >> +#if defined(CONFIG_KVM_MAX_VCPUS)
> >> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> >> #else
> >> #define KVM_MAX_VCPUS 0
> >> #endif
> >> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> >> index ebf5015508b52..de63bfccb3eb5 100644
> >> --- a/arch/arm/kvm/Kconfig
> >> +++ b/arch/arm/kvm/Kconfig
> >> @@ -40,16 +40,16 @@ config KVM_ARM_HOST
> >> ---help---
> >> Provides host support for ARM processors.
> >>
> >> -config KVM_ARM_MAX_VCPUS
> >> +config KVM_MAX_VCPUS
> >> int "Number maximum supported virtual CPUs per VM"
> >> depends on KVM_ARM_HOST
> >> default 4
> >> help
> >> Static number of max supported virtual CPUs per VM.
> >>
> >> - If you choose a high number, the vcpu structures will be quite
> >> - large, so only choose a reasonable number that you expect to
> >> - actually use.
> > I do no see why on ARM vcpu structure size depends on KVM_ARM_MAX_VCPUS.
> > Can somebody point me to it.
>
> There's a number of indirections, but here you go:
> arch/arm/include/asm/kvm_host.h:#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
>
> include/kvm/arm_vgic.h:#define VGIC_MAX_CPUS KVM_MAX_VCPUS
>
> include/kvm/arm_vgic.h:#if (VGIC_MAX_CPUS > 8)
> include/kvm/arm_vgic.h: } percpu[VGIC_MAX_CPUS];
> include/kvm/arm_vgic.h: u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
> include/kvm/arm_vgic.h: u8 irq_sgi_sources[VGIC_MAX_CPUS][VGIC_NR_SGIS];
> include/kvm/arm_vgic.h: struct vgic_bitmap irq_spi_target[VGIC_MAX_CPUS];
>
> Basically, the GIC emulation data structures are sized on the number of VCPUs
> (as well as the number of supported interrupts).
>
But vgic is per VM though, not per cpu as far as I can see, so the
overhead is also per VM, no? Is VGIC_MAX_CPUS limit architectural or
juts an implementation detail?

--
Gleb.

2013-09-16 08:49:10

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On 16/09/13 09:32, Gleb Natapov wrote:
> On Mon, Sep 16, 2013 at 09:09:27AM +0100, Marc Zyngier wrote:
>> On 15/09/13 10:30, Gleb Natapov wrote:
>>> On Sat, Sep 14, 2013 at 02:10:55PM +0200, Andrew Jones wrote:
>>>> Drop the _ARM_ part of the name. We can then introduce a config option
>>>> like this to aarch64 and other arches using the same name - allowing
>>>> grep to show them all. Also update the help text to describe the option
>>>> more completely.
>>>>
>>>> Signed-off-by: Andrew Jones <[email protected]>
>>>> ---
>>>> arch/arm/include/asm/kvm_host.h | 4 ++--
>>>> arch/arm/kvm/Kconfig | 8 ++++----
>>>> 2 files changed, 6 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
>>>> index 7d22517d80711..c614d3eb176c6 100644
>>>> --- a/arch/arm/include/asm/kvm_host.h
>>>> +++ b/arch/arm/include/asm/kvm_host.h
>>>> @@ -25,8 +25,8 @@
>>>> #include <asm/fpstate.h>
>>>> #include <kvm/arm_arch_timer.h>
>>>>
>>>> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
>>>> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
>>>> +#if defined(CONFIG_KVM_MAX_VCPUS)
>>>> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
>>>> #else
>>>> #define KVM_MAX_VCPUS 0
>>>> #endif
>>>> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
>>>> index ebf5015508b52..de63bfccb3eb5 100644
>>>> --- a/arch/arm/kvm/Kconfig
>>>> +++ b/arch/arm/kvm/Kconfig
>>>> @@ -40,16 +40,16 @@ config KVM_ARM_HOST
>>>> ---help---
>>>> Provides host support for ARM processors.
>>>>
>>>> -config KVM_ARM_MAX_VCPUS
>>>> +config KVM_MAX_VCPUS
>>>> int "Number maximum supported virtual CPUs per VM"
>>>> depends on KVM_ARM_HOST
>>>> default 4
>>>> help
>>>> Static number of max supported virtual CPUs per VM.
>>>>
>>>> - If you choose a high number, the vcpu structures will be quite
>>>> - large, so only choose a reasonable number that you expect to
>>>> - actually use.
>>> I do no see why on ARM vcpu structure size depends on KVM_ARM_MAX_VCPUS.
>>> Can somebody point me to it.
>>
>> There's a number of indirections, but here you go:
>> arch/arm/include/asm/kvm_host.h:#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
>>
>> include/kvm/arm_vgic.h:#define VGIC_MAX_CPUS KVM_MAX_VCPUS
>>
>> include/kvm/arm_vgic.h:#if (VGIC_MAX_CPUS > 8)
>> include/kvm/arm_vgic.h: } percpu[VGIC_MAX_CPUS];
>> include/kvm/arm_vgic.h: u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
>> include/kvm/arm_vgic.h: u8 irq_sgi_sources[VGIC_MAX_CPUS][VGIC_NR_SGIS];
>> include/kvm/arm_vgic.h: struct vgic_bitmap irq_spi_target[VGIC_MAX_CPUS];
>>
>> Basically, the GIC emulation data structures are sized on the number of VCPUs
>> (as well as the number of supported interrupts).
>>
> But vgic is per VM though, not per cpu as far as I can see, so the
> overhead is also per VM, no? Is VGIC_MAX_CPUS limit architectural or
> juts an implementation detail?

Indeed, the MAX_VCPU overhead is per VM. The MAX_IRQS overhead is both
per-vm and per-vcpu.

The GICv2 architecture has a limit of 8 CPUs - not at all an
implementation detail. GICv3 remove this limitation.

M.
--
Jazz is not dead. It just smells funny...

2013-09-17 21:34:32

by Christoffer Dall

[permalink] [raw]
Subject: Re: [PATCH 3/3] aarch64: kvm: introduce CONFIG_KVM_MAX_VCPUS

On Sat, Sep 14, 2013 at 02:10:56PM +0200, Andrew Jones wrote:
> Take CONFIG_KVM_MAX_VCPUS from arm32, but set the default to 8.
>
> Signed-off-by: Andrew Jones <[email protected]>
> ---
> arch/arm64/include/asm/kvm_host.h | 7 ++++++-
> arch/arm64/kvm/Kconfig | 11 +++++++++++
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 0859a4ddd1e7d..d1af8c49a5ca4 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -26,7 +26,12 @@
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
>
> -#define KVM_MAX_VCPUS 4
> +#if defined(CONFIG_KVM_MAX_VCPUS)
> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> +#else
> +#define KVM_MAX_VCPUS 0
> +#endif
> +
> #define KVM_USER_MEM_SLOTS 32
> #define KVM_PRIVATE_MEM_SLOTS 4
> #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 21e90820bd23c..c9924b02e84f7 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -35,6 +35,17 @@ config KVM_ARM_HOST
> ---help---
> Provides host support for ARM processors.
>
> +config KVM_MAX_VCPUS
> + int "Number maximum supported virtual CPUs per VM"
> + depends on KVM_ARM_HOST
> + default 8

That data structure gets really big with 8 max vcpus. I think we should
keep this at 4 for now, a quad-core VM is quite reasonable for now.

> + help
> + Static number of max supported virtual CPUs per VM.
> +
> + The default is set to the highest number of vcpus that
> + current hardware supports. Set to a lower number to save
> + some resources. Set to a higher number to test scalability.
> +
> config KVM_ARM_VGIC
> bool
> depends on KVM_ARM_HOST && OF
> --
> 1.8.1.4
>

--
Christoffer

2013-09-17 21:42:13

by Christoffer Dall

[permalink] [raw]
Subject: Re: [PATCH 2/3] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On Sat, Sep 14, 2013 at 02:10:55PM +0200, Andrew Jones wrote:
> Drop the _ARM_ part of the name. We can then introduce a config option
> like this to aarch64 and other arches using the same name - allowing
> grep to show them all. Also update the help text to describe the option
> more completely.
>
> Signed-off-by: Andrew Jones <[email protected]>
> ---
> arch/arm/include/asm/kvm_host.h | 4 ++--
> arch/arm/kvm/Kconfig | 8 ++++----
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 7d22517d80711..c614d3eb176c6 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -25,8 +25,8 @@
> #include <asm/fpstate.h>
> #include <kvm/arm_arch_timer.h>
>
> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
> +#if defined(CONFIG_KVM_MAX_VCPUS)
> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> #else
> #define KVM_MAX_VCPUS 0
> #endif
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index ebf5015508b52..de63bfccb3eb5 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -40,16 +40,16 @@ config KVM_ARM_HOST
> ---help---
> Provides host support for ARM processors.
>
> -config KVM_ARM_MAX_VCPUS
> +config KVM_MAX_VCPUS
> int "Number maximum supported virtual CPUs per VM"
> depends on KVM_ARM_HOST
> default 4
> help
> Static number of max supported virtual CPUs per VM.
>
> - If you choose a high number, the vcpu structures will be quite
> - large, so only choose a reasonable number that you expect to
> - actually use.
> + The default is set to the highest number of vcpus that
> + current hardware supports. Set to a lower number to save
> + some resources. Set to a higher number to test scalability.

Consider rephrasing the "Set to a lower number..." sentence to something
like "Choosing a lower number decreases the size of the VM data
structure.".

Also, please rephrase or get rid of the "Set to a higher number to test
scalability." sentence; you increase this value to be able to be able to
run more VCPUs (for whatever reason).

--
Christoffer

2013-09-18 06:22:49

by Gleb Natapov

[permalink] [raw]
Subject: Re: [PATCH 3/3] aarch64: kvm: introduce CONFIG_KVM_MAX_VCPUS

On Tue, Sep 17, 2013 at 10:34:28PM +0100, Christoffer Dall wrote:
> On Sat, Sep 14, 2013 at 02:10:56PM +0200, Andrew Jones wrote:
> > Take CONFIG_KVM_MAX_VCPUS from arm32, but set the default to 8.
> >
> > Signed-off-by: Andrew Jones <[email protected]>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 7 ++++++-
> > arch/arm64/kvm/Kconfig | 11 +++++++++++
> > 2 files changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 0859a4ddd1e7d..d1af8c49a5ca4 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -26,7 +26,12 @@
> > #include <asm/kvm_asm.h>
> > #include <asm/kvm_mmio.h>
> >
> > -#define KVM_MAX_VCPUS 4
> > +#if defined(CONFIG_KVM_MAX_VCPUS)
> > +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> > +#else
> > +#define KVM_MAX_VCPUS 0
> > +#endif
> > +
> > #define KVM_USER_MEM_SLOTS 32
> > #define KVM_PRIVATE_MEM_SLOTS 4
> > #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
> > diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> > index 21e90820bd23c..c9924b02e84f7 100644
> > --- a/arch/arm64/kvm/Kconfig
> > +++ b/arch/arm64/kvm/Kconfig
> > @@ -35,6 +35,17 @@ config KVM_ARM_HOST
> > ---help---
> > Provides host support for ARM processors.
> >
> > +config KVM_MAX_VCPUS
> > + int "Number maximum supported virtual CPUs per VM"
> > + depends on KVM_ARM_HOST
> > + default 8
>
> That data structure gets really big with 8 max vcpus. I think we should
> keep this at 4 for now, a quad-core VM is quite reasonable for now.
>
Which data structure? vgic? IIRC with 8 vcpus it will be something like
500 bytes bigger then it is now.

--
Gleb.

2013-09-24 14:10:37

by Andrew Jones

[permalink] [raw]
Subject: [PATCH v2] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

Drop the _ARM_ part of the name. We can then introduce a config option
like this to aarch64 and other arches using the same name - allowing
grep to show them all. Also update the help text to describe the option
more completely.

Signed-off-by: Andrew Jones <[email protected]>

---
v2: reword help text some more
---
arch/arm/include/asm/kvm_host.h | 4 ++--
arch/arm/kvm/Kconfig | 9 +++++----
2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 7d22517d80711..c614d3eb176c6 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -25,8 +25,8 @@
#include <asm/fpstate.h>
#include <kvm/arm_arch_timer.h>

-#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
-#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
+#if defined(CONFIG_KVM_MAX_VCPUS)
+#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
#else
#define KVM_MAX_VCPUS 0
#endif
diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index ebf5015508b52..8e56ccf45edce 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -40,16 +40,17 @@ config KVM_ARM_HOST
---help---
Provides host support for ARM processors.

-config KVM_ARM_MAX_VCPUS
+config KVM_MAX_VCPUS
int "Number maximum supported virtual CPUs per VM"
depends on KVM_ARM_HOST
default 4
help
Static number of max supported virtual CPUs per VM.

- If you choose a high number, the vcpu structures will be quite
- large, so only choose a reasonable number that you expect to
- actually use.
+ The default is set to the highest number of vcpus that
+ current hardware supports. Choosing a lower number decreases
+ the size of the VM data structure. This number may also be
+ increased.

config KVM_ARM_VGIC
bool "KVM support for Virtual GIC"
--
1.8.1.4

2013-09-25 19:31:17

by Christoffer Dall

[permalink] [raw]
Subject: Re: [PATCH v2] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On Tue, Sep 24, 2013 at 04:09:28PM +0200, Andrew Jones wrote:
> Drop the _ARM_ part of the name. We can then introduce a config option
> like this to aarch64 and other arches using the same name - allowing
> grep to show them all. Also update the help text to describe the option
> more completely.
>
> Signed-off-by: Andrew Jones <[email protected]>
>
> ---
> v2: reword help text some more
> ---
> arch/arm/include/asm/kvm_host.h | 4 ++--
> arch/arm/kvm/Kconfig | 9 +++++----
> 2 files changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 7d22517d80711..c614d3eb176c6 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -25,8 +25,8 @@
> #include <asm/fpstate.h>
> #include <kvm/arm_arch_timer.h>
>
> -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
> -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
> +#if defined(CONFIG_KVM_MAX_VCPUS)
> +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> #else
> #define KVM_MAX_VCPUS 0
> #endif
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index ebf5015508b52..8e56ccf45edce 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -40,16 +40,17 @@ config KVM_ARM_HOST
> ---help---
> Provides host support for ARM processors.
>
> -config KVM_ARM_MAX_VCPUS
> +config KVM_MAX_VCPUS
> int "Number maximum supported virtual CPUs per VM"
> depends on KVM_ARM_HOST
> default 4
> help
> Static number of max supported virtual CPUs per VM.
>
> - If you choose a high number, the vcpu structures will be quite
> - large, so only choose a reasonable number that you expect to
> - actually use.
> + The default is set to the highest number of vcpus that
> + current hardware supports. Choosing a lower number decreases
> + the size of the VM data structure. This number may also be
> + increased.
>
Maybe I'm being forgetful, but what do you mean by "This number may also
be increased" ?

-Christoffer

2013-09-26 07:55:13

by Andrew Jones

[permalink] [raw]
Subject: Re: [PATCH v2] arm32: kvm: rename CONFIG_KVM_ARM_MAX_VCPUS

On Wed, Sep 25, 2013 at 12:31:27PM -0700, Christoffer Dall wrote:
> On Tue, Sep 24, 2013 at 04:09:28PM +0200, Andrew Jones wrote:
> > Drop the _ARM_ part of the name. We can then introduce a config option
> > like this to aarch64 and other arches using the same name - allowing
> > grep to show them all. Also update the help text to describe the option
> > more completely.
> >
> > Signed-off-by: Andrew Jones <[email protected]>
> >
> > ---
> > v2: reword help text some more
> > ---
> > arch/arm/include/asm/kvm_host.h | 4 ++--
> > arch/arm/kvm/Kconfig | 9 +++++----
> > 2 files changed, 7 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> > index 7d22517d80711..c614d3eb176c6 100644
> > --- a/arch/arm/include/asm/kvm_host.h
> > +++ b/arch/arm/include/asm/kvm_host.h
> > @@ -25,8 +25,8 @@
> > #include <asm/fpstate.h>
> > #include <kvm/arm_arch_timer.h>
> >
> > -#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
> > -#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
> > +#if defined(CONFIG_KVM_MAX_VCPUS)
> > +#define KVM_MAX_VCPUS CONFIG_KVM_MAX_VCPUS
> > #else
> > #define KVM_MAX_VCPUS 0
> > #endif
> > diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> > index ebf5015508b52..8e56ccf45edce 100644
> > --- a/arch/arm/kvm/Kconfig
> > +++ b/arch/arm/kvm/Kconfig
> > @@ -40,16 +40,17 @@ config KVM_ARM_HOST
> > ---help---
> > Provides host support for ARM processors.
> >
> > -config KVM_ARM_MAX_VCPUS
> > +config KVM_MAX_VCPUS
> > int "Number maximum supported virtual CPUs per VM"
> > depends on KVM_ARM_HOST
> > default 4
> > help
> > Static number of max supported virtual CPUs per VM.
> >
> > - If you choose a high number, the vcpu structures will be quite
> > - large, so only choose a reasonable number that you expect to
> > - actually use.
> > + The default is set to the highest number of vcpus that
> > + current hardware supports. Choosing a lower number decreases
> > + the size of the VM data structure. This number may also be
> > + increased.
> >
> Maybe I'm being forgetful, but what do you mean by "This number may also
> be increased" ?

Originally I was proposing we write 'increase for scalability testing',
as would be the purpose on the x86 side. However, you suggested it can
be increased for other purposes, although you didn't mention what. I just
dropped the 'scalability testing' to allow users to fill the blank in
themselves. If it doesn't really make sense on arm to increase this
beyond what hardware supports, then we can drop the 'increase' completely
and add a

range 1 <same-as-default>

to the Kconfig entry instead.

drew