2013-04-25 16:57:17

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 0/7] xen/arm: move to mach-virt and support SMP

Hi all,
this patch series, based on 3.9-rc3, moves xenvm to mach-virt,
introduces SMP support in Xen on ARM and implements machine reboot and
power off via Xen sched_op hypercalls.

Each patch comes with a detailed changelog.


The merge window is approaching, this patch series only contains Xen
specific changes for ARM, if anybody has any objections speak now
please.

I would like an ack from an ARM maintainer on patch #4 and #5 since they
touch generic ARM Makefiles.


Changes in v5:
- allocate xen_vcpu_info dynamically, aligning it to the size of the
struct;
- use VCPUOP_register_vcpu_info on cpu0 too;
- set pm_power_off and arm_pm_restart from the Xen specific
intialization code.



Stefano Stabellini (7):
xen/arm: actually pass a non-NULL percpu pointer to request_percpu_irq
xen/arm: implement HYPERVISOR_vcpu_op
xen/arm: SMP support
xen: move the xenvm machine to mach-virt
xen/arm: XEN selects ARM_PSCI
xenvm: add a simple PSCI node and a second cpu
xen/arm: use sched_op hypercalls for machine reboot and power off

arch/arm/Kconfig | 1 +
arch/arm/boot/dts/Makefile | 4 +-
arch/arm/boot/dts/xenvm-4.2.dts | 13 ++++++
arch/arm/include/asm/xen/hypercall.h | 1 +
arch/arm/mach-vexpress/v2m.c | 1 -
arch/arm/mach-virt/virt.c | 1 +
arch/arm/xen/enlighten.c | 75 ++++++++++++++++++++++++++++++---
arch/arm/xen/hypercall.S | 1 +
8 files changed, 87 insertions(+), 10 deletions(-)


git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git 3.9-rc3-smp-5-tag

Cheers,

Stefano


2013-04-25 16:58:08

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 1/7] xen/arm: actually pass a non-NULL percpu pointer to request_percpu_irq

Signed-off-by: Stefano Stabellini <[email protected]>
Reviewed-by: Ian Campbell <[email protected]>
CC: [email protected]
---
arch/arm/xen/enlighten.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
index 8dc0605..99ce189 100644
--- a/arch/arm/xen/enlighten.c
+++ b/arch/arm/xen/enlighten.c
@@ -239,7 +239,7 @@ static int __init xen_init_events(void)
xen_init_IRQ();

if (request_percpu_irq(xen_events_irq, xen_arm_callback,
- "events", xen_vcpu)) {
+ "events", &xen_vcpu)) {
pr_err("Error requesting IRQ %d\n", xen_events_irq);
return -EINVAL;
}
--
1.7.2.5

2013-04-25 16:58:15

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 3/7] xen/arm: SMP support

Map vcpu_info using VCPUOP_register_vcpu_info on all the online vcpus,
make sure the allocated struct doesn't cross a page boundary.

Call enable_percpu_irq on every cpu.

Changes in v5:
- allocate xen_vcpu_info dynamically, aligning it to the size of the
struct;
- use VCPUOP_register_vcpu_info on cpu0 too.

Changed in v2:
- move the percpu variable argument fix to a separate patch;
- remove unused variable.

Signed-off-by: Stefano Stabellini <[email protected]>
---
arch/arm/xen/enlighten.c | 48 ++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 42 insertions(+), 6 deletions(-)

diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
index 82d5e63..6c87d11 100644
--- a/arch/arm/xen/enlighten.c
+++ b/arch/arm/xen/enlighten.c
@@ -2,6 +2,7 @@
#include <xen/events.h>
#include <xen/grant_table.h>
#include <xen/hvm.h>
+#include <xen/interface/vcpu.h>
#include <xen/interface/xen.h>
#include <xen/interface/memory.h>
#include <xen/interface/hvm/params.h>
@@ -32,6 +33,7 @@ struct shared_info xen_dummy_shared_info;
struct shared_info *HYPERVISOR_shared_info = (void *)&xen_dummy_shared_info;

DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
+static struct vcpu_info __percpu *xen_vcpu_info;

/* These are unused until we support booting "pre-ballooned" */
unsigned long xen_released_pages;
@@ -148,6 +150,29 @@ int xen_unmap_domain_mfn_range(struct vm_area_struct *vma,
}
EXPORT_SYMBOL_GPL(xen_unmap_domain_mfn_range);

+static int __init xen_secondary_init(unsigned int cpu)
+{
+ struct vcpu_register_vcpu_info info;
+ struct vcpu_info *vcpup;
+ int err;
+
+ pr_info("Xen: initializing cpu%d\n", cpu);
+ vcpup = per_cpu_ptr(xen_vcpu_info, cpu);
+
+ info.mfn = __pa(vcpup) >> PAGE_SHIFT;
+ info.offset = offset_in_page(vcpup);
+
+ err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info);
+ if (err) {
+ pr_debug("register_vcpu_info failed: err=%d\n", err);
+ } else {
+ /* This cpu is using the registered vcpu info, even if
+ later ones fail to. */
+ per_cpu(xen_vcpu, cpu) = vcpup;
+ }
+ return 0;
+}
+
/*
* see Documentation/devicetree/bindings/arm/xen.txt for the
* documentation of the Xen Device Tree format.
@@ -163,6 +188,7 @@ static int __init xen_guest_init(void)
const char *version = NULL;
const char *xen_prefix = "xen,xen-";
struct resource res;
+ int i;

node = of_find_compatible_node(NULL, NULL, "xen,xen");
if (!node) {
@@ -209,13 +235,18 @@ static int __init xen_guest_init(void)

/* xen_vcpu is a pointer to the vcpu_info struct in the shared_info
* page, we use it in the event channel upcall and in some pvclock
- * related functions. We don't need the vcpu_info placement
- * optimizations because we don't use any pv_mmu or pv_irq op on
- * HVM.
+ * related functions.
* The shared info contains exactly 1 CPU (the boot CPU). The guest
* is required to use VCPUOP_register_vcpu_info to place vcpu info
- * for secondary CPUs as they are brought up. */
- per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0];
+ * for secondary CPUs as they are brought up.
+ * For uniformity we use VCPUOP_register_vcpu_info even on cpu0.
+ */
+ xen_vcpu_info = __alloc_percpu(sizeof(struct vcpu_info),
+ sizeof(struct vcpu_info));
+ if (xen_vcpu_info == NULL)
+ return -ENOMEM;
+ for_each_online_cpu(i)
+ xen_secondary_init(i);

gnttab_init();
if (!xen_initial_domain())
@@ -231,6 +262,11 @@ static irqreturn_t xen_arm_callback(int irq, void *arg)
return IRQ_HANDLED;
}

+static __init void xen_percpu_enable_events(void *unused)
+{
+ enable_percpu_irq(xen_events_irq, 0);
+}
+
static int __init xen_init_events(void)
{
if (!xen_domain() || xen_events_irq < 0)
@@ -244,7 +280,7 @@ static int __init xen_init_events(void)
return -EINVAL;
}

- enable_percpu_irq(xen_events_irq, 0);
+ on_each_cpu(xen_percpu_enable_events, NULL, 0);

return 0;
}
--
1.7.2.5

2013-04-25 16:58:13

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 7/7] xen/arm: use sched_op hypercalls for machine reboot and power off

Changes in v5:
- set pm_power_off and arm_pm_restart from the Xen specific
intialization code.

Signed-off-by: Stefano Stabellini <[email protected]>
---
arch/arm/xen/enlighten.c | 24 ++++++++++++++++++++++++
1 files changed, 24 insertions(+), 0 deletions(-)

diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
index 6c87d11..9a0a917 100644
--- a/arch/arm/xen/enlighten.c
+++ b/arch/arm/xen/enlighten.c
@@ -10,9 +10,12 @@
#include <xen/platform_pci.h>
#include <xen/xenbus.h>
#include <xen/page.h>
+#include <xen/xen.h>
+#include <xen/interface/sched.h>
#include <xen/xen-ops.h>
#include <asm/xen/hypervisor.h>
#include <asm/xen/hypercall.h>
+#include <asm/system_misc.h>
#include <linux/interrupt.h>
#include <linux/irqreturn.h>
#include <linux/module.h>
@@ -173,6 +176,24 @@ static int __init xen_secondary_init(unsigned int cpu)
return 0;
}

+static void xen_restart(char str, const char *cmd)
+{
+ struct sched_shutdown r = { .reason = SHUTDOWN_reboot };
+ int rc;
+ rc = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r);
+ if (rc)
+ BUG();
+}
+
+static void xen_power_off(void)
+{
+ struct sched_shutdown r = { .reason = SHUTDOWN_poweroff };
+ int rc;
+ rc = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r);
+ if (rc)
+ BUG();
+}
+
/*
* see Documentation/devicetree/bindings/arm/xen.txt for the
* documentation of the Xen Device Tree format.
@@ -252,6 +273,9 @@ static int __init xen_guest_init(void)
if (!xen_initial_domain())
xenbus_probe(NULL);

+ pm_power_off = xen_power_off;
+ arm_pm_restart = xen_restart;
+
return 0;
}
core_initcall(xen_guest_init);
--
1.7.2.5

2013-04-25 16:58:11

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 2/7] xen/arm: implement HYPERVISOR_vcpu_op

Signed-off-by: Stefano Stabellini <[email protected]>
Reviewed-by: Ian Campbell <[email protected]>
---
arch/arm/include/asm/xen/hypercall.h | 1 +
arch/arm/xen/enlighten.c | 1 +
arch/arm/xen/hypercall.S | 1 +
3 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/arm/include/asm/xen/hypercall.h b/arch/arm/include/asm/xen/hypercall.h
index 8a82325..799f42e 100644
--- a/arch/arm/include/asm/xen/hypercall.h
+++ b/arch/arm/include/asm/xen/hypercall.h
@@ -46,6 +46,7 @@ int HYPERVISOR_event_channel_op(int cmd, void *arg);
unsigned long HYPERVISOR_hvm_op(int op, void *arg);
int HYPERVISOR_memory_op(unsigned int cmd, void *arg);
int HYPERVISOR_physdev_op(int cmd, void *arg);
+int HYPERVISOR_vcpu_op(int cmd, int vcpuid, void *extra_args);

static inline void
MULTI_update_va_mapping(struct multicall_entry *mcl, unsigned long va,
diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
index 99ce189..82d5e63 100644
--- a/arch/arm/xen/enlighten.c
+++ b/arch/arm/xen/enlighten.c
@@ -259,4 +259,5 @@ EXPORT_SYMBOL_GPL(HYPERVISOR_sched_op);
EXPORT_SYMBOL_GPL(HYPERVISOR_hvm_op);
EXPORT_SYMBOL_GPL(HYPERVISOR_memory_op);
EXPORT_SYMBOL_GPL(HYPERVISOR_physdev_op);
+EXPORT_SYMBOL_GPL(HYPERVISOR_vcpu_op);
EXPORT_SYMBOL_GPL(privcmd_call);
diff --git a/arch/arm/xen/hypercall.S b/arch/arm/xen/hypercall.S
index 71f7239..199cb2d 100644
--- a/arch/arm/xen/hypercall.S
+++ b/arch/arm/xen/hypercall.S
@@ -87,6 +87,7 @@ HYPERCALL2(event_channel_op);
HYPERCALL2(hvm_op);
HYPERCALL2(memory_op);
HYPERCALL2(physdev_op);
+HYPERCALL3(vcpu_op);

ENTRY(privcmd_call)
stmdb sp!, {r4}
--
1.7.2.5

2013-04-25 16:58:09

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 6/7] xenvm: add a simple PSCI node and a second cpu

Signed-off-by: Stefano Stabellini <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
CC: [email protected]
CC: [email protected]
CC: [email protected]
---
arch/arm/boot/dts/xenvm-4.2.dts | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/arch/arm/boot/dts/xenvm-4.2.dts b/arch/arm/boot/dts/xenvm-4.2.dts
index ec3f952..3369151 100644
--- a/arch/arm/boot/dts/xenvm-4.2.dts
+++ b/arch/arm/boot/dts/xenvm-4.2.dts
@@ -29,6 +29,19 @@
compatible = "arm,cortex-a15";
reg = <0>;
};
+
+ [email protected] {
+ device_type = "cpu";
+ compatible = "arm,cortex-a15";
+ reg = <1>;
+ };
+ };
+
+ psci {
+ compatible = "arm,psci";
+ method = "hvc";
+ cpu_off = <1>;
+ cpu_on = <2>;
};

[email protected] {
--
1.7.2.5

2013-04-25 16:59:17

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 4/7] xen: move the xenvm machine to mach-virt

xenvm is based on mach-vexpress, move it to mach-virt.

Changes in v4:
- update the dts Makefile too.

Signed-off-by: Stefano Stabellini <[email protected]>
CC: Marc Zyngier <[email protected]>
CC: [email protected]
CC: [email protected]
CC: [email protected]
---
arch/arm/boot/dts/Makefile | 4 ++--
arch/arm/mach-vexpress/v2m.c | 1 -
arch/arm/mach-virt/virt.c | 1 +
3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm/boot/dts/Makefile b/arch/arm/boot/dts/Makefile
index 9c62558..b6289b7 100644
--- a/arch/arm/boot/dts/Makefile
+++ b/arch/arm/boot/dts/Makefile
@@ -168,8 +168,8 @@ dtb-$(CONFIG_ARCH_TEGRA) += tegra20-harmony.dtb \
dtb-$(CONFIG_ARCH_VEXPRESS) += vexpress-v2p-ca5s.dtb \
vexpress-v2p-ca9.dtb \
vexpress-v2p-ca15-tc1.dtb \
- vexpress-v2p-ca15_a7.dtb \
- xenvm-4.2.dtb
+ vexpress-v2p-ca15_a7.dtb
+dtb-$(CONFIG_ARCH_VIRT) += xenvm-4.2.dtb
dtb-$(CONFIG_ARCH_VT8500) += vt8500-bv07.dtb \
wm8505-ref.dtb \
wm8650-mid.dtb \
diff --git a/arch/arm/mach-vexpress/v2m.c b/arch/arm/mach-vexpress/v2m.c
index 915683c..c43ec78 100644
--- a/arch/arm/mach-vexpress/v2m.c
+++ b/arch/arm/mach-vexpress/v2m.c
@@ -469,7 +469,6 @@ static void __init v2m_dt_init(void)

static const char * const v2m_dt_match[] __initconst = {
"arm,vexpress",
- "xen,xenvm",
NULL,
};

diff --git a/arch/arm/mach-virt/virt.c b/arch/arm/mach-virt/virt.c
index 31666f6..528c05e 100644
--- a/arch/arm/mach-virt/virt.c
+++ b/arch/arm/mach-virt/virt.c
@@ -40,6 +40,7 @@ static void __init virt_timer_init(void)

static const char *virt_dt_match[] = {
"linux,dummy-virt",
+ "xen,xenvm",
NULL
};

--
1.7.2.5

2013-04-25 16:59:46

by Stefano Stabellini

[permalink] [raw]
Subject: [PATCH v5 5/7] xen/arm: XEN selects ARM_PSCI

Signed-off-by: Stefano Stabellini <[email protected]>
---
arch/arm/Kconfig | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 2c3bdce..344e299 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1892,6 +1892,7 @@ config XEN
depends on ARM && AEABI && OF
depends on CPU_V7 && !CPU_V6
depends on !GENERIC_ATOMIC64
+ select ARM_PSCI
help
Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.

--
1.7.2.5

2013-04-26 10:41:41

by Stefano Stabellini

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] xen/arm: move to mach-virt and support SMP

Arnd, Olof,
during the last few merge windows Konrad has always sent the Xen ARM
patches to Linus via his tree, but this time (3.10 merge window), after
consulting with Konrad I am thinking of sending to Linus the pull
request myself.

Arnd, are you OK with this patch series?

It's all very Xen specific and it is already in linux-next.
The only non-Xen files that are touched are arch/arm/Kconfig, to select
ARM_PSCI on XEN, and arch/arm/boot/dts/Makefile, to build the xenvm DTB
if CONFIG_ARCH_VIRT.

Cheers,

Stefano


On Thu, 25 Apr 2013, Stefano Stabellini wrote:
> Hi all,
> this patch series, based on 3.9-rc3, moves xenvm to mach-virt,
> introduces SMP support in Xen on ARM and implements machine reboot and
> power off via Xen sched_op hypercalls.
>
> Each patch comes with a detailed changelog.
>
>
> The merge window is approaching, this patch series only contains Xen
> specific changes for ARM, if anybody has any objections speak now
> please.
>
> I would like an ack from an ARM maintainer on patch #4 and #5 since they
> touch generic ARM Makefiles.
>
>
> Changes in v5:
> - allocate xen_vcpu_info dynamically, aligning it to the size of the
> struct;
> - use VCPUOP_register_vcpu_info on cpu0 too;
> - set pm_power_off and arm_pm_restart from the Xen specific
> intialization code.
>
>
>
> Stefano Stabellini (7):
> xen/arm: actually pass a non-NULL percpu pointer to request_percpu_irq
> xen/arm: implement HYPERVISOR_vcpu_op
> xen/arm: SMP support
> xen: move the xenvm machine to mach-virt
> xen/arm: XEN selects ARM_PSCI
> xenvm: add a simple PSCI node and a second cpu
> xen/arm: use sched_op hypercalls for machine reboot and power off
>
> arch/arm/Kconfig | 1 +
> arch/arm/boot/dts/Makefile | 4 +-
> arch/arm/boot/dts/xenvm-4.2.dts | 13 ++++++
> arch/arm/include/asm/xen/hypercall.h | 1 +
> arch/arm/mach-vexpress/v2m.c | 1 -
> arch/arm/mach-virt/virt.c | 1 +
> arch/arm/xen/enlighten.c | 75 ++++++++++++++++++++++++++++++---
> arch/arm/xen/hypercall.S | 1 +
> 8 files changed, 87 insertions(+), 10 deletions(-)
>
>
> git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git 3.9-rc3-smp-5-tag
>
> Cheers,
>
> Stefano
>

2013-04-30 15:28:47

by Konrad Rzeszutek Wilk

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] xen/arm: move to mach-virt and support SMP

On Fri, Apr 26, 2013 at 11:41:32AM +0100, Stefano Stabellini wrote:
> Arnd, Olof,
> during the last few merge windows Konrad has always sent the Xen ARM
> patches to Linus via his tree, but this time (3.10 merge window), after
> consulting with Konrad I am thinking of sending to Linus the pull
> request myself.

<nods> They look good to me. If there are cross-platform patches
(say in drivers/xen) they would still need my Ack (or Jeremy's).

>
> Arnd, are you OK with this patch series?
>
> It's all very Xen specific and it is already in linux-next.
> The only non-Xen files that are touched are arch/arm/Kconfig, to select
> ARM_PSCI on XEN, and arch/arm/boot/dts/Makefile, to build the xenvm DTB
> if CONFIG_ARCH_VIRT.

I don't have much knowledge in the 'dts' Makefile but the rest looks
OK to me.
>
> Cheers,
>
> Stefano
>
>
> On Thu, 25 Apr 2013, Stefano Stabellini wrote:
> > Hi all,
> > this patch series, based on 3.9-rc3, moves xenvm to mach-virt,
> > introduces SMP support in Xen on ARM and implements machine reboot and
> > power off via Xen sched_op hypercalls.
> >
> > Each patch comes with a detailed changelog.
> >
> >
> > The merge window is approaching, this patch series only contains Xen
> > specific changes for ARM, if anybody has any objections speak now
> > please.
> >
> > I would like an ack from an ARM maintainer on patch #4 and #5 since they
> > touch generic ARM Makefiles.
> >
> >
> > Changes in v5:
> > - allocate xen_vcpu_info dynamically, aligning it to the size of the
> > struct;
> > - use VCPUOP_register_vcpu_info on cpu0 too;
> > - set pm_power_off and arm_pm_restart from the Xen specific
> > intialization code.
> >
> >
> >
> > Stefano Stabellini (7):
> > xen/arm: actually pass a non-NULL percpu pointer to request_percpu_irq
> > xen/arm: implement HYPERVISOR_vcpu_op
> > xen/arm: SMP support
> > xen: move the xenvm machine to mach-virt
> > xen/arm: XEN selects ARM_PSCI
> > xenvm: add a simple PSCI node and a second cpu
> > xen/arm: use sched_op hypercalls for machine reboot and power off
> >
> > arch/arm/Kconfig | 1 +
> > arch/arm/boot/dts/Makefile | 4 +-
> > arch/arm/boot/dts/xenvm-4.2.dts | 13 ++++++
> > arch/arm/include/asm/xen/hypercall.h | 1 +
> > arch/arm/mach-vexpress/v2m.c | 1 -
> > arch/arm/mach-virt/virt.c | 1 +
> > arch/arm/xen/enlighten.c | 75 ++++++++++++++++++++++++++++++---
> > arch/arm/xen/hypercall.S | 1 +
> > 8 files changed, 87 insertions(+), 10 deletions(-)
> >
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git 3.9-rc3-smp-5-tag
> >
> > Cheers,
> >
> > Stefano
> >