2008-01-21 21:53:39

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 0/4] paravirt_ops-64 compile fixes

This series contain fixes to make the paravirt_ops code compile and boot
on x86_64.

This is a follow-up for the previous series from Glauber.


2008-01-21 21:53:24

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 2/4] include/asm-x86/paravirt.h: x86_64 mmu operations

Add .set_pgd field to pv_mmu_ops.

Implement pud_val(), __pud(), set_pgd(), pud_clear(), pgd_clear().

pud_clear() and pgd_clear() are implemented simply using set_pud()
and set_pmd(). They don't have a field at pv_mmu_ops.

Signed-off-by: Eduardo Habkost <[email protected]>
---
include/asm-x86/paravirt.h | 55 ++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/include/asm-x86/paravirt.h b/include/asm-x86/paravirt.h
index 62cecb7..9f9ff57 100644
--- a/include/asm-x86/paravirt.h
+++ b/include/asm-x86/paravirt.h
@@ -259,6 +259,8 @@ struct pv_mmu_ops {
#if PAGETABLE_LEVELS == 4
pudval_t (*pud_val)(pud_t);
pud_t (*make_pud)(pudval_t pud);
+
+ void (*set_pgd)(pgd_t *pudp, pgd_t pgdval);
#endif /* PAGETABLE_LEVELS == 4 */
#endif /* PAGETABLE_LEVELS >= 3 */

@@ -1065,6 +1067,59 @@ static inline void set_pud(pud_t *pudp, pud_t pud)
PVOP_VCALL2(pv_mmu_ops.set_pud, pudp,
val);
}
+#if PAGETABLE_LEVELS == 4
+static inline pud_t __pud(pudval_t val)
+{
+ pudval_t ret;
+
+ if (sizeof(pudval_t) > sizeof(long))
+ ret = PVOP_CALL2(pudval_t, pv_mmu_ops.make_pud,
+ val, (u64)val >> 32);
+ else
+ ret = PVOP_CALL1(pudval_t, pv_mmu_ops.make_pud,
+ val);
+
+ return (pud_t) { ret };
+}
+
+static inline pudval_t pud_val(pud_t pud)
+{
+ pudval_t ret;
+
+ if (sizeof(pudval_t) > sizeof(long))
+ ret = PVOP_CALL2(pudval_t, pv_mmu_ops.pud_val,
+ pud.pud, (u64)pud.pud >> 32);
+ else
+ ret = PVOP_CALL1(pudval_t, pv_mmu_ops.pud_val,
+ pud.pud);
+
+ return ret;
+}
+
+static inline void set_pgd(pgd_t *pgdp, pgd_t pgd)
+{
+ pgdval_t val = native_pgd_val(pgd);
+
+ if (sizeof(pgdval_t) > sizeof(long))
+ PVOP_VCALL3(pv_mmu_ops.set_pgd, pgdp,
+ val, (u64)val >> 32);
+ else
+ PVOP_VCALL2(pv_mmu_ops.set_pgd, pgdp,
+ val);
+}
+
+static inline void pgd_clear(pgd_t *pgdp)
+{
+ set_pgd(pgdp, __pgd(0));
+}
+
+static inline void pud_clear(pud_t *pudp)
+{
+ set_pud(pudp, __pud(0));
+}
+
+#endif /* PAGETABLE_LEVELS == 4 */
+
#endif /* PAGETABLE_LEVELS >= 3 */

#ifdef CONFIG_X86_PAE
--
1.5.3.4

2008-01-21 21:53:56

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 1/4] Add missing semicolon to SWAPGS macro

Trivial compile fix.

Signed-off-by: Eduardo Habkost <[email protected]>
---
include/asm-x86/paravirt.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/asm-x86/paravirt.h b/include/asm-x86/paravirt.h
index 52bcd9d..62cecb7 100644
--- a/include/asm-x86/paravirt.h
+++ b/include/asm-x86/paravirt.h
@@ -1329,7 +1329,7 @@ static inline unsigned long __raw_local_irq_save(void)
#else
#define SWAPGS \
PARA_SITE(PARA_PATCH(pv_cpu_ops, PV_CPU_swapgs), CLBR_NONE, \
- PV_SAVE_REGS \
+ PV_SAVE_REGS; \
call *pv_cpu_ops+PV_CPU_swapgs; \
PV_RESTORE_REGS \
)
--
1.5.3.4

2008-01-21 21:54:17

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 4/4] Fill in missing pv_mmu_ops entries for PAGETABLE_LEVELS >= 3

This finally makes paravirt-ops able to compile and boot under x86_64.

Signed-off-by: Eduardo Habkost <[email protected]>
---
arch/x86/kernel/paravirt.c | 11 +++++++++--
1 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 37f38b7..075962c 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -398,16 +398,23 @@ struct pv_mmu_ops pv_mmu_ops = {
.kmap_atomic_pte = kmap_atomic,
#endif

+#if PAGETABLE_LEVELS >= 3
#ifdef CONFIG_X86_PAE
.set_pte_atomic = native_set_pte_atomic,
.set_pte_present = native_set_pte_present,
- .set_pud = native_set_pud,
.pte_clear = native_pte_clear,
.pmd_clear = native_pmd_clear,
-
+#endif
+ .set_pud = native_set_pud,
.pmd_val = native_pmd_val,
.make_pmd = native_make_pmd,
+
+#if PAGETABLE_LEVELS == 4
+ .pud_val = native_pud_val,
+ .make_pud = native_make_pud,
+ .set_pgd = native_set_pgd,
#endif
+#endif /* PAGETABLE_LEVELS >= 3 */

.pte_val = native_pte_val,
.pgd_val = native_pgd_val,
--
1.5.3.4

2008-01-21 21:54:33

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 3/4] Don't set pagetable_setup_{start,done} hooks on x86_64

paravirt_pagetable_setup_{start,done}() are not used (yet) under x86_64,
and native_pagetable_setup_{start,done}() don't exist on x86_64. So they
don't need to be set.

Signed-off-by: Eduardo Habkost <[email protected]>
---
arch/x86/kernel/paravirt.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index c67d331..37f38b7 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -367,8 +367,10 @@ struct pv_apic_ops pv_apic_ops = {
};

struct pv_mmu_ops pv_mmu_ops = {
+#ifndef CONFIG_X86_64
.pagetable_setup_start = native_pagetable_setup_start,
.pagetable_setup_done = native_pagetable_setup_done,
+#endif

.read_cr2 = native_read_cr2,
.write_cr2 = native_write_cr2,
--
1.5.3.4

2008-01-22 12:03:33

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH 0/4] paravirt_ops-64 compile fixes


* Eduardo Habkost <[email protected]> wrote:

> This series contain fixes to make the paravirt_ops code compile and
> boot on x86_64.
>
> This is a follow-up for the previous series from Glauber.

thanks Eduardo, i've applied your fixes.

could you perhaps send the missing bits that add the missing 64-bit
Kconfig entries? Even if the 64-bit side is not functional yet as a real
guest, it would help us find the build bugs and any runtime regressions
PARAVIRT might cause on the native 64-bit side of the kernel.

Or is it now just a matter of removing all the depends-on X86_32 bits in
arch/x86/Kconfig?

Ingo

2008-01-22 12:39:21

by Eduardo Habkost

[permalink] [raw]
Subject: Re: [PATCH 0/4] paravirt_ops-64 compile fixes

On Tue, Jan 22, 2008 at 01:02:07PM +0100, Ingo Molnar wrote:
>
> * Eduardo Habkost <[email protected]> wrote:
>
> > This series contain fixes to make the paravirt_ops code compile and
> > boot on x86_64.
> >
> > This is a follow-up for the previous series from Glauber.
>
> thanks Eduardo, i've applied your fixes.
>
> could you perhaps send the missing bits that add the missing 64-bit
> Kconfig entries? Even if the 64-bit side is not functional yet as a real
> guest, it would help us find the build bugs and any runtime regressions
> PARAVIRT might cause on the native 64-bit side of the kernel.
>
> Or is it now just a matter of removing all the depends-on X86_32 bits in
> arch/x86/Kconfig?

Removing the depends won't be enough because CONFIG_PARAVIRT won't be
selected unless one of Xen, VMI, or lguest is selected. (PARAVIRT is
not visible on menuconfig, only PARAVIRT_GUEST, that doesn't enable any
code).

When hacking, I remove the 32-bit depends and add "select PARAVIRT" to
"menuconfig PARAVIRT_GUEST".

I think neither of the three guest implementations are ready for 64-bit
yet. I am working on Xen and it is far from being ready for inclusion. I
don't know about VMI and lguest.

If this is desirable for broader testing, we can make config PARAVIRT
visible and selectable on menuconfig, even when there is no guest
implementation being enabled. Should I do it?

Maybe should this force-paravirt mode be a kernel debugging option?

--
Eduardo

2008-01-22 17:56:17

by Jeremy Fitzhardinge

[permalink] [raw]
Subject: Re: [PATCH 0/4] paravirt_ops-64 compile fixes

Eduardo Pereira Habkost wrote:
> If this is desirable for broader testing, we can make config PARAVIRT
> visible and selectable on menuconfig, even when there is no guest
> implementation being enabled. Should I do it?
>

Yes please.

> Maybe should this force-paravirt mode be a kernel debugging option?
>

No, just a plain CONFIG_PARAVIRT to turn it on and off independently of
everything else.

J

2008-01-22 18:55:28

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 1/2] Allow enabling PARAVIRT without any guest implementation


On Tue, Jan 22, 2008 at 09:55:41AM -0800, Jeremy Fitzhardinge wrote:
> Eduardo Pereira Habkost wrote:
> >If this is desirable for broader testing, we can make config PARAVIRT
> >visible and selectable on menuconfig, even when there is no guest
> >implementation being enabled. Should I do it?
> >
>
> Yes please.
>

This will allow people to enable the paravirt_ops code even when no
guest support is enabled, for broader testing of the paravirt_ops code.

Signed-off-by: Eduardo Habkost <[email protected]>
---
arch/x86/Kconfig | 18 +++++++++---------
1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 4e910d8..715bbcd 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -325,15 +325,6 @@ config SCHED_NO_NO_OMIT_FRAME_POINTER

If in doubt, say "Y".

-config PARAVIRT
- bool
- depends on X86_32 && !(X86_VISWS || X86_VOYAGER)
- help
- This changes the kernel so it can modify itself when it is run
- under a hypervisor, potentially improving performance significantly
- over full virtualization. However, when run without a hypervisor
- the kernel is theoretically slower and slightly larger.
-
menuconfig PARAVIRT_GUEST
bool "Paravirtualized guest support"
depends on X86_32
@@ -359,6 +350,15 @@ config VMI

source "arch/x86/lguest/Kconfig"

+config PARAVIRT
+ bool "Enable paravirtualization code"
+ depends on X86_32 && !(X86_VISWS || X86_VOYAGER)
+ help
+ This changes the kernel so it can modify itself when it is run
+ under a hypervisor, potentially improving performance significantly
+ over full virtualization. However, when run without a hypervisor
+ the kernel is theoretically slower and slightly larger.
+
endif

config ACPI_SRAT
--
1.5.3.4

--
Eduardo

2008-01-22 18:58:17

by Eduardo Habkost

[permalink] [raw]
Subject: [PATCH 2/2] Remove depends on X86_32 from PARAVIRT & PARAVIRT_GUEST


With this, the paravirt_ops code can be enabled on x86_64 also.

Each guest implementation (Xen, VMI, lguest) now depends on X86_32. The
dependencies can be dropped for each one when they start to support
x86_64.

Signed-off-by: Eduardo Habkost <[email protected]>
---
arch/x86/Kconfig | 4 ++--
arch/x86/lguest/Kconfig | 1 +
arch/x86/xen/Kconfig | 1 +
3 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 715bbcd..43d535a 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -327,7 +327,6 @@ config SCHED_NO_NO_OMIT_FRAME_POINTER

menuconfig PARAVIRT_GUEST
bool "Paravirtualized guest support"
- depends on X86_32
help
Say Y here to get to see options related to running Linux under
various hypervisors. This option alone does not add any kernel code.
@@ -341,6 +340,7 @@ source "arch/x86/xen/Kconfig"
config VMI
bool "VMI Guest support"
select PARAVIRT
+ depends on X86_32
depends on !(X86_VISWS || X86_VOYAGER)
help
VMI provides a paravirtualized interface to the VMware ESX server
@@ -352,7 +352,7 @@ source "arch/x86/lguest/Kconfig"

config PARAVIRT
bool "Enable paravirtualization code"
- depends on X86_32 && !(X86_VISWS || X86_VOYAGER)
+ depends on !(X86_VISWS || X86_VOYAGER)
help
This changes the kernel so it can modify itself when it is run
under a hypervisor, potentially improving performance significantly
diff --git a/arch/x86/lguest/Kconfig b/arch/x86/lguest/Kconfig
index 19626ac..964dfa3 100644
--- a/arch/x86/lguest/Kconfig
+++ b/arch/x86/lguest/Kconfig
@@ -1,6 +1,7 @@
config LGUEST_GUEST
bool "Lguest guest support"
select PARAVIRT
+ depends on X86_32
depends on !X86_PAE
depends on !(X86_VISWS || X86_VOYAGER)
select VIRTIO
diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
index fbfa55c..4d5f264 100644
--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -5,6 +5,7 @@
config XEN
bool "Xen guest support"
select PARAVIRT
+ depends on X86_32
depends on X86_CMPXCHG && X86_TSC && !NEED_MULTIPLE_NODES && !(X86_VISWS || X86_VOYAGER)
help
This is the Linux Xen port. Enabling this will allow the
--
1.5.3.4

2008-01-22 19:48:22

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH 0/4] paravirt_ops-64 compile fixes


* Jeremy Fitzhardinge <[email protected]> wrote:

>> Maybe should this force-paravirt mode be a kernel debugging option?
>
> No, just a plain CONFIG_PARAVIRT to turn it on and off independently
> of everything else.

it should be a debugging option in the way that it's dependent on
KERNEL_DEBUG and KERNEL_EXPERIMENTAL.

i dont think you shold worry about this - i think 64-bit guest support
for specific hypervisors could be added even after the merge window, if
it's reasonably isolated (which i'd expect it to be). It clearly cannot
break anything else so it's the same category as new driver or new
hardware support - more relaxed integration rules. Then we can remove
the KERNEL_DEBUG and KERNEL_EXPERIMENTAL dependency as well.

Ingo

2008-01-22 20:06:34

by Jeremy Fitzhardinge

[permalink] [raw]
Subject: Re: [PATCH 0/4] paravirt_ops-64 compile fixes

Ingo Molnar wrote:
> * Jeremy Fitzhardinge <[email protected]> wrote:
>
>
>>> Maybe should this force-paravirt mode be a kernel debugging option?
>>>
>> No, just a plain CONFIG_PARAVIRT to turn it on and off independently
>> of everything else.
>>
>
> it should be a debugging option in the way that it's dependent on
> KERNEL_DEBUG and KERNEL_EXPERIMENTAL.
>

EXPERIMENTAL certainly, but not DEBUG. Sure, CONFIG_PARAVIRT is
somewhat useless if you don't also configure Xen/lguest/etc, but running
a paravirt system on bare hardware will probably be the most common way
a paravirt kernel gets run (a distro will always enable it and compile
in whatever hypervisor support they want, but most people will end up
just running it on bare hardware). Therefore, we should try to be as
sure as possible that PARAVIRT works well (ie, indistinguishable from
non-PARAVIRT) on bare hardware.

> i dont think you shold worry about this - i think 64-bit guest support
> for specific hypervisors could be added even after the merge window, if
> it's reasonably isolated (which i'd expect it to be). It clearly cannot
> break anything else so it's the same category as new driver or new
> hardware support - more relaxed integration rules. Then we can remove
> the KERNEL_DEBUG and KERNEL_EXPERIMENTAL dependency as well.

Good, I'm glad you feel that way.

J