2008-02-03 14:11:39

by Jan Engelhardt

[permalink] [raw]
Subject: Commit f06e4ec breaks vmware

Hi,


sad to say, but f06e4ec... breaks booting the kernel in vmware
(bisected). Booting just stops after

Checking for 'hlt' instruction...

commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
Author: Ingo Molnar <[email protected]>
Date: Wed Jan 30 13:32:39 2008 +0100

x86: read_tsc sync

make native_read_tsc() always non-speculative.


Any debugging hintss?


thanks,
Jan


2008-02-04 04:19:20

by Petr Vandrovec

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware

Jan Engelhardt wrote:
> Hi,
>
>
> sad to say, but f06e4ec... breaks booting the kernel in vmware
> (bisected). Booting just stops after
>
> Checking for 'hlt' instruction...
>
> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> Author: Ingo Molnar <[email protected]>
> Date: Wed Jan 30 13:32:39 2008 +0100
>
> x86: read_tsc sync
>
> make native_read_tsc() always non-speculative.
>
>
> Any debugging hintss?

It does not make lot of sense... What is processor (AMD/Intel, VT/SVM,
...), kernel (32/64), and product?

Can you try adding

monitor_control.disable_rdtscopt_bt="TRUE"

or

monitor_control.virtual_rdtsc="FALSE"

to your configuration file?
Thanks,
Petr

2008-02-04 11:52:17

by Ingo Molnar

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


* Jan Engelhardt <[email protected]> wrote:

> sad to say, but f06e4ec... breaks booting the kernel in vmware
> (bisected). Booting just stops after
>
> Checking for 'hlt' instruction...
>
> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> Author: Ingo Molnar <[email protected]>
> Date: Wed Jan 30 13:32:39 2008 +0100
>
> x86: read_tsc sync
>
> make native_read_tsc() always non-speculative.

could you send us the /proc/cpuinfo data of the guest CPU?

if latest -git still fails, does the patch below solve the problem?

Ingo

---
include/asm-x86/system.h | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)

Index: linux-x86.q/include/asm-x86/system.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/system.h
+++ linux-x86.q/include/asm-x86/system.h
@@ -407,8 +407,7 @@ void default_idle(void);
*/
static inline void rdtsc_barrier(void)
{
- alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
- alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
+ smp_mb();
}

#endif

2008-02-04 14:02:21

by Jeff Chua

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware

On Feb 4, 2008 7:51 PM, Ingo Molnar <[email protected]> wrote:

> > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71

I had the same problem. But I bisect down to a earlier commit.
Reverting this patch, and I can boot up using vmware.

commit 8d947344c47a40626730bb80d136d8daac9f2060
Author: Glauber de Oliveira Costa <[email protected]>
Date: Wed Jan 30 13:31:12 2008 +0100

x86: change write_idt_entry signature


> could you send us the /proc/cpuinfo data of the guest CPU?

Here's my guest's /proc/cpuinfo

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 14
model name : Intel(R) Core(TM) Duo CPU L2500 @ 1.83GHz
stepping : 8
cpu MHz : 1828.645
cache size : 2048 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss constant_tsc up
arch_perfmon pebs bts pni
bogomips : 3546.54
clflush size : 64


> if latest -git still fails, does the patch below solve the problem?

Doesn't seem to work for me.


Thanks,
Jeff.

2008-02-04 14:48:48

by Ingo Molnar

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


* Jeff Chua <[email protected]> wrote:

> On Feb 4, 2008 7:51 PM, Ingo Molnar <[email protected]> wrote:
>
> > > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
>
> I had the same problem. But I bisect down to a earlier commit.
> Reverting this patch, and I can boot up using vmware.
>
> commit 8d947344c47a40626730bb80d136d8daac9f2060
> Author: Glauber de Oliveira Costa <[email protected]>
> Date: Wed Jan 30 13:31:12 2008 +0100
>
> x86: change write_idt_entry signature

32-bit or 64-bit guest kernel?

Ingo

2008-02-04 14:53:35

by Ingo Molnar

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


* Jeff Chua <[email protected]> wrote:

> On Feb 4, 2008 7:51 PM, Ingo Molnar <[email protected]> wrote:
>
> > > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
>
> I had the same problem. But I bisect down to a earlier commit.
> Reverting this patch, and I can boot up using vmware.
>
> commit 8d947344c47a40626730bb80d136d8daac9f2060
> Author: Glauber de Oliveira Costa <[email protected]>
> Date: Wed Jan 30 13:31:12 2008 +0100
>
> x86: change write_idt_entry signature

does the patch below ontop of x86.git#mm fix this?

Ingo

---
arch/x86/kernel/vmi_32.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-x86.q/arch/x86/kernel/vmi_32.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/vmi_32.c
+++ linux-x86.q/arch/x86/kernel/vmi_32.c
@@ -220,21 +220,21 @@ static void vmi_set_tr(void)
static void vmi_write_idt_entry(gate_desc *dt, int entry, const gate_desc *g)
{
u32 *idt_entry = (u32 *)g;
- vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[2]);
+ vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[1]);
}

static void vmi_write_gdt_entry(struct desc_struct *dt, int entry,
const void *desc, int type)
{
u32 *gdt_entry = (u32 *)desc;
- vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[2]);
+ vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[1]);
}

static void vmi_write_ldt_entry(struct desc_struct *dt, int entry,
const void *desc)
{
u32 *ldt_entry = (u32 *)desc;
- vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[2]);
+ vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[1]);
}

static void vmi_load_sp0(struct tss_struct *tss,

2008-02-04 14:58:21

by Jan Engelhardt

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


On Feb 3 2008 20:19, Petr Vandrovec wrote:
>>
>> sad to say, but f06e4ec... breaks booting the kernel in vmware (bisected).
>> Booting just stops after
>>
>> Checking for 'hlt' instruction...
>>
>> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
>> Author: Ingo Molnar <[email protected]>
>> Date: Wed Jan 30 13:32:39 2008 +0100
>>
>> x86: read_tsc sync
>>
>> make native_read_tsc() always non-speculative.
>>
>> Any debugging hints?
>
> It does not make lot of sense... What is processor (AMD/Intel, VT/SVM, ...),
> kernel (32/64), and product?

processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 8
model name : AMD Athlon(tm) XP 2000+
stepping : 0
cpu MHz : 1666.779
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow up ts
bogomips : 3334.92
clflush size : 32

No hardware virtualization.
Host: 32-bit 2.6.23.14, VMware Workstation 6.0.2-b59824.
Guest: 32-bit f06e4ec...


> Can you try adding
>
> monitor_control.disable_rdtscopt_bt="TRUE"

Nope.

> monitor_control.virtual_rdtsc="FALSE"

No change either.


But booting the guest kernel with 'notsc' works around the issue.


On Feb 4 2008 12:51, Ingo Molnar wrote:
>
>if latest -git still fails, does the patch below solve the problem?

Latest git (9135f1901ee6449dfe338adf6e40e9c2025b8150) works, as does the
patch on top of f06e4ec1c15691b0cfd2397ae32214fa36c90d71.


>@@ -407,8 +407,7 @@ void default_idle(void);
> */
> static inline void rdtsc_barrier(void)
> {
>- alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
>- alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
>+ smp_mb();
> }
>
> #endif
>

2008-02-04 15:30:30

by Jeff Chua

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware

On Feb 4, 2008 10:53 PM, Ingo Molnar <[email protected]> wrote:
> > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > Author: Glauber de Oliveira Costa <[email protected]>
> > Date: Wed Jan 30 13:31:12 2008 +0100
> >
> > x86: change write_idt_entry signature
>
> does the patch below ontop of x86.git#mm fix this?


> 32-bit or 64-bit guest kernel?

32-bit.

Yep, this fixed the problem.

You're amazingly fast!

Thank you,
Jeff.

2008-02-04 15:36:54

by Ingo Molnar

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


* Jeff Chua <[email protected]> wrote:

> On Feb 4, 2008 10:53 PM, Ingo Molnar <[email protected]> wrote:
> > > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > > Author: Glauber de Oliveira Costa <[email protected]>
> > > Date: Wed Jan 30 13:31:12 2008 +0100
> > >
> > > x86: change write_idt_entry signature
> >
> > does the patch below ontop of x86.git#mm fix this?
>
>
> > 32-bit or 64-bit guest kernel?
>
> 32-bit.
>
> Yep, this fixed the problem.

great! I've added:

Tested-by: Jeff Chua <[email protected]>

to the commit message as well, if you dont mind. Full patch is below.

> You're amazingly fast!

you did all the hard work by bisecting it down so fast - fixing it was
easy :)

Ingo

----------------------->
Subject: x86: VMI fix
From: Ingo Molnar <[email protected]>

Jeff Chua bisected down a vmware guest boot breakage (hang) to
this paravirt change:

commit 8d947344c47a40626730bb80d136d8daac9f2060
Author: Glauber de Oliveira Costa <[email protected]>
Date: Wed Jan 30 13:31:12 2008 +0100

x86: change write_idt_entry signature

fix the off-by-one indexing bug ...

Bisected-by: Jeff Chua <[email protected]>
Tested-by: Jeff Chua <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/kernel/vmi_32.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-x86.q/arch/x86/kernel/vmi_32.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/vmi_32.c
+++ linux-x86.q/arch/x86/kernel/vmi_32.c
@@ -220,21 +220,21 @@ static void vmi_set_tr(void)
static void vmi_write_idt_entry(gate_desc *dt, int entry, const gate_desc *g)
{
u32 *idt_entry = (u32 *)g;
- vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[2]);
+ vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[1]);
}

static void vmi_write_gdt_entry(struct desc_struct *dt, int entry,
const void *desc, int type)
{
u32 *gdt_entry = (u32 *)desc;
- vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[2]);
+ vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[1]);
}

static void vmi_write_ldt_entry(struct desc_struct *dt, int entry,
const void *desc)
{
u32 *ldt_entry = (u32 *)desc;
- vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[2]);
+ vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[1]);
}

static void vmi_load_sp0(struct tss_struct *tss,

2008-02-04 15:48:39

by Jeff Chua

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware

On Feb 4, 2008 11:36 PM, Ingo Molnar <[email protected]> wrote:
> great! I've added:
> you did all the hard work by bisecting it down so fast - fixing it was
> easy :)

Ingo,

Took me the whole of Friday night. I thought it was just me and my
vmware, so I didn't bother reporting until Jan reported it. Worried
that someone might just shoot me down for reporting a vmware problem
as it booted fine with kvm, and on X60s.

Thanks again for fixing it. Now I can test latest linux git version on
vmware again (host is running linux as well).

Jeff.

2008-02-05 01:28:17

by Zachary Amsden

[permalink] [raw]
Subject: Re: Commit f06e4ec breaks vmware


On Mon, 2008-02-04 at 16:36 +0100, Ingo Molnar wrote:
> * Jeff Chua <[email protected]> wrote:
>
> > On Feb 4, 2008 10:53 PM, Ingo Molnar <[email protected]> wrote:
> > > > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > > > Author: Glauber de Oliveira Costa <[email protected]>
> > > > Date: Wed Jan 30 13:31:12 2008 +0100
> > > >
> > > > x86: change write_idt_entry signature
> > >
> > > does the patch below ontop of x86.git#mm fix this?
> >
> >
> > > 32-bit or 64-bit guest kernel?
> >
> > 32-bit.
> >
> > Yep, this fixed the problem.
>
> great! I've added:
>
> Tested-by: Jeff Chua <[email protected]>
>
> to the commit message as well, if you dont mind. Full patch is below.

Acked-by: Zachary Amsden <[email protected]>

Thanks, Ingo!