2005-05-20 17:31:52

by Dimitris Zilaskos

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82


Unfortunately , it seems our conclusions were premature:

ksymoops 2.4.11 on i686 2.4.30. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.30/ (default)
-m /usr/src/linux/System.map (default)

Warning: You did not tell me where to find symbol information. I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc. ksymoops -h explains the options.

Warning (compare_maps): libafs-2.4.30.mp symbol kallsyms_address_to_symbol not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol kallsyms_symbol_to_address not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_chdir not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_exit not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_ioctl not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_open not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_wait4 not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
Warning (compare_maps): libafs-2.4.30.mp symbol sys_write not found in /usr/local/lib/openafs/libafs-2.4.30.mp.o. Ignoring /usr/local/lib/openafs/libafs-2.4.30.mp.o entry
May 20 15:31:15 system kernel: dcache hc<1>Unable to handle kernel paging request at virtual address ffffffff
May 20 15:31:15 system kernel: f09de170
May 20 15:31:15 system kernel: *pde = 00004063
May 20 15:31:15 system kernel: Oops: 0002
May 20 15:31:15 system kernel: CPU: 1
May 20 15:31:15 system kernel: EIP: 0010:[<f09de170>] Tainted: P
Using defaults from ksymoops -t elf32-i386 -a i386
May 20 15:31:15 system kernel: EFLAGS: 00010282
May 20 15:31:15 system kernel: eax: 00000009 ebx: 000a93f4 ecx: 00000082 edx: ef0c7f84
May 20 15:31:15 system kernel: esi: f1e0f5f8 edi: f0ae2000 ebp: 00000001 esp: c8d15dd4
May 20 15:31:15 system kernel: ds: 0018 es: 0018 ss: 0018
May 20 15:31:15 system kernel: Process afs_cachetrim (pid: 938, stackpage=c8d15000)
May 20 15:31:15 system kernel: Stack: f0a03fe1 f099fc43 00000010 f1e0f66c f1e0f5f8 0000000f 00000001 f099fc0f
May 20 15:31:15 system kernel: f0a03fe1 f099fc43 00000010 f1e0f66c 00528f57 f1e0f5f8 0000000f f099f897
May 20 15:31:15 system kernel: f1e0f5f8 00000000 00000000 000003aa ecf4a7f0 00000000 00000000 c8d14000
May 20 15:31:15 system kernel: Call Trace: [<f0a03fe1>] [<f099fc43>] [<f099fc0f>] [<f0a03fe1>] [<f099fc43>]
May 20 15:31:15 system kernel: [<f099f897>] [<f099ef5e>] [<f0a0caa0>] [<f0a0caa0>] [<f09efe39>] [<f0a0509c>]
May 20 15:31:15 system kernel: [<c010740e>] [<f09efad0>]
May 20 15:31:15 system kernel: Code: c6 05 ff ff ff ff 2a 83 c4 1c c3 90 8d 74 26 00 b8 f9 4a a0


>>EIP; f09de170 <[libafs-2.4.30.mp]osi_Panic+20/40> <=====

>>edx; ef0c7f84 <_end+2ec5fb44/3042bc20>
>>esp; c8d15dd4 <_end+88ad994/3042bc20>

Trace; f0a03fe1 <[libafs-2.4.30.mp].rodata.end+60e6/e545>
Trace; f099fc43 <[libafs-2.4.30.mp]afs_HashOutDCache+b3/120>
Trace; f099fc0f <[libafs-2.4.30.mp]afs_HashOutDCache+7f/120>
Trace; f0a03fe1 <[libafs-2.4.30.mp].rodata.end+60e6/e545>
Trace; f099fc43 <[libafs-2.4.30.mp]afs_HashOutDCache+b3/120>
Trace; f099f897 <[libafs-2.4.30.mp]afs_GetDownD+567/860>
Trace; f099ef5e <[libafs-2.4.30.mp]afs_CacheTruncateDaemon+10e/460>
Trace; f0a0caa0 <[libafs-2.4.30.mp]afs_global_lock+0/20>
Trace; f0a0caa0 <[libafs-2.4.30.mp]afs_global_lock+0/20>
Trace; f09efe39 <[libafs-2.4.30.mp]afsd_thread+369/5d0>
Trace; f0a0509c <[libafs-2.4.30.mp].rodata.end+71a1/e545>
Trace; c010740e <arch_kernel_thread+2e/40>
Trace; f09efad0 <[libafs-2.4.30.mp]afsd_thread+0/5d0>

Code; f09de170 <[libafs-2.4.30.mp]osi_Panic+20/40>
00000000 <_EIP>:
Code; f09de170 <[libafs-2.4.30.mp]osi_Panic+20/40> <=====
0: c6 05 ff ff ff ff 2a movb $0x2a,0xffffffff <=====
Code; f09de177 <[libafs-2.4.30.mp]osi_Panic+27/40>
7: 83 c4 1c add $0x1c,%esp
Code; f09de17a <[libafs-2.4.30.mp]osi_Panic+2a/40>
a: c3 ret
Code; f09de17b <[libafs-2.4.30.mp]osi_Panic+2b/40>
b: 90 nop
Code; f09de17c <[libafs-2.4.30.mp]osi_Panic+2c/40>
c: 8d 74 26 00 lea 0x0(%esi),%esi
Code; f09de180 <[libafs-2.4.30.mp]osi_Panic+30/40>
10: b8 f9 4a a0 00 mov $0xa04af9,%eax


9 warnings issued. Results may not be reliable.


--
=============================================================================

Dimitris Zilaskos

Department of Physics @ Aristotle University of Thessaloniki , Greece
PGP key : http://tassadar.physics.auth.gr/~dzila/pgp_public_key.asc
http://egnatia.ee.auth.gr/~dzila/pgp_public_key.asc
MD5sum : de2bd8f73d545f0e4caf3096894ad83f pgp_public_key.asc
=============================================================================


2005-05-20 18:06:37

by ted creedon

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82

What processor chip are you running on? A 686 has one different instruction
than a 586 does.

Running a 2.6 kernel here.
tedc

2005-05-20 18:20:38

by ted creedon

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82

If a new kernel was built, what was it named?

Looks like the existing kernel was overwritten. Normally the
CONIFG_LOCALVERSION is changed to differentiate from a working kernel. (I.e.
if the new kernel fails one can reboot with an existing stable kernel).
tedc

2005-05-20 18:53:41

by Dimitris Zilaskos

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82



> What processor chip are you running on? A 686 has one different instruction
> than a 586 does.

dual Pentium III (Coppermine) @ 600 Mhz


--
=============================================================================

Dimitris Zilaskos

Department of Physics @ Aristotle University of Thessaloniki , Greece
PGP key : http://tassadar.physics.auth.gr/~dzila/pgp_public_key.asc
http://egnatia.ee.auth.gr/~dzila/pgp_public_key.asc
MD5sum : de2bd8f73d545f0e4caf3096894ad83f pgp_public_key.asc
=============================================================================

2005-05-20 19:12:15

by ted creedon

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82

Gcc -dumpmachine #should prints out i586-suse-linux for a P III here.

I'd try a fresh single processor machine and force a 2.6 kernel, module and
afs recompile for a i586.

SuSE 9.3 costs $90 and it solved a similar problem noted in the mailings. In
fact the YasT installed openafs binaries ran fine.

The ksymoops man page has a script to tail -f /var/log/messages|ksymoops1
explained.

Are you sure there isn't a memory problem? I'm running out of ideas.
tedc

2005-05-20 19:48:04

by ted creedon

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82

/proc/cpuinfo shows that the cmov instruction is there for Durons, PIII's
and P4's.

Its not for the Via C3 processors which you don't have.

tedc

2005-05-20 19:53:43

by Jeffrey Hutzelman

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82



On Friday, May 20, 2005 12:12:05 PM -0700 ted creedon
<[email protected]> wrote:

> Gcc -dumpmachine #should prints out i586-suse-linux for a P III here.
>
> I'd try a fresh single processor machine and force a 2.6 kernel, module
> and afs recompile for a i586.
>
> SuSE 9.3 costs $90 and it solved a similar problem noted in the mailings.
> In fact the YasT installed openafs binaries ran fine.
>
> The ksymoops man page has a script to tail -f /var/log/messages|ksymoops1
> explained.
>
> Are you sure there isn't a memory problem? I'm running out of ideas.
> tedc

You're really trying too hard. The oops in question clearly shows EIP in
osi_Panic(), which is _supposed_ to result in a crash. So, stop trying to
analyze the mechanism by which osi_Panic forces a fault, and instead go
look at your logs and tell us what the panic string was.

2005-05-20 20:27:18

by Dimitris Zilaskos

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82


> If a new kernel was built, what was it named?

Linux system 2.4.30 #2 SMP Sat May 7 14:54:35 EEST 2005 i686 unknown
unknown GNU/Linux

> Looks like the existing kernel was overwritten. Normally the
> CONIFG_LOCALVERSION is changed to differentiate from a working kernel. (I.e.
> if the new kernel fails one can reboot with an existing stable kernel).
> tedc
>

that does not seem to exist in my 2.4 kernel...



--
=============================================================================

Dimitris Zilaskos

Department of Physics @ Aristotle University of Thessaloniki , Greece
PGP key : http://tassadar.physics.auth.gr/~dzila/pgp_public_key.asc
http://egnatia.ee.auth.gr/~dzila/pgp_public_key.asc
MD5sum : de2bd8f73d545f0e4caf3096894ad83f pgp_public_key.asc
=============================================================================

2005-05-20 20:34:48

by Dimitris Zilaskos

[permalink] [raw]
Subject: RE: [OpenAFS] Re: Openafs 1.3.78 and kernel 2.4.29 oopses , same for 2.4.30 and openafs 1.3.82


> Gcc -dumpmachine #should prints out i586-suse-linux for a P III here.

here it prints: i486-slackware-linux
>
> I'd try a fresh single processor machine and force a 2.6 kernel, module and
> afs recompile for a i586.
>
> SuSE 9.3 costs $90 and it solved a similar problem noted in the mailings. In
> fact the YasT installed openafs binaries ran fine.

Do you have any pointers to Suse's solution and source code?I
could check what they did and try it...

> The ksymoops man page has a script to tail -f /var/log/messages|ksymoops1
> explained.

I will try that , thnx :)

> Are you sure there isn't a memory problem? I'm running out of ideas.
> tedc

It's been a while since I memtested the box , I will try that also
when I return at the office ( in about a week).



--
=============================================================================

Dimitris Zilaskos

Department of Physics @ Aristotle University of Thessaloniki , Greece
PGP key : http://tassadar.physics.auth.gr/~dzila/pgp_public_key.asc
http://egnatia.ee.auth.gr/~dzila/pgp_public_key.asc
MD5sum : de2bd8f73d545f0e4caf3096894ad83f pgp_public_key.asc
=============================================================================