2008-11-23 20:59:58

by Arjan van de Ven

[permalink] [raw]
Subject: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

>From 57b0deb1ad706a94e3118baee4127676f465c4ab Mon Sep 17 00:00:00 2001
From: Arjan van de Ven <[email protected]>
Date: Wed, 5 Nov 2008 19:00:36 -0800
Subject: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

We're strugling all the time to figure out where the code came from
that oopsed.. The script below (a adaption from a script used by kerneloops.org)
can help developers quite a bit, at least for non-module cases.

It works and looks like this:

[/home/arjan/linux]$ dmesg | perl scripts/markup_oops.pl vmlinux
{
struct agp_memory *memory;

memory = agp_allocate_memory(agp_bridge, pg_count, type);
c055c10f: 89 c2 mov %eax,%edx
if (memory == NULL)
c055c111: 74 19 je c055c12c <agp_allocate_memory_wrap+0x30>
/* This function must only be called when current_controller != NULL */
static void agp_insert_into_pool(struct agp_memory * temp)
{
struct agp_memory *prev;

prev = agp_fe.current_controller->pool;
c055c113: a1 ec dc 8f c0 mov 0xc08fdcec,%eax
*c055c118: 8b 40 10 mov 0x10(%eax),%eax <----- faulting instruction

if (prev != NULL) {
c055c11b: 85 c0 test %eax,%eax
c055c11d: 74 05 je c055c124 <agp_allocate_memory_wrap+0x28>
prev->prev = temp;
c055c11f: 89 50 04 mov %edx,0x4(%eax)
temp->next = prev;
c055c122: 89 02 mov %eax,(%edx)
}
agp_fe.current_controller->pool = temp;
c055c124: a1 ec dc 8f c0 mov 0xc08fdcec,%eax
c055c129: 89 50 10 mov %edx,0x10(%eax)
if (memory == NULL)
return NULL;

agp_insert_into_pool(memory);

so in this case, we faulted while dereferencing agp_fe.current_controller pointer, and we get to see
exactly which function and line it affects... Personally I find this very useful, and I can
see value for having this script in the kernel for more-than-just-me to use.

Caveats:
* It only works for oopses not-in-modules
* It only works nicely for kernels compiled with CONFIG_DEBUG_INFO
* It's not very fast.
* It only works on x86

Signed-off-by: Arjan van de Ven <[email protected]>
---
scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 162 insertions(+), 0 deletions(-)
create mode 100644 scripts/markup_oops.pl

diff --git a/scripts/markup_oops.pl b/scripts/markup_oops.pl
new file mode 100644
index 0000000..700a7a6
--- /dev/null
+++ b/scripts/markup_oops.pl
@@ -0,0 +1,162 @@
+#!/usr/bin/perl -w
+
+# Copyright 2008, Intel Corporation
+#
+# This file is part of the Linux kernel
+#
+# This program file is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; version 2 of the License.
+#
+# Authors:
+# Arjan van de Ven <[email protected]>
+
+
+my $vmlinux_name = $ARGV[0];
+
+#
+# Step 1: Parse the oops to find the EIP value
+#
+
+my $target = "0";
+while (<STDIN>) {
+ if ($_ =~ /EIP: 0060:\[\<([a-z0-9]+)\>\]/) {
+ $target = $1;
+ }
+}
+
+if ($target =~ /^f8/) {
+ print "This script does not work on modules ... \n";
+ exit;
+}
+
+if ($target eq "0") {
+ print "No oops found!\n";
+ print "Usage: \n";
+ print " dmesg | perl scripts/markup_oops.pl vmlinux\n";
+ exit;
+}
+
+my $counter = 0;
+my $state = 0;
+my $center = 0;
+my @lines;
+
+sub InRange {
+ my ($address, $target) = @_;
+ my $ad = "0x".$address;
+ my $ta = "0x".$target;
+ my $delta = hex($ad) - hex($ta);
+
+ if (($delta > -4096) && ($delta < 4096)) {
+ return 1;
+ }
+ return 0;
+}
+
+
+
+# first, parse the input into the lines array, but to keep size down,
+# we only do this for 4Kb around the sweet spot
+
+my $filename;
+
+open(FILE, "objdump -dS $vmlinux_name |") || die "Cannot start objdump";
+
+while (<FILE>) {
+ my $line = $_;
+ chomp($line);
+ if ($state == 0) {
+ if ($line =~ /^([a-f0-9]+)\:/) {
+ if (InRange($1, $target)) {
+ $state = 1;
+ }
+ }
+ } else {
+ if ($line =~ /^([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+)\:/) {
+ my $val = $1;
+ if (!InRange($val, $target)) {
+ last;
+ }
+ if ($val eq $target) {
+ $center = $counter;
+ }
+ }
+ $lines[$counter] = $line;
+
+ $counter = $counter + 1;
+ }
+}
+
+close(FILE);
+
+if ($counter == 0) {
+ print "No matching code found \n";
+ exit;
+}
+
+if ($center == 0) {
+ print "No matching code found \n";
+ exit;
+}
+
+my $start;
+my $finish;
+my $codelines = 0;
+my $binarylines = 0;
+# now we go up and down in the array to find how much we want to print
+
+$start = $center;
+
+while ($start > 1) {
+ $start = $start - 1;
+ my $line = $lines[$start];
+ if ($line =~ /^([a-f0-9]+)\:/) {
+ $binarylines = $binarylines + 1;
+ } else {
+ $codelines = $codelines + 1;
+ }
+ if ($codelines > 10) {
+ last;
+ }
+ if ($binarylines > 20) {
+ last;
+ }
+}
+
+
+$finish = $center;
+$codelines = 0;
+$binarylines = 0;
+while ($finish < $counter) {
+ $finish = $finish + 1;
+ my $line = $lines[$finish];
+ if ($line =~ /^([a-f0-9]+)\:/) {
+ $binarylines = $binarylines + 1;
+ } else {
+ $codelines = $codelines + 1;
+ }
+ if ($codelines > 10) {
+ last;
+ }
+ if ($binarylines > 20) {
+ last;
+ }
+}
+
+
+my $i;
+
+my $fulltext = "";
+$i = $start;
+while ($i < $finish) {
+ if ($i == $center) {
+ $fulltext = $fulltext . "*$lines[$i] <----- faulting instruction\n";
+ } else {
+ $fulltext = $fulltext . " $lines[$i]\n";
+ }
+ $i = $i +1;
+}
+
+print $fulltext;
+
--
1.6.0.3



--
Arjan van de Ven Intel Open Source Technology Centre
For development, discussion and tips for power savings,
visit http://www.lesswatts.org


2008-11-26 07:25:25

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps


* Arjan van de Ven <[email protected]> wrote:

> From 57b0deb1ad706a94e3118baee4127676f465c4ab Mon Sep 17 00:00:00 2001
> From: Arjan van de Ven <[email protected]>
> Date: Wed, 5 Nov 2008 19:00:36 -0800
> Subject: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps
>
> We're strugling all the time to figure out where the code came from
> that oopsed.. The script below (a adaption from a script used by kerneloops.org)
> can help developers quite a bit, at least for non-module cases.
>
> It works and looks like this:
>
> [/home/arjan/linux]$ dmesg | perl scripts/markup_oops.pl vmlinux
> {
> struct agp_memory *memory;
>
> memory = agp_allocate_memory(agp_bridge, pg_count, type);
> c055c10f: 89 c2 mov %eax,%edx
> if (memory == NULL)
> c055c111: 74 19 je c055c12c <agp_allocate_memory_wrap+0x30>
> /* This function must only be called when current_controller != NULL */
> static void agp_insert_into_pool(struct agp_memory * temp)
> {
> struct agp_memory *prev;
>
> prev = agp_fe.current_controller->pool;
> c055c113: a1 ec dc 8f c0 mov 0xc08fdcec,%eax
> *c055c118: 8b 40 10 mov 0x10(%eax),%eax <----- faulting instruction
>
> if (prev != NULL) {
> c055c11b: 85 c0 test %eax,%eax
> c055c11d: 74 05 je c055c124 <agp_allocate_memory_wrap+0x28>
> prev->prev = temp;
> c055c11f: 89 50 04 mov %edx,0x4(%eax)
> temp->next = prev;
> c055c122: 89 02 mov %eax,(%edx)
> }
> agp_fe.current_controller->pool = temp;
> c055c124: a1 ec dc 8f c0 mov 0xc08fdcec,%eax
> c055c129: 89 50 10 mov %edx,0x10(%eax)
> if (memory == NULL)
> return NULL;
>
> agp_insert_into_pool(memory);
>
> so in this case, we faulted while dereferencing agp_fe.current_controller pointer, and we get to see
> exactly which function and line it affects... Personally I find this very useful, and I can
> see value for having this script in the kernel for more-than-just-me to use.
>
> Caveats:
> * It only works for oopses not-in-modules
> * It only works nicely for kernels compiled with CONFIG_DEBUG_INFO
> * It's not very fast.
> * It only works on x86
>
> Signed-off-by: Arjan van de Ven <[email protected]>
> ---
> scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 files changed, 162 insertions(+), 0 deletions(-)
> create mode 100644 scripts/markup_oops.pl

looks useful - Andrew, Sam, any objections to this?

Ingo

2008-11-26 08:13:49

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

On Wed, 26 Nov 2008 08:24:45 +0100 Ingo Molnar <[email protected]> wrote:

> > scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 files changed, 162 insertions(+), 0 deletions(-)
> > create mode 100644 scripts/markup_oops.pl
>
> looks useful - Andrew, Sam, any objections to this?

who-me-no-i-already-merged-it.

I didn't test it though - I don't have particularly high hopes given
the general deadness of `make mm/memory.lst' for the past numerous
years.

But it's worth trying. The challenge will be getting people to
remember that it exists, and to use it.

I often turn off CONFIG_DEUG_INFO because it slows builds down so much.
It is irritating when such a kernel goes splat.

2008-11-26 08:18:56

by Vegard Nossum

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

On Sun, Nov 23, 2008 at 10:00 PM, Arjan van de Ven <[email protected]> wrote:
> From 57b0deb1ad706a94e3118baee4127676f465c4ab Mon Sep 17 00:00:00 2001
> From: Arjan van de Ven <[email protected]>
> Date: Wed, 5 Nov 2008 19:00:36 -0800
> Subject: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps
>
> We're strugling all the time to figure out where the code came from
> that oopsed.. The script below (a adaption from a script used by kerneloops.org)
> can help developers quite a bit, at least for non-module cases.

Hi,

Just small style nitpicks below :-) (You pick which ones to heed.)


> ---
> scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 files changed, 162 insertions(+), 0 deletions(-)
> create mode 100644 scripts/markup_oops.pl
>
> diff --git a/scripts/markup_oops.pl b/scripts/markup_oops.pl
> new file mode 100644
> index 0000000..700a7a6
> --- /dev/null
> +++ b/scripts/markup_oops.pl
> @@ -0,0 +1,162 @@
> +#!/usr/bin/perl -w
> +
> +# Copyright 2008, Intel Corporation
> +#
> +# This file is part of the Linux kernel
> +#
> +# This program file is free software; you can redistribute it and/or modify it
> +# under the terms of the GNU General Public License as published by the
> +# Free Software Foundation; version 2 of the License.
> +#
> +# Authors:
> +# Arjan van de Ven <[email protected]>

use strict;
use warnings;

:-)

> +
> +
> +my $vmlinux_name = $ARGV[0];

I think the idiomatic way would be:

my $vmlinux_name = shift || 'vmlinux';

> +
> +#
> +# Step 1: Parse the oops to find the EIP value
> +#
> +
> +my $target = "0";
> +while (<STDIN>) {
> + if ($_ =~ /EIP: 0060:\[\<([a-z0-9]+)\>\]/) {

if (m/EIP:.../) {

is more elegant :-)

> + $target = $1;
> + }
> +}
> +
> +if ($target =~ /^f8/) {
> + print "This script does not work on modules ... \n";
> + exit;
> +}
> +
> +if ($target eq "0") {
> + print "No oops found!\n";
> + print "Usage: \n";
> + print " dmesg | perl scripts/markup_oops.pl vmlinux\n";
> + exit;
> +}
> +
> +my $counter = 0;
> +my $state = 0;
> +my $center = 0;
> +my @lines;
> +
> +sub InRange {
> + my ($address, $target) = @_;
> + my $ad = "0x".$address;
> + my $ta = "0x".$target;
> + my $delta = hex($ad) - hex($ta);
> +
> + if (($delta > -4096) && ($delta < 4096)) {
> + return 1;
> + }
> + return 0;

or just return ($delta > -4096) && ($delta < 4096);

> +}
> +
> +
> +
> +# first, parse the input into the lines array, but to keep size down,
> +# we only do this for 4Kb around the sweet spot
> +
> +my $filename;
> +
> +open(FILE, "objdump -dS $vmlinux_name |") || die "Cannot start objdump";

use open(my $fd, ...); for running with strict, and replace FILE with
$fd everywhere else

> +
> +while (<FILE>) {
> + my $line = $_;

Then you might as well do:

for my $line (<FILE>) {

> + chomp($line);
> + if ($state == 0) {
> + if ($line =~ /^([a-f0-9]+)\:/) {
> + if (InRange($1, $target)) {
> + $state = 1;
> + }
> + }
> + } else {
> + if ($line =~ /^([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+)\:/) {
> + my $val = $1;
> + if (!InRange($val, $target)) {
> + last;
> + }

last unless InRange($val, $target);

> + if ($val eq $target) {
> + $center = $counter;
> + }

$center = $counter if $val eq $target;

> + }
> + $lines[$counter] = $line;
> +
> + $counter = $counter + 1;
> + }
> +}
> +
> +close(FILE);
> +
> +if ($counter == 0) {
> + print "No matching code found \n";

Trailing space?

> + exit;
> +}
> +
> +if ($center == 0) {
> + print "No matching code found \n";

Also here.

> + exit;
> +}
> +
> +my $start;
> +my $finish;
> +my $codelines = 0;
> +my $binarylines = 0;
> +# now we go up and down in the array to find how much we want to print
> +
> +$start = $center;
> +
> +while ($start > 1) {
> + $start = $start - 1;
> + my $line = $lines[$start];
> + if ($line =~ /^([a-f0-9]+)\:/) {
> + $binarylines = $binarylines + 1;
> + } else {
> + $codelines = $codelines + 1;
> + }
> + if ($codelines > 10) {
> + last;
> + }
> + if ($binarylines > 20) {
> + last;
> + }

last if $codelines > 10 || $binarylines > 20;

> +}
> +
> +
> +$finish = $center;
> +$codelines = 0;
> +$binarylines = 0;
> +while ($finish < $counter) {
> + $finish = $finish + 1;
> + my $line = $lines[$finish];
> + if ($line =~ /^([a-f0-9]+)\:/) {
> + $binarylines = $binarylines + 1;
> + } else {
> + $codelines = $codelines + 1;
> + }
> + if ($codelines > 10) {
> + last;
> + }
> + if ($binarylines > 20) {
> + last;
> + }

last if $codelines > 10 || $binarylines > 20;

> +}
> +
> +
> +my $i;
> +
> +my $fulltext = "";
> +$i = $start;
> +while ($i < $finish) {
> + if ($i == $center) {
> + $fulltext = $fulltext . "*$lines[$i] <----- faulting instruction\n";
> + } else {
> + $fulltext = $fulltext . " $lines[$i]\n";
> + }
> + $i = $i +1;
> +}
> +
> +print $fulltext;
> +
> --


Vegard

--
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
-- E. W. Dijkstra, EWD1036

2008-11-26 08:28:15

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps


* Andrew Morton <[email protected]> wrote:

> On Wed, 26 Nov 2008 08:24:45 +0100 Ingo Molnar <[email protected]> wrote:
>
> > > scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
> > > 1 files changed, 162 insertions(+), 0 deletions(-)
> > > create mode 100644 scripts/markup_oops.pl
> >
> > looks useful - Andrew, Sam, any objections to this?
>
> who-me-no-i-already-merged-it.
>
> I didn't test it though - I don't have particularly high hopes given
> the general deadness of `make mm/memory.lst' for the past numerous
> years.
>
> But it's worth trying. The challenge will be getting people to
> remember that it exists, and to use it.
>
> I often turn off CONFIG_DEUG_INFO because it slows builds down so
> much. It is irritating when such a kernel goes splat.

i keep DEBUG_INFO turned off all the time, and only turn it on if
really needed. But in that case i can just turn it on and rebuild:
recent gccs seem better at keeping the image unmodified. (or at least
so i kid myself)

Years ago the image would mute a bit with DEBUG_INFO enabled, making
the splat info fuzzy. Seems better now.

Ingo

2008-11-26 14:09:32

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

Andrew Morton <[email protected]> writes:

> On Wed, 26 Nov 2008 08:24:45 +0100 Ingo Molnar <[email protected]> wrote:
>
>> > scripts/markup_oops.pl | 162 ++++++++++++++++++++++++++++++++++++++++++++++++
>> > 1 files changed, 162 insertions(+), 0 deletions(-)
>> > create mode 100644 scripts/markup_oops.pl
>>
>> looks useful - Andrew, Sam, any objections to this?
>
> who-me-no-i-already-merged-it.
>
> I didn't test it though - I don't have particularly high hopes given
> the general deadness of `make mm/memory.lst' for the past numerous
> years.

make mm/memory.lst should work again with latest binutils. The problem
earlier was that objdump -S was incompatible with unit-at-a-time
mode in gcc, but I fixed that and the binutils maintainers included
the patch.

-Andi

--
[email protected]

2008-11-26 14:48:30

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] scripts: script from kerneloops.org to pretty print oops dumps

On Wed, 26 Nov 2008 09:18:46 +0100
"Vegard Nossum" <[email protected]> wrote:

> On Sun, Nov 23, 2008 at 10:00 PM, Arjan van de Ven
> <[email protected]> wrote:
> > From 57b0deb1ad706a94e3118baee4127676f465c4ab Mon Sep 17 00:00:00
> > 2001 From: Arjan van de Ven <[email protected]>
> > Date: Wed, 5 Nov 2008 19:00:36 -0800
> > Subject: [PATCH] scripts: script from kerneloops.org to pretty
> > print oops dumps
> >
> > We're strugling all the time to figure out where the code came from
> > that oopsed.. The script below (a adaption from a script used by
> > kerneloops.org) can help developers quite a bit, at least for
> > non-module cases.
>
> Hi,
>
> Just small style nitpicks below :-) (You pick which ones to heed.)

Hi,

I'm sure your nitpicks make the perl script more perl like, and are
good suggestions from that view.
However, I'm not a perl hacker, and neither are most other kernel
hackers, and to be honest, for me, the more a perl script looks like
"bash with regexps" the better, because I can actually read, understand
and debug it in that case ;-)

Greetings,
Arjan van de Ven

--
Arjan van de Ven Intel Open Source Technology Centre
For development, discussion and tips for power savings,
visit http://www.lesswatts.org