2012-06-13 12:25:05

by Mahmood Naderan

[permalink] [raw]
Subject: logging on kernel panic

Hi everybody
If a system receives kernel panic during boot, will it be logged somewhere?
In our diskless cluster, clients boot from an image located at /nfsroot
Normally /nfsroot/var/log/syslog should contain messages. However
on a kernel panic, there is no log in /nfsroot/var/log/syslog/


~MN


2012-06-13 12:35:47

by Alan

[permalink] [raw]
Subject: Re: logging on kernel panic

On Wed, 13 Jun 2012 05:18:39 -0700 (PDT)
Mahmood Naderan <[email protected]> wrote:

> Hi everybody
> If a system receives kernel panic during boot, will it be logged somewhere?

Often not.. especially if it is fatal. Look up netconsole however.

2012-06-13 13:38:29

by Cong Wang

[permalink] [raw]
Subject: Re: logging on kernel panic

On Wed, Jun 13, 2012 at 8:18 PM, Mahmood Naderan <[email protected]> wrote:
> Hi everybody
> If a system receives kernel panic during boot, will it be logged somewhere?
> In our diskless cluster, clients boot from an image located at /nfsroot
> Normally /nfsroot/var/log/syslog should contain messages. However
> on a kernel panic, there is no log in /nfsroot/var/log/syslog/
>

The kernel log may not hit /var/log/syslog when kernel panics,
try other more reliable way to get the kernel log, there are some
new ways to store the kernel log at panic/oops, pstore filesystem
is one of them.

As Alan mentioned, netconsole may work too.

2012-06-13 13:54:36

by Jiri Kosina

[permalink] [raw]
Subject: Re: logging on kernel panic

On Wed, 13 Jun 2012, Alan Cox wrote:

> > Hi everybody
> > If a system receives kernel panic during boot, will it be logged somewhere?
>
> Often not.. especially if it is fatal. Look up netconsole however.

Well your machine's BIOS can still be providing non-volatile sotrage which
under recent kernels registers itself with 'pstore', and you can get the
dmesg+panic from there then (provided you have all the necessary bits,
such as APEI, compiled in and enabled).

--
Jiri Kosina
SUSE Labs

2012-06-13 14:53:38

by Mahmood Naderan

[permalink] [raw]
Subject: Re: logging on kernel panic

@jiri:
>and you can get the dmesg+panic from there
?Sorry where? in the bios?

@Cong:
>The kernel log may not hit /var/log/syslog when kernel panics,
Can I ask why. Just before a panic, kernel is still alive and can log.
Isn't that?


// Naderan *Mahmood;


----- Original Message -----
From: Jiri Kosina <[email protected]>
To: Alan Cox <[email protected]>
Cc: Mahmood Naderan <[email protected]>; "[email protected]" <[email protected]>
Sent: Wednesday, June 13, 2012 6:24 PM
Subject: Re: logging on kernel panic

On Wed, 13 Jun 2012, Alan Cox wrote:

> > Hi everybody
> > If a system receives kernel panic during boot, will it be logged somewhere?
>
> Often not.. especially if it is fatal. Look up netconsole however.

Well your machine's BIOS can still be providing non-volatile sotrage which
under recent kernels registers itself with 'pstore', and you can get the
dmesg+panic from there then (provided you have all the necessary bits,
such as APEI, compiled in and enabled).

--
Jiri Kosina
SUSE Labs

2012-06-13 15:15:04

by Mark Lord

[permalink] [raw]
Subject: Re: logging on kernel panic

On 12-06-13 10:52 AM, Mahmood Naderan wrote:
>
>> The kernel log may not hit /var/log/syslog when kernel panics,
> Can I ask why. Just before a panic, kernel is still alive and can log.
> Isn't that?

The kernel writes log messages to a circular in-RAM fifo (/proc/kmsg).
They then later get saved to a file such as /var/log/syslog by some
userspace task, eg. rsyslogd or syslogd.

At the point a Kernel Panic, the kernel is considered "dead",
or "non trustworthy" due to the error condition that triggered the panic.
Attempting to continue after the panic would be unwise,
with the possibilities including opening huge security holes/leaks
and/or corrupting user data.

So it is no longer alive, and will not be scheduling the userspace tasks
that might copy the log to a file for you. Thus, nothing gets saved.


2012-06-14 08:03:08

by Jiri Kosina

[permalink] [raw]
Subject: Re: logging on kernel panic

On Wed, 13 Jun 2012, Mahmood Naderan wrote:

> >and you can get the dmesg+panic from there
> ?Sorry where? in the bios?

Read Documentation/ABI/testing/pstore in the kernel sources.

> @Cong:
> >The kernel log may not hit /var/log/syslog when kernel panics,
> Can I ask why. Just before a panic, kernel is still alive and can log.

At the point of panic, the machine is too dead to be able to perform such
a complex operation as writing to filesystem.

--
Jiri Kosina
SUSE Labs