Hello!
We are running Debian with 2.6.22-2-686-bigmem on Dell Blade 1955 hardware
and get a Kernel Panic with oom + message that there are no processes
left to kill:
http://home.schottelius.org/~nico/unix/linux/oom_no_killable-2.6.22-1.jpeg
Anyone an idea, what's the cause for that? This error happened on two of
those machines,
What I can see in our analysis done with munin is that the number of
open inodes and inode table size decreased within some days from 40k
to next to zero. Munin uses
awk '{print "used.value " $1-$2 "\nmax.value " $1}' < /proc/sys/fs/inode-nr
to log those value (happened on both machines).
Thanks for any hint and CC as usual, please.
Nico
--
Think about Free and Open Source Software (FOSS).
http://nico.schottelius.org/documentations/foss/the-term-foss/
PGP: BFE4 C736 ABE5 406F 8F42 F7CF B8BE F92A 9885 188C
Nico Schottelius wrote:
> Hello!
>
> We are running Debian with 2.6.22-2-686-bigmem on Dell Blade 1955 hardware
> and get a Kernel Panic with oom + message that there are no processes
> left to kill:
>
> http://home.schottelius.org/~nico/unix/linux/oom_no_killable-2.6.22-1.jpeg
>
> Anyone an idea, what's the cause for that? This error happened on two of
> those machines,
>
> What I can see in our analysis done with munin is that the number of
> open inodes and inode table size decreased within some days from 40k
> to next to zero. Munin uses
>
> awk '{print "used.value " $1-$2 "\nmax.value " $1}' < /proc/sys/fs/inode-nr
>
> to log those value (happened on both machines).
>
> Thanks for any hint and CC as usual, please.
How much RAM is in these machines? If you're running tons of memory, it
really is better to run a 64-bit kernel if possible. I believe there are
some cases where low memory can be pretty easily exhausted on machines
with lots of high memory.
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
Hello Robert,
Robert Hancock [Tue, Dec 18, 2007 at 06:43:22PM -0600]:
> How much RAM is in these machines?
8 GiB
> If you're running tons of memory, it
> really is better to run a 64-bit kernel if possible.
Sure? Afaik that results in a bit slower access to memory and appart
from being able to address MUCH more memory doesn't change the
situation.
Perhaps I missed something.
And updating to 64 Bit kernel also implies changing the userland, so
a reinstall would be necessary, wouldn't it?
> I believe there are
> some cases where low memory can be pretty easily exhausted on machines with
> lots of high memory.
Some minutes ago the third machine of that type freezed.
with 7 GiB of *free* memory, before getting oom+panic.
The thing that looks pretty the same is that the inode table / open
inodes shrinked very heavy from 41k to about 2k before the freeze
in about 3 days.
Same kernel, same Dell 1955 hardware, different machine.
Is it possible to debug this issue somehow?
Nico
--
Think about Free and Open Source Software (FOSS).
http://nico.schottelius.org/documentations/foss/the-term-foss/
PGP: BFE4 C736 ABE5 406F 8F42 F7CF B8BE F92A 9885 188C
On Wed, 19 Dec 2007, Nico Schottelius wrote:
>> If you're running tons of memory, it
>> really is better to run a 64-bit kernel if possible.
>
> Sure? Afaik that results in a bit slower access to memory and appart
> from being able to address MUCH more memory doesn't change the
> situation.
No, generally 64-bit x86 does not slow down things - it might speed things
up a bit due to more accessible registers in 64-bit mode. And it does
change the situation from VM PoV a lot. In (bearded) God's own words -
"... run a 64bit kernel and then the kernel isn't trying the software equivalent of
managing a filing cabinet through the keyhole."
You will find this recent thread on same topic useful -
http://lkml.org/lkml/2007/12/17/7 .
> Perhaps I missed something.
>
> And updating to 64 Bit kernel also implies changing the userland, so
> a reinstall would be necessary, wouldn't it?
May not be necessary - I recently tried installing Fedora 8 32-bit/PAE on
a 8Gb Core 2 box and then upgrading just the kernel to 64-bit and it worked
fine except for the 32-bit iptables service having problems running on
64-bit kernel. [Also I hear the closed source nVidia binaries will not
work on this setup - so if you don't need such things you will be able to
go to a 64-bit kernel w/o reinstalling userspace.]
>> I believe there are
>> some cases where low memory can be pretty easily exhausted on machines with
>> lots of high memory.
>
> Some minutes ago the third machine of that type freezed.
> with 7 GiB of *free* memory, before getting oom+panic.
>
> Is it possible to debug this issue somehow?
Like said in the above thread I referred to you might have better luck
with RHEL/CentOS or other "Enterprise" kernels if you want to stick to
32-bit. But the pain is not worthwhile - give 64-bit kernel a shot.
HTH
Parag
On 12/19/2007 06:55 AM, Nico Schottelius wrote:
> And updating to 64 Bit kernel also implies changing the userland, so
> a reinstall would be necessary, wouldn't it?
No, if you enable ia32 support.
On 12/19/2007 09:56 AM, Parag Warudkar wrote:
> running on 64-bit kernel. [Also I hear the closed source nVidia binaries
> will not work on this setup - so if you don't need such things you will
> be able to go to a 64-bit kernel w/o reinstalling userspace.]
False, I'm running 100-* nvidia drivers on such set up system. However some
drivers might be broken due to unimplemented (not filled) compat_ioctl.
