Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756831AbZGFO4t (ORCPT ); Mon, 6 Jul 2009 10:56:49 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1755679AbZGFO4g (ORCPT ); Mon, 6 Jul 2009 10:56:36 -0400 Received: from vms173003pub.verizon.net ([206.46.173.3]:51919 "EHLO vms173003pub.verizon.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756127AbZGFO4e convert rfc822-to-8bit (ORCPT ); Mon, 6 Jul 2009 10:56:34 -0400 From: Gene Heskett Organization: Organization? Not detectable To: Linux Kernel list Subject: OOM killer in 2.6.31-rc2 Date: Mon, 06 Jul 2009 10:56:00 -0400 User-Agent: KMail/1.11.4 (Linux/2.6.31-rc2; KDE/4.2.4; i686; ; ) MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 8BIT Content-disposition: inline Message-id: <200907061056.00229.gene.heskett@verizon.net> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 16148 Lines: 253 Greetings all; I had to hard reset this box just now as there was no response to a ctl-alt- bksp when X was un-responsive this morning. I had built a 2nd version of the 2.6.31-rc2 kernel last night when I found the video stuff appeared to have been moved in the .config and my pcHDTV-3000 cards modules were not being built, but are now. That kernel was installed, and if at some time in the night a module was needed, it would have been available, but I can't make a solid connection. This machine will always be marked as 'tainted' because any bios update that fixes the very early boot time oops, also leaves me with a machine that will crash hard in 30 seconds to 3 or 4 hours. The fixes done by the oops make it generally dead stable for weeks. That oops: Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-provided physical RAM map: Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009f000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000000100000 - 00000000dfee0000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfee0000 - 00000000dfee3000 (ACPI NVS) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfee3000 - 00000000dfef0000 (ACPI data) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfef0000 - 00000000dff00000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000f0000000 - 00000000f4000000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000100000000 - 0000000120000000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] DMI 2.4 present. Jul 6 10:03:58 coyote kernel: [ 0.000000] Phoenix BIOS detected: BIOS may corrupt low RAM, working around it. Jul 6 10:03:58 coyote kernel: [ 0.000000] last_pfn = 0x120000 max_arch_pfn = 0x1000000 Jul 6 10:03:58 coyote kernel: [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 Jul 6 10:03:58 coyote kernel: [ 0.000000] ------------[ cut here ]------------ Jul 6 10:03:58 coyote kernel: [ 0.000000] WARNING: at arch/x86/kernel/cpu/mtrr/generic.c:456 generic_get_mtrr+0x12c/0x150() Jul 6 10:03:58 coyote kernel: [ 0.000000] Hardware name: System Product Name Jul 6 10:03:58 coyote kernel: [ 0.000000] mtrr: your BIOS has set up an incorrect mask, fixing it up. Jul 6 10:03:58 coyote kernel: [ 0.000000] Modules linked in: Jul 6 10:03:58 coyote kernel: [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.31-rc2 #2 Jul 6 10:03:58 coyote kernel: [ 0.000000] Call Trace: Jul 6 10:03:58 coyote kernel: [ 0.000000] [] ? generic_get_mtrr+0x12c/0x150 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] warn_slowpath_common+0x7d/0xe0 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] ? generic_get_mtrr+0x12c/0x150 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] warn_slowpath_fmt+0x33/0x50 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] generic_get_mtrr+0x12c/0x150 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] mtrr_trim_uncached_memory+0x85/0x368 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] ? mtrr_bp_init+0x1d9/0x2bb Jul 6 10:03:58 coyote kernel: [ 0.000000] [] setup_arch+0x52c/0xa33 Jul 6 10:03:58 coyote kernel: [ 0.000000] [] ? thermal_get_trip_type+0x0/0x9c Jul 6 10:03:58 coyote kernel: [ 0.000000] [] start_kernel+0xb2/0x38b Jul 6 10:03:58 coyote kernel: [ 0.000000] [] i386_start_kernel+0x84/0xb0 Jul 6 10:03:58 coyote kernel: [ 0.000000] ---[ end trace a7919e7f17c0a725 ]--- Jul 6 10:03:58 coyote kernel: [ 0.000000] Scanning 0 areas for low memory corruption Jul 6 10:03:58 coyote kernel: [ 0.000000] modified physical RAM map: Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000000000 - 0000000000010000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000010000 - 000000000009f000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 000000000009f000 - 00000000000a0000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000000f0000 - 0000000000100000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000100000 - 00000000dfee0000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfee0000 - 00000000dfee3000 (ACPI NVS) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfee3000 - 00000000dfef0000 (ACPI data) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfef0000 - 00000000dff00000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000f0000000 - 00000000f4000000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000fec00000 - 0000000100000000 (reserved) Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000100000000 - 0000000120000000 (usable) Jul 6 10:03:58 coyote kernel: [ 0.000000] init_memory_mapping: 0000000000000000-00000000379fe000 Jul 6 10:03:58 coyote kernel: [ 0.000000] NX (Execute Disable) protection: active You all have seen this one before, several times. I have asked that since its a good fix, that the kernel not be marked tainted in that instance. I would run the asus bios that didn't do that _IF_ it was stable. 2 newer versions are _not_ stable, this is stable after the fix. The machine has 4G of ram & is I believe 'pae' The oom's first stanza: Jul 6 06:45:01 coyote kernel: [78748.106803] X invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0 Jul 6 06:45:01 coyote kernel: [78748.106808] Pid: 3068, comm: X Tainted: G W 2.6.31-rc2 #1 Jul 6 06:45:01 coyote kernel: [78748.106811] Call Trace: Jul 6 06:45:01 coyote kernel: [78748.106818] [] ? printk+0x23/0x40 Jul 6 06:45:01 coyote kernel: [78748.106823] [] oom_kill_process+0x178/0x270 Jul 6 06:45:01 coyote kernel: [78748.106827] [] ? badness+0x14d/0x220 Jul 6 06:45:01 coyote kernel: [78748.106830] [] __out_of_memory+0x142/0x170 Jul 6 06:45:01 coyote kernel: [78748.106834] [] out_of_memory+0x59/0xc0 Jul 6 06:45:01 coyote kernel: [78748.106837] [] __alloc_pages_nodemask+0x4f7/0x510 Jul 6 06:45:01 coyote kernel: [78748.106841] [] __get_free_pages+0x23/0x50 Jul 6 06:45:01 coyote kernel: [78748.106845] [] __pollwait+0xb2/0xf0 Jul 6 06:45:01 coyote kernel: [78748.106848] [] unix_poll+0x28/0xc0 Jul 6 06:45:01 coyote kernel: [78748.106851] [] sock_poll+0x1e/0x40 Jul 6 06:45:01 coyote kernel: [78748.106853] [] do_select+0x34e/0x6b0 Jul 6 06:45:01 coyote kernel: [78748.106871] [] ? __pollwait+0x0/0xf0 Jul 6 06:45:01 coyote kernel: [78748.106874] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106877] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106879] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106882] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106884] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106887] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106890] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106892] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106895] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106897] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106900] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106902] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106905] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106907] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106910] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106913] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106915] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106918] [] ? pollwake+0x0/0x90 Jul 6 06:45:01 coyote kernel: [78748.106920] [] core_sys_select+0x212/0x350 Jul 6 06:45:01 coyote kernel: [78748.106925] [] ? poll_select_set_timeout+0x82/0x90 Jul 6 06:45:01 coyote kernel: [78748.106928] [] sys_select+0x51/0xf0 Jul 6 06:45:01 coyote kernel: [78748.106931] [] sysenter_do_call+0x12/0x22 Jul 6 06:45:01 coyote kernel: [78748.106933] Mem-Info: Jul 6 06:45:01 coyote kernel: [78748.106935] DMA per-cpu: Jul 6 06:45:01 coyote kernel: [78748.106937] CPU 0: hi: 0, btch: 1 usd: 0 Jul 6 06:45:01 coyote kernel: [78748.106939] CPU 1: hi: 0, btch: 1 usd: 0 Jul 6 06:45:01 coyote kernel: [78748.106941] CPU 2: hi: 0, btch: 1 usd: 0 Jul 6 06:45:01 coyote kernel: [78748.106943] CPU 3: hi: 0, btch: 1 usd: 0 Jul 6 06:45:01 coyote kernel: [78748.106944] Normal per-cpu: Jul 6 06:45:01 coyote kernel: [78748.106946] CPU 0: hi: 186, btch: 31 usd: 118 Jul 6 06:45:01 coyote kernel: [78748.106948] CPU 1: hi: 186, btch: 31 usd: 171 Jul 6 06:45:01 coyote kernel: [78748.106950] CPU 2: hi: 186, btch: 31 usd: 159 Jul 6 06:45:01 coyote kernel: [78748.106952] CPU 3: hi: 186, btch: 31 usd: 172 Jul 6 06:45:01 coyote kernel: [78748.106954] HighMem per-cpu: Jul 6 06:45:01 coyote kernel: [78748.106955] CPU 0: hi: 186, btch: 31 usd: 56 Jul 6 06:45:01 coyote kernel: [78748.106957] CPU 1: hi: 186, btch: 31 usd: 20 Jul 6 06:45:01 coyote kernel: [78748.106959] CPU 2: hi: 186, btch: 31 usd: 53 Jul 6 06:45:01 coyote kernel: [78748.106961] CPU 3: hi: 186, btch: 31 usd: 180 Jul 6 06:45:01 coyote kernel: [78748.106965] Active_anon:90702 active_file:136927 inactive_anon:26328 Jul 6 06:45:01 coyote kernel: [78748.106966] inactive_file:1956 unevictable:25 dirty:4 writeback:0 unstable:0 Jul 6 06:45:01 coyote kernel: [78748.106967] free:560899 slab:206505 mapped:19048 pagetables:3220 bounce:0 Jul 6 06:45:01 coyote kernel: [78748.106971] DMA free:3496kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB acti ve_file:12kB inactive_file:8kB unevictable:0kB present:15804kB pages_scanned:0 all_unreclaimable? yes Jul 6 06:45:01 coyote kernel: [78748.106983] lowmem_reserve[]: 0 0 25406 25406 Jul 6 06:45:01 coyote kernel: [78748.106988] HighMem free:2236464kB min:512kB low:3928kB high:7348kB active_anon:362808kB inact ive_anon:105308kB active_file:547220kB inactive_file:7704kB unevictable:100kB present:3252052kB pages_scanned:0 all_unreclaimabl e? no Jul 6 06:45:01 coyote kernel: [78748.106991] lowmem_reserve[]: 0 0 0 0 Jul 6 06:45:01 coyote kernel: [78748.106994] DMA: 310*4kB 204*8kB 27*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048 kB 0*4096kB = 3496kB Jul 6 06:45:01 coyote kernel: [78748.107002] Normal: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3636kB Jul 6 06:45:01 coyote kernel: [78748.107009] HighMem: 45888*4kB 44682*8kB 32844*16kB 18479*32kB 6641*64kB 1114*128kB 37*256kB 1 *512kB 1*1024kB 0*2048kB 0*4096kB = 2236464kB Jul 6 06:45:01 coyote kernel: [78748.107017] 139771 total pagecache pages Jul 6 06:45:01 coyote kernel: [78748.107019] 616 pages in swap cache Jul 6 06:45:01 coyote kernel: [78748.107021] Swap cache stats: add 7937, delete 7321, find 2362/2515 Jul 6 06:45:01 coyote kernel: [78748.107023] Free swap = 8360652kB Jul 6 06:45:01 coyote kernel: [78748.107024] Total swap = 8385912kB Jul 6 06:45:01 coyote kernel: [78748.121323] 1179632 pages RAM Jul 6 06:45:01 coyote kernel: [78748.121325] 951810 pages HighMem Jul 6 06:45:01 coyote kernel: [78748.121327] 146165 pages reserved Jul 6 06:45:01 coyote kernel: [78748.121328] 149491 pages shared Jul 6 06:45:01 coyote kernel: [78748.121329] 441615 pages non-shared Jul 6 06:45:01 coyote kernel: [78748.121332] Out of memory: kill process 2385 (mysqld) score 15451 or a child Jul 6 06:45:01 coyote kernel: [78748.121334] Killed process 2385 (mysqld) and continued to: Jul 6 06:45:01 coyote kernel: [78748.137525] Killed process 30192 (spamd) Jul 6 06:45:01 coyote kernel: [78748.154292] Killed process 2506 (httpd) Jul 6 06:45:01 coyote kernel: [78748.170851] Killed process 2507 (httpd) Jul 6 06:45:01 coyote kernel: [78748.187519] Killed process 2508 (httpd) Jul 6 06:45:01 coyote kernel: [78748.320880] Killed process 2510 (httpd) Jul 6 06:45:01 coyote kernel: [78748.337529] Killed process 2511 (httpd) Jul 6 06:45:05 coyote kernel: [78751.467158] Killed process 948 (spamd) Jul 6 06:45:05 coyote kernel: [78751.483535] Killed process 963 (httpd) Jul 6 06:45:05 coyote kernel: [78751.500194] Killed process 964 (httpd) Jul 6 06:45:05 coyote kernel: [78751.502688] Killed process 965 (httpd) Jul 6 06:45:05 coyote kernel: [78751.506025] Killed process 2512 (httpd) Jul 6 06:45:05 coyote kernel: [78751.509376] Killed process 2513 (httpd) Jul 6 06:45:05 coyote kernel: [78751.512713] Killed process 2514 (httpd) Jul 6 06:45:05 coyote kernel: [78751.516057] Killed process 1995 (krunner_lock) Jul 6 06:45:05 coyote kernel: [78751.519481] Killed process 3391 (kwin) Jul 6 06:45:05 coyote kernel: [78751.522674] Killed process 3396 (plasma) Jul 6 06:45:05 coyote kernel: [78751.526006] Killed process 6931 (kmail) Jul 6 06:45:05 coyote kernel: [78751.542786] Killed process 16309 (spamd) Jul 6 06:45:05 coyote kernel: [78751.546042] Killed process 3421 (krunner) Jul 6 06:45:05 coyote kernel: [78751.549521] Killed process 3357 (klauncher) Jul 6 06:45:05 coyote kernel: [78751.552760] Killed process 3526 (kcalc) Jul 6 06:45:05 coyote kernel: [78751.759570] Killed process 3068 (X) Jul 6 06:45:05 coyote kernel: [78751.765648] Killed process 3389 (ksmserver) The oom started at 6:45:01 this morning. mysqld wasn't doing anything & the only reason its even started is for mythtv, which is how I found my pcHDTV-3000 was on the missing list even if occupying a slot. I've no idea if the rebuilt (with v4l drivers now) will also crash. It feels normal. And looks normal in htop's display, using 538M of 4096M, no swap used yet. >From my .config: # grep MEM .config CONFIG_SHMEM=y # CONFIG_MEMTEST is not set # CONFIG_NOHIGHMEM is not set # CONFIG_HIGHMEM4G is not set CONFIG_HIGHMEM64G=y CONFIG_HIGHMEM=y CONFIG_ARCH_FLATMEM_ENABLE=y CONFIG_ARCH_SPARSEMEM_ENABLE=y CONFIG_ARCH_SELECT_MEMORY_MODEL=y CONFIG_SELECT_MEMORY_MODEL=y CONFIG_FLATMEM_MANUAL=y # CONFIG_DISCONTIGMEM_MANUAL is not set # CONFIG_SPARSEMEM_MANUAL is not set CONFIG_FLATMEM=y CONFIG_FLAT_NODE_MEM_MAP=y CONFIG_SPARSEMEM_STATIC=y CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y # CONFIG_BLK_DEV_UMEM is not set # CONFIG_INPUT_FF_MEMLESS is not set CONFIG_DEVKMEM=y CONFIG_FIX_EARLYCON_MEM=y # CONFIG_HW_RANDOM_TIMERIOMEM is not set # CONFIG_MEMSTICK is not set CONFIG_FIRMWARE_MEMMAP=y CONFIG_DEBUG_MEMORY_INIT=y CONFIG_HAVE_ARCH_KMEMCHECK=y CONFIG_STRICT_DEVMEM=y CONFIG_HAS_IOMEM=y In case I have some option miss-set in that, plz advise. Thanks. -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) The NRA is offering FREE Associate memberships to anyone who wants them. Mal: "Shouldn't have been you - they shoulda hit us." Book: "That crossed my mind..." --"Serenity" -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/