Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755874AbYLDD2Q (ORCPT ); Wed, 3 Dec 2008 22:28:16 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752709AbYLDD15 (ORCPT ); Wed, 3 Dec 2008 22:27:57 -0500 Received: from colobus.isomerica.net ([216.93.242.10]:53788 "EHLO colobus.isomerica.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751753AbYLDD1y convert rfc822-to-8bit (ORCPT ); Wed, 3 Dec 2008 22:27:54 -0500 Date: Wed, 3 Dec 2008 22:27:50 -0500 From: Dan =?UTF-8?B?Tm/DqQ==?= To: linux-kernel@vger.kernel.org Subject: Page alloc failures under network/disk IO load Message-ID: <20081203222750.391e8890@tuna> Organization: isomerica.net X-Mailer: Claws Mail 3.5.0 (GTK+ 2.14.4; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 88182 Lines: 1676 This is on Linux 2.6.28-rc7, on a Core 2 Duo. The system has plenty of memory: total used free shared buffers cached Mem: 1893 1822 70 0 0 1573 -/+ buffers/cache: 249 1644 Swap: 1906 37 1868 I am using rsync to transfer data onto this system. The filesystem is XFS, and the target drive is a 1TB Western Digital on ata_piix. The system files are on a RAID 1 (Linux md, also on ata_piix). Periodically I get page allocation failures, from __netdev_alloc_skb. I suppose this causes the driver to drop packets and thus hurts performance. Below are the logs from the entire event. Cheers, Dan [112854.339298] rsync: page allocation failure. order:1, mode:0x4020 [112854.339338] Pid: 20229, comm: rsync Not tainted 2.6.28-rc7 #3 [112854.339365] Call Trace: [112854.339387] [] __alloc_pages_internal+0x40e/0x530 [112854.339433] [] new_slab+0x274/0x350 [112854.339461] [] __slab_alloc+0x2c0/0x440 [112854.339498] [] ? __netdev_alloc_skb+0x38/0x60 [112854.339527] [] ? __netdev_alloc_skb+0x38/0x60 [112854.339555] [] __kmalloc_node_track_caller+0x94/0x110 [112854.339583] [] ? __netdev_alloc_skb+0x38/0x60 [112854.339610] [] __alloc_skb+0x7c/0x160 [112854.339637] [] __netdev_alloc_skb+0x38/0x60 [112854.339684] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [112854.339716] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [112854.339747] [] rtl8169_poll+0x41/0x260 [r8169] [112854.339775] [] net_rx_action+0x17b/0x290 [112854.339801] [] ? net_rx_action+0x102/0x290 [112854.339829] [] __do_softirq+0xac/0x190 [112854.339866] [] call_softirq+0x1c/0x50 [112854.339895] [] do_softirq+0x75/0xb0 [112854.339923] [] irq_exit+0xa5/0xb0 [112854.339949] [] do_IRQ+0x111/0x1d0 [112854.339979] [] ret_from_intr+0x0/0xf [112854.340004] [] ? touch_atime+0x21/0x140 [112854.340008] [] ? mnt_want_write+0x7/0xa0 [112854.340008] [] ? stack_trace_call+0x68/0x1c0 [112854.340008] [] ? ftrace_call+0x5/0x2b [112854.340008] [] ? mnt_want_write+0xc/0xa0 [112854.340008] [] ? touch_atime+0x21/0x140 [112854.340008] [] ? generic_file_aio_read+0x22d/0x6c0 [112854.340008] [] ? xfs_ilock+0x75/0xb0 [xfs] [112854.340008] [] ? xfs_read+0x12f/0x2c0 [xfs] [112854.340008] [] ? xfs_file_aio_read+0x56/0x60 [xfs] [112854.340008] [] ? do_sync_read+0xf9/0x140 [112854.340008] [] ? autoremove_wake_function+0x0/0x40 [112854.340008] [] ? ftrace_call+0x5/0x2b [112854.340008] [] ? cap_file_permission+0x9/0x10 [112854.340008] [] ? security_file_permission+0x16/0x20 [112854.340008] [] ? vfs_read+0xc8/0x180 [112854.340008] [] ? sys_read+0x55/0x90 [112854.340008] [] ? system_call_fastpath+0x16/0x1b [112854.340008] Mem-Info: [112854.340008] Node 0 DMA per-cpu: [112854.340008] CPU 0: hi: 0, btch: 1 usd: 0 [112854.340008] CPU 1: hi: 0, btch: 1 usd: 0 [112854.340008] Node 0 DMA32 per-cpu: [112854.340008] CPU 0: hi: 186, btch: 31 usd: 49 [112854.340008] CPU 1: hi: 186, btch: 31 usd: 170 [112854.340008] Active_anon:11937 active_file:27370 inactive_anon:4025 [112854.340008] inactive_file:227401 unevictable:0 dirty:12448 writeback:4774 unstable:0 [112854.340008] free:2168 slab:201706 mapped:1364 pagetables:1914 bounce:0 [112854.340008] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? yes [112854.340008] lowmem_reserve[]: 0 1976 1976 1976 [112854.340008] Node 0 DMA32 free:7180kB min:5664kB low:7080kB high:8496kB active_anon:47748kB inactive_anon:16100kB active_file:109480kB inactive_file:909604kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [112854.340008] lowmem_reserve[]: 0 0 0 0 [112854.340008] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [112854.340008] Node 0 DMA32: 1465*4kB 13*8kB 31*16kB 12*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 7228kB [112854.340008] 255838 total pagecache pages [112854.340008] 948 pages in swap cache [112854.340008] Swap cache stats: add 10447, delete 9499, find 82578/82707 [112854.340008] Free swap = 1913948kB [112854.340008] Total swap = 1951800kB [112854.340008] 522160 pages RAM [112854.340008] 37425 pages reserved [112854.340008] 180511 pages shared [112854.340008] 308921 pages non-shared [116409.444195] rsync: page allocation failure. order:1, mode:0x4020 [116409.444242] Pid: 20801, comm: rsync Not tainted 2.6.28-rc7 #3 [116409.444269] Call Trace: [116409.444290] [] __alloc_pages_internal+0x40e/0x530 [116409.444339] [] new_slab+0x274/0x350 [116409.444366] [] __slab_alloc+0x2c0/0x440 [116409.444395] [] ? __netdev_alloc_skb+0x38/0x60 [116409.444423] [] ? __netdev_alloc_skb+0x38/0x60 [116409.444450] [] __kmalloc_node_track_caller+0x94/0x110 [116409.444478] [] ? __netdev_alloc_skb+0x38/0x60 [116409.444507] [] __alloc_skb+0x7c/0x160 [116409.444536] [] __netdev_alloc_skb+0x38/0x60 [116409.444585] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116409.444617] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116409.444649] [] rtl8169_poll+0x41/0x260 [r8169] [116409.444679] [] net_rx_action+0x17b/0x290 [116409.444705] [] ? net_rx_action+0x102/0x290 [116409.444736] [] __do_softirq+0xac/0x190 [116409.444763] [] call_softirq+0x1c/0x50 [116409.444788] [] do_softirq+0x75/0xb0 [116409.444814] [] irq_exit+0xa5/0xb0 [116409.444840] [] do_IRQ+0x111/0x1d0 [116409.444868] [] ret_from_intr+0x0/0xf [116409.444893] <6>Mem-Info: [116409.444924] Node 0 DMA per-cpu: [116409.444956] CPU 0: hi: 0, btch: 1 usd: 0 [116409.444979] CPU 1: hi: 0, btch: 1 usd: 0 [116409.445003] Node 0 DMA32 per-cpu: [116409.445032] CPU 0: hi: 186, btch: 31 usd: 14 [116409.445055] CPU 1: hi: 186, btch: 31 usd: 39 [116409.445082] Active_anon:10553 active_file:44722 inactive_anon:4935 [116409.445084] inactive_file:342297 unevictable:0 dirty:20031 writeback:5988 unstable:0 [116409.445087] free:37099 slab:34695 mapped:775 pagetables:2327 bounce:0 [116409.445168] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116409.445832] lowmem_reserve[]: 0 1976 1976 1976 [116409.445892] Node 0 DMA32 free:146736kB min:5664kB low:7080kB high:8496kB active_anon:42212kB inactive_anon:19740kB active_file:178888kB inactive_file:1369188kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116409.445966] lowmem_reserve[]: 0 0 0 0 [116409.446026] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116409.446164] Node 0 DMA32: 36391*4kB 18*8kB 7*16kB 0*32kB 0*64kB 4*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 146588kB [116409.446349] 387905 total pagecache pages [116409.446372] 704 pages in swap cache [116409.446395] Swap cache stats: add 10504, delete 9800, find 88920/89050 [116409.446422] Free swap = 1913748kB [116409.446443] Total swap = 1951800kB [116409.449229] 522160 pages RAM [116409.449229] 37425 pages reserved [116409.449229] 314570 pages shared [116409.449229] 140539 pages non-shared [116409.463369] rsync: page allocation failure. order:1, mode:0x4020 [116409.463402] Pid: 20801, comm: rsync Not tainted 2.6.28-rc7 #3 [116409.463427] Call Trace: [116409.463447] [] __alloc_pages_internal+0x40e/0x530 [116409.463491] [] new_slab+0x274/0x350 [116409.463517] [] __slab_alloc+0x2c0/0x440 [116409.463544] [] ? __netdev_alloc_skb+0x38/0x60 [116409.463571] [] ? __netdev_alloc_skb+0x38/0x60 [116409.463598] [] __kmalloc_node_track_caller+0x94/0x110 [116409.463625] [] ? __netdev_alloc_skb+0x38/0x60 [116409.463652] [] __alloc_skb+0x7c/0x160 [116409.463678] [] __netdev_alloc_skb+0x38/0x60 [116409.463720] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116409.463749] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116409.463780] [] rtl8169_poll+0x41/0x260 [r8169] [116409.463808] [] net_rx_action+0x17b/0x290 [116409.463833] [] ? net_rx_action+0x102/0x290 [116409.463872] [] ? __kfree_skb+0x47/0xa0 [116409.463901] [] __do_softirq+0xac/0x190 [116409.463927] [] call_softirq+0x1c/0x50 [116409.463959] [] do_softirq+0x75/0xb0 [116409.463984] [] irq_exit+0xa5/0xb0 [116409.464009] [] do_IRQ+0x111/0x1d0 [116409.464040] [] ret_from_intr+0x0/0xf [116409.464064] <6>Mem-Info: [116409.464098] Node 0 DMA per-cpu: [116409.464123] CPU 0: hi: 0, btch: 1 usd: 0 [116409.464123] CPU 1: hi: 0, btch: 1 usd: 0 [116409.464123] Node 0 DMA32 per-cpu: [116409.464123] CPU 0: hi: 186, btch: 31 usd: 25 [116409.464123] CPU 1: hi: 186, btch: 31 usd: 0 [116409.464123] Active_anon:10553 active_file:44690 inactive_anon:4935 [116409.464123] inactive_file:342106 unevictable:0 dirty:19659 writeback:6360 unstable:0 [116409.464123] free:36903 slab:35099 mapped:775 pagetables:2327 bounce:0 [116409.464123] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116409.464123] lowmem_reserve[]: 0 1976 1976 1976 [116409.464123] Node 0 DMA32 free:146120kB min:5664kB low:7080kB high:8496kB active_anon:42212kB inactive_anon:19740kB active_file:178760kB inactive_file:1368424kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116409.464516] lowmem_reserve[]: 0 0 0 0 [116409.464516] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116409.464516] Node 0 DMA32: 36259*4kB 22*8kB 16*16kB 0*32kB 0*64kB 3*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 146108kB [116409.464516] 387719 total pagecache pages [116409.464516] 704 pages in swap cache [116409.464516] Swap cache stats: add 10504, delete 9800, find 88920/89050 [116409.464516] Free swap = 1913748kB [116409.464516] Total swap = 1951800kB [116409.476530] 522160 pages RAM [116409.476530] 37425 pages reserved [116409.476530] 314464 pages shared [116409.476530] 140767 pages non-shared [116419.900921] pdflush: page allocation failure. order:1, mode:0x4020 [116419.900965] Pid: 20863, comm: pdflush Not tainted 2.6.28-rc7 #3 [116419.900993] Call Trace: [116419.901014] [] __alloc_pages_internal+0x40e/0x530 [116419.901063] [] new_slab+0x274/0x350 [116419.901090] [] __slab_alloc+0x2c0/0x440 [116419.901119] [] ? __netdev_alloc_skb+0x38/0x60 [116419.901148] [] ? __netdev_alloc_skb+0x38/0x60 [116419.901178] [] __kmalloc_node_track_caller+0x94/0x110 [116419.901208] [] ? __netdev_alloc_skb+0x38/0x60 [116419.901237] [] __alloc_skb+0x7c/0x160 [116419.901265] [] __netdev_alloc_skb+0x38/0x60 [116419.901319] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116419.901349] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116419.901381] [] rtl8169_poll+0x41/0x260 [r8169] [116419.901410] [] net_rx_action+0x17b/0x290 [116419.901437] [] ? net_rx_action+0x102/0x290 [116419.901465] [] __do_softirq+0xac/0x190 [116419.901493] [] call_softirq+0x1c/0x50 [116419.901519] [] do_softirq+0x75/0xb0 [116419.901547] [] irq_exit+0xa5/0xb0 [116419.901577] [] do_IRQ+0x111/0x1d0 [116419.901605] [] ? mempool_alloc_slab+0x16/0x20 [116419.901634] [] ret_from_intr+0x0/0xf [116419.901658] [] ? kmem_cache_alloc+0xb5/0xe0 [116419.901696] [] ? kmem_cache_alloc+0xb8/0xe0 [116419.901724] [] ? mempool_alloc_slab+0x16/0x20 [116419.901752] [] ? mempool_alloc+0x5b/0x150 [116419.901778] [] ? ftrace_call+0x5/0x2b [116419.901807] [] ? bvec_alloc_bs+0x74/0x150 [116419.901835] [] ? bio_alloc_bioset+0x5e/0x100 [116419.901864] [] ? bio_alloc+0x15/0x30 [116419.901929] [] ? xfs_alloc_ioend_bio+0x2e/0x60 [xfs] [116419.901973] [] ? xfs_submit_ioend+0xe3/0xf0 [xfs] [116419.902014] [] ? xfs_page_state_convert+0x3a0/0x760 [xfs] [116419.902058] [] ? xfs_vm_writepage+0x71/0x120 [xfs] [116419.902087] [] ? __writepage+0x1a/0x60 [116419.902115] [] ? write_cache_pages+0x2dd/0x450 [116419.902145] [] ? __writepage+0x0/0x60 [116419.902176] [] ? generic_writepages+0x27/0x30 [116419.902218] [] ? xfs_vm_writepages+0x5b/0x70 [xfs] [116419.902248] [] ? do_writepages+0x30/0x50 [116419.902276] [] ? __writeback_single_inode+0xa6/0x3f0 [116419.902305] [] ? ftrace_call+0x5/0x2b [116419.902332] [] ? generic_sync_sb_inodes+0x29c/0x3a0 [116419.902361] [] ? writeback_inodes+0x75/0x110 [116419.902388] [] ? background_writeout+0xb8/0xf0 [116419.902416] [] ? pdflush+0x151/0x230 [116419.902443] [] ? background_writeout+0x0/0xf0 [116419.902472] [] ? pdflush+0x0/0x230 [116419.902502] [] ? kthread+0x4e/0x90 [116419.902528] [] ? child_rip+0xa/0x11 [116419.902559] [] ? _spin_unlock_irq+0x30/0x40 [116419.902588] [] ? restore_args+0x0/0x30 [116419.902615] [] ? kthread+0x0/0x90 [116419.902640] [] ? child_rip+0x0/0x11 [116419.902664] Mem-Info: [116419.902685] Node 0 DMA per-cpu: [116419.902713] CPU 0: hi: 0, btch: 1 usd: 0 [116419.902738] CPU 1: hi: 0, btch: 1 usd: 0 [116419.902761] Node 0 DMA32 per-cpu: [116419.902791] CPU 0: hi: 186, btch: 31 usd: 28 [116419.902815] CPU 1: hi: 186, btch: 31 usd: 27 [116419.902841] Active_anon:10553 active_file:47172 inactive_anon:4935 [116419.902844] inactive_file:342800 unevictable:0 dirty:35963 writeback:5148 unstable:0 [116419.902847] free:36055 slab:32747 mapped:753 pagetables:2327 bounce:0 [116419.902931] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116419.903010] lowmem_reserve[]: 0 1976 1976 1976 [116419.903079] Node 0 DMA32 free:142728kB min:5664kB low:7080kB high:8496kB active_anon:42212kB inactive_anon:19740kB active_file:188688kB inactive_file:1371200kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116419.903155] lowmem_reserve[]: 0 0 0 0 [116419.903225] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116419.903396] Node 0 DMA32: 35413*4kB 1*8kB 8*16kB 4*32kB 3*64kB 3*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 142748kB [116419.903531] 390851 total pagecache pages [116419.903553] 704 pages in swap cache [116419.903577] Swap cache stats: add 10504, delete 9800, find 88920/89050 [116419.903603] Free swap = 1913748kB [116419.903625] Total swap = 1951800kB [116419.916522] 522160 pages RAM [116419.916522] 37425 pages reserved [116419.916522] 326518 pages shared [116419.916522] 129291 pages non-shared [116809.273641] swapper: page allocation failure. order:1, mode:0x4020 [116809.273684] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.273709] Call Trace: [116809.273748] [] __alloc_pages_internal+0x40e/0x530 [116809.273794] [] new_slab+0x274/0x350 [116809.273820] [] __slab_alloc+0x2c0/0x440 [116809.273849] [] ? __netdev_alloc_skb+0x38/0x60 [116809.273875] [] ? __netdev_alloc_skb+0x38/0x60 [116809.273902] [] __kmalloc_node_track_caller+0x94/0x110 [116809.273931] [] ? __netdev_alloc_skb+0x38/0x60 [116809.273958] [] __alloc_skb+0x7c/0x160 [116809.273984] [] __netdev_alloc_skb+0x38/0x60 [116809.274033] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.274064] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.274103] [] rtl8169_poll+0x41/0x260 [r8169] [116809.274133] [] net_rx_action+0x17b/0x290 [116809.274736] [] ? net_rx_action+0x102/0x290 [116809.274764] [] __do_softirq+0xac/0x190 [116809.274791] [] call_softirq+0x1c/0x50 [116809.274817] [] do_softirq+0x75/0xb0 [116809.274843] [] irq_exit+0xa5/0xb0 [116809.274868] [] do_IRQ+0x111/0x1d0 [116809.274894] [] ret_from_intr+0x0/0xf [116809.274918] [] ? mwait_idle+0x43/0x60 [116809.274955] [] ? mwait_idle+0x4c/0x60 [116809.274989] [] ? mwait_idle+0x43/0x60 [116809.275015] [] ? cpu_idle+0x6b/0xc0 [116809.275043] [] ? rest_init+0x66/0x70 [116809.275069] Mem-Info: [116809.275088] Node 0 DMA per-cpu: [116809.275116] CPU 0: hi: 0, btch: 1 usd: 0 [116809.275139] CPU 1: hi: 0, btch: 1 usd: 0 [116809.275163] Node 0 DMA32 per-cpu: [116809.275192] CPU 0: hi: 186, btch: 31 usd: 28 [116809.275215] CPU 1: hi: 186, btch: 31 usd: 0 [116809.275248] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.275250] inactive_file:390025 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.275253] free:18660 slab:44429 mapped:757 pagetables:2291 bounce:0 [116809.275333] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.275402] lowmem_reserve[]: 0 1976 1976 1976 [116809.275458] Node 0 DMA32 free:73148kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1560100kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116809.275539] lowmem_reserve[]: 0 0 0 0 [116809.275596] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.275738] Node 0 DMA32: 18028*4kB 112*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 73152kB [116809.275889] 392307 total pagecache pages [116809.275926] 704 pages in swap cache [116809.275948] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.275973] Free swap = 1913748kB [116809.276000] Total swap = 1951800kB [116809.276039] 522160 pages RAM [116809.276039] 37425 pages reserved [116809.276039] 394179 pages shared [116809.276039] 78983 pages non-shared [116809.289171] swapper: page allocation failure. order:1, mode:0x4020 [116809.289207] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.289231] Call Trace: [116809.289251] [] __alloc_pages_internal+0x40e/0x530 [116809.289299] [] new_slab+0x274/0x350 [116809.289326] [] __slab_alloc+0x2c0/0x440 [116809.289359] [] ? __netdev_alloc_skb+0x38/0x60 [116809.289385] [] ? __netdev_alloc_skb+0x38/0x60 [116809.289412] [] __kmalloc_node_track_caller+0x94/0x110 [116809.289446] [] ? __netdev_alloc_skb+0x38/0x60 [116809.289473] [] __alloc_skb+0x7c/0x160 [116809.289499] [] __netdev_alloc_skb+0x38/0x60 [116809.289553] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.289583] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.289613] [] rtl8169_poll+0x41/0x260 [r8169] [116809.289646] [] net_rx_action+0x17b/0x290 [116809.289671] [] ? net_rx_action+0x102/0x290 [116809.289699] [] __do_softirq+0xac/0x190 [116809.289732] [] call_softirq+0x1c/0x50 [116809.289758] [] do_softirq+0x75/0xb0 [116809.289784] [] irq_exit+0xa5/0xb0 [116809.289815] [] do_IRQ+0x111/0x1d0 [116809.289841] [] ret_from_intr+0x0/0xf [116809.289864] [] ? mwait_idle+0x43/0x60 [116809.289914] [] ? mwait_idle+0x4c/0x60 [116809.289940] [] ? mwait_idle+0x43/0x60 [116809.289971] [] ? cpu_idle+0x6b/0xc0 [116809.289997] [] ? rest_init+0x66/0x70 [116809.290021] Mem-Info: [116809.290046] Node 0 DMA per-cpu: [116809.290073] CPU 0: hi: 0, btch: 1 usd: 0 [116809.290103] CPU 1: hi: 0, btch: 1 usd: 0 [116809.290127] Node 0 DMA32 per-cpu: [116809.290168] CPU 0: hi: 186, btch: 31 usd: 2 [116809.290192] CPU 1: hi: 186, btch: 31 usd: 17 [116809.290216] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.290218] inactive_file:390025 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.290221] free:18616 slab:44493 mapped:757 pagetables:2291 bounce:0 [116809.290307] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.290381] lowmem_reserve[]: 0 1976 1976 1976 [116809.290463] Node 0 DMA32 free:72888kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1560100kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116809.290540] lowmem_reserve[]: 0 0 0 0 [116809.290606] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.290763] Node 0 DMA32: 17966*4kB 95*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 72768kB [116809.290932] 392307 total pagecache pages [116809.290955] 704 pages in swap cache [116809.290982] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.291008] Free swap = 1913748kB [116809.291029] Total swap = 1951800kB [116809.293012] 522160 pages RAM [116809.293012] 37425 pages reserved [116809.293012] 394088 pages shared [116809.293012] 79069 pages non-shared [116809.308343] swapper: page allocation failure. order:1, mode:0x4020 [116809.308377] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.308402] Call Trace: [116809.308423] [] __alloc_pages_internal+0x40e/0x530 [116809.308467] [] new_slab+0x274/0x350 [116809.308493] [] __slab_alloc+0x2c0/0x440 [116809.308523] [] ? __netdev_alloc_skb+0x38/0x60 [116809.308550] [] ? __netdev_alloc_skb+0x38/0x60 [116809.308577] [] __kmalloc_node_track_caller+0x94/0x110 [116809.308605] [] ? __netdev_alloc_skb+0x38/0x60 [116809.308634] [] __alloc_skb+0x7c/0x160 [116809.308661] [] __netdev_alloc_skb+0x38/0x60 [116809.308702] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.308733] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.308764] [] rtl8169_poll+0x41/0x260 [r8169] [116809.308793] [] net_rx_action+0x17b/0x290 [116809.308819] [] ? net_rx_action+0x102/0x290 [116809.308847] [] __do_softirq+0xac/0x190 [116809.308875] [] call_softirq+0x1c/0x50 [116809.308901] [] do_softirq+0x75/0xb0 [116809.308927] [] irq_exit+0xa5/0xb0 [116809.308953] [] do_IRQ+0x111/0x1d0 [116809.308980] [] ret_from_intr+0x0/0xf [116809.309005] [] ? mwait_idle+0x43/0x60 [116809.309043] [] ? mwait_idle+0x4c/0x60 [116809.309070] [] ? mwait_idle+0x43/0x60 [116809.309095] [] ? cpu_idle+0x6b/0xc0 [116809.309122] [] ? rest_init+0x66/0x70 [116809.309147] Mem-Info: [116809.309167] Node 0 DMA per-cpu: [116809.309196] CPU 0: hi: 0, btch: 1 usd: 0 [116809.309219] CPU 1: hi: 0, btch: 1 usd: 0 [116809.309242] Node 0 DMA32 per-cpu: [116809.309270] CPU 0: hi: 186, btch: 31 usd: 49 [116809.309294] CPU 1: hi: 186, btch: 31 usd: 0 [116809.309319] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.309321] inactive_file:389905 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.309324] free:18635 slab:44515 mapped:757 pagetables:2291 bounce:0 [116809.309412] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.309480] lowmem_reserve[]: 0 1976 1976 1976 [116809.309538] Node 0 DMA32 free:73048kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1559620kB unevictable:0kB present:2023748kB pages_scanned:33 all_unreclaimable? no [116809.309616] lowmem_reserve[]: 0 0 0 0 [116809.309673] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.309814] Node 0 DMA32: 17997*4kB 119*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 73084kB [116809.309960] 392214 total pagecache pages [116809.309987] 704 pages in swap cache [116809.310016] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.310042] Free swap = 1913748kB [116809.310068] Total swap = 1951800kB [116809.312229] 522160 pages RAM [116809.312229] 37425 pages reserved [116809.312229] 394111 pages shared [116809.312229] 79071 pages non-shared [116809.323440] swapper: page allocation failure. order:1, mode:0x4020 [116809.323476] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.323501] Call Trace: [116809.323521] [] __alloc_pages_internal+0x40e/0x530 [116809.323571] [] new_slab+0x274/0x350 [116809.323596] [] __slab_alloc+0x2c0/0x440 [116809.323629] [] ? __netdev_alloc_skb+0x38/0x60 [116809.323656] [] ? __netdev_alloc_skb+0x38/0x60 [116809.323683] [] __kmalloc_node_track_caller+0x94/0x110 [116809.323717] [] ? __netdev_alloc_skb+0x38/0x60 [116809.323743] [] __alloc_skb+0x7c/0x160 [116809.323770] [] __netdev_alloc_skb+0x38/0x60 [116809.323814] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.323845] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.323876] [] rtl8169_poll+0x41/0x260 [r8169] [116809.323909] [] net_rx_action+0x17b/0x290 [116809.324491] [] ? net_rx_action+0x102/0x290 [116809.324520] [] __do_softirq+0xac/0x190 [116809.324553] [] call_softirq+0x1c/0x50 [116809.324586] [] do_softirq+0x75/0xb0 [116809.324612] [] irq_exit+0xa5/0xb0 [116809.324643] [] do_IRQ+0x111/0x1d0 [116809.324669] [] ret_from_intr+0x0/0xf [116809.324693] [] ? mwait_idle+0x43/0x60 [116809.324737] [] ? mwait_idle+0x4c/0x60 [116809.324763] [] ? mwait_idle+0x43/0x60 [116809.324794] [] ? cpu_idle+0x6b/0xc0 [116809.324820] [] ? rest_init+0x66/0x70 [116809.324844] Mem-Info: [116809.324869] Node 0 DMA per-cpu: [116809.324905] CPU 0: hi: 0, btch: 1 usd: 0 [116809.324934] CPU 1: hi: 0, btch: 1 usd: 0 [116809.324958] Node 0 DMA32 per-cpu: [116809.324992] CPU 0: hi: 186, btch: 31 usd: 25 [116809.325015] CPU 1: hi: 186, btch: 31 usd: 53 [116809.325040] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.325042] inactive_file:389905 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.325045] free:18571 slab:44558 mapped:757 pagetables:2291 bounce:0 [116809.325130] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.325213] lowmem_reserve[]: 0 1976 1976 1976 [116809.325280] Node 0 DMA32 free:72792kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1559620kB unevictable:0kB present:2023748kB pages_scanned:33 all_unreclaimable? no [116809.325358] lowmem_reserve[]: 0 0 0 0 [116809.325432] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.325595] Node 0 DMA32: 17935*4kB 108*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 72748kB [116809.325761] 392214 total pagecache pages [116809.325783] 704 pages in swap cache [116809.325811] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.325838] Free swap = 1913748kB [116809.325858] Total swap = 1951800kB [116809.337024] 522160 pages RAM [116809.337024] 37425 pages reserved [116809.337024] 394113 pages shared [116809.337024] 79132 pages non-shared [116809.339350] swapper: page allocation failure. order:1, mode:0x4020 [116809.339381] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.339423] Call Trace: [116809.339444] [] __alloc_pages_internal+0x40e/0x530 [116809.339494] [] new_slab+0x274/0x350 [116809.339520] [] __slab_alloc+0x2c0/0x440 [116809.339549] [] ? __netdev_alloc_skb+0x38/0x60 [116809.339581] [] ? __netdev_alloc_skb+0x38/0x60 [116809.339608] [] __kmalloc_node_track_caller+0x94/0x110 [116809.339636] [] ? __netdev_alloc_skb+0x38/0x60 [116809.339669] [] __alloc_skb+0x7c/0x160 [116809.339695] [] __netdev_alloc_skb+0x38/0x60 [116809.339736] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.339781] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.339812] [] rtl8169_poll+0x41/0x260 [r8169] [116809.339841] [] net_rx_action+0x17b/0x290 [116809.339873] [] ? net_rx_action+0x102/0x290 [116809.339902] [] __do_softirq+0xac/0x190 [116809.339929] [] call_softirq+0x1c/0x50 [116809.339961] [] do_softirq+0x75/0xb0 [116809.339986] [] irq_exit+0xa5/0xb0 [116809.340011] [] do_IRQ+0x111/0x1d0 [116809.340043] [] ret_from_intr+0x0/0xf [116809.340067] [] ? mwait_idle+0x43/0x60 [116809.340118] [] ? mwait_idle+0x4c/0x60 [116809.340144] [] ? mwait_idle+0x43/0x60 [116809.340169] [] ? cpu_idle+0x6b/0xc0 [116809.340201] [] ? rest_init+0x66/0x70 [116809.340226] Mem-Info: [116809.340246] Node 0 DMA per-cpu: [116809.340282] CPU 0: hi: 0, btch: 1 usd: 0 [116809.340306] CPU 1: hi: 0, btch: 1 usd: 0 [116809.340334] Node 0 DMA32 per-cpu: [116809.340363] CPU 0: hi: 186, btch: 31 usd: 16 [116809.340401] CPU 1: hi: 186, btch: 31 usd: 0 [116809.340426] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.340429] inactive_file:389815 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.340431] free:18611 slab:44665 mapped:757 pagetables:2291 bounce:0 [116809.340517] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.340592] lowmem_reserve[]: 0 1976 1976 1976 [116809.340668] Node 0 DMA32 free:72952kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1559260kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116809.340740] lowmem_reserve[]: 0 0 0 0 [116809.340807] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.340975] Node 0 DMA32: 17945*4kB 114*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 72836kB [116809.341160] 392121 total pagecache pages [116809.341183] 704 pages in swap cache [116809.341205] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.341237] Free swap = 1913748kB [116809.341259] Total swap = 1951800kB [116809.342446] 522160 pages RAM [116809.342446] 37425 pages reserved [116809.342446] 394015 pages shared [116809.342446] 79247 pages non-shared [116809.354340] swapper: page allocation failure. order:1, mode:0x4020 [116809.354371] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.354397] Call Trace: [116809.354426] [] __alloc_pages_internal+0x40e/0x530 [116809.354477] [] new_slab+0x274/0x350 [116809.354503] [] __slab_alloc+0x2c0/0x440 [116809.354536] [] ? __netdev_alloc_skb+0x38/0x60 [116809.354563] [] ? __netdev_alloc_skb+0x38/0x60 [116809.354590] [] __kmalloc_node_track_caller+0x94/0x110 [116809.354625] [] ? __netdev_alloc_skb+0x38/0x60 [116809.354653] [] __alloc_skb+0x7c/0x160 [116809.354680] [] __netdev_alloc_skb+0x38/0x60 [116809.354725] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.354755] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.354795] [] rtl8169_poll+0x41/0x260 [r8169] [116809.354829] [] net_rx_action+0x17b/0x290 [116809.354856] [] ? net_rx_action+0x102/0x290 [116809.354884] [] __do_softirq+0xac/0x190 [116809.354916] [] call_softirq+0x1c/0x50 [116809.354942] [] do_softirq+0x75/0xb0 [116809.354969] [] irq_exit+0xa5/0xb0 [116809.355000] [] do_IRQ+0x111/0x1d0 [116809.355027] [] ret_from_intr+0x0/0xf [116809.355050] [] ? mwait_idle+0x43/0x60 [116809.355094] [] ? mwait_idle+0x4c/0x60 [116809.355128] [] ? mwait_idle+0x43/0x60 [116809.355161] [] ? cpu_idle+0x6b/0xc0 [116809.355187] [] ? rest_init+0x66/0x70 [116809.355212] Mem-Info: [116809.355238] Node 0 DMA per-cpu: [116809.355266] CPU 0: hi: 0, btch: 1 usd: 0 [116809.355295] CPU 1: hi: 0, btch: 1 usd: 0 [116809.355319] Node 0 DMA32 per-cpu: [116809.355352] CPU 0: hi: 186, btch: 31 usd: 7 [116809.355377] CPU 1: hi: 186, btch: 31 usd: 30 [116809.355409] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.355411] inactive_file:389815 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.355414] free:18546 slab:44729 mapped:757 pagetables:2291 bounce:0 [116809.355501] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.355576] lowmem_reserve[]: 0 1976 1976 1976 [116809.355644] Node 0 DMA32 free:72692kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1559260kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116809.355731] lowmem_reserve[]: 0 0 0 0 [116809.355805] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.355969] Node 0 DMA32: 17883*4kB 101*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 72484kB [116809.356130] 392121 total pagecache pages [116809.356152] 704 pages in swap cache [116809.356190] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.356216] Free swap = 1913748kB [116809.356238] Total swap = 1951800kB [116809.357787] 522160 pages RAM [116809.357787] 37425 pages reserved [116809.357787] 393894 pages shared [116809.357787] 79340 pages non-shared [116809.369472] swapper: page allocation failure. order:1, mode:0x4020 [116809.369502] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116809.369529] Call Trace: [116809.369560] [] __alloc_pages_internal+0x40e/0x530 [116809.369604] [] new_slab+0x274/0x350 [116809.369636] [] __slab_alloc+0x2c0/0x440 [116809.369664] [] ? __netdev_alloc_skb+0x38/0x60 [116809.369690] [] ? __netdev_alloc_skb+0x38/0x60 [116809.369724] [] __kmalloc_node_track_caller+0x94/0x110 [116809.369753] [] ? __netdev_alloc_skb+0x38/0x60 [116809.369780] [] __alloc_skb+0x7c/0x160 [116809.369813] [] __netdev_alloc_skb+0x38/0x60 [116809.369854] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.369884] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.369921] [] rtl8169_poll+0x41/0x260 [r8169] [116809.369949] [] net_rx_action+0x17b/0x290 [116809.370551] [] ? net_rx_action+0x102/0x290 [116809.370583] [] ? __kfree_skb+0x47/0xa0 [116809.370611] [] __do_softirq+0xac/0x190 [116809.370639] [] call_softirq+0x1c/0x50 [116809.370671] [] do_softirq+0x75/0xb0 [116809.370705] [] irq_exit+0xa5/0xb0 [116809.370731] [] do_IRQ+0x111/0x1d0 [116809.370763] [] ret_from_intr+0x0/0xf [116809.370787] [] ? mwait_idle+0x43/0x60 [116809.370831] [] ? mwait_idle+0x4c/0x60 [116809.370858] [] ? mwait_idle+0x43/0x60 [116809.370883] [] ? cpu_idle+0x6b/0xc0 [116809.370916] [] ? rest_init+0x66/0x70 [116809.370940] Mem-Info: [116809.370960] Node 0 DMA per-cpu: [116809.371001] CPU 0: hi: 0, btch: 1 usd: 0 [116809.371024] CPU 1: hi: 0, btch: 1 usd: 0 [116809.371053] Node 0 DMA32 per-cpu: [116809.371082] CPU 0: hi: 186, btch: 31 usd: 17 [116809.371111] CPU 1: hi: 186, btch: 31 usd: 0 [116809.371136] Active_anon:14659 active_file:1394 inactive_anon:5119 [116809.371139] inactive_file:389780 unevictable:0 dirty:18876 writeback:3071 unstable:0 [116809.371142] free:18524 slab:44815 mapped:757 pagetables:2291 bounce:0 [116809.371228] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116809.371303] lowmem_reserve[]: 0 1976 1976 1976 [116809.371381] Node 0 DMA32 free:72604kB min:5664kB low:7080kB high:8496kB active_anon:58636kB inactive_anon:20476kB active_file:5576kB inactive_file:1559120kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116809.371454] lowmem_reserve[]: 0 0 0 0 [116809.371528] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116809.371696] Node 0 DMA32: 17893*4kB 114*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 72628kB [116809.371873] 392059 total pagecache pages [116809.371896] 704 pages in swap cache [116809.371918] Swap cache stats: add 10504, delete 9800, find 89821/89951 [116809.371950] Free swap = 1913748kB [116809.371971] Total swap = 1951800kB [116809.372831] 522160 pages RAM [116809.372831] 37425 pages reserved [116809.372831] 393797 pages shared [116809.372831] 79424 pages non-shared [116809.389158] events/0: page allocation failure. order:1, mode:0x4020 [116809.389195] Pid: 9, comm: events/0 Not tainted 2.6.28-rc7 #3 [116809.389221] Call Trace: [116809.389243] [] __alloc_pages_internal+0x40e/0x530 [116809.389296] [] new_slab+0x274/0x350 [116809.389333] [] __slab_alloc+0x2c0/0x440 [116809.389370] [] ? __netdev_alloc_skb+0x38/0x60 [116809.389397] [] ? __netdev_alloc_skb+0x38/0x60 [116809.389424] [] __kmalloc_node_track_caller+0x94/0x110 [116809.389459] [] ? __netdev_alloc_skb+0x38/0x60 [116809.389486] [] __alloc_skb+0x7c/0x160 [116809.389513] [] __netdev_alloc_skb+0x38/0x60 [116809.389562] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116809.389592] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116809.389623] [] rtl8169_poll+0x41/0x260 [r8169] [116809.389656] [] net_rx_action+0x17b/0x290 [116809.389682] [] ? net_rx_action+0x102/0x290 [116809.389719] [] __do_softirq+0xac/0x190 [116809.389753] [] call_softirq+0x1c/0x50 [116809.389779] [] do_softirq+0x75/0xb0 [116809.389804] [] irq_exit+0xa5/0xb0 [116809.389836] [] do_IRQ+0x111/0x1d0 [116809.389862] [] ret_from_intr+0x0/0xf [116809.389886] [] ? queue_process+0xe5/0x160 [116809.389930] [] ? queue_process+0xe9/0x160 [116809.389957] [] ? queue_process+0x0/0x160 [116809.389990] [] ? run_workqueue+0x104/0x240 [116809.390018] [] ? run_workqueue+0xb2/0x240 [116809.390053] [] ? worker_thread+0xaf/0x130 [116809.390087] [] ? autoremove_wake_function+0x0/0x40 [116809.390114] [] ? worker_thread+0x0/0x130 [116809.390141] [] ? kthread+0x4e/0x90 [116809.390172] [] ? child_rip+0xa/0x11 [116809.390198] [] ? restore_args+0x0/0x30 [116809.390224] [] ? kthread+0x0/0x90 [116809.390254] [] ? child_rip+0x0/0x11 [116969.994551] swapper: page allocation failure. order:1, mode:0x4020 [116969.994595] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116969.994621] Call Trace: [116969.994642] [] __alloc_pages_internal+0x40e/0x530 [116969.994687] [] new_slab+0x274/0x350 [116969.994713] [] __slab_alloc+0x2c0/0x440 [116969.994741] [] ? __netdev_alloc_skb+0x38/0x60 [116969.994769] [] ? __netdev_alloc_skb+0x38/0x60 [116969.994796] [] __kmalloc_node_track_caller+0x94/0x110 [116969.994823] [] ? __netdev_alloc_skb+0x38/0x60 [116969.994850] [] __alloc_skb+0x7c/0x160 [116969.994877] [] __netdev_alloc_skb+0x38/0x60 [116969.994923] [] rtl8169_rx_fill+0xca/0x1f0 [r8169] [116969.994953] [] rtl8169_rx_interrupt+0x251/0x540 [r8169] [116969.994984] [] rtl8169_poll+0x41/0x260 [r8169] [116969.995012] [] net_rx_action+0x17b/0x290 [116969.995039] [] ? net_rx_action+0x102/0x290 [116969.995069] [] __do_softirq+0xac/0x190 [116969.995096] [] call_softirq+0x1c/0x50 [116969.995122] [] do_softirq+0x75/0xb0 [116969.995147] [] irq_exit+0xa5/0xb0 [116969.995172] [] do_IRQ+0x111/0x1d0 [116969.995200] [] ret_from_intr+0x0/0xf [116969.995225] [] ? mwait_idle+0x43/0x60 [116969.995262] [] ? mwait_idle+0x4c/0x60 [116969.995289] [] ? mwait_idle+0x43/0x60 [116969.995315] [] ? cpu_idle+0x6b/0xc0 [116969.995341] [] ? rest_init+0x66/0x70 [116969.995365] Mem-Info: [116969.995386] Node 0 DMA per-cpu: [116969.995415] CPU 0: hi: 0, btch: 1 usd: 0 [116969.995439] CPU 1: hi: 0, btch: 1 usd: 0 [116969.995462] Node 0 DMA32 per-cpu: [116969.995490] CPU 0: hi: 186, btch: 31 usd: 0 [116969.995514] CPU 1: hi: 186, btch: 31 usd: 0 [116969.995540] Active_anon:15548 active_file:2650 inactive_anon:5247 [116969.995542] inactive_file:370800 unevictable:0 dirty:20048 writeback:3434 unstable:0 [116969.995545] free:39798 slab:40301 mapped:725 pagetables:2297 bounce:0 [116969.995626] Node 0 DMA free:1492kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15144kB pages_scanned:0 all_unreclaimable? no [116969.995695] lowmem_reserve[]: 0 1976 1976 1976 [116969.995770] Node 0 DMA32 free:157616kB min:5664kB low:7080kB high:8496kB active_anon:62192kB inactive_anon:20988kB active_file:10600kB inactive_file:1483200kB unevictable:0kB present:2023748kB pages_scanned:0 all_unreclaimable? no [116969.995842] lowmem_reserve[]: 0 0 0 0 [116969.995910] Node 0 DMA: 1*4kB 0*8kB 3*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1492kB [116969.996072] Node 0 DMA32: 39144*4kB 28*8kB 13*16kB 1*32kB 1*64kB 2*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 157616kB [116969.996237] 374321 total pagecache pages [116969.996259] 704 pages in swap cache [116969.996280] Swap cache stats: add 10504, delete 9800, find 89979/90109 [116969.996312] Free swap = 1913748kB [116969.996334] Total swap = 1951800kB [116969.998448] 522160 pages RAM [116969.998448] 37425 pages reserved [116969.998448] 342755 pages shared [116969.998448] 109288 pages non-shared [116970.009917] swapper: page allocation failure. order:1, mode:0x4020 [116970.009948] Pid: 0, comm: swapper Not tainted 2.6.28-rc7 #3 [116970.009973] Call Trace: [116970.009999] [] __alloc_pages_internal+0x40e/0x530 [116970.010043] [