Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754099Ab0KHQ00 (ORCPT ); Mon, 8 Nov 2010 11:26:26 -0500 Received: from mail-ww0-f44.google.com ([74.125.82.44]:37859 "EHLO mail-ww0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753942Ab0KHQ0Y convert rfc822-to-8bit (ORCPT ); Mon, 8 Nov 2010 11:26:24 -0500 MIME-Version: 1.0 X-Originating-IP: [172.22.89.164] In-Reply-To: <1283718304.3402.370.camel@edumazet-laptop> References: <20100604092025.GA96690@trillian.comsick.at> <1275656014.2482.169.camel@edumazet-laptop> <20100604161644.GA98048@trillian.local> <20100606095640.GA4436@trillian.comsick.at> <1275820945.5238.987.camel@edumazet-laptop> <1282769089.2681.159.camel@edumazet-laptop>

<1283718304.3402.370.camel@edumazet-laptop> From: =?UTF-8?B?TmlraGlsIFNldGhpICjgpKjgpL/gpJbgpL/gpLIg4KS44KWH4KSg4KWAKQ==?= Date: Mon, 8 Nov 2010 08:25:40 -0800 Message-ID: Subject: Re: [2.6.35-rc1] page alloc failure order:1, mode:0x4020 To: Eric Dumazet Cc: Michael Guntsche , linux-kernel , netdev Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 12215 Lines: 298 Thanks Eric, for all the help. I get very occasional errors with vanilla 2.6.36 kernel with your patch. Error messages are slightly different, and very rare compared to without your patch. 2.6.35.7 seems fine, and does not show this error. rsync: page allocation failure. order:1, mode:0x4020 Pid: 31479, comm: rsync Not tainted 2.6.36-rc1 #1 Call Trace: [] ? __alloc_pages_nodemask+0x3f1/0x52e [] ? __slab_alloc+0x40f/0x45a [] ? __kmalloc_track_caller+0x91/0x97 [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? __alloc_skb+0x57/0x100 [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? ath_rx_tasklet+0x680/0xcfe [ath9k] [] ? __queue_work+0xe4/0x1a2 [] ? ath9k_tasklet+0xd0/0x134 [ath9k] [] ? tasklet_action+0x5f/0x65 [] ? __do_softirq+0x60/0xc6 [] ? do_softirq+0x2e/0x30 [] ? irq_exit+0x54/0x56 [] ? do_IRQ+0x3a/0x72 [] ? common_interrupt+0x29/0x30 [] ? cfq_may_queue+0x0/0x92 [] ? get_request+0x121/0x23e [] ? get_request_wait+0x20/0xfd [] ? cfq_merge+0x0/0x69 [] ? elv_merge+0x14b/0x170 [] ? __make_request+0x71/0x364 [] ? generic_make_request+0x14f/0x318 [] ? mempool_alloc+0x35/0xb9 [] ? submit_bio+0x3e/0x90 [] ? bio_alloc_bioset+0x33/0x9e [] ? submit_bh+0xbc/0xec [] ? ll_rw_block+0x72/0x74 [] ? __breadahead+0x20/0x30 [] ? __ext4_get_inode_loc+0x308/0x416 [] ? ext4_iget+0x52/0x652 [] ? ext4_lookup+0x65/0xbf [] ? d_alloc_and_lookup+0x30/0x58 [] ? do_lookup+0xc5/0xfe [] ? link_path_walk+0x4d1/0x8d6 [] ? path_walk+0x48/0xa9 [] ? do_path_lookup+0x3b/0x3f [] ? user_path_at+0x30/0x64 [] ? cp_new_stat64+0xee/0x100 [] ? vfs_fstatat+0x25/0x63 [] ? vfs_lstat+0x13/0x15 [] ? sys_lstat64+0xf/0x23 [] ? switched_to_idle+0x1a/0x43 [] ? syscall_call+0x7/0xb [] ? quirk_ich7_lpc+0x46/0x4d Mem-Info: DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Normal per-cpu: CPU 0: hi: 90, btch: 15 usd: 84 active_anon:14957 inactive_anon:15137 isolated_anon:0 active_file:11820 inactive_file:11640 isolated_file:0 unevictable:470 dirty:1378 writeback:0 unstable:0 free:975 slab_reclaimable:5606 slab_unreclaimable:2343 mapped:2786 shmem:72 pagetables:394 bounce:0 DMA free:1000kB min:124kB low:152kB high:184kB active_anon:2896kB inactive_anon:3552kB active_file:2116kB inactive_file:1768kB unevictable:348kB isolated(anon):0kB isolated(file):0kB present:15864kB mlocked:348kB dirty:16kB writeback:0kB mapped:228kB shmem:32kB slab_reclaimable:1136kB slab_unreclaimable:608kB kernel_stack:144kB pagetables:52kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no lowmem_reserve[]: 0 238 238 Normal free:2900kB min:1908kB low:2384kB high:2860kB active_anon:56932kB inactive_anon:56996kB active_file:45164kB inactive_file:44792kB unevictable:1532kB isolated(anon):0kB isolated(file):0kB present:243840kB mlocked:1532kB dirty:5496kB writeback:0kB mapped:10916kB shmem:256kB slab_reclaimable:21288kB slab_unreclaimable:8764kB kernel_stack:1056kB pagetables:1524kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 DMA: 6*4kB 14*8kB 16*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1000kB Normal: 691*4kB 17*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2900kB 28310 total pagecache pages 4385 pages in swap cache Swap cache stats: add 144387, delete 140002, find 98398/115667 Free swap = 889116kB Total swap = 1023996kB 65535 pages RAM 1245 pages reserved 27369 pages shared 42219 pages non-shared SLUB: Unable to allocate memory on node -1 (gfp=0x20) cache: kmalloc-8192, object size: 8192, buffer size: 8192, default order: 3, min order: 1 node 0: slabs: 133, objs: 529, free: 0 skbuff alloc of size 3872 failed rsync: page allocation failure. order:1, mode:0x4020 Pid: 31479, comm: rsync Not tainted 2.6.36-rc1 #1 Call Trace: [] ? __alloc_pages_nodemask+0x3f1/0x52e [] ? __slab_alloc+0x40f/0x45a [] ? __kmalloc_track_caller+0x91/0x97 [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? __alloc_skb+0x57/0x100 [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] [] ? ath_rx_tasklet+0x680/0xcfe [ath9k] [] ? __queue_work+0xe4/0x1a2 [] ? ath9k_tasklet+0xd0/0x134 [ath9k] [] ? tasklet_action+0x5f/0x65 [] ? __do_softirq+0x60/0xc6 [] ? do_softirq+0x2e/0x30 [] ? irq_exit+0x54/0x56 [] ? do_IRQ+0x3a/0x72 [] ? common_interrupt+0x29/0x30 [] ? cfq_may_queue+0x0/0x92 [] ? get_request+0x121/0x23e [] ? get_request_wait+0x20/0xfd [] ? cfq_merge+0x0/0x69 [] ? elv_merge+0x14b/0x170 [] ? __make_request+0x71/0x364 [] ? generic_make_request+0x14f/0x318 [] ? mempool_alloc+0x35/0xb9 [] ? submit_bio+0x3e/0x90 [] ? bio_alloc_bioset+0x33/0x9e [] ? submit_bh+0xbc/0xec [] ? ll_rw_block+0x72/0x74 [] ? __breadahead+0x20/0x30 [] ? __ext4_get_inode_loc+0x308/0x416 [] ? ext4_iget+0x52/0x652 [] ? ext4_lookup+0x65/0xbf [] ? d_alloc_and_lookup+0x30/0x58 [] ? do_lookup+0xc5/0xfe [] ? link_path_walk+0x4d1/0x8d6 [] ? path_walk+0x48/0xa9 [] ? do_path_lookup+0x3b/0x3f [] ? user_path_at+0x30/0x64 [] ? cp_new_stat64+0xee/0x100 [] ? vfs_fstatat+0x25/0x63 [] ? vfs_lstat+0x13/0x15 [] ? sys_lstat64+0xf/0x23 [] ? switched_to_idle+0x1a/0x43 [] ? syscall_call+0x7/0xb [] ? quirk_ich7_lpc+0x46/0x4d Mem-Info: DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Normal per-cpu: CPU 0: hi: 90, btch: 15 usd: 84 active_anon:14957 inactive_anon:15137 isolated_anon:0 active_file:11820 inactive_file:11640 isolated_file:0 unevictable:470 dirty:1378 writeback:0 unstable:0 free:975 slab_reclaimable:5606 slab_unreclaimable:2343 mapped:2786 shmem:72 pagetables:394 bounce:0 DMA free:1000kB min:124kB low:152kB high:184kB active_anon:2896kB inactive_anon:3552kB active_file:2116kB inactive_file:1768kB unevictable:348kB isolated(anon):0kB isolated(file):0kB present:15864kB mlocked:348kB dirty:16kB writeback:0kB mapped:228kB shmem:32kB slab_reclaimable:1136kB slab_unreclaimable:608kB kernel_stack:144kB pagetables:52kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no lowmem_reserve[]: 0 238 238 Normal free:2900kB min:1908kB low:2384kB high:2860kB active_anon:56932kB inactive_anon:56996kB active_file:45164kB inactive_file:44792kB unevictable:1532kB isolated(anon):0kB isolated(file):0kB present:243840kB mlocked:1532kB dirty:5496kB writeback:0kB mapped:10916kB shmem:256kB slab_reclaimable:21288kB slab_unreclaimable:8764kB kernel_stack:1056kB pagetables:1524kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 DMA: 6*4kB 14*8kB 16*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1000kB Normal: 691*4kB 17*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2900kB 28310 total pagecache pages 4385 pages in swap cache Swap cache stats: add 144387, delete 140002, find 98398/115667 Free swap = 889116kB Total swap = 1023996kB 65535 pages RAM 1245 pages reserved 27369 pages shared 42219 pages non-shared SLUB: Unable to allocate memory on node -1 (gfp=0x20) cache: kmalloc-8192, object size: 8192, buffer size: 8192, default order: 3, min order: 1 node 0: slabs: 133, objs: 529, free: 0 skbuff alloc of size 3872 failed 2010/9/5 Eric Dumazet > > Le dimanche 05 septembre 2010 à 08:51 -0700, Nikhil Sethi (निखिल सेठी) a > écrit : > > This came back. Here is the dmesg output. It might be the problem only > > happens if I have wmm_enabled=1 in my hostapd.conf. That is the only > > thing that I changed yesterday, and today I saw these messages in my > > logs. I will try to conform this. > > > > > > I am running 2.6.35.4 with this patch: > > > > > > *** 225,234 **** > > int error = 0, i; > > u32 size; > > > > ! > > ! common->rx_bufsize = roundup(IEEE80211_MAX_MPDU_LEN + > > ! ah->caps.rx_status_len, > > ! min(common->cachelsz, (u16)64)); > > > > ath9k_hw_set_rx_bufsize(ah, common->rx_bufsize - > > ah->caps.rx_status_len); > > --- 225,234 ---- > > int error = 0, i; > > u32 size; > > > > ! size = roundup(IEEE80211_MAX_MPDU_LEN + ah->caps.rx_status_len, > > ! min(common->cachelsz, (u16)64)); > > ! common->rx_bufsize = min_t(u32, size, > > ! SKB_MAX_ORDER(NET_SKB_PAD + > > common->cachelsz, 0)); > > > > ath9k_hw_set_rx_bufsize(ah, common->rx_bufsize - > > ah->caps.rx_status_len); > > > > > > dmesg output: > > > > > > skbuff alloc of size 3872 failed > > ps: page allocation failure. order:1, mode:0x4020 > > Pid: 16022, comm: ps Tainted: G W 2.6.35.4 #1 > > Call Trace: > > [] ? __alloc_pages_nodemask+0x3e6/0x513 > > [] ? slob_new_pages+0xc/0x23 > > Unfortunately you use SLOB, and it seems to ask order-1 pages, while > we want less than a page. > > Please try SLAB or SLUB > > > [] ? __kmalloc_node+0x39/0x72 > > [] ? __alloc_skb+0x5c/0x105 > > [] ? ath_rxbuf_alloc+0x1a/0x78 [ath] > > [] ? ath_rx_tasklet+0x2fb/0x808 [ath9k] > > [] ? scsi_next_command+0x27/0x34 > > [] ? ath9k_tasklet+0xcc/0x107 [ath9k] > > [] ? tasklet_action+0x5f/0x65 > > [] ? __do_softirq+0x60/0xc6 > > [] ? do_softirq+0x2e/0x30 > > [] ? irq_exit+0x53/0x55 > > [] ? do_IRQ+0x3a/0x72 > > [] ? common_interrupt+0x29/0x30 > > [] ? __remove_mapping+0x4b/0x8d > > [] ? try_to_release_page+0x20/0x30 > > [] ? shrink_page_list+0x2b8/0x46c > > [] ? sd_prep_fn+0x1a5/0x92b > > [] ? shrink_inactive_list+0x281/0x552 > > [] ? determine_dirtyable_memory+0xc/0x12 > > [] ? get_dirty_limits+0x18/0x2b0 > > [] ? ata_bmdma_port_intr+0x21/0xcd > > [] ? shrink_zone+0x247/0x38f > > [] ? __do_softirq+0x75/0xc6 > > [] ? shrink_slab+0x13c/0x186 > > [] ? try_to_free_pages+0x1ab/0x2be > > [] ? try_to_free_pages+0x1ab/0x2be > > [] ? __alloc_pages_nodemask+0x2b7/0x513 > > [] ? slob_new_pages+0xc/0x23 > > [] ? kmem_cache_alloc_node+0x23/0x53 > > [] ? getname+0x19/0xae > > [] ? user_path_at+0x11/0x64 > > [] ? access_process_vm+0xe0/0x1e7 > > [] ? access_process_vm+0x19f/0x1e7 > > [] ? vfs_fstatat+0x25/0x63 > > [] ? vfs_stat+0x10/0x12 > > [] ? sys_stat64+0xf/0x23 > > [] ? fput+0x88/0x191 > > [] ? dput+0x92/0xd4 > > [] ? fput+0x11c/0x191 > > [] ? mntput_no_expire+0x13/0x60 > > [] ? filp_close+0x3b/0x54 > > [] ? sys_close+0x52/0x72 > > [] ? syscall_call+0x7/0xb > > [] ? unix_dgram_sendmsg+0x258/0x3c1 > > Mem-Info: > > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/