Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759151AbZFKQPT (ORCPT ); Thu, 11 Jun 2009 12:15:19 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754148AbZFKQPI (ORCPT ); Thu, 11 Jun 2009 12:15:08 -0400 Received: from lucidpixels.com ([75.144.35.66]:57721 "EHLO lucidpixels.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751621AbZFKQPG (ORCPT ); Thu, 11 Jun 2009 12:15:06 -0400 Date: Thu, 11 Jun 2009 12:15:07 -0400 (EDT) From: Justin Piszcz To: linux-kernel@vger.kernel.org cc: xfs@oss.sgi.com Subject: Kernel 2.6.30: Memory/XFS leak, OOM killer kills many processes Message-ID: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 23772 Lines: 404 Hello, I have a daily cron that backs up my root filesystem using xfsdump, it has remain unchanged for at least 7-10 kernel versions. When I migrated to 2.6.30, when the xfsdump ran at its scheduled time, nearly all of my processes were killed due to an OOM situation, I can reproduce the situation. Kernel: 2.6.30 Dist: Debian Testing xfsdump: 2.2.48-1 Full dmesg: http://home.comcast.net/~jpiszcz/20090611/dmesg.txt Regular boot messages: [ 12.641483] DVB: registering adapter 0 frontend 0 (Samsung S5H1411 QAM/8VSB Frontend)... [ 12.647108] tda829x 1-0042: type set to tda8295 [ 12.685552] tda18271 1-0060: attaching existing instance Then I ran xfsdump: xfsdump -e -L title -M title -f /nfs/path/root.img Soon after: [ 94.950729] xfsdump invoked oom-killer: gfp_mask=0x2d0, order=3, oomkilladj=0 [ 94.954690] Pid: 2736, comm: xfsdump Not tainted 2.6.30 #1 [ 94.958662] Call Trace: [ 94.962581] [] ? oom_kill_process+0xac/0x240 [ 94.966544] [] ? badness+0x17d/0x2b0 [ 94.970454] [] ? __out_of_memory+0x144/0x170 [ 94.974351] [] ? out_of_memory+0x54/0xb0 [ 94.978196] [] ? __alloc_pages_internal+0x442/0x480 [ 94.982076] [] ? cache_alloc_refill+0x2b2/0x500 [ 94.985966] [] ? __kmalloc+0xc9/0xd0 [ 94.989846] [] ? kmem_alloc+0x59/0xf0 [ 94.993727] [] ? kmem_zalloc+0x11/0x40 [ 94.997575] [] ? xfs_getbmap+0x19a/0x7a0 [ 95.001398] [] ? xfs_bmap_search_multi_extents+0x7b/0x100 [ 95.005251] [] ? xfs_bmap_search_extents+0x67/0x120 [ 95.009075] [] ? io_schedule+0x29/0x40 [ 95.012884] [] ? xfs_getbmapx_format+0x0/0x30 [ 95.016690] [] ? __wait_on_bit_lock+0x5b/0x80 [ 95.020502] [] ? file_move+0x1c/0x60 [ 95.024342] [] ? __dentry_open+0xf1/0x2c0 [ 95.028200] [] ? xfs_ioc_getbmapx+0x6e/0x90 [ 95.032076] [] ? journal_destroy+0x148/0x1e0 [ 95.035954] [] ? xfs_file_ioctl+0x6c2/0x7d0 [ 95.039835] [] ? xfs_read+0x171/0x290 [ 95.043707] [] ? xfs_iunlock+0x70/0xb0 [ 95.047550] [] ? xfs_free_eofblocks+0x122/0x310 [ 95.051383] [] ? xfs_file_ioctl+0x0/0x7d0 [ 95.055209] [] ? journal_destroy+0x148/0x1e0 [ 95.059049] [] ? vfs_ioctl+0x2b/0x90 [ 95.062866] [] ? do_vfs_ioctl+0x7b/0x590 [ 95.066668] [] ? _atomic_dec_and_lock+0x47/0x70 [ 95.070487] [] ? dput+0xd5/0x120 [ 95.074231] [] ? mntput_no_expire+0x18/0x110 [ 95.077912] [] ? sys_ioctl+0x3d/0x70 [ 95.081589] [] ? syscall_call+0x7/0xb [ 95.085252] [] ? journal_destroy+0x148/0x1e0 [ 95.088922] [] ? vmx_create_vcpu+0x100/0x800 [ 95.092534] Mem-Info: [ 95.096071] DMA per-cpu: [ 95.099563] CPU 0: hi: 0, btch: 1 usd: 0 [ 95.103056] CPU 1: hi: 0, btch: 1 usd: 0 [ 95.106458] Normal per-cpu: [ 95.109747] CPU 0: hi: 186, btch: 31 usd: 0 [ 95.113018] CPU 1: hi: 186, btch: 31 usd: 24 [ 95.116248] HighMem per-cpu: [ 95.119387] CPU 0: hi: 186, btch: 31 usd: 0 [ 95.122488] CPU 1: hi: 186, btch: 31 usd: 29 [ 95.125561] Active_anon:4735 active_file:1703 inactive_anon:972 [ 95.125562] inactive_file:376179 unevictable:0 dirty:370 writeback:0 unstable:272 [ 95.125563] free:445875 slab:202789 mapped:3133 pagetables:302 bounce:0 [ 95.135127] DMA free:3764kB min:284kB low:352kB high:424kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15868kB pages_scanned:0 all_unreclaimable? no [ 95.142051] lowmem_reserve[]: 0 867 4038 4038 [ 95.145533] Normal free:35440kB min:16096kB low:20120kB high:24144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:12kB unevictable:0kB present:887976kB pages_scanned:0 all_unreclaimable? no [ 95.152926] lowmem_reserve[]: 0 0 25374 25374 [ 95.156652] HighMem free:1744296kB min:512kB low:15228kB high:29948kB active_anon:18940kB inactive_anon:3888kB active_file:6812kB inactive_file:1504704kB unevictable:0kB present:3247956kB pages_scanned:0 all_unreclaimable? no [ 95.164621] lowmem_reserve[]: 0 0 0 0 [ 95.168632] DMA: 7*4kB 5*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3764kB [ 95.172797] Normal: 1948*4kB 1448*8kB 861*16kB 12*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 35456kB [ 95.177043] HighMem: 349*4kB 3582*8kB 5821*16kB 7689*32kB 5889*64kB 3020*128kB 113*256kB 2*512kB 2*1024kB 3*2048kB 140*4096kB = 1744276kB [ 95.181388] 377922 total pagecache pages [ 95.185665] 0 pages in swap cache [ 95.189858] Swap cache stats: add 0, delete 0, find 0/0 [ 95.194006] Free swap = 2056280kB [ 95.198125] Total swap = 2056280kB [ 95.211202] 1310720 pages RAM [ 95.215282] 1082882 pages HighMem [ 95.219354] 274370 pages reserved [ 95.223398] 13360 pages shared [ 95.227421] 412265 pages non-shared [ 95.231430] Out of memory: kill process 2623 (sshd) score 2840 or a child [ 95.235525] Killed process 2624 (bash) [ 95.246873] xfsdump invoked oom-killer: gfp_mask=0x2d0, order=3, oomkilladj=0 [ 95.251069] Pid: 2736, comm: xfsdump Not tainted 2.6.30 #1 [ 95.255273] Call Trace: [ 95.259430] [] ? oom_kill_process+0xac/0x240 [ 95.263650] [] ? badness+0x17d/0x2b0 [ 95.267849] [] ? __out_of_memory+0x144/0x170 [ 95.272052] [] ? out_of_memory+0x54/0xb0 [ 95.276249] [] ? __alloc_pages_internal+0x442/0x480 [ 95.280467] [] ? cache_alloc_refill+0x2b2/0x500 [ 95.284699] [] ? __kmalloc+0xc9/0xd0 [ 95.288900] [] ? kmem_alloc+0x59/0xf0 [ 95.293096] [] ? kmem_zalloc+0x11/0x40 [ 95.297278] [] ? xfs_getbmap+0x19a/0x7a0 [ 95.301462] [] ? xfs_bmap_search_multi_extents+0x7b/0x100 [ 95.305690] [] ? xfs_bmap_search_extents+0x67/0x120 [ 95.309907] [] ? io_schedule+0x29/0x40 [ 95.314124] [] ? xfs_getbmapx_format+0x0/0x30 [ 95.318354] [] ? __wait_on_bit_lock+0x5b/0x80 [ 95.322580] [] ? file_move+0x1c/0x60 [ 95.326805] [] ? __dentry_open+0xf1/0x2c0 [ 95.331029] [] ? xfs_ioc_getbmapx+0x6e/0x90 [ 95.335252] [] ? journal_destroy+0x148/0x1e0 [ 95.339471] [] ? xfs_file_ioctl+0x6c2/0x7d0 [ 95.343688] [] ? xfs_read+0x171/0x290 [ 95.347902] [] ? xfs_iunlock+0x70/0xb0 [ 95.352106] [] ? xfs_free_eofblocks+0x122/0x310 [ 95.356329] [] ? xfs_file_ioctl+0x0/0x7d0 [ 95.360523] [] ? journal_destroy+0x148/0x1e0 [ 95.364744] [] ? vfs_ioctl+0x2b/0x90 [ 95.368964] [] ? do_vfs_ioctl+0x7b/0x590 [ 95.373215] [] ? _atomic_dec_and_lock+0x47/0x70 [ 95.377486] [] ? dput+0xd5/0x120 [ 95.381746] [] ? mntput_no_expire+0x18/0x110 [ 95.386050] [] ? sys_ioctl+0x3d/0x70 [ 95.390346] [] ? syscall_call+0x7/0xb [ 95.394632] [] ? journal_destroy+0x148/0x1e0 [ 95.398961] [] ? vmx_create_vcpu+0x100/0x800 [ 95.403256] Mem-Info: [ 95.407483] DMA per-cpu: [ 95.411649] CPU 0: hi: 0, btch: 1 usd: 0 [ 95.415715] CPU 1: hi: 0, btch: 1 usd: 0 [ 95.419606] Normal per-cpu: [ 95.423454] CPU 0: hi: 186, btch: 31 usd: 11 [ 95.427208] CPU 1: hi: 186, btch: 31 usd: 13 [ 95.430786] HighMem per-cpu: [ 95.434324] CPU 0: hi: 186, btch: 31 usd: 158 [ 95.437762] CPU 1: hi: 186, btch: 31 usd: 0 [ 95.441028] Active_anon:3884 active_file:1703 inactive_anon:972 [ 95.441029] inactive_file:376429 unevictable:0 dirty:518 writeback:0 unstable:272 [ 95.441030] free:446299 slab:202878 mapped:3133 pagetables:302 bounce:0 [ 95.450772] DMA free:3764kB min:284kB low:352kB high:424kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15868kB pages_scanned:0 all_unreclaimable? no [ 95.457350] lowmem_reserve[]: 0 867 4038 4038 [ 95.460630] Normal free:35236kB min:16096kB low:20120kB high:24144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:887976kB pages_scanned:0 all_unreclaimable? no [ 95.467655] lowmem_reserve[]: 0 0 25374 25374 [ 95.471228] HighMem free:1746196kB min:512kB low:15228kB high:29948kB active_anon:15536kB inactive_anon:3888kB active_file:6812kB inactive_file:1505740kB unevictable:0kB present:3247956kB pages_scanned:0 all_unreclaimable? no [ 95.478934] lowmem_reserve[]: 0 0 0 0 [ 95.482849] DMA: 7*4kB 5*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3764kB [ 95.486958] Normal: 1957*4kB 1449*8kB 863*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 35212kB [ 95.491174] HighMem: 409*4kB 3632*8kB 5839*16kB 7713*32kB 5891*64kB 3021*128kB 113*256kB 2*512kB 2*1024kB 3*2048kB 140*4096kB = 1746228kB [ 95.495476] 378156 total pagecache pages [ 95.499695] 0 pages in swap cache [ 95.503863] Swap cache stats: add 0, delete 0, find 0/0 [ 95.508025] Free swap = 2056280kB [ 95.512156] Total swap = 2056280kB [ 95.525018] 1310720 pages RAM [ 95.529107] 1082882 pages HighMem [ 95.533174] 274370 pages reserved [ 95.537230] 12393 pages shared [ 95.541262] 411399 pages non-shared [ 95.545282] Out of memory: kill process 2506 (sshd) score 2839 or a child [ 95.549384] Killed process 2508 (bash) [ 95.554123] xfsdump invoked oom-killer: gfp_mask=0x2d0, order=3, oomkilladj=0 [ 95.558391] Pid: 2736, comm: xfsdump Not tainted 2.6.30 #1 [ 95.562595] Call Trace: [ 95.566756] [] ? oom_kill_process+0xac/0x240 [ 95.570971] [] ? badness+0x17d/0x2b0 [ 95.575168] [] ? __out_of_memory+0x144/0x170 [ 95.579370] [] ? out_of_memory+0x54/0xb0 [ 95.583564] [] ? __alloc_pages_internal+0x442/0x480 [ 95.587778] [] ? cache_alloc_refill+0x2b2/0x500 [ 95.591997] [] ? __kmalloc+0xc9/0xd0 [ 95.596205] [] ? kmem_alloc+0x59/0xf0 [ 95.600408] [] ? kmem_zalloc+0x11/0x40 [ 95.604600] [] ? xfs_getbmap+0x19a/0x7a0 [ 95.608790] [] ? xfs_bmap_search_multi_extents+0x7b/0x100 [ 95.613022] [] ? xfs_bmap_search_extents+0x67/0x120 [ 95.617243] [] ? io_schedule+0x29/0x40 [ 95.621467] [] ? xfs_getbmapx_format+0x0/0x30 [ 95.625702] [] ? __wait_on_bit_lock+0x5b/0x80 [ 95.629936] [] ? file_move+0x1c/0x60 [ 95.634169] [] ? __dentry_open+0xf1/0x2c0 [ 95.638400] [] ? xfs_ioc_getbmapx+0x6e/0x90 [ 95.642628] [] ? journal_destroy+0x148/0x1e0 [ 95.646853] [] ? xfs_file_ioctl+0x6c2/0x7d0 [ 95.651078] [] ? xfs_read+0x171/0x290 [ 95.655300] [] ? xfs_iunlock+0x70/0xb0 [ 95.659511] [] ? xfs_free_eofblocks+0x122/0x310 [ 95.663741] [] ? xfs_file_ioctl+0x0/0x7d0 [ 95.667939] [] ? journal_destroy+0x148/0x1e0 [ 95.672165] [] ? vfs_ioctl+0x2b/0x90 [ 95.676390] [] ? do_vfs_ioctl+0x7b/0x590 [ 95.680651] [] ? _atomic_dec_and_lock+0x47/0x70 [ 95.684929] [] ? dput+0xd5/0x120 [ 95.689194] [] ? mntput_no_expire+0x18/0x110 [ 95.693505] [] ? sys_ioctl+0x3d/0x70 [ 95.697806] [] ? syscall_call+0x7/0xb [ 95.702093] [] ? journal_destroy+0x148/0x1e0 [ 95.706426] [] ? vmx_create_vcpu+0x100/0x800 [ 95.710722] Mem-Info: [ 95.714947] DMA per-cpu: [ 95.719110] CPU 0: hi: 0, btch: 1 usd: 0 [ 95.723172] CPU 1: hi: 0, btch: 1 usd: 0 [ 95.727058] Normal per-cpu: [ 95.730902] CPU 0: hi: 186, btch: 31 usd: 0 [ 95.734648] CPU 1: hi: 186, btch: 31 usd: 7 [ 95.738221] HighMem per-cpu: [ 95.741750] CPU 0: hi: 186, btch: 31 usd: 0 [ 95.745178] CPU 1: hi: 186, btch: 31 usd: 109 [ 95.748434] Active_anon:3070 active_file:1777 inactive_anon:972 [ 95.748435] inactive_file:376392 unevictable:0 dirty:518 writeback:0 unstable:272 [ 95.748436] free:447166 slab:202878 mapped:3133 pagetables:277 bounce:0 [ 95.758139] DMA free:3764kB min:284kB low:352kB high:424kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15868kB pages_scanned:0 all_unreclaimable? no [ 95.764686] lowmem_reserve[]: 0 867 4038 4038 [ 95.767953] Normal free:35304kB min:16096kB low:20120kB high:24144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:887976kB pages_scanned:0 all_unreclaimable? no [ 95.774949] lowmem_reserve[]: 0 0 25374 25374 [ 95.778505] HighMem free:1749596kB min:512kB low:15228kB high:29948kB active_anon:12280kB inactive_anon:3888kB active_file:7108kB inactive_file:1505592kB unevictable:0kB present:3247956kB pages_scanned:0 all_unreclaimable? no [ 95.786177] lowmem_reserve[]: 0 0 0 0 [ 95.790072] DMA: 7*4kB 5*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3764kB [ 95.794159] Normal: 1961*4kB 1445*8kB 866*16kB 6*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 35372kB [ 95.798354] HighMem: 740*4kB 3815*8kB 5877*16kB 7713*32kB 5891*64kB 3021*128kB 113*256kB 2*512kB 2*1024kB 3*2048kB 140*4096kB = 1749624kB [ 95.802636] 378181 total pagecache pages [ 95.806838] 0 pages in swap cache [ 95.810988] Swap cache stats: add 0, delete 0, find 0/0 [ 95.815132] Free swap = 2056280kB [ 95.819246] Total swap = 2056280kB [ 95.831982] 1310720 pages RAM [ 95.836054] 1082882 pages HighMem [ 95.840099] 274370 pages reserved [ 95.844136] 11446 pages shared [ 95.848146] 410690 pages non-shared [ 95.852142] ------------[ cut here ]------------ [ 95.856153] WARNING: at mm/oom_kill.c:327 __oom_kill_task+0x73/0xa0() [ 95.860229] Hardware name: [ 95.864297] Modules linked in: s5h1411 tda18271 tda8290 tuner cx25840 snd_usb_audio snd_usb_lib snd_seq_midi snd_rawmidi snd_hwdep pvrusb2 dvb_core cx2341x v4l2_common pwc videodev tveeprom v4l1_compat [ 95.873044] Pid: 2736, comm: xfsdump Not tainted 2.6.30 #1 [ 95.877428] Call Trace: [ 95.881772] [] ? __oom_kill_task+0x73/0xa0 [ 95.886159] [] ? __oom_kill_task+0x73/0xa0 [ 95.890511] [] ? warn_slowpath_common+0x76/0xd0 [ 95.894866] [] ? __oom_kill_task+0x73/0xa0 [ 95.899220] [] ? warn_slowpath_null+0x13/0x20 [ 95.903575] [] ? __oom_kill_task+0x73/0xa0 [ 95.907932] [] ? oom_kill_process+0x41/0x240 [ 95.912290] [] ? badness+0x17d/0x2b0 [ 95.916643] [] ? __out_of_memory+0x144/0x170 [ 95.921017] [] ? out_of_memory+0x54/0xb0 [ 95.925383] [] ? __alloc_pages_internal+0x442/0x480 [ 95.929781] [] ? cache_alloc_refill+0x2b2/0x500 [ 95.934179] [] ? __kmalloc+0xc9/0xd0 [ 95.938564] [] ? kmem_alloc+0x59/0xf0 [ 95.942949] [] ? kmem_zalloc+0x11/0x40 [ 95.947314] [] ? xfs_getbmap+0x19a/0x7a0 [ 95.951668] [] ? xfs_bmap_search_multi_extents+0x7b/0x100 [ 95.956056] [] ? xfs_bmap_search_extents+0x67/0x120 [ 95.960430] [] ? io_schedule+0x29/0x40 [ 95.964810] [] ? xfs_getbmapx_format+0x0/0x30 [ 95.969197] [] ? __wait_on_bit_lock+0x5b/0x80 [ 95.973593] [] ? file_move+0x1c/0x60 [ 95.977969] [] ? __dentry_open+0xf1/0x2c0 [ 95.982370] [] ? xfs_ioc_getbmapx+0x6e/0x90 [ 95.986782] [] ? journal_destroy+0x148/0x1e0 [ 95.991225] [] ? xfs_file_ioctl+0x6c2/0x7d0 [ 95.995674] [] ? xfs_read+0x171/0x290 [ 96.000107] [] ? xfs_iunlock+0x70/0xb0 [ 96.004569] [] ? xfs_free_eofblocks+0x122/0x310 [ 96.009042] [] ? xfs_file_ioctl+0x0/0x7d0 [ 96.013496] [] ? journal_destroy+0x148/0x1e0 [ 96.017988] [] ? vfs_ioctl+0x2b/0x90 [ 96.022449] [] ? do_vfs_ioctl+0x7b/0x590 [ 96.026867] [] ? _atomic_dec_and_lock+0x47/0x70 [ 96.031257] [] ? dput+0xd5/0x120 [ 96.035513] [] ? mntput_no_expire+0x18/0x110 [ 96.039642] [] ? sys_ioctl+0x3d/0x70 [ 96.043751] [] ? syscall_call+0x7/0xb [ 96.047727] [] ? journal_destroy+0x148/0x1e0 [ 96.051573] [] ? vmx_create_vcpu+0x100/0x800 [ 96.055393] ---[ end trace 05f373a7e53c7489 ]--- [ 96.059088] tried to kill an mm-less task! -- I re-ran the xfsdump without having any modules loaded: Then I ran xfsdump (again): xfsdump -e -L title -M title -f /nfs/path/root.img http://home.comcast.net/~jpiszcz/20090611/dmesg2.txt dmesg: [ 1.388253] sd 1:0:0:0: [sdb] 1465149168 512-byte hardware sectors: (750 GB/698 GiB) [ 1.392302] sd 1:0:0:0: [sdb] Write Protect is off [ 1.396317] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00 [ 1.396329] scsi 2:0:0:0: CD-ROM TSSTcorp CD/DVDW SH-S183L SB02 PQ: 0 ANSI: 5 [ 1.396331] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 1.396373] sdb: sdb1 [ 1.408771] sd 1:0:0:0: [sdb] Attached SCSI disk [ 1.418982] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray [ 1.423106] Uniform CD-ROM driver Revision: 3.20 [ 1.427232] sr 2:0:0:0: Attached scsi CD-ROM sr0 [ 1.610651] UDF-fs: No partition found (1) [ 1.616011] XFS mounting filesystem sda2 [ 1.629087] usb 2-1: configuration #1 chosen from 1 choice [ 1.677313] Ending clean XFS mount for filesystem: sda2 [ 1.677344] VFS: Mounted root (xfs filesystem) readonly on device 8:2. [ 1.681438] Freeing unused kernel memory: 316k freed [ 1.843013] usb 2-2: new low speed USB device using uhci_hcd and address 3 [ 2.017021] usb 2-2: configuration #1 chosen from 1 choice [ 2.039154] input: Microsoft Microsoft 5-Button Mouse with IntelliEye(TM) as /class/input/input4 [ 2.043605] generic-usb 0003:045E:0039.0001: input: USB HID v1.10 Mouse [Microsoft Microsoft 5-Button Mouse with IntelliEye(TM)] on usb-0000:00:1d.0-2/input0 [ 2.255013] usb 3-2: new low speed USB device using uhci_hcd and address 2 [ 2.425812] usb 3-2: configuration #1 chosen from 1 choice [ 2.705758] generic-usb 0003:050D:1100.0002: hiddev96: USB HID v1.11 Device [Belkin Belkin UPS] on usb-0000:00:1d.1-2/input0 [ 2.959805] Adding 2056280k swap on /dev/sda1. Priority:-1 extents:1 across:2056280k SS [ 4.053399] XFS mounting filesystem sdb1 [ 4.173486] Ending clean XFS mount for filesystem: sdb1 [ 4.472295] e1000e 0000:03:00.0: irq 30 for MSI/MSI-X [ 4.524118] e1000e 0000:03:00.0: irq 30 for MSI/MSI-X [ 7.468987] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ 123.857734] xfsdump invoked oom-killer: gfp_mask=0x2d0, order=3, oomkilladj=0 [ 123.861449] Pid: 2491, comm: xfsdump Not tainted 2.6.30 #1 [ 123.865191] Call Trace: [ 123.868812] [] ? oom_kill_process+0xac/0x240 [ 123.872490] [] ? badness+0x17d/0x2b0 [ 123.875995] [] ? __out_of_memory+0x144/0x170 [ 123.879524] [] ? out_of_memory+0x54/0xb0 [ 123.883047] [] ? __alloc_pages_internal+0x442/0x480 [ 123.886622] [] ? cache_alloc_refill+0x2b2/0x500 [ 123.890202] [] ? __kmalloc+0xc9/0xd0 [ 123.893771] [] ? kmem_alloc+0x59/0xf0 [ 123.897252] [] ? kmem_zalloc+0x11/0x40 [ 123.900643] [] ? xfs_getbmap+0x19a/0x7a0 [ 123.904044] [] ? xfs_bmap_search_multi_extents+0x7b/0x100 [ 123.907483] [] ? xfs_bmap_search_extents+0x67/0x120 [ 123.910914] [] ? io_schedule+0x29/0x40 [ 123.914354] [] ? xfs_getbmapx_format+0x0/0x30 [ 123.917778] [] ? __wait_on_bit_lock+0x5b/0x80 [ 123.921182] [] ? file_move+0x1c/0x60 [ 123.924574] [] ? __dentry_open+0xf1/0x2c0 [ 123.927952] [] ? xfs_ioc_getbmapx+0x6e/0x90 [ 123.931325] [] ? journal_destroy+0x148/0x1e0 [ 123.934679] [] ? xfs_file_ioctl+0x6c2/0x7d0 [ 123.938040] [] ? xfs_read+0x171/0x290 [ 123.941421] [] ? xfs_iunlock+0x70/0xb0 [ 123.944810] [] ? xfs_free_eofblocks+0x122/0x310 [ 123.948223] [] ? xfs_file_ioctl+0x0/0x7d0 [ 123.951624] [] ? journal_destroy+0x148/0x1e0 [ 123.955045] [] ? vfs_ioctl+0x2b/0x90 [ 123.958454] [] ? do_vfs_ioctl+0x7b/0x590 [ 123.961841] [] ? _atomic_dec_and_lock+0x47/0x70 [ 123.965221] [] ? dput+0xd5/0x120 [ 123.968596] [] ? mntput_no_expire+0x18/0x110 [ 123.971995] [] ? sys_ioctl+0x3d/0x70 [ 123.975370] [] ? syscall_call+0x7/0xb [ 123.978721] [] ? journal_destroy+0x148/0x1e0 [ 123.982087] [] ? vmx_create_vcpu+0x100/0x800 [ 123.985442] Mem-Info: [ 123.988765] DMA per-cpu: [ 123.992064] CPU 0: hi: 0, btch: 1 usd: 0 [ 123.995385] CPU 1: hi: 0, btch: 1 usd: 0 [ 123.998622] Normal per-cpu: [ 124.001779] CPU 0: hi: 186, btch: 31 usd: 0 [ 124.004945] CPU 1: hi: 186, btch: 31 usd: 25 [ 124.008051] HighMem per-cpu: [ 124.011138] CPU 0: hi: 186, btch: 31 usd: 0 [ 124.014257] CPU 1: hi: 186, btch: 31 usd: 58 [ 124.017336] Active_anon:3977 active_file:1686 inactive_anon:822 [ 124.017337] inactive_file:367773 unevictable:0 dirty:444 writeback:0 unstable:1283 [ 124.017338] free:458376 slab:200088 mapped:3121 pagetables:272 bounce:0 [ 124.026921] DMA free:3780kB min:284kB low:352kB high:424kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15868kB pages_scanned:0 all_unreclaimable? no [ 124.033860] lowmem_reserve[]: 0 867 4038 4038 [ 124.037351] Normal free:46460kB min:16096kB low:20120kB high:24144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:887976kB pages_scanned:0 all_unreclaimable? no [ 124.044760] lowmem_reserve[]: 0 0 25374 25374 [ 124.048491] HighMem free:1783264kB min:512kB low:15228kB high:29948kB active_anon:15908kB inactive_anon:3288kB active_file:6744kB inactive_file:1471092kB unevictable:0kB present:3247956kB pages_scanned:0 all_unreclaimable? no [ 124.056476] lowmem_reserve[]: 0 0 0 0 [ 124.060497] DMA: 9*4kB 12*8kB 12*16kB 4*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3780kB [ 124.064676] Normal: 3898*4kB 1834*8kB 885*16kB 2*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 46472kB [ 124.068914] HighMem: 872*4kB 3147*8kB 7562*16kB 8323*32kB 5774*64kB 2973*128kB 107*256kB 2*512kB 1*1024kB 3*2048kB 142*4096kB = 1783288kB [ 124.073250] 369489 total pagecache pages [ 124.077516] 0 pages in swap cache [ 124.081697] Swap cache stats: add 0, delete 0, find 0/0 [ 124.085833] Free swap = 2056280kB [ 124.089939] Total swap = 2056280kB [ 124.102973] 1310720 pages RAM [ 124.107035] 1082882 pages HighMem [ 124.111095] 274370 pages reserved [ 124.115123] 13146 pages shared [ 124.119127] 407410 pages non-shared [ 124.123114] Out of memory: kill process 2393 (sshd) score 2839 or a child [ 124.127189] Killed process 2395 (bash) Any clues? Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/