Hello,
I can reproducible crash my machine by writing bulk data from a
socket to an encrypted partition. It always crashes after a few
Gigabytes have been written.
The Partition in charge is using dm-crypt+xfs filesystem.
Here is what I captured using a serial console cable:
--cut--
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813a5aa4>] ? tcp_v4_rcv+0x448/0x6a5
[<ffffffff81347f32>] ? __netdev_alloc_skb+0x15/0x2f
[<ffffffff81241147>] ? e1000_alloc_rx_buffers+0x90/0x374
[<ffffffff81241812>] ? e1000_clean_rx_irq+0x3e7/0x429
[<ffffffff812440a1>] ? e1000_clean+0x318/0x4af
[<ffffffff8138d4df>] ? ip_local_deliver_finish+0xf3/0x192
[<ffffffff8138d3b9>] ? ip_rcv+0x236/0x269
[<ffffffff8134d2e8>] ? net_rx_action+0x71/0x136
[<ffffffff8104287c>] ? getnstimeofday+0x55/0xaf
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff81347f32>] ? __netdev_alloc_skb+0x15/0x2f
[<ffffffff81255cd7>] ? rhine_napipoll+0x381/0x490
[<ffffffff8134d2e8>] ? net_rx_action+0x71/0x136
[<ffffffff8104287c>] ? getnstimeofday+0x55/0xaf
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff8123bf37>] ? ata_bmdma_setup+0x53/0x6d
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81189468>] ? __blk_run_queue+0x41/0x6e
[<ffffffff81197f2c>] ? rb_insert_color+0x66/0xe2
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff8102897e>] ? check_preempt_wakeup+0x22/0x110
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81189468>] ? __blk_run_queue+0x41/0x6e
[<ffffffff81197f2c>] ? rb_insert_color+0x66/0xe2
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff8102897e>] ? check_preempt_wakeup+0x22/0x110
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810323e5>] ? irq_exit+0x40/0x6d
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81189468>] ? __blk_run_queue+0x41/0x6e
[<ffffffff81197f2c>] ? rb_insert_color+0x66/0xe2
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff8102897e>] ? check_preempt_wakeup+0x22/0x110
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81040063>] ? hrtimer_interrupt+0xf1/0x106
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81040063>] ? hrtimer_interrupt+0xf1/0x106
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff8102897e>] ? check_preempt_wakeup+0x22/0x110
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810323e5>] ? irq_exit+0x40/0x6d
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810323e5>] ? irq_exit+0x40/0x6d
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810323e5>] ? irq_exit+0x40/0x6d
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff81251858>] ? mdio_ctrl_hw+0xbf/0x122
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813b007f>] ? inet_sk_rebuild_header+0x1b/0x2fa
[<ffffffff81347a0d>] ? pskb_copy+0x19/0x166
[<ffffffff813a0a69>] ? tcp_transmit_skb+0x84/0x623
[<ffffffff813a05b7>] ? tcp_current_mss+0x3c/0x57
[<ffffffff813a27f2>] ? tcp_retransmit_skb+0x4a1/0x591
[<ffffffff813a3f29>] ? tcp_write_timer+0x3b1/0x590
[<ffffffff813a3b78>] ? tcp_write_timer+0x0/0x590
[<ffffffff81035b8c>] ? run_timer_softirq+0x16c/0x1cb
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103af23>] ? worker_thread+0x106/0x187
[<ffffffff8103d99d>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x187
[<ffffffff8103d639>] ? kthread+0x85/0x8d
[<ffffffff8102b6ab>] ? finish_task_switch+0x2a/0x69
[<ffffffff8100ba6a>] ? child_rip+0xa/0x20
[<ffffffff8103d5b4>] ? kthread+0x0/0x8d
[<ffffffff8100ba60>] ? child_rip+0x0/0x20
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 184
Active_anon:11837 active_file:39855 inactive_anon:18739
inactive_file:40700 unevictable:0 dirty:7416 writeback:4093 unstable:0
free:759 slab:10144 mapped:2777 pagetables:2524 bounce:0
DMA free:1992kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:124kB active_file:4904kB inactive_file:8236kB unevictable:0kB present:15308kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 489 489 489
DMA32 free:1044kB min:2784kB low:3480kB high:4176kB active_anon:47348kB inactive_anon:74832kB active_file:154516kB inactive_file:154564kB unevictable:0kB present:500768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
DMA32: 261*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1044kB
95554 total pagecache pages
14741 pages in swap cache
Swap cache stats: add 40531, delete 25790, find 57320/57805
Free swap = 856060kB
Total swap = 987832kB
131008 pages RAM
4139 pages reserved
92164 pages shared
45322 pages non-shared
kcryptd: page allocation failure. order:0, mode:0x20
Pid: 4904, comm: kcryptd Not tainted 2.6.31.4-ztdummy #1
Call Trace:
<IRQ> [<ffffffff810594ae>] ? __alloc_pages_nodemask+0x4c5/0x50c
[<ffffffff810753eb>] ? cache_alloc_refill+0x218/0x454
[<ffffffff8107569c>] ? __kmalloc+0x75/0x9e
[<ffffffff81347361>] ? __alloc_skb+0x59/0x149
[<ffffffff813a5aa4>] ? tcp_v4_rcv+0x448/0x6a5
[<ffffffff81347f32>] ? __netdev_alloc_skb+0x15/0x2f
[<ffffffff81241147>] ? e1000_alloc_rx_buffers+0x90/0x374
[<ffffffff812417b1>] ? e1000_clean_rx_irq+0x386/0x429
[<ffffffff8134dce0>] ? dev_queue_xmit+0x2f3/0x321
[<ffffffff812440a1>] ? e1000_clean+0x318/0x4af
[<ffffffff81257bc7>] ? ppp_start_xmit+0x134/0x150
[<ffffffff8134d2e8>] ? net_rx_action+0x71/0x136
[<ffffffff81032676>] ? __do_softirq+0x7e/0x108
[<ffffffff81241b08>] ? e1000_intr+0x10a/0x148
[<ffffffff8100bb6a>] ? call_softirq+0x1a/0x24
[<ffffffff8100d084>] ? do_softirq+0x28/0x62
[<ffffffff810323de>] ? irq_exit+0x39/0x6d
[<ffffffff8100cbb2>] ? do_IRQ+0x86/0x9a
[<ffffffff8100b593>] ? ret_from_intr+0x0/0x9
<EOI> [<ffffffff8117541f>] ? crypto_cbc_encrypt+0x10f/0x15a
[<ffffffff81025a02>] ? aes_encrypt+0x0/0x9
[<ffffffff811668f5>] ? async_encrypt+0x35/0x3a
[<ffffffff812ef115>] ? crypt_convert+0x1e9/0x266
[<ffffffff812ef57d>] ? kcryptd_crypt+0x3eb/0x40d
[<ffffffff812ef192>] ? kcryptd_crypt+0x0/0x40d
[<ffffffff8103ae1d>] ? worker_thread+0x0/0x18?
--cut--
This is repeating over and over again until I did a hard reboot.
Sven
--
"Those who do not understand Unix are condemned to reinvent it, poorly"
(Henry Spencer)
/me is giggls@ircnet, http://sven.gegg.us/ on the Web
Hello Sven,
Sven Geggus wrote:
> I can reproducible crash my machine by writing bulk data from a
> socket to an encrypted partition. It always crashes after a few
> Gigabytes have been written.
>
> The Partition in charge is using dm-crypt+xfs filesystem.
This is without any doubt related to an issue that's already being
investigated. I have to warn you that the thread is very long:
http://thread.gmane.org/gmane.linux.kernel/896714
What is the _exact_ command sequence you use to reproduce it? I already
have a testcase, but a second test case, or a simpler one, may be useful.
In all cases reported so far, and also in your case, networking is involved
in the actual allocation errors.
It would also be useful if you could try to bisect the issue independently.
For me bisection has proven difficult because the symptoms change
between .30 and .31. The suspicion is that more than one change is
involved in the regression.
Cheers,
FJP
Frans Pop <[email protected]> wrote:
> What is the _exact_ command sequence you use to reproduce it? I already
> have a testcase, but a second test case, or a simpler one, may be useful.
Not a particular easy testcase.
This is what I did:
On the crashing machine with the dm-encrypted xfs volume:
ionice -c 3 socat TCP4-LISTEN:5555 - >backup.tar
On the source machine:
tar cv dir |socat - TCP4:targetmachine:5555
You will certainly not need to use tar.
socat /dev/zero TCP4:targetmachine:5555 should work as well.
I don't know if TCP traffic is really needed probably it is.
Regards
Sven
--
"We don't know the OS that God uses, but the Vatican uses Linux"
(Sister Judith Zoebelein, Vatican Webmaster)
/me is giggls@ircnet, http://sven.gegg.us/ on the Web
(Sven: For kernel mailing lists please always do a "reply to all".
Although some other communities do not want that, it is standard for the
kernel community. It is needed because otherwise, with the huge amount of
traffic on the linux-kernel list, people are too likely to miss replies.)
Sven Geggus wrote:
> Frans Pop <[email protected]> wrote:
>
>> What is the _exact_ command sequence you use to reproduce it? I already
>> have a testcase, but a second test case, or a simpler one, may be
>> useful.
>
> Not a particular easy testcase. This is what I did:
>
> On the crashing machine with the dm-encrypted xfs volume:
> ionice -c 3 socat TCP4-LISTEN:5555 - >backup.tar
>
> On the source machine:
> tar cv dir |socat - TCP4:targetmachine:5555
>
> You will certainly not need to use tar.
>
> socat /dev/zero TCP4:targetmachine:5555 should work as well.
>
> I don't know if TCP traffic is really needed probably it is.
Thanks.
In the mean time I've been able to trace the culprit. Could you please try
if reverting 373c0a7e + 8aa7e847 [1] on top of 2.6.31 fixes the issue for
you?
Cheers,
FJP
[1] The first commit is a build fix for the second.
Frans Pop schrieb am Montag, den 19. Oktober um 01:41 Uhr:
> In the mean time I've been able to trace the culprit. Could you please try
> if reverting 373c0a7e + 8aa7e847 [1] on top of 2.6.31 fixes the issue for
> you?
Unfortunately not :(
Starting from 2.6.31.4 I did
git revert 373c0a7e
git revert 8aa7e847 and build a new kernel.
The problem persists. The Kernel crashed again, this
time in "swapper".
Regards
Sven
--
"I'm a bastard, and proud of it"
(Linus Torvalds, Wednesday Sep 6, 2000)
/me is giggls@ircnet, http://sven.gegg.us/ on the Web
On Tue, Oct 20, 2009 at 09:16:57PM +0200, Sven Geggus wrote:
> Frans Pop schrieb am Montag, den 19. Oktober um 01:41 Uhr:
>
> > In the mean time I've been able to trace the culprit. Could you please try
> > if reverting 373c0a7e + 8aa7e847 [1] on top of 2.6.31 fixes the issue for
> > you?
>
> Unfortunately not :(
>
> Starting from 2.6.31.4 I did
> git revert 373c0a7e
> git revert 8aa7e847 and build a new kernel.
>
> The problem persists. The Kernel crashed again, this
> time in "swapper".
>
Can you please try with this patch also applied? i.e. this patch with
the two reverts. I'm looking for either allocation failures or the
WARN_ON triggering.
Thanks
==== CUT HERE ====
page-allocator: Always wake kswapd when restarting an allocation attempt after direct reclaim failed
If a direct reclaim makes no forward progress, it considers whether it
should go OOM or not. Whether OOM is triggered or not, it may retry the
application afterwards. In times past, this would always wake kswapd as well
but currently, kswapd is not woken up after direct reclaim fails. For order-0
allocations, this makes little difference but if there is a heavy mix of
higher-order allocations that direct reclaim is failing for, it might mean
that kswapd is not rewoken for higher orders as much as it did previously.
This patch wakes up kswapd when an allocation is being retried after a direct
reclaim failure. It would be expected that kswapd is already awake, but
this has the effect of telling kswapd to reclaim at the higher order as well.
Signed-off-by: Mel Gorman <[email protected]>
---
mm/page_alloc.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0b3c6cb..e07b2f2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1763,16 +1763,17 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
if (NUMA_BUILD && (gfp_mask & GFP_THISNODE) == GFP_THISNODE)
goto nopage;
- wake_all_kswapd(order, zonelist, high_zoneidx);
-
/*
- * OK, we're below the kswapd watermark and have kicked background
- * reclaim. Now things get more complex, so set up alloc_flags according
- * to how we want to proceed.
+ * OK, we're below the kswapd watermark and now things get more
+ * complex, so set up alloc_flags according to how we want to
+ * proceed.
*/
alloc_flags = gfp_to_alloc_flags(gfp_mask);
restart:
+ /* Kick background reclaim */
+ wake_all_kswapd(order, zonelist, high_zoneidx);
+
/* This is the last chance, in general, before the goto nopage. */
page = get_page_from_freelist(gfp_mask, nodemask, order, zonelist,
high_zoneidx, alloc_flags & ~ALLOC_NO_WATERMARKS,
@@ -1802,6 +1803,9 @@ rebalance:
if (test_thread_flag(TIF_MEMDIE) && !(gfp_mask & __GFP_NOFAIL))
goto nopage;
+ /* This shouldn't be possible but needs to be eliminated */
+ WARN_ON_ONCE(alloc_flags & ALLOC_NO_WATERMARKS);
+
/* Try direct reclaim and then allocating */
page = __alloc_pages_direct_reclaim(gfp_mask, order,
zonelist, high_zoneidx,