Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758283AbWK0PzA (ORCPT ); Mon, 27 Nov 2006 10:55:00 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1758334AbWK0PzA (ORCPT ); Mon, 27 Nov 2006 10:55:00 -0500 Received: from mail.gmx.net ([213.165.64.20]:17867 "HELO mail.gmx.net") by vger.kernel.org with SMTP id S1758283AbWK0Py7 (ORCPT ); Mon, 27 Nov 2006 10:54:59 -0500 X-Authenticated: #5358227 Date: Mon, 27 Nov 2006 16:54:54 +0100 From: Matthias Lederhofer To: Milan Broz Cc: Linux Kernel , dm-crypt@saout.de Subject: Re: freeze with swap on dm-crypt on smp system (v2.6.18-g23541d2) Message-ID: <20061127155454.GA21409@moooo.ath.cx> References: <20061127000234.GA2084@moooo.ath.cx> <456AF7A8.8020901@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <456AF7A8.8020901@redhat.com> X-Y-GMX-Trusted: 0 Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 36306 Lines: 769 Milan Broz wrote: > [23541d2] dm crypt: move io to workqueue > > So this patch is the source of problems ? That's it. > Please, if you can send task list when system stops responding (sysrq + t) > and output of > dmsetup table > dmsetup info -c sysrq+t output is at the end of the mail. Is there any way I'm supposed to post such long stuff? E.g. put it on a webserver, paste site, in an attachment? # dmsetup table swap2: 0 1959867 crypt cbc(aes)-cbc-plain \ c4992e55743a12068f1767e9b855a6388b9bf0c7191ff22823a2cf74ea0dcde8 0 8:37 0 swap1: 0 1959867 crypt cbc(aes)-cbc-plain \ cb922dc385d0d1df09139eabfb32220a968a715428c91708411affbbfecaba08 0 8:21 0 swap0: 0 1959867 crypt cbc(aes)-cbc-plain \ 93c36d63556fb9a42308349d18f62a78553445add2424577e39ce1fbf175dcf7 0 8:5 0 # dmsetup info -c Name Maj Min Stat Open Targ Event UUID swap2 253 2 L--w 1 1 0 swap1 253 1 L--w 1 1 0 swap0 253 0 L--w 1 1 0 > Are there any messages in syslog related to this hang ? I can't find anything, there is also nothing on the console with loglevel set to 9. > > While running mprime with this options I got this warning: > >> warning: many lost ticks. > >> Your time source seems to be instable or some driver is hogging interupts > >> rip xor_128+0x2/0x20 > > Not sure if this can be related... did you try compile it without config_preempt ? > (and dynamic overclocking in BIOS is disabled ?) It still happens without config_preempt and afaik I don't have dynamic overclocking enabled (I did not find anything in the bios setup, it's a GA-M55PLUS-S3G). SysRq : Show State sibling task PC pid father child younger older init D 000000000000000a 0 1 0 2 (NOTLB) ffff81003ef83b78 0000000000000082 0000000000000296 ffffffff801f2d31 0000000000000000 ffff81003ef814e0 ffff81003ef8e080 00000000000164f8 ffff81003ef816b8 0000000139e071c0 0000000000000000 0000000000000046 Call Trace: [] __up_read+0x21/0xb0 [] __generic_unplug_device+0x13/0x30 [] sync_page+0x0/0x50 [] io_schedule+0x28/0x40 [] sync_page+0x45/0x50 [] __wait_on_bit_lock+0x40/0x80 [] __lock_page+0x5f/0x70 [] wake_bit_function+0x0/0x30 [] grab_swap_token+0xa0/0x160 [] __handle_mm_fault+0x885/0xb40 [] proc_delete_inode+0x0/0x40 [] invalidate_inode_buffers+0xe/0x60 [] do_page_fault+0x4b7/0x8d0 [] default_wake_function+0x0/0x10 [] compat_core_sys_select+0x203/0x240 [] vfs_getattr+0x4b/0xa0 [] error_exit+0x0/0x84 [] compat_set_fd_set+0x4a/0x50 [] compat_core_sys_select+0x1d3/0x240 [] compat_sys_select+0xba/0x1a0 [] ia32_sysret+0x0/0xa migration/0 S 0000000000000001 0 2 1 3 (L-TLB) ffff81003ef87e90 0000000000000046 0000000000000082 ffff81003ca77e98 0000000000000001 ffff81003ef80e00 ffffffff802f73c0 0000000000000d17 ffff81003ef80fd8 0000000080125124 0000000300000000 ffff81003ca77e88 Call Trace: [] migration_thread+0x193/0x2c0 [] migration_thread+0x0/0x2c0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] vgacon_cursor+0x0/0x1d0 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 ksoftirqd/0 S 0000000000000001 0 3 1 4 2 (L-TLB) ffff81003ef8bef0 0000000000000046 ffff81003ef8bf00 0000000000000046 0000000000000086 ffff81003ef80720 ffff81003e6f77a0 00000000000005e4 ffff81003ef808f8 000000008012584c 0000000000000013 ffff81003e6f77a0 Call Trace: [] ksoftirqd+0x0/0xc0 [] ksoftirqd+0x51/0xc0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 watchdog/0 S 0000000000000001 0 4 1 5 3 (L-TLB) ffff81003ef8dee0 0000000000000046 0000000000000001 ffffffff802bcb83 ffff81003ef8df00 ffff81003ef80040 ffffffff802f73c0 0000000000000157 ffff81003ef80218 0000000000000001 ffff810001dd4b18 ffffffff802f73c0 Call Trace: [] thread_return+0x0/0xfd [] __wake_up_common+0x44/0x80 [] watchdog+0x0/0x70 [] watchdog+0x59/0x70 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 migration/1 S 0000000000000001 0 5 1 6 4 (L-TLB) ffff81003ef91e90 0000000000000046 0000000000000082 ffff81003d5a1e98 0000000000000001 ffff81003ef8f520 ffff81003ef8e080 0000000000001022 ffff81003ef8f6f8 0000000180125124 0000000300000000 ffff81003d5a1e88 Call Trace: [] migration_thread+0x193/0x2c0 [] migration_thread+0x0/0x2c0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 ksoftirqd/1 S 000000000000000a 0 6 1 7 5 (L-TLB) ffff81003ef99ef0 0000000000000046 ffff810001f818e0 000000000001325b ffff81003ef8f020 ffff81003ef8ee40 ffff81003ef8e080 0000000000000386 ffff81003ef8f018 000000018012584c ffff810001dd97f8 ffff81003ef1ee80 Call Trace: [] ksoftirqd+0x0/0xc0 [] ksoftirqd+0x51/0xc0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 watchdog/1 S 0000000000000009 0 7 1 8 6 (L-TLB) ffff81003ef9bee0 0000000000000046 000000000000000a ffffffff802bcb83 ffff81003ef9bf00 ffff81003ef8e760 ffff81003ef8e080 00000000000003ff ffff81003ef8e938 0000000100000001 ffff810001dda0d8 ffff81003ef8e080 Call Trace: [] thread_return+0x0/0xfd [] __wake_up_common+0x44/0x80 [] watchdog+0x0/0x70 [] watchdog+0x59/0x70 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 events/0 S 000000000000000a 0 8 1 9 7 (L-TLB) ffff81003efb1e70 0000000000000046 0000000000000086 0000000000000003 ffff81003efb1e20 ffff81003ef1f560 ffffffff802f73c0 0000000000005a8a ffff81003ef1f738 0000000000000000 0000000000000286 ffffffff8016c250 Call Trace: [] cache_reap+0x0/0x130 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 events/1 S 000000000000000a 0 9 1 10 8 (L-TLB) ffff81003efb3e70 0000000000000046 0000000000000086 0000000000000003 ffff81003efb3e20 ffff81003ef1ee80 ffff81003ef8e080 00000000000058f9 ffff81003ef1f058 0000000100000000 0000000000000286 ffffffff8016c250 Call Trace: [] cache_reap+0x0/0x130 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 khelper S 0000000000000006 0 10 1 11 9 (L-TLB) ffff81003efb5e70 0000000000000046 0000000000000086 0000000000000003 ffff81003efb5e20 ffff81003ef1e7a0 ffffffff802f73c0 0000000000000577 ffff81003ef1e978 000000003ef83cb8 0000000000000286 ffffffff8013ce00 Call Trace: [] __call_usermodehelper+0x0/0x80 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] flat_send_IPI_mask+0x0/0x50 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kthread S 000000000000000a 0 11 1 35 321 10 (L-TLB) ffff81003efb9e70 0000000000000046 0000000000000086 0000000000000003 ffff81003efb9e20 ffff81003ef1e0c0 ffffffff802f73c0 00000000000007d8 ffff81003ef1e298 000000003c51f9b8 0000000000000286 ffffffff80140a60 Call Trace: [] keventd_create_kthread+0x0/0x80 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kblockd/0 S ffff81003ef02540 0 35 11 36 (L-TLB) ffff81003e517e70 0000000000000046 0000000000000086 0000000000000003 ffff81003e517e20 ffff81003e62c240 ffff810001f81200 0000000000001637 ffff81003e62c418 000000003ee1b2d8 0000000000000282 ffffffff801ec410 Call Trace: [] cfq_kick_queue+0x0/0x50 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kblockd/1 S 000000000000000a 0 36 11 37 35 (L-TLB) ffff81003e51be70 0000000000000046 0000000000000086 0000000000000003 ffff81003e51be20 ffff81003e519720 ffff81003ef8e080 000000000000d34a ffff81003e5198f8 000000013ee1b2d8 0000000000000282 ffffffff801ec410 Call Trace: [] cfq_kick_queue+0x0/0x50 [] run_workqueue+0xd8/0x110 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kseriod S ffffffff803187c0 0 37 11 92 36 (L-TLB) ffff81003ef5deb0 0000000000000046 ffffffff80314f80 ffff81003ef1c788 00000000ffffffef ffff81003e519040 ffff81003ef1ee80 0000000000507007 ffff81003e519218 0000000180314f80 ffff810001dd97f8 ffffffff80319670 Call Trace: [] driver_create_file+0x3d/0x60 [] serio_thread+0x0/0x330 [] keventd_create_kthread+0x0/0x80 [] serio_thread+0x2f3/0x330 [] autoremove_wake_function+0x0/0x30 [] serio_thread+0x0/0x330 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 pdflush S 000000000000000a 0 92 11 93 37 (L-TLB) ffff81003ef29ec0 0000000000000046 0000000000000400 0000000000000000 ffffffff80153940 ffff810001f80b20 ffffffff802f73c0 0000000000000195 ffff810001f80cf8 0000000000000400 0000000000000000 0000000000000000 Call Trace: [] pdflush+0x0/0x1d0 [] pdflush+0x0/0x1d0 [] keventd_create_kthread+0x0/0x80 [] pdflush+0xd0/0x1d0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 pdflush S 000000000000000a 0 93 11 94 92 (L-TLB) ffff81003ef2bec0 0000000000000046 0000000000000282 0000000000000286 00000000000004e2 ffff810001f81200 ffffffff802f73c0 000000000000063f ffff810001f813d8 00000000801534d5 0000000000000000 0000000000000000 Call Trace: [] pdflush+0x0/0x1d0 [] keventd_create_kthread+0x0/0x80 [] pdflush+0xd0/0x1d0 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kswapd0 D 000000000000000a 0 94 11 95 93 (L-TLB) ffff81003ef27a40 0000000000000046 ffff81003ef279e0 ffffffff80300a10 ffff810001f818e0 ffff810001f818e0 ffff81003ef8e080 00000000000008bd ffff810001f81ab8 000000018015218b 0000000000000286 ffffffff80135f89 Call Trace: [] lock_timer_base+0x29/0x60 [] schedule_timeout+0x9a/0xd0 [] process_timeout+0x0/0x10 [] io_schedule_timeout+0x28/0x40 [] mempool_alloc+0xd3/0x110 [] autoremove_wake_function+0x0/0x30 [] bio_alloc_bioset+0xd1/0x150 [] end_swap_bio_write+0x0/0x90 [] bio_alloc+0x10/0x30 [] get_swap_bio+0x2f/0xd0 [] swap_writepage+0x4e/0xd0 [] shrink_zone+0x9a8/0xe20 [] kswapd+0x30f/0x460 [] autoremove_wake_function+0x0/0x30 [] keventd_create_kthread+0x0/0x80 [] kswapd+0x0/0x460 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 aio/0 S ffff810001f65440 0 95 11 96 94 (L-TLB) ffff81003e543e70 0000000000000046 0000000000000064 0000000000000000 ffff81003e542000 ffff81003e676400 ffff81003ef1e0c0 000000000000098a ffff81003e6765d8 000000003e676400 ffff810001dd4238 ffffffff80136d0b Call Trace: [] do_sigaction+0x1ab/0x1d0 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 aio/1 S ffff810001f653c0 0 96 11 193 95 (L-TLB) ffff81003efbde70 0000000000000046 ffff81003efbde40 ffff81003efbddf0 ffffffff80124fad ffff81003e676ae0 ffff81003ef1e7a0 00000000000012d4 ffff81003e676cb8 000000013ef1e0c0 ffff810001dd97f8 0000000000000000 Call Trace: [] deactivate_task+0x1d/0x30 [] do_sigaction+0x172/0x1d0 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 ata/0 S ffff81003e5a2640 0 193 11 194 96 (L-TLB) ffff81003ee17e70 0000000000000046 0000000000000064 0000000000000000 ffff81003ee16000 ffff81003ee14aa0 ffff81003ef1e0c0 0000000000000b4a ffff81003ee14c78 000000003ee14aa0 ffff810001dd4238 ffffffff80136d0b Call Trace: [] do_sigaction+0x1ab/0x1d0 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 ata/1 S 0000000000000008 0 194 11 195 193 (L-TLB) ffff81003ee0fe70 0000000000000046 ffff81003ee0fe40 ffff81003ee0fdf0 ffffffff80124fad ffff81003ee15180 ffff81003ef8e080 00000000000009e4 ffff81003ee15358 000000013ef1e0c0 0000000000000086 0000000000000000 Call Trace: [] deactivate_task+0x1d/0x30 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 ata_aux S 0000000000000008 0 195 11 197 194 (L-TLB) ffff81003ee0be70 0000000000000046 ffff81003ee0be40 ffff81003ee0bdf0 ffffffff80124fad ffff81003ee15860 ffffffff802f73c0 00000000000003f6 ffff81003ee15a38 000000003ef1e0c0 0000000000000086 0000000000000000 Call Trace: [] deactivate_task+0x1d/0x30 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0xf6/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 scsi_eh_0 S 0000000000000246 0 197 11 198 195 (L-TLB) ffff81003e62be70 0000000000000046 ffff81003e62be10 ffffffff80125783 0000000000000296 ffff81003ee08a60 ffff81003ef814e0 00000000002df9e2 ffff81003ee08c38 000000013e684098 0000000000000000 0000000000000001 Call Trace: [] __wake_up+0x43/0x70 [] __scsi_iterate_devices+0x5a/0x80 [] scsi_error_handler+0x6f/0x730 [] keventd_create_kthread+0x0/0x80 [] scsi_error_handler+0x0/0x730 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 scsi_eh_1 S 0000000000000246 0 198 11 219 197 (L-TLB) ffff81003ef2fe70 0000000000000046 ffff81003ef2fe10 ffffffff80125783 0000000000000296 ffff81003ee09140 ffff81003ef814e0 00000000002dce40 ffff81003ee09318 000000013ee1c098 0000000000000000 0000000000000001 Call Trace: [] __wake_up+0x43/0x70 [] __scsi_iterate_devices+0x5a/0x80 [] scsi_error_handler+0x6f/0x730 [] keventd_create_kthread+0x0/0x80 [] scsi_error_handler+0x0/0x730 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 scsi_eh_2 S 0000000000000246 0 219 11 220 198 (L-TLB) ffff81003e627e70 0000000000000046 ffff81003e627e10 ffffffff80125783 0000000000000296 ffff81003e6b6180 ffff81003ef814e0 00000000002df821 ffff81003e6b6358 000000013ee10098 0000000000000000 0000000000000001 Call Trace: [] __wake_up+0x43/0x70 [] __scsi_iterate_devices+0x5a/0x80 [] scsi_error_handler+0x6f/0x730 [] keventd_create_kthread+0x0/0x80 [] scsi_error_handler+0x0/0x730 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 scsi_eh_3 S 000000000000000a 0 220 11 237 219 (L-TLB) ffff81003e645e70 0000000000000046 ffff81003e645e10 ffffffff80125783 0000000000000296 ffff81003ee08380 ffffffff802f73c0 000000000116007a ffff81003ee08558 000000003e66c098 0000000000000000 0000000000000001 Call Trace: [] __wake_up+0x43/0x70 [] __scsi_iterate_devices+0x5a/0x80 [] scsi_error_handler+0x6f/0x730 [] keventd_create_kthread+0x0/0x80 [] scsi_error_handler+0x0/0x730 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kcryptd/0 D 000000000000000a 0 237 11 238 220 (L-TLB) ffff81003e6a7c50 0000000000000046 ffff810022780ff0 0000000000000001 ffff81003ef36340 ffff81003ef36340 ffffffff802f73c0 0000000000000956 ffff81003ef36518 000000008015227d 0000000000000286 ffffffff80135f89 Call Trace: [] lock_timer_base+0x29/0x60 [] schedule_timeout+0x9a/0xd0 [] process_timeout+0x0/0x10 [] io_schedule_timeout+0x28/0x40 [] mempool_alloc+0xd3/0x110 [] autoremove_wake_function+0x0/0x30 [] bio_alloc_bioset+0xd1/0x150 [] bio_alloc+0x10/0x30 [] kcryptd_do_work+0x1ea/0x370 [] kcryptd_do_work+0x0/0x370 [] run_workqueue+0xb2/0x110 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0x121/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kcryptd/1 D 000000000000000a 0 238 11 242 237 (L-TLB) ffff81003e69dc50 0000000000000046 ffff810033338ff0 0000000000000001 ffff81003ee09820 ffff81003ee09820 ffff81003ef8e080 0000000000000877 ffff81003ee099f8 000000018015227d 0000000000000286 ffffffff80135f89 Call Trace: [] lock_timer_base+0x29/0x60 [] schedule_timeout+0x9a/0xd0 [] process_timeout+0x0/0x10 [] io_schedule_timeout+0x28/0x40 [] mempool_alloc+0xd3/0x110 [] autoremove_wake_function+0x0/0x30 [] bio_alloc_bioset+0xd1/0x150 [] bio_alloc+0x10/0x30 [] kcryptd_do_work+0x1ea/0x370 [] kcryptd_do_work+0x0/0x370 [] run_workqueue+0xb2/0x110 [] worker_thread+0x0/0x160 [] keventd_create_kthread+0x0/0x80 [] worker_thread+0x121/0x160 [] default_wake_function+0x0/0x10 [] worker_thread+0x0/0x160 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 md11_raid1 S 7fffffffffffffff 0 242 11 244 238 (L-TLB) ffff81003e6dde50 0000000000000046 0000000000000002 ffff81003e6ce800 ffff81003e6dddf0 ffff81003ef36a20 ffff81003ef2cf40 0000000000000235 ffff81003ef36bf8 000000008012584c ffff81003e6dde10 ffff81003ef2cf40 Call Trace: [] keventd_create_kthread+0x0/0x80 [] schedule_timeout+0x25/0xd0 [] prepare_to_wait+0x23/0x80 [] md_thread+0xee/0x130 [] autoremove_wake_function+0x0/0x30 [] md_thread+0x0/0x130 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 md10_raid1 S 000000000000000a 0 244 11 245 242 (L-TLB) ffff81003e6dfe50 0000000000000046 0000000000000002 ffff81003e6ce400 ffff81003e6dfdf0 ffff81003ef37100 ffff81003ef8e080 0000000000001338 ffff81003ef372d8 000000018012584c ffff810001dd97f8 ffff81003ef8e080 Call Trace: [] keventd_create_kthread+0x0/0x80 [] schedule_timeout+0x25/0xd0 [] prepare_to_wait+0x23/0x80 [] md_thread+0xee/0x130 [] autoremove_wake_function+0x0/0x30 [] md_thread+0x0/0x130 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 kjournald S 000000000000000a 0 245 11 696 244 (L-TLB) ffff81003d61fe80 0000000000000046 ffff81003e6be000 0000000000000000 ffff81003a329024 ffff81003e661820 ffffffff802f73c0 0000000000002b21 ffff81003e6619f8 000000003e6c5a90 0000000000000000 0000000000000001 Call Trace: [] kjournald+0x1a8/0x230 [] autoremove_wake_function+0x0/0x30 [] keventd_create_kthread+0x0/0x80 [] kjournald+0x0/0x230 [] keventd_create_kthread+0x0/0x80 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 udevd S 0000000000000001 0 321 1 977 11 (NOTLB) ffff81003d321b08 0000000000000082 00000000fff57000 ffff81003d770740 0000000000000001 ffff81003e660380 ffffffff802f73c0 000000000000c060 ffff81003e660558 00000000802db5d9 0000000000000003 ffff81003e6d7000 Call Trace: [] schedule_timeout+0x25/0xd0 [] add_wait_queue+0x1c/0x60 [] inotify_poll+0x5d/0x80 [] do_select+0x477/0x530 [] __pollwait+0x0/0x120 [] default_wake_function+0x0/0x10 [] default_wake_function+0x0/0x10 [] default_wake_function+0x0/0x10 [] default_wake_function+0x0/0x10 [] mntput_no_expire+0x24/0xa0 [] __link_path_walk+0x6e5/0xd90 [] d_lookup+0x24/0x50 [] mntput_no_expire+0x24/0xa0 [] link_path_walk+0x100/0x140 [] remove_wait_queue+0x19/0x60 [] do_wait+0xabd/0xbb0 [] vfs_read+0x11b/0x160 [] default_wake_function+0x0/0x10 [] compat_core_sys_select+0x19d/0x240 [] compat_sys_select+0xde/0x1a0 [] ia32_sysret+0x0/0xa md126_raid1 S 000000000000000a 0 696 11 245 (L-TLB) ffff81003ce01e50 0000000000000046 0000000000000002 ffff81003d2eb400 0000000000000000 ffff81003e6c01c0 ffff81003ef8e080 000000000000073e ffff81003e6c0398 000000018012584c ffff81003ce01e10 ffff81003ef8e080 Call Trace: [] keventd_create_kthread+0x0/0x80 [] schedule_timeout+0x25/0xd0 [] prepare_to_wait+0x23/0x80 [] md_thread+0xee/0x130 [] autoremove_wake_function+0x0/0x30 [] md_thread+0x0/0x130 [] kthread+0xd9/0x120 [] child_rip+0xa/0x12 [] keventd_create_kthread+0x0/0x80 [] kthread+0x0/0x120 [] child_rip+0x0/0x12 syslogd D 0000000000000009 0 977 1 983 321 (NOTLB) ffff81003aa45628 0000000000000082 0000000000000000 ffffffff80300a10 ffff81003e6f69e0 ffff81003e6f69e0 ffff81003ef8e080 0000000000000995 ffff81003e6f6bb8 000000018015218b 0000000000000286 ffffffff80135f89 Call Trace: [] lock_timer_base+0x29/0x60 [] schedule_timeout+0x9a/0xd0 [] process_timeout+0x0/0x10 [] io_schedule_timeout+0x28/0x40 [] mempool_alloc+0xd3/0x110 [] autoremove_wake_function+0x0/0x30 [] try_to_wake_up+0x38c/0x3b0 [] bio_alloc_bioset+0xd1/0x150 [] end_swap_bio_write+0x0/0x90 [