Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757832AbXFMLQy (ORCPT ); Wed, 13 Jun 2007 07:16:54 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1756858AbXFMLQq (ORCPT ); Wed, 13 Jun 2007 07:16:46 -0400 Received: from s2.ukfsn.org ([217.158.120.143]:48783 "EHLO mail.ukfsn.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756187AbXFMLQm (ORCPT ); Wed, 13 Jun 2007 07:16:42 -0400 Message-ID: <466FD214.9070603@dgreaves.com> Date: Wed, 13 Jun 2007 12:16:36 +0100 From: David Greaves User-Agent: Mozilla-Thunderbird 2.0.0.0 (X11/20070601) MIME-Version: 1.0 To: Linus Torvalds Cc: David Chinner , Tejun Heo , "Rafael J. Wysocki" , xfs@oss.sgi.com, "'linux-kernel@vger.kernel.org'" , linux-pm , Neil Brown , Jeff Garzik Subject: Re: 2.6.22-rc3 hibernate(?) fails totally - regression (xfs on raid6) References: <200706020122.49989.rjw@sisk.pl> <4661EFBB.5010406@dgreaves.com> <4662D852.4000005@dgreaves.com> <46667160.80905@gmail.com> <46668EE0.2030509@dgreaves.com> <46679D56.7040001@gmail.com> <4667DE2D.6050903@dgreaves.com> <20070607110708.GS86004887@sgi.com> <46680F5E.6070806@dgreaves.com> <20070607222813.GG85884050@sgi.com> <4669A965.20403@dgreaves.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 73570 Lines: 1634 Linus Torvalds wrote: > > On Fri, 8 Jun 2007, David Greaves wrote: >> positive: I can now get sysrq-t :) > > Ok, so color me confused, So what do you think that makes me > and maybe I have missed some of the emails or > skimmed over them too fast (there's been too many of them ;), You may have missed these 'tests' with rc4+Tejun's fix: * clean boot, unmounting the xfs fs : normal hibernate/resume * clean boot, remount ro xfs fs : normal hibernate/resume * clean boot, touch; sync; echo 1 > /proc/sys/vm/drop_caches: normal hibernate/resume * clean boot, touch; sync; echo 2 > /proc/sys/vm/drop_caches: hang hibernating * clean boot, touch; sync; echo 3 > /proc/sys/vm/drop_caches: hang hibernating Dave asked me to do them but hasn't responded yet. > but > > - I haven't actually seen any traces for this (netconsole apparently > doesn't work for you, and I'm not surprised: it never really worked > well for me over suspend/resume either, but I think I saw a mention of > serial console?) Well, I got netconsole working but needed to buildin the skge and that changed the behaviour a bit... let me know if that's interesting. I've left skge as a module as originally reported. I have however configured serial and serial console and plugged in a cable so I can capture data there. Sysrq at the end. > - You apparently bisected it down to the range > > 0a3fd051c7036ef71b58863f8e5da7c3dabd9d3f <- works > 1d30c33d8d07868199560b24f10ed6280e78a89c <- breaks > > but some of the intermediates in that range didn't compile. Correct? Yes, then applying the sata_via patch confirmed 9666f4009c22f6520ac3fb8a19c9e32ab973e828 libata: reimplement suspend/resume support using sdev->manage_start_stop was the first to cause the problems. However.... > Can you try to bisect down a bit more, despite the compile error? Just do > > git bisect start > git bisect good 0a3fd051c7036ef71b58863f8e5da7c3dabd9d3f > git bisect bad 1d30c33d8d07868199560b24f10ed6280e78a89c > > and it should pick > > f4d6d004: libata: ignore EH scheduling during initialization > > for you to test. It will apparently break on the fact that "sata_via.c" > wants "ata_scsi_device_resume/suspend" for the initialization of the > resume/suspend things in the scsi_host_template, but you should just > remove those lines, and the compile hopefully completes cleanly after > that. > > IOW, it *should* be easy enough to pinpoint this from 9 changes down to > just one. ... let me reconfirm - there's been a lot of testing and I don't want my recollection causing problems... These tests have had the config changed to include serial+console I also configured CONFIG_DISABLE_CONSOLE_SUSPEND=y 2.6.21-gf4d6d004-dirty : bad 2.6.21-g920a4b10-dirty : bad 2.6.21-g9666f400-dirty : bad git-bisect bad 9666f4009c22f6520ac3fb8a19c9e32ab973e828 is first bad commit commit 9666f4009c22f6520ac3fb8a19c9e32ab973e828 Author: Tejun Heo Date: Fri May 4 21:27:47 2007 +0200 libata: reimplement suspend/resume support using sdev->manage_start_stop Good. > Jeff added to the Cc, since he may not have noticed that one of the most > long-running issues is apparently sata-related. > > (Jeff: David Greaves _also_ had issues with -rc4 due to the SETFXSR > change, but that should hopefully be resolved and is presumably an > independent bug. Apart from the fact that "sata_via.c" seems problematic) > > Linus So here's a sysrq-t from a failed resume. Ask if you'd like anything else... SysRq : Show State free sibling task PC stack pid father child younger older init D 00000001 0 1 0 (NOTLB) c1941ea0 00000082 00000000 00000001 00000001 00000000 466fc747 28d2d41a 466fc747 28d2d41a 9120edc4 000001b3 000943a3 00000000 c192e030 c192eb3c 00000086 00001182 9136dece 000001b3 00000000 00000000 00000000 c1941f08 Call Trace: [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] copy_to_user+0x32/0x50 [] sys_select+0x149/0x170 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= kthreadd S F6BA3F04 0 2 0 (L-TLB) c1943fd0 00000046 00000001 f6ba3f04 c1943fb0 c0117627 00000000 00000000 ffffffff c0104a04 f6ba3f04 00000000 00000003 00000296 f72e15b0 c192e63c c1943fd0 00000048 6e2deb31 0000000c 00000a74 c0431298 00000000 00000000 Call Trace: [] __wake_up_common+0x37/0x60 [] kernel_thread_helper+0x0/0x3c [] kthreadd+0x71/0xa0 [] kthreadd+0x0/0xa0 [] kernel_thread_helper+0x7/0x3c ======================= ksoftirqd/0 S 000001B3 0 3 2 (L-TLB) c1945fc0 00000046 68967778 000001b3 000000fc 00000000 f72eaad0 c192e140 00000073 c192ea30 68967778 000001b3 c1945f90 00000000 f73a5a50 c192e13c c1945fb0 000000a5 9136e544 000001b3 c1945fc0 00000000 c011ef70 fffffffc Call Trace: [] ksoftirqd+0x0/0x90 [] ksoftirqd+0x7b/0x90 [] ksoftirqd+0x0/0x90 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= watchdog/0 S C0369725 0 4 2 (L-TLB) c1947fc0 00000046 c1943f70 c0369725 c1947fd0 00000046 c048e0e0 c1932a50 c1947f90 00000296 2dbc9200 000012ae 5716de49 00000009 c1932550 c1932b5c fffffffc 00000ab5 ab8b6f40 00000004 c1947fd0 00000000 c0140290 fffffffc Call Trace: [] schedule+0x2e5/0x580 [] watchdog+0x0/0x70 [] watchdog+0x4e/0x70 [] watchdog+0x0/0x70 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= events/0 R running 0 5 2 (L-TLB) khelper S 00000286 0 6 2 (L-TLB) c194bf80 00000046 00000060 00000286 00000000 00000000 c1914c20 c0127a80 c194bf60 00000001 f66ebb60 c0127a80 00000000 c0127aa5 f73a5a50 c193215c c1914c20 000008c7 b1f02986 0000000f c194bfd0 c1914c20 c194bfa8 c1914c28 Call Trace: [] __call_usermodehelper+0x0/0x70 [] __call_usermodehelper+0x0/0x70 [] __call_usermodehelper+0x25/0x70 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kblockd/0 S C021F890 0 35 2 (L-TLB) c19a7f80 00000046 c19146e0 c021f890 00000000 c021f85b c19146e0 c0225790 f7ea2438 c021f869 5fe478a8 00000009 028a6b4f c021f89b c19b7a70 c19616dc 0000006e 0000003c 8c1ad4be 00000013 ffffff10 c19146e0 c19a7fa8 c19146e8 Call Trace: [] blk_unplug_work+0x0/0x10 [] __generic_unplug_device+0x2b/0x30 [] as_work_handler+0x0/0x20 [] generic_unplug_device+0x9/0x10 [] blk_unplug_work+0xb/0x10 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kacpid S 000012B0 0 36 2 (L-TLB) c19a9f80 00000046 8e56e600 000012b0 58472b73 00000009 c192e530 ac2395b9 c19a9f60 c0116ce7 ac239cdd 00000004 00000077 00000000 c192e530 c19611dc 00000078 00000117 ac239df7 00000004 c19a9fd0 c19145e0 c19a9fa8 c19145e8 Call Trace: [] activate_task+0x37/0xb0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kacpi_notify S 000012B0 0 37 2 (L-TLB) c19abf80 00000046 8ef1f200 000012b0 584778f9 00000009 c192e530 ac23bc7c c19abf60 c0116ce7 ac6306e3 00000004 00000154 00000000 c1932050 c1969b3c 00000078 000002f9 ac630a09 00000004 c19abfd0 c19145a0 c19abfa8 c19145a8 Call Trace: [] activate_task+0x37/0xb0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= ata/0 S 0000000A 0 121 2 (L-TLB) c19ddf80 00000046 00000060 0000000a 00000000 00000009 f7f40000 f7f41e90 00000000 c02c9acb 64c028bc 00000009 00023d66 00000000 c19615d0 c19b3b5c 0000006e 00003906 64c028bc 00000009 c19ddfd0 c19989e0 c19ddfa8 c19989e8 Call Trace: [] ata_pio_task+0x5b/0xe0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= ata_aux D F65F09A4 0 122 2 (L-TLB) c19dfd20 00000046 f786c800 f65f09a4 f7ea23b8 c02bcdef ce2e3b9f 4d64703f ffffcfff f7ea23b8 00000082 f7ea2418 0005eb1d 00000082 f7c9f0b0 c196913c f786c800 000003d4 605db977 00000009 f7ea23b8 c19dfe08 f65f09a4 c19dfd3c Call Trace: [] scsi_prep_fn+0x8f/0x130 [] wait_for_completion+0x64/0xa0 [] default_wake_function+0x0/0x10 [] __generic_unplug_device+0x2b/0x30 [] default_wake_function+0x0/0x10 [] blk_execute_rq+0xa7/0xe0 [] blk_end_sync_rq+0x0/0x30 [] buffered_rmqueue+0x9f/0x100 [] get_page_from_freelist+0x80/0xc0 [] scsi_execute+0xb8/0x110 [] scsi_execute_req+0x6b/0x90 [] sd_spinup_disk+0x76/0x440 [] sd_revalidate_disk+0x6e/0x160 [] __scsi_disk_get+0x34/0x40 [] sd_rescan+0x1d/0x40 [] scsi_rescan_device+0x40/0x50 [] ata_scsi_dev_rescan+0x5c/0x70 [] ata_scsi_dev_rescan+0x0/0x70 [] run_workqueue+0x4a/0xf0 [] worker_thread+0xcd/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kseriod D C0369725 0 123 2 (L-TLB) c1971f60 00000046 f6d0ff90 c0369725 c1971f70 00000046 c0443c08 c1971fa8 00000000 c0444de0 91c4e988 000001b3 00002028 00000000 c19b3050 c196963c 00000073 00000120 91c535b2 000001b3 00000000 00000000 c02da2d0 c1971fa8 Call Trace: [] schedule+0x2e5/0x580 [] serio_thread+0x0/0x100 [] refrigerator+0x3c/0x50 [] serio_thread+0xf8/0x100 [] __wake_up_common+0x37/0x60 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] serio_thread+0x0/0x100 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= pdflush D C0369725 0 145 2 (L-TLB) c19cdf70 00000046 c1971f60 c0369725 c19cdf80 00000046 ef006e00 000012ca 65778037 00000009 91c4eb5e 000001b3 00002124 00000000 c19b7a70 c19b315c 00000073 0000006a 91c539de 000001b3 00000000 00000000 c19cdfa8 fffffffc Call Trace: [] schedule+0x2e5/0x580 [] refrigerator+0x3c/0x50 [] __pdflush+0x145/0x150 [] pdflush+0x0/0x30 [] pdflush+0x0/0x30 [] pdflush+0x25/0x30 [] pdflush+0x0/0x30 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= pdflush D C0369725 0 146 2 (L-TLB) c19cff70 00000046 c19cdf70 c0369725 c19cff80 00000046 00000001 c19cff48 c19cff74 c19615d0 91c4f11f 000001b3 00002016 00000000 f7d5f550 c19b7b7c 00000076 00000067 4270ffb6 0000000a 00000000 00000000 c19cffa8 fffffffc Call Trace: [] schedule+0x2e5/0x580 [] refrigerator+0x3c/0x50 [] __pdflush+0x145/0x150 [] pdflush+0x0/0x30 [] pdflush+0x0/0x30 [] pdflush+0x25/0x30 [] wb_kupdate+0x0/0xf0 [] pdflush+0x0/0x30 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kswapd0 D C0369725 0 147 2 (L-TLB) c1981f50 00000046 f67a5e90 c0369725 c1981f60 00000046 c1955550 00000000 c1981f78 c018dcac 91c4dbf7 000001b3 00000c71 00000000 f7d61070 c19b767c 0000006e 000000c5 91c4f96f 000001b3 00000000 00000000 c0434f44 00000000 Call Trace: [] schedule+0x2e5/0x580 [] proc_flush_task+0x4c/0x1a0 [] refrigerator+0x3c/0x50 [] kswapd+0xe1/0x110 [] schedule+0x2e5/0x580 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] kswapd+0x0/0x110 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= aio/0 S 000012CA 0 148 2 (L-TLB) c1a71f80 00000046 f91aea00 000012ca 657c8d75 00000009 c192e530 b2be46ba c1a71f60 c0116ce7 b2c07abe 00000004 000000b0 00000000 c192e530 c19b717c 00000078 000001a2 b2c07c5f 00000004 c1a71fd0 c199be20 c1a71fa8 c199be28 Call Trace: [] activate_task+0x37/0xb0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfslogd/0 S F74D6EE0 0 149 2 (L-TLB) c1a73f80 00000046 f74d2500 f74d6ee0 f78a64e0 c01f3421 00000000 b2c09d9a c1a73f60 00000246 f6f22840 c199b9e0 f6f22cc0 f6f22d1c c199b9e0 c19bbb9c 00000000 0000004a 91980244 000001b3 c1a73fd0 c199b9e0 c1a73fa8 c199b9e8 Call Trace: [] xlog_iodone+0x51/0xd0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfsdatad/0 S 00000000 0 150 2 (L-TLB) c1a75f80 00000046 00000000 00000000 00000000 00000246 00000000 c19e17f0 00000000 c199e820 c199b9a0 c020c690 00000000 c014557a c19b7a70 c19bb69c c199b9a0 00000049 91b5c447 000001b3 c1a75fd0 c199b9a0 c1a75fa8 c199b9a8 Call Trace: [] xfs_end_bio_read+0x0/0x10 [] mempool_free+0x2a/0x60 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_0 S F7C98084 0 774 2 (L-TLB) f7fa1fc0 00000046 00000246 f7c98084 f7c98000 00000246 c1a6f044 00000001 00000003 00000000 a35b398a 00000005 00000e17 00000000 c1932550 f7cbab5c 0000006e 0019a81a a35b398a 00000005 00000000 c1a6f000 c02bb740 fffffffc Call Trace: [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_1 S F7C64084 0 776 2 (L-TLB) c1ad3fc0 00000046 00000246 f7c64084 f7c64000 00000246 f78b3c44 00000001 00000003 00000000 00000292 fffffffc c1ad3fb0 00000246 c192ea30 f7d6167c 00000000 0019d570 c305aee5 00000005 00000000 f78b3c00 c02bb740 fffffffc Call Trace: [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_2 S C1AD4084 0 778 2 (L-TLB) f7d5dfc0 00000046 00000246 c1ad4084 c1ad4000 00000246 f78b3844 00000001 00000003 00000000 00000292 fffffffc f7d5dfb0 00000246 c192ea30 f7efd65c 00000000 001a0997 e2bc559e 00000005 00000000 f78b3800 c02bb740 fffffffc Call Trace: [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_3 S F7FC4084 0 780 2 (L-TLB) c1a77fc0 00000046 00000246 f7fc4084 f7fc4000 00000246 f78b3444 00000001 00000003 00000000 02d2bfed 00000006 0000067f 00000000 c1932550 f7d61b7c 0000006e 001c7d7b 02d2bfed 00000006 00000000 f78b3400 c02bb740 fffffffc Call Trace: [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_4 S F786C800 0 803 2 (L-TLB) f7fc3fc0 00000046 f786c800 f786c800 f7ea23b8 c02bd166 f7964044 f7ea23b8 00000292 c021f7f2 6154d436 00000009 002f304e 00000000 c1932550 f7d0915c 0000006e 000ec1d6 6154d436 00000009 00000000 f7964000 c02bb740 fffffffc Call Trace: [] scsi_request_fn+0x196/0x280 [] blk_remove_plug+0x32/0x70 [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= scsi_eh_5 S F786C400 0 805 2 (L-TLB) f7e3ffc0 00000046 f786c400 f786c400 f7ea2730 c02bd166 f786cc44 f7ea2730 00000292 c021f7f2 67583eed 00000009 002f0e42 00000000 c19615d0 f7c9f1bc 0000006e 000ec010 67583eed 00000009 00000000 f786cc00 c02bb740 fffffffc Call Trace: [] scsi_request_fn+0x196/0x280 [] blk_remove_plug+0x32/0x70 [] scsi_error_handler+0x0/0xa0 [] scsi_error_handler+0x41/0xa0 [] scsi_error_handler+0x0/0xa0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kpsmoused S 000019B6 0 826 2 (L-TLB) f7e81f80 00000046 b68df400 000019b6 db5b46fa 0000000c c192e530 6dada37d f7e81f60 c0116ce7 6dadbfd3 00000006 000000a8 00000000 c1932050 f7d5fb5c 00000078 00000029 6dadc163 00000006 f7e81fd0 f7fa9c20 f7e81fa8 f7fa9c28 Call Trace: [] activate_task+0x37/0xb0 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= md0_raid5 D 1D3836C7 0 836 2 (L-TLB) f7cbfdf0 00000046 f7ea3888 1d3836c7 00000008 c0221036 06a47600 00011210 c191ede0 c01454ba 5324bddc 00000009 0002fb08 00000000 f7d09050 f7e2c6bc 0000006e 0000320f 532bcd0b 00000009 f7ea3888 c1a4f000 f7cbfe18 c1a4f13c Call Trace: [] generic_make_request+0x146/0x1d0 [] mempool_alloc+0x2a/0xc0 [] md_super_wait+0x7e/0xc0 [] autoremove_wake_function+0x0/0x50 [] bio_clone+0x31/0x40 [] autoremove_wake_function+0x0/0x50 [] write_sb_page+0x50/0x80 [] write_page+0x112/0x120 [] sync_sbs+0x77/0xe0 [] bitmap_update_sb+0x69/0xa0 [] md_update_sb+0x138/0x2c0 [] schedule+0x2e5/0x580 [] md_check_recovery+0x2dd/0x360 [] raid5d+0x10/0xe0 [] md_thread+0x55/0x110 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] md_thread+0x0/0x110 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfsbufd D F7D6117C 0 839 2 (L-TLB) f74dffa0 00000046 00000282 f7d6117c 0005eadc 000000de 00000282 f74dff88 00000000 00000282 91c4e16c 000001b3 00000eb5 00000000 f7c51a30 f7d6117c 0000006e 00000115 91c50443 000001b3 00000000 00000000 00000000 f74ccfa0 Call Trace: [] refrigerator+0x3c/0x50 [] xfsbufd+0xf0/0x100 [] xfsbufd+0x0/0x100 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfssyncd D F7D5F65C 0 840 2 (L-TLB) f775ff90 00000046 00000282 f7d5f65c 00060a54 00001d76 00000282 f775ff78 00000000 00000282 91c4fd9e 000001b3 00001d19 00000000 f7348090 f7d5f65c 00000078 000002b5 91c5428a 000001b3 00000000 00000000 f775ffb8 f78a93dc Call Trace: [] refrigerator+0x3c/0x50 [] xfssyncd+0x15b/0x160 [] xfssyncd+0x0/0x160 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= udevd D 00000010 0 921 1 (NOTLB) f7043ea0 00000082 f66ec590 00000010 c1913ba0 f67ba22c 407ef268 00000010 c1913ba0 f7fa5cb4 9133dae2 000001b3 00006df1 00000000 f7348590 f7cba65c 00000075 00000567 9134df46 000001b3 00000000 00000000 00000000 f7043f08 Call Trace: [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_wait+0x1bb/0x3b0 [] shrink_dcache_parent+0xd/0x30 [] default_wake_function+0x0/0x10 [] sys_select+0xa9/0x170 [] sys_wait4+0x31/0x40 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= ksuspend_usbd S 00002061 0 1822 2 (L-TLB) f6c81f80 00000046 f9766400 00002061 30fcbb32 00000010 c192e530 187e5d99 f6c81f60 c0116ce7 c048e0e0 c192e530 f6c81f60 c0116be1 f7c51a30 f735867c f7358570 000003e1 19639b4d 00000008 f6c81fd0 f71611a0 f6c81fa8 f71611a8 Call Trace: [] activate_task+0x37/0xb0 [] __activate_task+0x21/0x40 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= khubd D C0369725 0 1825 2 (L-TLB) f75adf70 00000046 f74dffa0 c0369725 f75adf80 00000046 00000023 00000011 00000002 f882a3a5 91c4e5a2 000001b3 000010e3 00000000 f73a25d0 f7c51b3c 0000006e 000000f0 91c50da4 000001b3 00000000 00000000 f882d180 f75adfa8 Call Trace: [] schedule+0x2e5/0x580 [] usb_get_intf+0x15/0x20 [usbcore] [] hub_thread+0x0/0xf0 [usbcore] [] refrigerator+0x3c/0x50 [] hub_thread+0x55/0xf0 [usbcore] [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] hub_thread+0x0/0xf0 [usbcore] [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= ksnapd S 000025EF 0 2132 2 (L-TLB) f7237f80 00000046 6f8a7a00 000025ef f7b7c53d 00000012 c192e530 7bdbe29e f7237f60 c0116ce7 c048e0e0 c192e530 f7237f60 c0116be1 c19bb090 f730e67c f730e570 0000007c 7bea9794 00000009 f7237fd0 f71c9360 f7237fa8 f71c9368 Call Trace: [] activate_task+0x37/0xb0 [] __activate_task+0x21/0x40 [] worker_thread+0xe4/0xf0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] worker_thread+0x0/0xf0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= kjournald D C0369725 0 2178 2 (L-TLB) f6d71f40 00000046 f75adf70 c0369725 f6d71f50 00000046 00000000 f6d0dd38 00000000 c0117627 91c4e939 000001b3 000012d7 00000000 c19bb090 f73a26dc 0000006e 000000d2 91c515db 000001b3 00000000 00000000 f73a25d0 00000001 Call Trace: [] schedule+0x2e5/0x580 [] __wake_up_common+0x37/0x60 [] refrigerator+0x3c/0x50 [] kjournald+0xd1/0x1d0 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] kjournald+0x0/0x1d0 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfsbufd D C19BB19C 0 2179 2 (L-TLB) f6d29fa0 00000046 00000282 c19bb19c 0005eb4b 00000085 00000282 f6d29f88 00000000 00000282 91c4ec56 000001b3 0000145d 00000000 c1961ad0 c19bb19c 0000006e 000000ab 91c51c91 000001b3 00000000 00000000 00000000 f6d44aa0 Call Trace: [] refrigerator+0x3c/0x50 [] xfsbufd+0xf0/0x100 [] xfsbufd+0x0/0x100 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfssyncd D C1961BDC 0 2183 2 (L-TLB) f6881f90 00000046 00000282 c1961bdc 0005f177 00000269 00000282 f6881f78 00000000 00000282 91c4efba 000001b3 000015d2 00000000 f7cb10b0 c1961bdc 0000006e 000000af 91c52369 000001b3 00000000 00000000 f6881fb8 f6d45c3c Call Trace: [] refrigerator+0x3c/0x50 [] xfssyncd+0x15b/0x160 [] xfssyncd+0x0/0x160 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfsbufd D F7CB11BC 0 2184 2 (L-TLB) f6a33fa0 00000046 00000282 f7cb11bc 0005eb9d 00000067 00000282 f6a33f88 00000000 00000282 91c4f278 000001b3 0000171e 00000000 f73945b0 f7cb11bc 0000006e 00000094 91c52939 000001b3 00000000 00000000 00000000 f74cc720 Call Trace: [] refrigerator+0x3c/0x50 [] xfsbufd+0xf0/0x100 [] xfsbufd+0x0/0x100 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= xfssyncd D F73946BC 0 2185 2 (L-TLB) f6d0ff90 00000046 00000282 f73946bc 0005f197 00000153 00000282 f6d0ff78 00000000 00000282 91c4e688 000001b3 00001e93 00000000 c1969530 f73946bc 00000072 00000092 91c52ef2 000001b3 00000000 00000000 f6d0ffb8 f6d4573c Call Trace: [] refrigerator+0x3c/0x50 [] xfssyncd+0x15b/0x160 [] xfssyncd+0x0/0x160 [] kthread+0x6a/0x70 [] kthread+0x0/0x70 [] kernel_thread_helper+0x7/0x3c ======================= portmap D F6BA6684 0 2341 1 (NOTLB) f6d0dea0 00000086 00000282 f6ba6684 f6ba64a0 c033b7e6 00000282 f6ba64a0 f6ba64a0 c034d144 913389cf 000001b3 00002e10 00000000 f7358070 f730cb9c 00000073 000002ae 9133f6e9 000001b3 00000000 00000000 0804ff78 f6d0df08 Call Trace: [] inet_csk_clear_xmit_timers+0x36/0x50 [] tcp_v4_destroy_sock+0x14/0x150 [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] invalidate_inode_buffers+0xd/0xa0 [] d_kill+0x40/0x60 [] dput+0x1c/0xe0 [] __fput+0xf4/0x160 [] mntput_no_expire+0x1c/0x70 [] filp_close+0x43/0x70 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= syslogd D F77A1F28 0 2478 1 (NOTLB) f77a1ea0 00000086 f7fbe804 f77a1f28 00000000 00000001 ffffffff f7c20940 00000000 00000000 91338d55 000001b3 00002e6f 00000000 f7358a70 f735817c 00000073 00000070 9133fb50 000001b3 00000000 00000000 00000000 f77a1f08 Call Trace: [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_readv_writev+0xaa/0x190 [] pipe_write+0x0/0x400 [] sys_select+0xa9/0x170 [] sigprocmask+0x45/0xb0 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= klogd D C04953A0 0 2484 1 (NOTLB) f6809ea0 00000086 f68d0a20 c04953a0 f6809dc8 f6809e58 00000000 00000000 f6809eb0 00000001 913380eb 000001b3 0000267d 00000000 f730ca90 f7c9e69c 00000073 000000b5 9133dc16 000001b3 00000000 00000000 00000000 f6809f08 Call Trace: [] refrigerator+0x3c/0x50 [] smp_apic_timer_interrupt+0x30/0x40 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_syslog+0xf2/0x350 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] vfs_read+0xe4/0x110 [] sys_read+0x47/0x80 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= sshd D F7215BF8 0 2506 1 (NOTLB) f6a2dea0 00200082 f687e360 f7215bf8 00000000 f6a2de58 00000000 00000000 f6a2deb0 00000001 91339d89 000001b3 00002dd0 00000000 f730c090 f7358b7c 00000073 00000179 91340a0c 000001b3 00000000 00000000 00000000 f6a2df08 Call Trace: [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_wp_page+0x299/0x3b0 [] invalidate_inode_buffers+0xd/0xa0 [] d_kill+0x40/0x60 [] sys_select+0xa9/0x170 [] mntput_no_expire+0x1c/0x70 [] filp_close+0x43/0x70 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= exim4 D 00000000 0 2546 1 (NOTLB) f6835ea0 00000086 f6520a30 00000000 00000000 f6520a30 f6520a30 00000000 bf896808 c018dcac f6835e88 0000000d c03f1d32 00000f42 f72e1ab0 f7e2cbbc f6835e88 00000235 9134f609 000001b3 00000000 00000000 00000000 f6835f08 Call Trace: [] proc_flush_task+0x4c/0x1a0 [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_wait+0x1bb/0x3b0 [] do_setitimer+0x1f1/0x280 [] sys_select+0xa9/0x170 [] sys_wait4+0x31/0x40 [] do_notify_resume+0x3c/0x40 [] work_notifysig+0x13/0x19 ======================= inetd D 00000000 0 2552 1 (NOTLB) f6a29ea0 00000082 00000000 00000000 662f6984 c0437200 f6845bc0 f6a29eb8 c195f8c0 c02101b8 00000001 00000000 00000001 00000000 f7394ab0 f72e1bbc f6a29eb0 000004f5 913518bf 000001b3 00000000 00000000 00000000 f6a29f08 Call Trace: [] xfs_file_aio_read+0x78/0x90 [] refrigerator+0x3c/0x50 [] get_signal_to_deliver+0x227/0x230 [] do_signal+0x65/0x140 [] do_no_page+0x162/0x280 [] __handle_mm_fault+0xfa/0x1e0 [] do_page_fault+0x39b/0x590 [] sys_select+0xa9/0x170 [] do_notify_resume+0x3c/0x40 [