Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754443AbZKBLCq (ORCPT ); Mon, 2 Nov 2009 06:02:46 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754065AbZKBLCp (ORCPT ); Mon, 2 Nov 2009 06:02:45 -0500 Received: from devils.ext.ti.com ([198.47.26.153]:56127 "EHLO devils.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753471AbZKBLCo convert rfc822-to-8bit (ORCPT ); Mon, 2 Nov 2009 06:02:44 -0500 From: "Dasgupta, Romit" To: "viro@zeniv.linux.org.uk" , "rjw@sisk.pl" CC: "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pm@lists.linux-foundation.org" , "linux-omap@vger.kernel.org" Date: Mon, 2 Nov 2009 16:32:13 +0530 Subject: [PATCH 0/1] PM: Making bdi threads non-freezable Thread-Topic: [PATCH 0/1] PM: Making bdi threads non-freezable Thread-Index: Acpbq+6/nJ9rhkNJSSGwee60PVnNTA== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 7077 Lines: 100 Hello all, For last few days I am facing an interesting suspend resume issue when I have a SD card inserted in a development platform. My kernel is built without CONFIG_MMC_UNSAFE_RESUME. (Most of the problems don't appear with CONFIG_MMC_UNSAFE_RESUME=y but that option seems to be not-recommended). When I try to issue system suspend (S2R) I can see that my shell is hung. Enabling CONFIG_DETECT_HUNG_TASK would reveal the following: # echo mem > /sys/power/state PM: Syncing filesystems ... done. Freezing user space processes ... (elapsed 0.00 seconds) done. Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done. platform_legacy_suspend(): serial8250_suspend+0x0/0x54 returns 1 mmc1: card 0001 removed platform_legacy_suspend(): omap_hsmmc_suspend+0x0/0x104 returns 1 mmc0: card 25b7 removed INFO: task sh:387 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. sh D c027e89c 0 387 1 0x00000000 [] (schedule+0x2e0/0x36c) from [] (bdi_sched_wait+0x8/0x10) [] (bdi_sched_wait+0x8/0x10) from [] (__wait_on_bit+0x5c/0xa8) [] (__wait_on_bit+0x5c/0xa8) from [] (out_of_line_wait_on_bit+0x74/0x80) [] (out_of_line_wait_on_bit+0x74/0x80) from [] (sync_inodes_sb+0x88/0x178) [] (sync_inodes_sb+0x88/0x178) from [] (__sync_filesystem+0x5c/0x88) [] (__sync_filesystem+0x5c/0x88) from [] (fsync_bdev+0x18/0x38) [] (fsync_bdev+0x18/0x38) from [] (invalidate_partition+0x18/0x34) [] (invalidate_partition+0x18/0x34) from [] (del_gendisk+0x24/0xb4) [] (del_gendisk+0x24/0xb4) from [] (mmc_blk_remove+0x24/0x44) [] (mmc_blk_remove+0x24/0x44) from [] (mmc_bus_remove+0x18/0x20) [] (mmc_bus_remove+0x18/0x20) from [] (__device_release_driver+0x64/0xa4) [] (__device_release_driver+0x64/0xa4) from [] (device_release_driver+0x1c/0x28) [] (device_release_driver+0x1c/0x28) from [] (bus_remove_device+0x7c/0x90) [] (bus_remove_device+0x7c/0x90) from [] (device_del+0x110/0x160) [] (device_del+0x110/0x160) from [] (mmc_remove_card+0x50/0x64) [] (mmc_remove_card+0x50/0x64) from [] (mmc_sd_remove+0x24/0x30) [] (mmc_sd_remove+0x24/0x30) from [] (mmc_suspend_host+0x110/0x1a8) [] (mmc_suspend_host+0x110/0x1a8) from [] (omap_hsmmc_suspend+0x74/0x104) [] (omap_hsmmc_suspend+0x74/0x104) from [] (platform_pm_suspend+0x60/0x8c) [] (platform_pm_suspend+0x60/0x8c) from [] (pm_op+0x30/0x74) [] (pm_op+0x30/0x74) from [] (dpm_suspend_start+0x3b4/0x518) [] (dpm_suspend_start+0x3b4/0x518) from [] (suspend_devices_and_enter+0x3c/0x1c4) [] (suspend_devices_and_enter+0x3c/0x1c4) from [] (enter_state+0xe0/0x138) [] (enter_state+0xe0/0x138) from [] (state_store+0x94/0xbc) [] (state_store+0x94/0xbc) from [] (kobj_attr_store+0x18/0x1c) [] (kobj_attr_store+0x18/0x1c) from [] (sysfs_write_file+0x108/0x13c) [] (sysfs_write_file+0x108/0x13c) from [] (vfs_write+0xac/0x154) [] (vfs_write+0xac/0x154) from [] (sys_write+0x3c/0x68) [] (sys_write+0x3c/0x68) from [] (ret_fast_syscall+0x0/0x2c) A closer investigation showed that when this happens the 'bdi' tasks (i.e. forker and the individual flush kthreads) are already in the 'refrigerator' hence we are blocked. I made those tasks as non-freezable and things were fine until I hit yet another deeper issue. I have attached the patch for the first fix in the next part of the mail. The second problem shows up when I have filesystem(s) mounted on the MMC card and I try the following: 1) I successfully suspend/resume followed by 2) attempt to next suspend/resume cycle. This time again I get blocked. khungd outputs the following: INFO: task sh:387 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. sh D c027e83c 0 387 1 0x00000000 [] (schedule+0x2e0/0x36c) from [] (bdi_sched_wait+0x8/0x10) [] (bdi_sched_wait+0x8/0x10) from [] (__wait_on_bit+0x5c/0xa8) [] (__wait_on_bit+0x5c/0xa8) from [] (out_of_line_wait_on_bit+0x74/0x80) [] (out_of_line_wait_on_bit+0x74/0x80) from [] (sync_inodes_sb+0x88/0x178) [] (sync_inodes_sb+0x88/0x178) from [] (__sync_filesystem+0x5c/0x88) [] (__sync_filesystem+0x5c/0x88) from [] (sync_filesystems+0xd0/0x140) [] (sync_filesystems+0xd0/0x140) from [] (sys_sync+0x1c/0x3c) [] (sys_sync+0x1c/0x3c) from [] (enter_state+0x38/0x138) [] (enter_state+0x38/0x138) from [] (state_store+0x94/0xbc) [] (state_store+0x94/0xbc) from [] (kobj_attr_store+0x18/0x1c) [] (kobj_attr_store+0x18/0x1c) from [] (sysfs_write_file+0x108/0x13c) [] (sysfs_write_file+0x108/0x13c) from [] (vfs_write+0xac/0x154) [] (vfs_write+0xac/0x154) from [] (sys_write+0x3c/0x68) [] (sys_write+0x3c/0x68) from [] (ret_fast_syscall+0x0/0x2c) After some investigation I could see that on the first successful suspend 'bdi_unregister' was called as a part of MMC removal. However the vfat filesystem was still mounted out of MMC and the superblock had a stale value for the 's_bdi' field pointing to the just removed struct backing_dev_info. On the next attempt for system suspend, 'sync_inodes_sb' was trying to queue work to the bdi work list for an invalid bdi while waking up the 'forker' task. The forker task would never find this bdi on the 'bdi_list' and hence we see this apparent lockup. So how do we handle unsafe removal while filesystem is still mounted? This is perhaps a bigger discussion. However for fixing this issue I will suggest the following (I am not even close to a FS internals beginner but I will try): i) On the suspend path save the information (disk + partition) of the disk being deleted and the superblocks that were mounted on this device. ii) On resume path when we try to add a newly detected disk we would compare the disk info and the partition info. iii) If the saved values and the detected values are same then update the 's_bdi' fields of the superblocks which were mounted on the partitions of this device. Please let me know if this is totally irrelevant or a brain dead idea? Regards, -Romit -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/