Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754800Ab3JMPuJ (ORCPT ); Sun, 13 Oct 2013 11:50:09 -0400 Received: from frost.carfax.org.uk ([85.119.82.111]:38738 "EHLO frost.carfax.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752617Ab3JMPuG (ORCPT ); Sun, 13 Oct 2013 11:50:06 -0400 Date: Sun, 13 Oct 2013 16:50:00 +0100 From: Hugo Mills To: John Drescher Cc: linux-kernel@vger.kernel.org, Btrfs mailing list Subject: Re: btrfs hung tasks Message-ID: <20131013155000.GC28147@carfax.org.uk> Mail-Followup-To: Hugo Mills , John Drescher , linux-kernel@vger.kernel.org, Btrfs mailing list References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="snImcH6YoPvXmXT+" Content-Disposition: inline In-Reply-To: X-GPG-Fingerprint: B25A 5A66 69D3 1E15 2DD8 0998 BF73 F4E5 65E7 4AC0 X-GPG-Key: 65E74AC0 X-Parrot: It is no more. It has joined the choir invisible. X-IRC-Nicks: darksatanic darkersatanic darkling darkthing User-Agent: Mutt/1.5.21 (2010-09-15) X-frost.carfax.org.uk-Spam-Score: 0.0 (/) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 20182 Lines: 354 --snImcH6YoPvXmXT+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline [cc'd linux-btrfs] On Sat, Oct 12, 2013 at 09:36:42AM -0400, John Drescher wrote: > I am trying to debug this problem since it appears this is causing > mythtv to truncate recordings on top of other very long delays. > > # uname -a > Linux jmd0.comcast.net 3.11.4-gentoo-jmd0 #1 SMP PREEMPT Mon Oct 7 > 21:04:44 EDT 2013 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz > GenuineIntel GNU/Linux > > Here is 1 example of a hung task: > > [ 3000.576427] INFO: task TFWWrite:11329 blocked for more than 120 seconds. > [ 3000.576430] "echo 0 > /proc/sys/kernel/hung_task_ > timeout_secs" disables this message. > [ 3000.576433] TFWWrite D ffff8800a7ab5ac0 0 11329 1 0x00000000 > [ 3000.576437] ffff8800a7ab5a18 0000000000000082 0000000000000002 > ffff8800a7ab5fd8 > [ 3000.576442] ffff8800a7ab5fd8 0000000000004000 ffff8801a6cfade0 > ffff880099782de0 > [ 3000.576445] 0000000000000000 00000000000000d3 00000000000001a6 > 0000000000000001 > [ 3000.576449] Call Trace: > [ 3000.576458] [] ? load_balance+0xfe/0x800 > [ 3000.576462] [] ? sched_clock_local+0x25/0xa0 > [ 3000.576466] [] ? __lock_page+0x70/0x70 > [ 3000.576471] [] schedule+0x24/0x70 > [ 3000.576474] [] io_schedule+0x8a/0xd0 > [ 3000.576477] [] sleep_on_page+0x9/0x10 > [ 3000.576480] [] __wait_on_bit_lock+0x53/0xb0 > [ 3000.576484] [] __lock_page+0x62/0x70 > [ 3000.576488] [] ? wake_atomic_t_function+0x30/0x30 > [ 3000.576492] [] find_lock_page+0x51/0x80 > [ 3000.576495] [] find_or_create_page+0x3a/0xc0 > [ 3000.576499] [] prepare_pages.isra.23+0x18c/0x340 > [ 3000.576504] [] ? block_rsv_add_bytes+0x50/0x70 > [ 3000.576508] [] __btrfs_buffered_write+0x29a/0x4e0 > [ 3000.576511] [] btrfs_file_aio_write+0x215/0x580 > [ 3000.576515] [] ? kmem_cache_free+0xae/0x180 > [ 3000.576519] [] ? final_putname+0x21/0x50 > [ 3000.576523] [] do_sync_write+0x7b/0xb0 > [ 3000.576527] [] vfs_write+0xc0/0x1f0 > [ 3000.576530] [] SyS_write+0x4d/0xa0 > [ 3000.576533] [] system_call_fastpath+0x16/0x1b > [ 3000.576556] INFO: task btrfs-transacti:6770 blocked for more than > 120 seconds. > [ 3000.576558] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3000.576559] btrfs-transacti D ffff8800a6d6f0c0 0 6770 2 0x00000000 > [ 3000.576562] ffff8801500f5dd8 0000000000000046 ffff8801500f5d18 > ffff8801500f5fd8 > [ 3000.576566] ffff8801500f5fd8 0000000000004000 ffff88017f08dbc0 > ffff88015eeb2de0 > [ 3000.576569] ffff8801a86c0000 ffff8801a86b3990 0000000000000000 > 0000000000000000 > [ 3000.576573] Call Trace: > [ 3000.576577] [] ? btrfs_free_path+0x25/0x30 > [ 3000.576580] [] ? _raw_spin_lock+0x16/0x40 > [ 3000.576583] [] ? _raw_spin_unlock+0x11/0x40 > [ 3000.576587] [] ? btrfs_run_ordered_operations+0x20f/0x2c0 > [ 3000.576590] [] ? _raw_spin_lock_irqsave+0x18/0x50 > [ 3000.576593] [] ? _raw_spin_unlock_irqrestore+0x13/0x40 > [ 3000.576596] [] schedule+0x24/0x70 > [ 3000.576600] [] btrfs_commit_transaction+0x33d/0x960 > [ 3000.576603] [] ? abort_exclusive_wait+0xb0/0xb0 > [ 3000.576607] [] transaction_kthread+0x19d/0x220 > [ 3000.576610] [] ? btrfs_alloc_root+0x30/0x30 > [ 3000.576613] [] kthread+0xbb/0xc0 > [ 3000.576617] [] ? __schedule+0x4c0/0xa10 > [ 3000.576620] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3000.576623] [] ret_from_fork+0x7c/0xb0 > [ 3000.576626] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3000.576652] INFO: task mythpreviewgen:12052 blocked for more than > 120 seconds. > [ 3000.576654] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3000.576655] mythpreviewgen D ffff88015f6e8670 0 12052 12051 0x00000004 > [ 3000.576659] ffff8801066adc08 0000000000000082 ffff8801066adb08 > ffff8801066adfd8 > [ 3000.576662] ffff8801066adfd8 0000000000004000 ffff8801a6cf96f0 > ffff88018080ade0 > [ 3000.576665] ffff8801a655d0c0 ffff8801a655d0c0 000000006d5f7675 > 0000000000000000 > [ 3000.576669] Call Trace: > [ 3000.576676] [] ? do_expire_wait+0x17a/0x190 [autofs4] > [ 3000.576680] [] ? _raw_spin_lock+0x16/0x40 > [ 3000.576683] [] ? _raw_spin_lock+0x16/0x40 > [ 3000.576686] [] ? _raw_spin_unlock+0x11/0x40 > [ 3000.576689] [] schedule+0x24/0x70 > [ 3000.576692] [] schedule_preempt_disabled+0x13/0x20 > [ 3000.576695] [] __mutex_lock_slowpath+0x12c/0x210 > [ 3000.576699] [] mutex_lock+0x1d/0x40 > [ 3000.576702] [] do_last+0x268/0xd20 > [ 3000.576705] [] ? inode_permission+0x13/0x50 > [ 3000.576708] [] ? link_path_walk+0x23d/0x8e0 > [ 3000.576711] [] path_openat+0xbd/0x6b0 > [ 3000.576715] [] do_filp_open+0x3e/0xa0 > [ 3000.576718] [] ? _raw_spin_unlock+0x11/0x40 > [ 3000.576722] [] ? __alloc_fd+0xcb/0x120 > [ 3000.576725] [] do_sys_open+0x13c/0x230 > [ 3000.576729] [] SyS_open+0x1d/0x20 > [ 3000.576731] [] system_call_fastpath+0x16/0x1b > [ 3000.576734] INFO: task mythpreviewgen:12054 blocked for more than > 120 seconds. > [ 3000.576736] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3000.576737] mythpreviewgen D ffff8801a86c06e0 0 12054 12053 0x00000004 > [ 3000.576740] ffff880121141b28 0000000000000086 ffff880121141aa8 > ffff880121141fd8 > [ 3000.576744] ffff880121141fd8 0000000000004000 ffff8801a6cfade0 > ffff88000a795bc0 > [ 3000.576747] ffff880121141b58 ffffffff8125c7c7 ffff880121141a68 > ffffffff81561ca1 > [ 3000.576751] Call Trace: > [ 3000.576755] [] ? reserve_metadata_bytes+0x177/0x880 > [ 3000.576758] [] ? _raw_read_unlock+0x11/0x40 > [ 3000.576762] [] ? btrfs_set_lock_blocking_rw+0xd8/0x110 > [ 3000.576766] [] ? btrfs_search_slot+0x3ef/0x900 > [ 3000.576769] [] ? _raw_spin_lock_irqsave+0x18/0x50 > [ 3000.576772] [] ? _raw_spin_unlock_irqrestore+0x13/0x40 > [ 3000.576775] [] schedule+0x24/0x70 > [ 3000.576779] [] wait_current_trans.isra.34+0xb7/0x120 > [ 3000.576782] [] ? start_transaction+0x11c/0x4b0 > [ 3000.576785] [] ? abort_exclusive_wait+0xb0/0xb0 > [ 3000.576789] [] start_transaction+0x2e6/0x4b0 > [ 3000.576792] [] btrfs_start_transaction+0x16/0x20 > [ 3000.576795] [] btrfs_create+0x42/0x210 > [ 3000.576799] [] ? btrfs_permission+0x37/0x70 > [ 3000.576802] [] vfs_create+0x8e/0xf0 > [ 3000.576805] [] do_last+0x8e6/0xd20 > [ 3000.576808] [] ? inode_permission+0x13/0x50 > [ 3000.576812] [] path_openat+0xbd/0x6b0 > [ 3000.576815] [] do_filp_open+0x3e/0xa0 > [ 3000.576818] [] ? _raw_spin_unlock+0x11/0x40 > [ 3000.576821] [] ? __alloc_fd+0xcb/0x120 > [ 3000.576825] [] do_sys_open+0x13c/0x230 > [ 3000.576828] [] SyS_open+0x1d/0x20 > [ 3000.576831] [] system_call_fastpath+0x16/0x1b > [ 3120.576415] INFO: task TFWWrite:11329 blocked for more than 120 seconds. > [ 3120.576419] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3120.576421] TFWWrite D ffff8800a7ab5ac0 0 11329 1 0x00000000 > [ 3120.576426] ffff8800a7ab5a18 0000000000000082 0000000000000002 > ffff8800a7ab5fd8 > [ 3120.576431] ffff8800a7ab5fd8 0000000000004000 ffff8801a6cfade0 > ffff880099782de0 > [ 3120.576434] 0000000000000000 00000000000000d3 00000000000001a6 > 0000000000000001 > [ 3120.576438] Call Trace: > [ 3120.576447] [] ? load_balance+0xfe/0x800 > [ 3120.576451] [] ? sched_clock_local+0x25/0xa0 > [ 3120.576455] [] ? __lock_page+0x70/0x70 > [ 3120.576460] [] schedule+0x24/0x70 > [ 3120.576463] [] io_schedule+0x8a/0xd0 > [ 3120.576466] [] sleep_on_page+0x9/0x10 > [ 3120.576469] [] __wait_on_bit_lock+0x53/0xb0 > [ 3120.576473] [] __lock_page+0x62/0x70 > [ 3120.576477] [] ? wake_atomic_t_function+0x30/0x30 > [ 3120.576480] [] find_lock_page+0x51/0x80 > [ 3120.576483] [] find_or_create_page+0x3a/0xc0 > [ 3120.576487] [] prepare_pages.isra.23+0x18c/0x340 > [ 3120.576491] [] ? block_rsv_add_bytes+0x50/0x70 > [ 3120.576495] [] __btrfs_buffered_write+0x29a/0x4e0 > [ 3120.576499] [] btrfs_file_aio_write+0x215/0x580 > [ 3120.576502] [] ? kmem_cache_free+0xae/0x180 > [ 3120.576506] [] ? final_putname+0x21/0x50 > [ 3120.576510] [] do_sync_write+0x7b/0xb0 > [ 3120.576514] [] vfs_write+0xc0/0x1f0 > [ 3120.576517] [] SyS_write+0x4d/0xa0 > [ 3120.576521] [] system_call_fastpath+0x16/0x1b > [ 3120.576543] INFO: task btrfs-transacti:6770 blocked for more than > 120 seconds. > [ 3120.576545] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3120.576546] btrfs-transacti D ffff8800a6d6f0c0 0 6770 2 0x00000000 > [ 3120.576549] ffff8801500f5dd8 0000000000000046 ffff8801500f5d18 > ffff8801500f5fd8 > [ 3120.576553] ffff8801500f5fd8 0000000000004000 ffff88017f08dbc0 > ffff88015eeb2de0 > [ 3120.576556] ffff8801a86c0000 ffff8801a86b3990 0000000000000000 > 0000000000000000 > [ 3120.576560] Call Trace: > [ 3120.576563] [] ? btrfs_free_path+0x25/0x30 > [ 3120.576567] [] ? _raw_spin_lock+0x16/0x40 > [ 3120.576570] [] ? _raw_spin_unlock+0x11/0x40 > [ 3120.576573] [] ? btrfs_run_ordered_operations+0x20f/0x2c0 > [ 3120.576577] [] ? _raw_spin_lock_irqsave+0x18/0x50 > [ 3120.576580] [] ? _raw_spin_unlock_irqrestore+0x13/0x40 > [ 3120.576583] [] schedule+0x24/0x70 > [ 3120.576587] [] btrfs_commit_transaction+0x33d/0x960 > [ 3120.576590] [] ? abort_exclusive_wait+0xb0/0xb0 > [ 3120.576594] [] transaction_kthread+0x19d/0x220 > [ 3120.576597] [] ? btrfs_alloc_root+0x30/0x30 > [ 3120.576600] [] kthread+0xbb/0xc0 > [ 3120.576604] [] ? __schedule+0x4c0/0xa10 > [ 3120.576607] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3120.576610] [] ret_from_fork+0x7c/0xb0 > [ 3120.576613] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3120.576634] INFO: task kworker/u8:0:9957 blocked for more than 120 seconds. > [ 3120.576636] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3120.576637] kworker/u8:0 D ffff8801a4716d20 0 9957 2 0x00000000 > [ 3120.576644] Workqueue: writeback bdi_writeback_workfn (flush-btrfs-7) > [ 3120.576646] ffff88010080b6f8 0000000000000046 0000000000000000 > ffff88010080bfd8 > [ 3120.576649] ffff88010080bfd8 0000000000004000 ffff8801758e16f0 > ffff88014e4f2de0 > [ 3120.576653] ffff88010080b668 ffffffff8111b477 ffffffff810dadf0 > 0001120000000010 > [ 3120.576656] Call Trace: > [ 3120.576659] [] ? kmem_cache_alloc+0x57/0xf0 > [ 3120.576663] [] ? mempool_alloc_slab+0x10/0x20 > [ 3120.576666] [] ? mempool_alloc_slab+0x10/0x20 > [ 3120.576668] [] ? mempool_alloc+0x55/0x150 > [ 3120.576672] [] ? _raw_spin_lock_irqsave+0x18/0x50 > [ 3120.576675] [] ? _raw_spin_unlock_irqrestore+0x13/0x40 > [ 3120.576678] [] schedule+0x24/0x70 > [ 3120.576682] [] wait_current_trans.isra.34+0xb7/0x120 > [ 3120.576685] [] ? abort_exclusive_wait+0xb0/0xb0 > [ 3120.576688] [] start_transaction+0x288/0x4b0 > [ 3120.576692] [] btrfs_join_transaction+0x12/0x20 > [ 3120.576695] [] cow_file_range+0x40/0xe0 > [ 3120.576698] [] run_delalloc_range+0x35b/0x390 > [ 3120.576702] [] __extent_writepage+0x2b9/0x730 > [ 3120.576706] [] ? end_buffer_async_read+0x1e0/0x1e0 > [ 3120.576709] [] ? I_BDEV+0x10/0x10 > [ 3120.576713] [] ? merge+0x50/0xa0 > [ 3120.576718] [] > extent_write_cache_pages.isra.40.constprop.57+0x292/0x3d0 > [ 3120.576722] [] extent_writepages+0x48/0x60 > [ 3120.576725] [] ? btrfs_fiemap+0x60/0x60 > [ 3120.576728] [] btrfs_writepages+0x23/0x30 > [ 3120.576732] [] do_writepages+0x19/0x40 > [ 3120.576736] [] __writeback_single_inode+0x3e/0x140 > [ 3120.576739] [] writeback_sb_inodes+0x276/0x3c0 > [ 3120.576742] [] __writeback_inodes_wb+0x97/0xd0 > [ 3120.576746] [] wb_writeback+0x1cb/0x1e0 > [ 3120.576749] [] bdi_writeback_workfn+0x15e/0x310 > [ 3120.576754] [] process_one_work+0x140/0x390 > [ 3120.576757] [] worker_thread+0x119/0x370 > [ 3120.576761] [] ? rescuer_thread+0x2e0/0x2e0 > [ 3120.576764] [] kthread+0xbb/0xc0 > [ 3120.576767] [] ? __schedule+0x4c0/0xa10 > [ 3120.576771] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3120.576774] [] ret_from_fork+0x7c/0xb0 > [ 3120.576777] [] ? kthread_freezable_should_stop+0x60/0x60 > [ 3120.576782] INFO: task mythpreviewgen:12052 blocked for more than > 120 seconds. > [ 3120.576784] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3120.576785] mythpreviewgen D ffff88015f6e8670 0 12052 12051 0x00000004 > [ 3120.576789] ffff8801066adc08 0000000000000082 ffff8801066adb08 > ffff8801066adfd8 > [ 3120.576792] ffff8801066adfd8 0000000000004000 ffff8801a6cf96f0 > ffff88018080ade0 > [ 3120.576796] ffff8801a655d0c0 ffff8801a655d0c0 000000006d5f7675 > 0000000000000000 > [ 3120.576799] Call Trace: > [ 3120.576807] [] ? do_expire_wait+0x17a/0x190 [autofs4] > [ 3120.576810] [] ? _raw_spin_lock+0x16/0x40 > [ 3120.576813] [] ? _raw_spin_lock+0x16/0x40 > [ 3120.576816] [] ? _raw_spin_unlock+0x11/0x40 > [ 3120.576819] [] schedule+0x24/0x70 > [ 3120.576823] [] schedule_preempt_disabled+0x13/0x20 > [ 3120.576826] [] __mutex_lock_slowpath+0x12c/0x210 > [ 3120.576829] [] mutex_lock+0x1d/0x40 > [ 3120.576832] [] do_last+0x268/0xd20 > [ 3120.576835] [] ? inode_permission+0x13/0x50 > [ 3120.576838] [] ? link_path_walk+0x23d/0x8e0 > [ 3120.576841] [] path_openat+0xbd/0x6b0 > [ 3120.576845] [] do_filp_open+0x3e/0xa0 > [ 3120.576848] [] ? _raw_spin_unlock+0x11/0x40 > [ 3120.576852] [] ? __alloc_fd+0xcb/0x120 > [ 3120.576855] [] do_sys_open+0x13c/0x230 > [ 3120.576859] [] SyS_open+0x1d/0x20 > [ 3120.576861] [] system_call_fastpath+0x16/0x1b > [ 3120.576864] INFO: task mythpreviewgen:12054 blocked for more than > 120 seconds. > [ 3120.576866] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 3120.576867] mythpreviewgen D ffff8801a86c06e0 0 12054 12053 0x00000004 > [ 3120.576870] ffff880121141b28 0000000000000086 ffff880121141aa8 > ffff880121141fd8 > [ 3120.576874] ffff880121141fd8 0000000000004000 ffff8801a6cfade0 > ffff88000a795bc0 > [ 3120.576877] ffff880121141b58 ffffffff8125c7c7 ffff880121141a68 > ffffffff81561ca1 > [ 3120.576881] Call Trace: > [ 3120.576885] [] ? reserve_metadata_bytes+0x177/0x880 > [ 3120.576888] [] ? _raw_read_unlock+0x11/0x40 > [ 3120.576892] [] ? btrfs_set_lock_blocking_rw+0xd8/0x110 > [ 3120.576895] [] ? btrfs_search_slot+0x3ef/0x900 > [ 3120.576899] [] ? _raw_spin_lock_irqsave+0x18/0x50 > [ 3120.576902] [] ? _raw_spin_unlock_irqrestore+0x13/0x40 > [ 3120.576905] [] schedule+0x24/0x70 > [ 3120.576908] [] wait_current_trans.isra.34+0xb7/0x120 > [ 3120.576911] [] ? start_transaction+0x11c/0x4b0 > [ 3120.576915] [] ? abort_exclusive_wait+0xb0/0xb0 > [ 3120.576918] [] start_transaction+0x2e6/0x4b0 > [ 3120.576922] [] btrfs_start_transaction+0x16/0x20 > [ 3120.576925] [] btrfs_create+0x42/0x210 > [ 3120.576928] [] ? btrfs_permission+0x37/0x70 > [ 3120.576931] [] vfs_create+0x8e/0xf0 > [ 3120.576934] [] do_last+0x8e6/0xd20 > [ 3120.576937] [] ? inode_permission+0x13/0x50 > [ 3120.576941] [] path_openat+0xbd/0x6b0 > [ 3120.576944] [] do_filp_open+0x3e/0xa0 > [ 3120.576947] [] ? _raw_spin_unlock+0x11/0x40 > [ 3120.576951] [] ? __alloc_fd+0xcb/0x120 > [ 3120.576954] [] do_sys_open+0x13c/0x230 > [ 3120.576957] [] SyS_open+0x1d/0x20 > [ 3120.576960] [] system_call_fastpath+0x16/ > -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Jazz is the sort of music where no-one plays anything the --- same way once. --snImcH6YoPvXmXT+ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) iQIVAwUBUlrBKFheFHXiqx3kAQJW1g/+InGStOXexOAUqzaysZ7PnFzH5ruqe7zV Vw7zwi7g7FTwK4+ClU+yIl0/nwEop6xGyBX4Tqyly6uNTMFLcHB6P8n+4TAKHCy6 3427qwETK/F2duha8R7SeYeBexXKDMGcJUBQuDDvhP96oPMg7vOyoDu9Ed5nUHZy on1neLklBl+DvAK21WaMIev8F4qGMpRgDEr8kBaGP/19ldqmuwO+1H+BuhID2/0G cV/8181zqV67E+HGUC4rbzVZRkhsdR1r1EZPu7VX/SGrK7+5x1VitOMkXu0gZfNu x8dnAnodzy8r2RT+vvLiSja7NH+f02VnRg0S70Wcw47QvACulZXHxT0i31Jjq9Vt KFjTIlUZh/qx9rmW+NOj6mPW/nIEcwFs2oFgvqRqShxj1H3YwVHXvRsuNol1PaN2 oZoVvucPBoEdwgWkhoFAEFkORA5q5zJ3u7xhcfQLkKQhVo0HPak3vA0ZgUYD5sK2 pB1KvOo0X8F8M/nNe+LFR/7cM+l5nG/pZns+BXH10s3G1VS74CUIZ5lmu4uGLwtR GBDLDjoEDXqx0Dv3fHn7smbf4j+Mc+U5QdMExIVSHaS17wgLqEET7M+DtJ+4RFca oyny5gutTmlB7r2PXBZv85PCBnobYPJOZGUoVVbETsKXWYj+mbsApZd9X4XRIJVy S+zLr2xLC/s= =snSH -----END PGP SIGNATURE----- --snImcH6YoPvXmXT+-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/