Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754948Ab0L3RO5 (ORCPT ); Thu, 30 Dec 2010 12:14:57 -0500 Received: from metis.ext.pengutronix.de ([92.198.50.35]:51678 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754662Ab0L3RO4 (ORCPT ); Thu, 30 Dec 2010 12:14:56 -0500 Date: Thu, 30 Dec 2010 18:14:53 +0100 From: Uwe =?iso-8859-1?Q?Kleine-K=F6nig?= To: Linus Torvalds Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: still nfs problems [Was: Linux 2.6.37-rc8] Message-ID: <20101230171453.GA5787@pengutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) X-SA-Exim-Connect-IP: 2001:6f8:1178:2:215:17ff:fe12:23b0 X-SA-Exim-Mail-From: ukl@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-kernel@vger.kernel.org Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 16540 Lines: 252 Hello, I wonder if the nfs-stuff is considered to be solved, because I still see strange things. During boot my machine sometimes (approx one out of two times) hangs with the output pasted below on Sysctl-l. The irq I'm not 100% sure it's related, but at least it seems to hang in nfs_readdir. (When the serial irq happend that triggered the sysrq the program counter was at 0xc014601c, which is fs/nfs/dir.c:647 for me.) This is on 2.6.37-rc8 plus some patches for machine support on an ARM machine. Best regards Uwe [ 2700.100000] SysRq : Show State [ 2700.100000] task PC stack pid father [ 2700.100000] init S c0285d80 0 1 0 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (do_wait+0x1a4/0x20c) [ 2700.100000] [] (do_wait+0x0/0x20c) from [] (sys_wait4+0xa8/0xc0) [ 2700.100000] [] (sys_wait4+0x0/0xc0) from [] (ret_fast_syscall+0x0/0x38) [ 2700.100000] r8:c0034088 r7:00000072 r6:00000001 r5:0000001b r4:0140b228 [ 2700.100000] kthreadd S c0285d80 0 2 0 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (kthreadd+0x70/0xfc) [ 2700.100000] [] (kthreadd+0x0/0xfc) from [] (do_exit+0x0/0x658) [ 2700.100000] ksoftirqd/0 S c0285d80 0 3 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (run_ksoftirqd+0x5c/0x110) [ 2700.100000] [] (run_ksoftirqd+0x0/0x110) from [] (kthread+0x8c/0x94) [ 2700.100000] r8:00000000 r7:c00526b8 r6:00000000 r5:c7843f1c r4:c7859fac [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f1c [ 2700.100000] kworker/0:0 S c0285d80 0 4 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (worker_thread+0x41c/0x444) [ 2700.100000] [] (worker_thread+0x0/0x444) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843edc [ 2700.100000] kworker/u:0 S c0285d80 0 5 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (worker_thread+0x41c/0x444) [ 2700.100000] [] (worker_thread+0x0/0x444) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843edc [ 2700.100000] watchdog/0 S c0285d80 0 6 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (watchdog+0xc0/0x110) [ 2700.100000] [] (watchdog+0x0/0x110) from [] (kthread+0x8c/0x94) [ 2700.100000] r6:00000000 r5:c7843efc r4:c785ffac [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843efc [ 2700.100000] khelper S c0285d80 0 7 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f1c [ 2700.100000] sync_supers S c0285d80 0 8 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (bdi_sync_supers+0x38/0x50) [ 2700.100000] [] (bdi_sync_supers+0x0/0x50) from [] (kthread+0x8c/0x94) [ 2700.100000] r5:c7843f2c r4:c7895fac [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f2c [ 2700.100000] bdi-default S c0285d80 0 9 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (schedule_timeout+0x22c/0x27c) [ 2700.100000] [] (schedule_timeout+0x0/0x27c) from [] (bdi_forker_thread+0x3a8/0x41c) [ 2700.100000] r8:c0363f80 r7:00000000 r6:00000000 r5:c03641e8 r4:00000000 [ 2700.100000] [] (bdi_forker_thread+0x0/0x41c) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843efc [ 2700.100000] kintegrityd S c0285d80 0 10 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843e9c [ 2700.100000] kblockd S c0285d80 0 11 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843ec4 [ 2700.100000] rpciod S c0285d80 0 12 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843ebc [ 2700.100000] kworker/0:1 S c0285d80 0 13 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (worker_thread+0x41c/0x444) [ 2700.100000] [] (worker_thread+0x0/0x444) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c785be94 [ 2700.100000] khungtaskd S c0285d80 0 14 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (schedule_timeout+0x22c/0x27c) [ 2700.100000] [] (schedule_timeout+0x0/0x27c) from [] (schedule_timeout_interruptible+0x28/0x2c) [ 2700.100000] r8:00000078 r7:00007fe9 r6:000003e9 r5:c034eef0 r4:00000064 [ 2700.100000] [] (schedule_timeout_interruptible+0x0/0x2c) from [] (watchdog+0x54/0x2e8) [ 2700.100000] [] (watchdog+0x0/0x2e8) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f2c [ 2700.100000] kswapd0 S c0285d80 0 15 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (kswapd+0x210/0x74c) [ 2700.100000] [] (kswapd+0x0/0x74c) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f1c [ 2700.100000] fsnotify_mark S c0285d80 0 16 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (fsnotify_mark_destroy+0x11c/0x144) [ 2700.100000] [] (fsnotify_mark_destroy+0x0/0x144) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843f34 [ 2700.100000] aio S c0285d80 0 17 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843edc [ 2700.100000] nfsiod S c0285d80 0 18 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843edc [ 2700.100000] crypto S c0285d80 0 19 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (rescuer_thread+0x1b8/0x1c4) [ 2700.100000] [] (rescuer_thread+0x0/0x1c4) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c7843ee4 [ 2700.100000] kworker/u:1 S c0285d80 0 24 2 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (worker_thread+0x41c/0x444) [ 2700.100000] [] (worker_thread+0x0/0x444) from [] (kthread+0x8c/0x94) [ 2700.100000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x658) [ 2700.100000] r7:00000013 r6:c004f4d8 r5:c006a208 r4:c785de94 [ 2700.100000] rcS S c0285d80 0 27 1 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (do_wait+0x1a4/0x20c) [ 2700.100000] [] (do_wait+0x0/0x20c) from [] (sys_wait4+0xa8/0xc0) [ 2700.100000] [] (sys_wait4+0x0/0xc0) from [] (ret_fast_syscall+0x0/0x38) [ 2700.100000] r8:c0034088 r7:00000072 r6:ffffffff r5:bee7880c r4:00000000 [ 2700.100000] run-parts S c0285d80 0 35 27 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (schedule+0x0/0x534) from [] (do_wait+0x1a4/0x20c) [ 2700.100000] [] (do_wait+0x0/0x20c) from [] (sys_wait4+0xa8/0xc0) [ 2700.100000] [] (sys_wait4+0x0/0xc0) from [] (ret_fast_syscall+0x0/0x38) [ 2700.100000] r8:c0034088 r7:00000072 r6:00000024 r5:bef7dcc4 r4:00000000 [ 2700.100000] S00splashutil R running 0 36 35 0x00000000 [ 2700.100000] Backtrace: [ 2700.100000] [] (dump_backtrace+0x0/0x110) from [] (show_stack+0x1c/0x20) [ 2700.100000] r7:c79cfd64 r6:00000000 r5:c7954600 r4:00000000 [ 2700.100000] [] (show_stack+0x0/0x20) from [] (sched_show_task+0xb0/0xcc) [ 2700.100000] [] (sched_show_task+0x0/0xcc) from [] (show_state_filter+0x5c/0xc8) [ 2700.100000] r5:c7954600 r4:c7954600 [ 2700.100000] [] (show_state_filter+0x0/0xc8) from [] (sysrq_handle_showstate+0x18/0x1c) [ 2700.100000] r8:20000093 r7:00000007 r6:00000001 r5:00000074 r4:c036ec5c [ 2700.100000] [] (sysrq_handle_showstate+0x0/0x1c) from [] (__handle_sysrq+0xe0/0x190) [ 2700.100000] [] (__handle_sysrq+0x0/0x190) from [] (handle_sysrq+0x38/0x44) [ 2700.100000] r8:c7999000 r7:00000100 r6:c7973640 r5:00010074 r4:c7864300 [ 2700.100000] [] (handle_sysrq+0x0/0x44) from [] (pl011_int+0x18c/0x5a4) [ 2700.100000] [] (pl011_int+0x0/0x5a4) from [] (handle_IRQ_event+0x7c/0x1a8) [ 2700.100000] [] (handle_IRQ_event+0x0/0x1a8) from [] (handle_level_irq+0xc8/0x148) [ 2700.100000] [] (handle_level_irq+0x0/0x148) from [] (asm_do_IRQ+0x80/0xa4) [ 2700.100000] r7:c74a05a4 r6:c74a0508 r5:00000000 r4:0000002f [ 2700.100000] [] (asm_do_IRQ+0x0/0xa4) from [] (__irq_svc+0x38/0x80) [ 2700.100000] Exception stack(0xc79cfe88 to 0xc79cfed0) [ 2700.100000] fe80: c74a0508 00000000 c0145d24 c7487e60 00000000 c79cfee8 [ 2700.100000] fea0: c74a0508 c74a05a4 c7487e60 c79cfee8 c74a0508 c79cff4c c79ce000 c79cfed0 [ 2700.100000] fec0: c016ff10 c014601c 60000013 ffffffff [ 2700.100000] r5:f5000000 r4:ffffffff [ 2700.100000] [] (nfs_readdir+0x0/0x458) from [] (vfs_readdir+0x7c/0xb0) [ 2700.100000] [] (vfs_readdir+0x0/0xb0) from [] (sys_getdents+0x70/0xb8) [ 2700.100000] [] (sys_getdents+0x0/0xb8) from [] (ret_fast_syscall+0x0/0x38) [ 2700.100000] r7:0000008d r6:00000000 r5:402ed00c r4:402ed020 [ 2700.100000] Sched Debug Version: v0.09, 2.6.37-rc8-00065-g1cd48e3-dirty #35 [ 2700.100000] now at 2701202.749966 msecs [ 2700.100000] .jiffies : 240010 [ 2700.100000] .sysctl_sched_latency : 6.000000 [ 2700.100000] .sysctl_sched_min_granularity : 0.750000 [ 2700.100000] .sysctl_sched_wakeup_granularity : 1.000000 [ 2700.100000] .sysctl_sched_child_runs_first : 0 [ 2700.100000] .sysctl_sched_features : 31855 [ 2700.100000] .sysctl_sched_tunable_scaling : 1 (logaritmic) [ 2700.100000] [ 2700.100000] cpu#0 [ 2700.100000] .nr_running : 1 [ 2700.100000] .load : 1024 [ 2700.100000] .nr_switches : 11875 [ 2700.100000] .nr_load_updates : 269696 [ 2700.100000] .nr_uninterruptible : 0 [ 2700.100000] .next_balance : 0.000000 [ 2700.100000] .curr->pid : 36 [ 2700.100000] .clock : 2700100.000000 [ 2700.100000] .cpu_load[0] : 1024 [ 2700.100000] .cpu_load[1] : 1024 [ 2700.100000] .cpu_load[2] : 1024 [ 2700.100000] .cpu_load[3] : 1024 [ 2700.100000] .cpu_load[4] : 1024 [ 2700.100000] [ 2700.100000] cfs_rq[0]: [ 2700.100000] .exec_clock : 0.000000 [ 2700.100000] .MIN_vruntime : 0.000001 [ 2700.100000] .min_vruntime : 2695651.938408 [ 2700.100000] .max_vruntime : 0.000001 [ 2700.100000] .spread : 0.000000 [ 2700.100000] .spread0 : 0.000000 [ 2700.100000] .nr_running : 1 [ 2700.100000] .load : 1024 [ 2700.100000] .nr_spread_over : 0 [ 2700.100000] [ 2700.100000] rt_rq[0]: [ 2700.100000] .rt_nr_running : 0 [ 2700.100000] .rt_throttled : 0 [ 2700.100000] .rt_time : 0.000000 [ 2700.100000] .rt_runtime : 950.000000 [ 2700.100000] [ 2700.100000] runnable tasks: [ 2700.100000] task PID tree-key switches prio exec-runtime sum-exec sum-sleep [ 2700.100000] ---------------------------------------------------------------------------------------------------------- [ 2700.100000] R S00splashutils 36 2695651.938408 5397 120 0 0 0.000000 0.000000 0.000000 [ 2700.100000] [ 2700.100000] [ 2700.100000] Showing all locks held in the system: [ 2700.100000] 4 locks held by S00splashutils/36: [ 2700.100000] #0: (&sb->s_type->i_mutex_key#8){+.+.+.}, at: [] vfs_readdir+0x4c/0xb0 [ 2700.100000] #1: (&port_lock_key){-.-...}, at: [] pl011_int+0x20/0x5a4 [ 2700.100000] #2: (sysrq_key_table_lock){-.....}, at: [] __handle_sysrq+0x24/0x190 [ 2700.100000] #3: (tasklist_lock){.?.+..}, at: [] debug_show_all_locks+0x40/0x1a4 [ 2700.100000] [ 2700.100000] ============================================= [ 2700.100000] -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/