I do have a user mode linux image (stable 32 bit Gentoo Linux ) which erratically crashes
while fuzz tested with trinity if the victim files are located on a NFS share.
The back trace of the core dumps always looks like the attached.
To bisect it is hard. However after few attempts in the last weeks the following
commit is either the first bad commit or at least the upper limit (less likely).
commit 8aac62706adaaf0fab02c4327761561c8bda9448
Author: Oleg Nesterov <[email protected]>
Date: Fri Jun 14 21:09:49 2013 +0200
move exit_task_namespaces() outside of exit_notify()
tfoerste@n22 ~/devel/linux $ gdb --core=/mnt/ramdisk/core /home/tfoerste/devel/linux/linux -n -batch -ex bt
[New LWP 20802]
Core was generated by `/home/tfoerste/devel/linux/linux earlyprintk ubda=/home/tfoerste/virtual/uml/tr'.
Program terminated with signal 6, Aborted.
#0 0xb778e424 in __kernel_vsyscall ()
#0 0xb778e424 in __kernel_vsyscall ()
#1 0x08396175 in kill ()
#2 0x0807155d in uml_abort () at arch/um/os-Linux/util.c:93
#3 0x08071845 in os_dump_core () at arch/um/os-Linux/util.c:138
#4 0x08061197 in panic_exit (self=0x8591518 <panic_exit_notifier>, unused1=0, unused2=0x85c5d60 <buf.12251>) at arch/um/kernel/um_arch.c:240
#5 0x0809daf8 in notifier_call_chain (nl=0x0, val=0, v=0x85c5d60 <buf.12251>, nr_to_call=-2, nr_calls=0x0) at kernel/notifier.c:93
#6 0x0809dc43 in __atomic_notifier_call_chain (nr_calls=<optimized out>, nr_to_call=<optimized out>, v=<optimized out>, val=<optimized out>, nh=<optimized out>) at kernel/notifier.c:182
#7 atomic_notifier_call_chain (nh=0x85c5d44 <panic_notifier_list>, val=0, v=0x85c5d60 <buf.12251>) at kernel/notifier.c:191
#8 0x083f34f8 in panic (fmt=0x0) at kernel/panic.c:127
#9 0x08060b5e in segv (fi=<incomplete type>, ip=136527369, is_user=0, regs=0x858f85c <cpu0_irqstack+30812>) at arch/um/kernel/trap.c:209
#10 0x08060e13 in segv_handler (sig=11, unused_si=0x858fb0c <cpu0_irqstack+31500>, regs=0x858f85c <cpu0_irqstack+30812>) at arch/um/kernel/trap.c:185
#11 0x080706a8 in sig_handler_common (sig=11, si=0x858fb0c <cpu0_irqstack+31500>, mc=0x858fba0 <cpu0_irqstack+31648>) at arch/um/os-Linux/signal.c:44
#12 0x080707ed in sig_handler (sig=0, si=0x858fb0c <cpu0_irqstack+31500>, mc=0x858fba0 <cpu0_irqstack+31648>) at arch/um/os-Linux/signal.c:231
#13 0x0807033b in hard_handler (sig=6, si=0x858fb0c <cpu0_irqstack+31500>, p=0x858fba0 <cpu0_irqstack+31648>) at arch/um/os-Linux/signal.c:165
#14 <signal handler called>
#15 nlmclnt_setlockargs (req=0x48e18860, fl=0x48f27c8c) at fs/lockd/clntproc.c:131
#16 0x08234892 in nlmclnt_proc (host=0x48e18860, cmd=7, fl=0x48f27c8c) at fs/lockd/clntproc.c:170
#17 0x081d91ae in nfs_proc_lock (filp=0x0, cmd=0, fl=0x0) at fs/nfs/proc.c:667
#18 0x081ca386 in do_unlk (filp=0x48fbe0c0, cmd=7, fl=0x48f27c8c, is_local=0) at fs/nfs/file.c:773
#19 0x081ca572 in nfs_flock (filp=0x48fbe0c0, cmd=7, fl=0x0) at fs/nfs/file.c:902
#20 0x0813ee6e in locks_remove_flock (filp=0x48fbe0c0) at fs/locks.c:2074
#21 0x080fe438 in __fput (file=0x48fbe0c0) at fs/file_table.c:240
#22 0x080fe55b in ____fput (work=0x48fbe0c0) at fs/file_table.c:285
#23 0x08095f3e in task_work_run () at kernel/task_work.c:87
#24 0x08080c9d in exit_task_work (task=<optimized out>) at include/linux/task_work.h:21
#25 do_exit (code=1224150016) at kernel/exit.c:798
#26 0x080812a7 in do_group_exit (exit_code=11) at kernel/exit.c:931
#27 0x0808bc2d in get_signal_to_deliver (info=0x48f27e34, return_ka=0x48f27eb4, regs=0x48db31d4, cookie=0x0) at kernel/signal.c:2370
#28 0x0805f6ec in kern_do_signal (regs=0x48db31d4) at arch/um/kernel/signal.c:77
#29 0x0805f7ed in do_signal () at arch/um/kernel/signal.c:123
#30 0x0805e6b7 in interrupt_end () at arch/um/kernel/process.c:107
#31 0x08073c1b in userspace (regs=0x48db31d4) at arch/um/os-Linux/skas/process.c:464
#32 0x0805e44c in fork_handler () at arch/um/kernel/process.c:160
#33 0x5a5a5a5a in ?? ()
--
MfG/Sincerely
Toralf Förster
pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
On 09/22, Toralf F?rster wrote:
>
> On 07/27/2013 07:00 PM, Oleg Nesterov wrote:
> >
> > So nlmclnt_setlockargs()->utsname() crashes and we probably need
> > the patch below.
> >
> > But is it correct? I know _absolutely_ nothing about nfs/sunrpc/etc and
> > I never looked into this code before, most probably I am wrong.
> >
> > But it seems that __nlm_async_call() relies on workqueues.
> > nlmclnt_async_call() does rpc_wait_for_completion_task(), but what if
> > the caller is killed?
> >
> > nlm_rqst can't go away, ->a_count was incremented. But can't the caller
> > exit before call->name is used? In this case the memory it points to
> > can be already freed.
> >
> > Oleg.
> >
> > --- x/kernel/exit.c
> > +++ x/kernel/exit.c
> > @@ -783,8 +783,8 @@ void do_exit(long code)
> > exit_shm(tsk);
> > exit_files(tsk);
> > exit_fs(tsk);
> > - exit_task_namespaces(tsk);
> > exit_task_work(tsk);
> > + exit_task_namespaces(tsk);
> > check_stack_usage();
> > exit_thread();
> >
> >
> >
> /me wonders if/when this will go in the main kernel ?
I think this was fixed by 9a1b6bf818e74 ?
Oleg.