2003-05-07 13:58:54

by Chuck Ebbert

[permalink] [raw]
Subject: Problem: strace -ff fails on 2.4.21-rc1


Is this a bug or a feature? Machine only has 1 CPU, if that makes
any difference:


# strace -q -o minicom.trc -tt -ff minicom
upeek: ptrace(PTRACE_PEEKUSER, ... ): Operation not permitted
detach: ptrace(PTRACE_CONT, ...): Operation not permitted
Device /dev/ttyS1 lock failed: No child processes.
# uname -a
Linux d2 2.4.21-rc1 #1 SMP Wed May 7 06:05:31 EDT 2003 i686 unknown


(Ignore that minicom error message about the lock failing -- that's
a separate problem...)



2003-05-09 07:40:19

by Chuck Ebbert

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

Ingo Oeser wrote:

>> # strace -q -o minicom.trc -tt -ff minicom
>> upeek: ptrace(PTRACE_PEEKUSER, ... ): Operation not permitted
>> detach: ptrace(PTRACE_CONT, ...): Operation not permitted
>> Device /dev/ttyS1 lock failed: No child processes.
>> # uname -a
>> Linux d2 2.4.21-rc1 #1 SMP Wed May 7 06:05:31 EDT 2003 i686 unknown
>
> Is minicom a SUID binary, like on my machine? E.g. does
>
> ls -la $(which minicom)
>
> show
>
> -rwxr-sr-x 1 root uucp 130664 Feb 21 1998 /usr/bin/minicom
>
> ?
>
> SUID binaries cannot be ptrace()d under Linux for security reasons.


No, it's not suid. Rather than do that I changed the permissions
on /dev/ttyS1 so my normal userid could open it.

And the strace works -- until minicom forks and then the trace fails
to attach to the child. Both root and a normal userid fail with the
above messages; it works on 2.5.68.

(BTW does minicom work for you on 2.5? It fails with the "No child
processes" message on 2.5.6x here but works on 2.4 when it's not being
traced. Just the very act of running it under strace on 2.4 makes it
fail the same way it does on 2.5 here. And stracing it on 2.5.66 made
it start working again! Something very strange is going on...)

2003-05-09 09:01:08

by Chuck Ebbert

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

Ingo Oeser wrote:

> SUID binaries cannot be ptrace()d under Linux for security reasons.

I just found out minicom is spawing /sbin/lockdev which is setgrp
'lock'. Would that cause ptrace failure??


2003-05-09 12:03:14

by Carl-Daniel Hailfinger

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

Chuck Ebbert wrote:
> Ingo Oeser wrote:
>
>
>>SUID binaries cannot be ptrace()d under Linux for security reasons.
>
>
> I just found out minicom is spawing /sbin/lockdev which is setgrp
> 'lock'. Would that cause ptrace failure??

AFAIK that could have caused the failure. Please test 2.4.21-rc2 whcih
has fixes for many ptrace problems.


Carl-Daniel
--
http://www.hailfinger.org/

2003-05-11 15:28:50

by Chuck Ebbert

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

Carl-Daniel Hailfinger wrote:

> Chuck Ebbert wrote:
> > I just found out minicom is spawing /sbin/lockdev which is setgrp
> > 'lock'. Would that cause ptrace failure??
>
> AFAIK that could have caused the failure. Please test 2.4.21-rc2 whcih
> has fixes for many ptrace problems.

I can now strace minicom and its children with 2.4.21-rc2-ac1 but it
hangs on exit. Both child processes exit successfully:

# tail -5 minicom.trc.5775

10:16:49.299253 kill(5774, SIG_0) = 0
10:16:49.299418 unlink("/var/lock/LOCKDEV") = 0
10:16:49.299649 close(3) = 0
10:16:49.299849 umask(022) = 02
10:16:49.300043 _exit(0) = ?

# tail -8 minicom.trc.5776

10:17:05.497676 kill(5774, SIG_0) = 0
10:17:05.497845 unlink("/var/lock/LCK...5774") = 0
10:17:05.498068 unlink("/var/lock/LCK..ttyS1") = 0
10:17:05.498283 unlink("/var/lock/LCK.004.065") = 0
10:17:05.498616 unlink("/var/lock/LOCKDEV") = 0
10:17:05.498870 close(4) = 0
10:17:05.499065 umask(022) = 02
10:17:05.499337 _exit(0) = ?

However strace and minicom are hung up somehow and the screen is
black with a blinking cursor at row 1 column 1. The other ttys all
work OK and killing minicom cleans everything up.

# tail -3 minicom.trc

10:17:05.499678 --- SIGCHLD (Child exited) ---
10:17:05.499760 rt_sigaction(SIGCHLD, {SIG_DFL}, {SIG_IGN}, 8) = 0
10:17:05.499971 close(3


strace S C015FF82 960 5773 822 5774 (NOTLB)
Call Trace: [<c015ff82>] [<c011dba8>] [<c0108b73>]
minicom S 00000000 2400 5774 5773 (NOTLB)
Call Trace: [<c0122f24>] [<c0124095>] [<c021195b>] [<c0220d79>] [<c020d997>]
[<c0124095>] [<c0124159>] [<c020e01a>] [<c013c52e>] [<c01183a1>] [<c013b1d5>]
[<c013b23b>] [<c0108bdf>]

Proc; strace
>>EIP; c015ff82 <ext3_file_write+22/b0> <=====
Trace; c015ff82 <ext3_file_write+22/b0>
Trace; c011dba8 <sys_wait4+3c8/400>
Trace; c0108b73 <system_call+33/38>
Proc; minicom
>>EIP; 00000000 Before first symbol
Trace; c0122f24 <schedule_timeout+14/a0>
Trace; c0124095 <wake_up_parent+25/40>
Trace; c021195b <tty_wait_until_sent+9b/e0>
Trace; c0220d79 <rs_close+129/1f0>
Trace; c020d997 <release_dev+247/500>
Trace; c0124095 <wake_up_parent+25/40>
Trace; c0124159 <do_notify_parent+a9/c0>
Trace; c020e01a <tty_release+2a/60>
Trace; c013c52e <fput+4e/100>
Trace; c01183a1 <schedule+351/3b0>
Trace; c013b1d5 <filp_close+95/a0>
Trace; c013b23b <sys_close+5b/70>
Trace; c0108bdf <tracesys+1f/23>


Configuration:
2.4.21-rc2-ac1
SMP kernel on 1-CPU SMP machine (PII Xeon, 440GX)

To reproduce:
# strace -ff -q -tt -o minicom.trc minicom

Note:
I also get lockups while minicom is running:

Proc; strace
>>EIP; c015ff82 <ext3_file_write+22/b0> <=====
Trace; c015ff82 <ext3_file_write+22/b0>
Trace; c011dba8 <sys_wait4+3c8/400>
Trace; c0108b73 <system_call+33/38>
Proc; minicom
>>EIP; 00000000 Before first symbol
Trace; c0122f24 <schedule_timeout+14/a0>
Trace; c021105a <read_chan+37a/720>
Trace; c020cf5f <tty_read+cf/120>
Trace; c013b706 <sys_read+96/110>
Trace; c0108bdf <tracesys+1f/23>

2003-05-11 23:34:53

by Bernhard Kaindl

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

On Sun, 11 May 2003, Chuck Ebbert wrote:
> Carl-Daniel Hailfinger wrote:
> > Chuck Ebbert wrote:
> > > I just found out minicom is spawing /sbin/lockdev which is setgrp
> > > 'lock'. Would that cause ptrace failure??
> >
> > AFAIK that could have caused the failure. Please test 2.4.21-rc2 whcih
> > has fixes for many ptrace problems.
>
> I can now strace minicom and its children with 2.4.21-rc2-ac1 but it
> hangs on exit. Both child processes exit successfully:

Very strange, does this work with 2.4.20? What's your version of strace?

> However strace and minicom are hung up somehow and the screen is
> black with a blinking cursor at row 1 column 1. The other ttys all
> work OK and killing minicom cleans everything up.

Hm, this sounds like there could be some error/loop opening the device,
which could be the effect of another side effect of the original ptrace
fix, which my fixes which are included in 2.4.21-rc2 don't fix.

This side effect causes that if a system call needs a module loaded,
it is not loaded and only an error from request_module() is sent to
the kernel log. The attached patch on top of 2.4.21-rc2, fixes this
remaining problem.

I'm not writing much info about it now, except that I think that it does
not open any securiy hole, but I would like to give it a little more
testing on SMP machines. On single CPU it fixed the "modprobe rejected"
problem fine for me, without sacrifying ptrace securitey.

I'm uncertain if it would help in your case.

In your first message, you wrote:

> (BTW does minicom work for you on 2.5? It fails with the "No child
> processes" message on 2.5.6x here but works on 2.4 when it's not being
> traced. Just the very act of running it under strace on 2.4 makes it
> fail the same way it does on 2.5 here. And stracing it on 2.5.66 made
> it start working again! Something very strange is going on...)

Very strange, maybe a "tail minicom.trc.*" at the time when it's hanging
helps to get some idea.

Another note: suid is ignored when you are tracing the task which runs
exec() for a setuid program.

So if minicom relies on having the setgid gid of /sbin/lockdev honored
has in your case, the only ways I can find to get it working inder strace are:

- Change the locking config (temporary, for the debug) so that /sbin/lockdev
does not need to be setgid.

- Don't have ptrace follow fork mode activated when forking the child
which exec()'s /sbin/lockdev.

Bernd


Attachments:
ptrace-kmod-2.4.21-rc2.diff (662.00 B)

2003-05-12 00:21:10

by Daniel Jacobowitz

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

On Mon, May 12, 2003 at 01:50:01AM +0200, Bernhard Kaindl wrote:
> On Sun, 11 May 2003, Chuck Ebbert wrote:
> > Carl-Daniel Hailfinger wrote:
> > > Chuck Ebbert wrote:
> > > > I just found out minicom is spawing /sbin/lockdev which is setgrp
> > > > 'lock'. Would that cause ptrace failure??
> > >
> > > AFAIK that could have caused the failure. Please test 2.4.21-rc2 whcih
> > > has fixes for many ptrace problems.
> >
> > I can now strace minicom and its children with 2.4.21-rc2-ac1 but it
> > hangs on exit. Both child processes exit successfully:
>
> Very strange, does this work with 2.4.20? What's your version of strace?
>
> > However strace and minicom are hung up somehow and the screen is
> > black with a blinking cursor at row 1 column 1. The other ttys all
> > work OK and killing minicom cleans everything up.
>
> Hm, this sounds like there could be some error/loop opening the device,
> which could be the effect of another side effect of the original ptrace
> fix, which my fixes which are included in 2.4.21-rc2 don't fix.
>
> This side effect causes that if a system call needs a module loaded,
> it is not loaded and only an error from request_module() is sent to
> the kernel log. The attached patch on top of 2.4.21-rc2, fixes this
> remaining problem.
>
> I'm not writing much info about it now, except that I think that it does
> not open any securiy hole, but I would like to give it a little more
> testing on SMP machines. On single CPU it fixed the "modprobe rejected"
> problem fine for me, without sacrifying ptrace securitey.
>
> I'm uncertain if it would help in your case.
>
> In your first message, you wrote:
>
> > (BTW does minicom work for you on 2.5? It fails with the "No child
> > processes" message on 2.5.6x here but works on 2.4 when it's not being
> > traced. Just the very act of running it under strace on 2.4 makes it
> > fail the same way it does on 2.5 here. And stracing it on 2.5.66 made
> > it start working again! Something very strange is going on...)
>
> Very strange, maybe a "tail minicom.trc.*" at the time when it's hanging
> helps to get some idea.
>
> Another note: suid is ignored when you are tracing the task which runs
> exec() for a setuid program.
>
> So if minicom relies on having the setgid gid of /sbin/lockdev honored
> has in your case, the only ways I can find to get it working inder strace are:
>
> - Change the locking config (temporary, for the debug) so that /sbin/lockdev
> does not need to be setgid.
>
> - Don't have ptrace follow fork mode activated when forking the child
> which exec()'s /sbin/lockdev.

Run strace as root, and use strace -u? Then it should not remove the
setuid-ness, I think.

--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer

2003-05-12 00:54:01

by Chuck Ebbert

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

> Another note: suid is ignored when you are tracing the task which runs
> exec() for a setuid program.

You're right -- an ordinary user cannot trace minicom's setgroupid
children with -ff (but root now can, where with rc1 that failed too.)

And this is no longer an strace/ptrace problem at all.

Minicom hangs on exit (ctrl-a, q, enter) even when strace is not
running (can't believe I just now thought to try that.)

minicom S C01183A1 4472 1197 834 (NOTLB)
Using defaults from ksymoops -t elf32-i386 -a i386
Call Trace: [<c0122f24>] [<c0134a31>] [<c021195b>] [<c0220d79>] [<c020d997>]
[<c011690d>] [<c020e01a>] [<c013c52e>] [<c013b1d5>] [<c013b23b>] [<c0108b73>]
Warning (Oops_read): Code line not seen, dumping what data is available

Proc; minicom
>>EIP; c01183a1 <schedule+351/3b0> <===== ignore this bogus address
Trace; c0122f24 <schedule_timeout+14/a0>
Trace; c0134a31 <__alloc_pages+41/170>
Trace; c021195b <tty_wait_until_sent+9b/e0>
Trace; c0220d79 <rs_close+129/1f0>
Trace; c020d997 <release_dev+247/500>
Trace; c011690d <do_page_fault+11d/43b>
Trace; c020e01a <tty_release+2a/60>
Trace; c013c52e <fput+4e/100>
Trace; c013b1d5 <filp_close+95/a0>
Trace; c013b23b <sys_close+5b/70>
Trace; c0108b73 <system_call+33/38>

It's hung up somewhere inside schedule().


2003-05-12 08:09:59

by Russell King

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

On Sun, May 11, 2003 at 09:03:12PM -0400, Chuck Ebbert wrote:
> Proc; minicom
> >>EIP; c01183a1 <schedule+351/3b0> <===== ignore this bogus address
> Trace; c0122f24 <schedule_timeout+14/a0>
> Trace; c0134a31 <__alloc_pages+41/170>
> Trace; c021195b <tty_wait_until_sent+9b/e0>
> Trace; c0220d79 <rs_close+129/1f0>
> Trace; c020d997 <release_dev+247/500>
> Trace; c011690d <do_page_fault+11d/43b>
> Trace; c020e01a <tty_release+2a/60>
> Trace; c013c52e <fput+4e/100>
> Trace; c013b1d5 <filp_close+95/a0>
> Trace; c013b23b <sys_close+5b/70>
> Trace; c0108b73 <system_call+33/38>
>
> It's hung up somewhere inside schedule().

Wait 30 seconds and see if it exits by itself. I bet you have hardware
RTS/CTS handshaking enabled on the serial port, but without anything
connected.

When a port is closed, we wait up to 30 seconds (or a user specified time
period) for any characters in the transmit queue to be sent. If CTS is
inactive and we're using RTS/CTS handshaking, we can't send any characters,
so we'll wait the full timeout.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2003-05-12 11:40:55

by Chuck Ebbert

[permalink] [raw]
Subject: Re: Problem: strace -ff fails on 2.4.21-rc1

Russell King wrote:

>On Sun, May 11, 2003 at 09:03:12PM -0400, Chuck Ebbert wrote:
>> It's hung up somewhere inside schedule().
>
>Wait 30 seconds and see if it exits by itself. I bet you have hardware
>RTS/CTS handshaking enabled on the serial port, but without anything
>connected.

It hangs forever (over an hour anyway.) And it's definitely taking the
MAX_TIMEOUT case (wait forever) in schedule_timeout().

Yes, RTS/CTS is enabled but nothing is connected... and just attaching
a nullmodem adapter that lights up CTS, DTR and CD makes the problem go
away.

Does that mean this is a minicom problem because it didn't specify
a timeout?