2002-09-10 16:27:42

by David Brownell

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

> In short:
>
> Either you want debugging (in which case BUG() is the wrong thing to
> do), or you don't want debugging (in which case BUG() is the wrong thing
> to do). You can choose either, but in neither case is BUG() acceptable.

Or in even shorter sound bite format: "Just say no to BUG()s."

:)




2002-09-10 16:46:28

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On Tue, 10 Sep 2002, David Brownell wrote:
> > In short:
> >
> > Either you want debugging (in which case BUG() is the wrong thing to
> > do), or you don't want debugging (in which case BUG() is the wrong thing
> > to do). You can choose either, but in neither case is BUG() acceptable.
>
> Or in even shorter sound bite format: "Just say no to BUG()s."

Well, the thing is, BUG() _is_ sometimes useful. It's a dense and very
convenient way to say that something catastrophic happened.

And actually, outside of drivers and filesystems you can often know (or
control) the number of locks the surrounding code is holding, and then a
BUG() may not be as lethal. At which point the normal "oops and kill the
process" action is clearly fine - the machine is still perfectly usable.

(In fact, on UP a BUG() tends to be quite usable just about anywhere
except in an interrupt handler: there may be some local locks like
directory semaphores etc that are held and not released, but _most_ of the
time the machine is quite usable. SMP really does make things harder to
debug even quite apart from the races it introduces. Sad.)

Linus

2002-09-10 18:37:54

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, 2002-09-10 at 17:51, Linus Torvalds wrote:
> (In fact, on UP a BUG() tends to be quite usable just about anywhere
> except in an interrupt handler: there may be some local locks like
> directory semaphores etc that are held and not released, but _most_ of the
> time the machine is quite usable. SMP really does make things harder to
> debug even quite apart from the races it introduces. Sad.)

It drops you politely into the kernel debugger, you fix up the values
and step over it. If you want to debug with zen mind power and printk
feel free. For the rest of us BUG() is fine on SMP

2002-09-10 18:58:26

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On 10 Sep 2002, Alan Cox wrote:
>
> It drops you politely into the kernel debugger, you fix up the values
> and step over it. If you want to debug with zen mind power and printk
> feel free. For the rest of us BUG() is fine on SMP

Ok, a show of hands..

Of the millions (whatever) of Linux machines, how many have a kernel
debugger attached? Count them.

In other words, if a user is faced with a dead machine with no other way
to even know what BUG() triggered than to try to set up a cross debugger,
just how useful is that BUG()? I claim it is pretty useless - simply
because 99+% of all people won't even make a bug report in that case,
they'll just push the reset button and consider Linux unreliable.

In other news, the approach that shows up in the kernel logs might just
eventually be noticed and acted upon (especially if the machine acts
strange and kills processes).

So I claim a BUG() that locks up the machine is useless. If the user can't
just run ksymoops and email out the BUG message, that BUG() is _not_ fine
on SMP.

It has nothing to do with zen mind power or printk's. It has everything to
do with the fact that a working machine is about a million times easier to
debug on than a dead one, _and_ is a lot more likely to get acted upon by
most users.

Linus

2002-09-10 19:22:43

by Rik van Riel

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, 10 Sep 2002, Linus Torvalds wrote:

> So I claim a BUG() that locks up the machine is useless. If the user
> can't just run ksymoops and email out the BUG message, that BUG() is
> _not_ fine on SMP.

Agreed. Along those same lines, it would be nice if the kernel
could spit out symbolic names so the user can't screw up the
backtrace and we've got a better chance of extracting a useful
bug report from our users ;)

regards,

Rik
--
Spamtrap of the month: [email protected]

http://www.surriel.com/ http://distro.conectiva.com/

2002-09-10 19:33:23

by David Miller

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


The way I see it, FATAL() is the only thing that really should escape
from my local tree and end up ever being a permanent fixture of the
code that is in the standard sources.

The other WARN/DEBUG variants seem, at least to me, to be a prime
candidate for unmaintained clutter.

Also, another big win of OHSHIT(), or BUG_ON() or whatever you
want to call it is that the thing is so cheap. Both in terms
of space (no strings, no calls with varags arguments, 1 instruction)
and time (a cycle or two).

With these DEBUG/WARN/FATAL things with the strings, I am much less
likely to ever add them to code I am writing. Simply because I'll
say "hmmm that's expensive, the check isn't that important"

On the other hand I do recognize your user side arguments.

But to counter the "BUG with locks freezes box" argument, 9 times out
of 10 the thing that you are BUG()'ing on will freeze up the box
anyways with the same locks held when some other thread of execution
derefences some bad pointer created by, or as a side effect of, the
erroneous state.

2002-09-10 20:11:30

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, 2002-09-10 at 20:27, Rik van Riel wrote:
> On Tue, 10 Sep 2002, Linus Torvalds wrote:
>
> > So I claim a BUG() that locks up the machine is useless. If the user
> > can't just run ksymoops and email out the BUG message, that BUG() is
> > _not_ fine on SMP.
>
> Agreed. Along those same lines, it would be nice if the kernel
> could spit out symbolic names so the user can't screw up the
> backtrace and we've got a better chance of extracting a useful
> bug report from our users ;)

There is a patch for this. However its fairly useless since all the
users are in X11 and while -ac will give you morse as well thats not
terribly friendly.

There are some real mode patches that try and get you back into a sane
video mode and dump you into a saner environment.

2002-09-10 21:55:58

by David Woodhouse

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


[email protected] said:
> There is a patch for this. However its fairly useless since all the
> users are in X11 and while -ac will give you morse as well thats not
> terribly friendly.

> There are some real mode patches that try and get you back into a sane
> video mode and dump you into a saner environment.

Why change video mode? As long as X told the kernel what it's doing, we
know how to drive a linear framebuffer and in fact a few other types of
framebuffers. In the 'oh shit' case we could at least _attempt_ to spew our
dying splutter onto the screen.

--
dwmw2


2002-09-10 22:17:11

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, 2002-09-10 at 23:00, David Woodhouse wrote:
> Why change video mode? As long as X told the kernel what it's doing, we
> know how to drive a linear framebuffer and in fact a few other types of
> framebuffers. In the 'oh shit' case we could at least _attempt_ to spew our
> dying splutter onto the screen.

It changes video mode to get the hardware into a known or roughly known
state. Linear framebuffer is a bit of an assumption both at the low end
(windowed) and high end (no framebuffer or tiled)

2002-09-10 22:22:07

by David Woodhouse

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


[email protected] said:
> It changes video mode to get the hardware into a known or roughly
> known state. Linear framebuffer is a bit of an assumption both at the
> low end (windowed) and high end (no framebuffer or tiled)

We shouldn't _assume_ it. XFree86 should tell us. And it can tell us about
the other cases too; some of those we can deal with anyway.

--
dwmw2


2002-09-10 22:55:23

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, 2002-09-10 at 23:26, David Woodhouse wrote:
>
> [email protected] said:
> > It changes video mode to get the hardware into a known or roughly
> > known state. Linear framebuffer is a bit of an assumption both at the
> > low end (windowed) and high end (no framebuffer or tiled)
>
> We shouldn't _assume_ it. XFree86 should tell us. And it can tell us about
> the other cases too; some of those we can deal with anyway.

You could make a usermode app that makes a DGA query and feeds the
results to the kernel I guess

2002-09-15 04:54:51

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tuesday 10 September 2002 18:51, Linus Torvalds wrote:
> On Tue, 10 Sep 2002, David Brownell wrote:
> > Or in even shorter sound bite format: "Just say no to BUG()s."
>
> Well, the thing is, BUG() _is_ sometimes useful. It's a dense and very
> convenient way to say that something catastrophic happened.

There's an important case you're overlooking that takes us out of the
"sometimes" zone, and that is where you want to load up some piece of
heavily-context-dependent code with assertions, just to have confidence
that the many assumptions actually hold. And once you have those hooks
in the code, it often makes little sense to take them out, because
they'll just have to go back in again the next time some remote part
of the kernel violates one of the assumptions.

What you *want* to do, is just turn them off, not remove them. (Sure,
there are lots that shouldn't survive the alpha version of any code,
but still lots of good ones that should be kept, just stubbed out.)

So this is just a name problem: define MYSUBSYSTEM_BUG_ON which is nil
in production, but equivalent to BUG_ON for alpha or beta code. In the
former case it means the code is going to run a little faster, plus the
system is going to be a little more resilient, as you say.

Otherwise, completely agreed.

> And actually, outside of drivers and filesystems you can often know (or
> control) the number of locks the surrounding code is holding, and then a
> BUG() may not be as lethal. At which point the normal "oops and kill the
> process" action is clearly fine - the machine is still perfectly usable.

Eventually we could try some fancy trick that involves keeping track
of the locks in a systematic way, so that they can be analyzed
automatically by a tool that generates lock-breaking code. Then a
subsystem could use the generated code in its error path. Sure, it's a
lot of work just to add another "9" to reliability, but when was anything
ever easy?

--
Daniel

2002-09-15 05:03:40

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tuesday 10 September 2002 21:03, Linus Torvalds wrote:
> On 10 Sep 2002, Alan Cox wrote:
> >
> > It drops you politely into the kernel debugger, you fix up the values
> > and step over it. If you want to debug with zen mind power and printk
> > feel free. For the rest of us BUG() is fine on SMP
>
> Ok, a show of hands..
>
> Of the millions (whatever) of Linux machines, how many have a kernel
> debugger attached? Count them.

Eh, mine is getting one attached to it right now. It's getting more
popular, and it would be more popular yet if it weren't considered some
dirty little secret, or somehow unmanly.

Let's try a different show of hands: How many users would be happier if
they knew that kernel developers are using modern techniques to improve
the quality of the kernel?

Of course, I use the term "modern" here loosely, since kdb and kgdb are
really only 80's technology. Without them, we're stuck in the 60's.

--
Daniel

2002-09-15 05:28:47

by Daniel Berlin

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34



On Sun, 15 Sep 2002, Daniel Phillips wrote:

> On Tuesday 10 September 2002 21:03, Linus Torvalds wrote:
> > On 10 Sep 2002, Alan Cox wrote:
> > >
> > > It drops you politely into the kernel debugger, you fix up the values
> > > and step over it. If you want to debug with zen mind power and printk
> > > feel free. For the rest of us BUG() is fine on SMP
> >
> > Ok, a show of hands..
> >
> > Of the millions (whatever) of Linux machines, how many have a kernel
> > debugger attached? Count them.
>
> Eh, mine is getting one attached to it right now.

Me too.

> It's getting more
> popular, and it would be more popular yet if it weren't considered some
> dirty little secret, or somehow unmanly.

Reminds me of "Suns boot fast" (do a google search on it, and read the
first thing that comes up).

>
> Let's try a different show of hands: How many users would be happier if
> they knew that kernel developers are using modern techniques to improve
> the quality of the kernel?
>
> Of course, I use the term "modern" here loosely, since kdb and kgdb are
> really only 80's technology. Without them, we're stuck in the 60's.
Moving from the stone to the bronze age, one day at a time.


>
>

2002-09-15 06:02:56

by Pete Zaitcev

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

> From: Daniel Phillips <[email protected]>
> Date: Sun, 15 Sep 2002 07:10:00 +0200

>[...]
> Let's try a different show of hands: How many users would be happier if
> they knew that kernel developers are using modern techniques to improve
> the quality of the kernel?

I do not see how using a debugger improves a quality of the kernel.
Good thinking and coding does improve kernel quality. Debugger
certainly does not help if someone cannot code.

A debugger can do some good things. Some people argue that it
improves productivity, which I think may be true under some
circomstances. If your build system sucks and/or slow, and
if you work with a binary only software, debugger helps.
If you work with something like Linux, and compile on something
better than a 333MHz x86, it probably does not help your
productivity. This is all wonderful, but has nothing to do
with the code quality.

And to think that your users would be happier with a crap produced
by a debugger touting Windows graduate than with a quality code
debugged with observation simply defies any reason.

-- Pete

2002-09-15 13:50:01

by Rogier Wolff

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Tue, Sep 10, 2002 at 12:03:00PM -0700, Linus Torvalds wrote:
> In other words, if a user is faced with a dead machine with no other way
> to even know what BUG() triggered than to try to set up a cross debugger,
> just how useful is that BUG()? I claim it is pretty useless - simply
> because 99+% of all people won't even make a bug report in that case,
> they'll just push the reset button and consider Linux unreliable.


I just had an idea: There are the periodic discussions about
dumping the oops-report somewhere where you can log it on the next
boot, right?

Disk drivers may be unreliable etc etc.

Main memory is probably cleared on boot, right? (would have been too
obvious).

I'm pretty sure that my system has 16M of non-cleared memory: The
video RAM. The state of my screen before the crash always shows up
while X is starting.... (I was running a foolishly old 2.4.3-ac3 until
a couple of weeks ago)

Is my system special in that it doesn't clear video RAM? Can we
grab a pointer to video memory when we boot, check a magic
value, if OK, print the oops report from last crash, put in a new
magic value ("No oops report"), and when we crash write in the
crash report with the special magic value?



Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currenly in such an *
* excursion: The stable situation does not include humans. ***************

2002-09-15 14:45:45

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 08:07, Pete Zaitcev wrote:
> > From: Daniel Phillips <[email protected]>
> > Date: Sun, 15 Sep 2002 07:10:00 +0200
>
> >[...]
> > Let's try a different show of hands: How many users would be happier if
> > they knew that kernel developers are using modern techniques to improve
> > the quality of the kernel?
>
> I do not see how using a debugger improves a quality of the kernel.

It improves my quality of life, that would be enough by itself. And
since I am doing this purely for my own satisfaction at the moment, I
*will not* waste my time screwing around with 60's tools because
somebody who is paid whether or not they work productively thinks they
imbue the kernel with some kind of airy-fairy zen quality.

Look, we tried the zen state thing. It didn't work. Think about the
madness in the period between 2.3 and 2.4, with one oops after another
reported to the list, each taking days or weeks to track down. Sure,
it allowed Linus and Al to show off their superior powers of cerebration,
solving the things given only a few lines of oops as a result, but it
sure fucked everything else up. As I recall, it wasn't fun at all in
the end, not for Linus or Al either, and some folks got pretty close
to burned out.

The answer to the question "is this sillyness slowing down development
and reducing the quality of the kernel?" is "yes". I don't have to
speculate about that any more, I've seen it enough with my own eyes.
Now ask yourself who the most productive hackers are today, and ask
yourself if they are using the good ol zen state blunt edged tools.

> Good thinking and coding does improve kernel quality. Debugger
> certainly does not help if someone cannot code.

Sorry, but you are full of shit.

--
Daniel

2002-09-15 16:35:41

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 07:33, Daniel Berlin wrote:
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> > It's getting more
> > popular, and it would be more popular yet if it weren't considered some
> > dirty little secret, or somehow unmanly.
>
> Reminds me of "Suns boot fast" (do a google search on it, and read the
> first thing that comes up).

Yes. Using a really nice development environment is an "aha" experience.
You can listen to people talk about it as much as you want, but if you
have never actually used one you will never really understand what
they're talking about, and why they get worked up about it.

Linux is WAY WAY far from being a really nice development environment.
It's really nice in many other ways, but not in that way.

For example, there is no excuse for ever needing more than 1/4 second
to get from the function you're looking at on the screen to its
definition and source text, ready to read or edit.

There's no excuse for having to copy down an oops from a screen by
hand, either. It's nice to know you can fall back to this if you have
to, but having that be the default way of working is just broken.

There's no excuse for having to pepper low level code with printk's
to bracket a simple segfault.

OK, I'll stop there. Actually, the only thing I'm really irritated
about at the moment is the attitude of people who should know better,
promoting the fiction that this veggie-zen-tools-made-out-of-wood
thing is actually helping the kernel progress faster.

I suppose I'm going about this the wrong way, since directly pointing
out wrong-headed thinking usually just causes a defensive reaction
and puts people in a position where they're reluctant to back down
for fear of losing karma. Well, then. Listen to Andrew. He knows
whereof he speaks, and plus he's nice about it, unlike me.

--
Daniel

2002-09-15 18:01:10

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On Sun, 15 Sep 2002, Daniel Phillips wrote:
>
> Let's try a different show of hands: How many users would be happier if
> they knew that kernel developers are using modern techniques to improve
> the quality of the kernel?

You're all talk and no action.

The last time I looked, the people who really _do_ improve the quality of
the kernel don't tend to care too much about debuggers, or are at least
capable to add a patch on their own.

In fact, of the people who opened their mouth about the BUG() issue, how
many actually ended up personally _debugging_ the BUG() that we were
talking about?

I did. Matt probably did. But I didn't see you fixing it with your
debugger.

So next time you bring up the kernel debugger issue, show some code and
real improvement first. Until then, don't spout off.

Linus

2002-09-15 18:18:21

by Pete Zaitcev

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

> From: Daniel Phillips <[email protected]>
> Date: Sun, 15 Sep 2002 16:53:15 +0200

> On Sunday 15 September 2002 08:07, Pete Zaitcev wrote:
> > > From: Daniel Phillips <[email protected]>
> > > Date: Sun, 15 Sep 2002 07:10:00 +0200
> >
> > >[...]
> > > Let's try a different show of hands: How many users would be happier if
> > > they knew that kernel developers are using modern techniques to improve
> > > the quality of the kernel?
> >
> > I do not see how using a debugger improves a quality of the kernel.
>
> It improves my quality of life, that would be enough by itself. [...]
>[...]
> The answer to the question "is this sillyness slowing down development
> and reducing the quality of the kernel?" is "yes". I don't have to
> speculate about that any more, I've seen it enough with my own eyes.
> Now ask yourself who the most productive hackers are today, and ask
> yourself if they are using the good ol zen state blunt edged tools.

OK, so you actually do not care about users getting happier.
You should have added a smilie to the quote about the show
of hands then.

The agrument about your quality of life does hold some water,
at least I do not doubt that kdb makes you and Andrew happier.
This is a wonderful thing. I do strongly suspect though, that
any gains you get on the productivity front are NOT going to
be used to improve code quality.

> Look, we tried the zen state thing. It didn't work. Think about the
> madness in the period between 2.3 and 2.4, with one oops after another
> reported to the list, each taking days or weeks to track down. [...]

This has nothing to do with a debugger, this is a different topic.
You actually want a crash dump analyzis tool, and so do I.
So, let's discuss that. I happen to get e-mails with oops in USB
callbacks pretty often, and they are always useless. It would be
possible to track them if off-stack memory was saved, perhaps.
However, to expect users to use debugger to collect this off-stack
information is a delusion.

This is why Red Hat stopped shipping kdb and started to ship
netdump (or so I think, anyway). It is a much more effective
tool for the crash analysis, and it can be operated by a user.
I think it beats a debugger fair and square. N.B. The data
that netdump collects may be an image to be examined by a
debugger (such as gdb), together with dedicated analysis tools.
That's entirely different debugger, so no hypocrisicy here.

-- Pete

2002-09-15 18:32:00

by Roman Zippel

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

Hi,

On Sun, 15 Sep 2002, Linus Torvalds wrote:

> The last time I looked, the people who really _do_ improve the quality of
> the kernel don't tend to care too much about debuggers, or are at least
> capable to add a patch on their own.

Um, several kernel hackers here use UML also because of the gdb interface.

bye, Roman

2002-09-15 19:03:27

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On Sun, 15 Sep 2002, Linus Torvalds wrote:

> ... or are at least capable to add a patch on their own.

Actually, in BK-current you don't even need to do that, as UML has been
integrated into 2.5.x for a few days now.

I'll make a 2.5.35 later today, assuming no new threading bugs crop up.
And then you can use a debugger on a bog-standard kernel, and maybe this
issue won't come up ever again.

Yeah, right.

Linus

2002-09-15 19:01:12

by Daniel Jacobowitz

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 11:06:09AM -0700, Linus Torvalds wrote:
>
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> >
> > Let's try a different show of hands: How many users would be happier if
> > they knew that kernel developers are using modern techniques to improve
> > the quality of the kernel?
>
> You're all talk and no action.
>
> The last time I looked, the people who really _do_ improve the quality of
> the kernel don't tend to care too much about debuggers, or are at least
> capable to add a patch on their own.
>
> In fact, of the people who opened their mouth about the BUG() issue, how
> many actually ended up personally _debugging_ the BUG() that we were
> talking about?
>
> I did. Matt probably did. But I didn't see you fixing it with your
> debugger.
>
> So next time you bring up the kernel debugger issue, show some code and
> real improvement first. Until then, don't spout off.

[I haven't read the beginning of this thread. I don't know which
debugger you're talking about, although I suspect from context it's
KDB. I'm just going to pretend it's KGDB because that's the one that I
consider most useful.]

Let me offer a different perspective, with example. Perhaps you
remember the paste-o fix I sent you last week. A patch had changed:

list_entry(current->ptrace_children.next,struct task_struct,ptrace_list)
to:
list_entry(current->ptrace_children.next,struct task_struct,sibling)

I usually develop on 2.4. More, I usually develop on a tree that
already has i386 kgdb merged, or on non-i386 architecture which has a
kgdb stub as standard - MIPS or PowerPC. I find it a great deal more
convenient, for a whole lot of reasons. For instance:

Tracking this bug down took me about six hours. Someone more familiar
with that particular segment of code could, I assume, have done it more
quickly. One advantage of a debugger is that it's easier to look at
parts of the code you aren't intimately familiar with, and still find
the problem. With access to a debugger, my process would have gone
something like this:
- establish that exit_notify was crashing the first time that an
initial thread exited (half an hour. because my i386 test machine
reboots very slowly.)
- step through it (five minutes)
- see it step into zap_thread, the incorrect line right there on my
screen, the corrupted argument immediately obvious

As it was, there is no 2.5 / i386 port of KGDB as far as I know;
George's patches are only for 2.4. Porting KGDB to 2.5 would have
taken me more than the hours I wasted with printks and oops traces.

Plus, in my experience the work model that BitKeeper encourages puts a
significant penalty on including unrelated patches in the tree while
you're debugging. It can be gotten around but it's exceptionally
awkward. Adding in KGDB means time spent merging it into my tree and
time spend merging it cleanly out when I'm done with it.

I even have ulterior motives in suggesting that KGDB be included - with
a proper and complete KGDB in the kernel, the features that make KGDB
more than just the minimal stubs that other platforms have could move
to common code. MIPS, for instance, could have a thread-aware KGDB,
for instance.

I've always thought it would be useful. Sure, everyone debugs
differently; but a number of people seem to agree with me that KGDB is
convenient.

--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer

2002-09-15 19:03:53

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 20:06, Linus Torvalds wrote:
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> >
> > Let's try a different show of hands: How many users would be happier if
> > they knew that kernel developers are using modern techniques to improve
> > the quality of the kernel?
>
> You're all talk and no action.
>
> The last time I looked, the people who really _do_ improve the quality of
> the kernel don't tend to care too much about debuggers, or are at least
> capable to add a patch on their own.

Oh sure. So last week's page release race counts as no action, and htree
is not the fastest directory index in the known universe. I could go on.
What I do not do is rub people's faces in every little thing I do. Please
remember who found the last truncate race of the year, just before
2.4.0-prerelease.

> In fact, of the people who opened their mouth about the BUG() issue, how
> many actually ended up personally _debugging_ the BUG() that we were
> talking about?

I am personally working on a different problem.

> I did. Matt probably did. But I didn't see you fixing it with your
> debugger.

I am setting up the debugger to work on the DAC960.

> So next time you bring up the kernel debugger issue, show some code and
> real improvement first. Until then, don't spout off.

You're shooting way wide of the target.

--
Daniel

2002-09-15 19:07:07

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 21:08, Linus Torvalds wrote:
> On Sun, 15 Sep 2002, Linus Torvalds wrote:
>
> > ... or are at least capable to add a patch on their own.
>
> Actually, in BK-current you don't even need to do that, as UML has been
> integrated into 2.5.x for a few days now.
>
> I'll make a 2.5.35 later today, assuming no new threading bugs crop up.
> And then you can use a debugger on a bog-standard kernel, and maybe this
> issue won't come up ever again.
>
> Yeah, right.

I use UML all the time. It's great, but it doesn't work for SMP debugging.

By the way, thanks much for integrating UML.

--
Daniel

2002-09-15 19:27:54

by Daniel Jacobowitz

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 12:26:02PM -0700, Linus Torvalds wrote:
>
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> >
> > I use UML all the time. It's great, but it doesn't work for SMP debugging.
>
> That should not be something fundamental, though. It should be perfectly
> doable with threading. "SMOP".

I run into problems fairly often that I can't reproduce in UML - timing
sensitive, hardware sensitive, etc. Some of them KGDB perturbs too
much to be useful, but most of the time I can get it to work. UML also
doesn't use a lot of the code under arch/i386/ (or didn't at least)
which makes debugging that code under UML a bit futile.

> Yeah, and gdb (not to mention all the grapical nice stuff) sucks in a
> threaded environment. At least it used to.

Well, yeah. It's getting a little bit better - a lot better for some
cases - but no one's really sure where it needs to go to keep
improving. I'm making a little progress.

--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer

2002-09-15 19:21:00

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On Sun, 15 Sep 2002, Daniel Phillips wrote:
>
> I use UML all the time. It's great, but it doesn't work for SMP debugging.

That should not be something fundamental, though. It should be perfectly
doable with threading. "SMOP".

Yeah, and gdb (not to mention all the grapical nice stuff) sucks in a
threaded environment. At least it used to.

Linus

2002-09-15 19:32:08

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 21:26, Linus Torvalds wrote:
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> >
> > I use UML all the time. It's great, but it doesn't work for SMP debugging.
>
> That should not be something fundamental, though. It should be perfectly
> doable with threading. "SMOP".

Jeff and I occasionally kick this around and it always ends with "yeah,
not hard at all, when I have more time on my hands than I know what to
do with maybe I'll do that", or words to that effect. It's understandable
why Jeff hasn't gotten around to doing it, given the workload he's had
just maintaining UML in both the 2.4 and 2.5 kernel series. Plus Jeff
does not get paid of any of this.

Now that it's integrated, nice things like that can happen, and maybe
we won't have to impose on Jeff to do 100% of the work on UML in the
future. (Well, actually I did about 0.00001% of the work on UML, to
wit, a patch to change the bogus virtual contents of ptes to "physical"
addresses, bringing UML in line with the other arches in that regard.
<- part of my new policy of making sure everybody knows about every
little contribution I make.)

> Yeah, and gdb (not to mention all the grapical nice stuff) sucks in a
> threaded environment. At least it used to.

It still does. That's yet another source of irritation.

--
Daniel

2002-09-15 19:45:53

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 21:32, Daniel Jacobowitz wrote:
> On Sun, Sep 15, 2002 at 12:26:02PM -0700, Linus Torvalds wrote:
> >
> > On Sun, 15 Sep 2002, Daniel Phillips wrote:
> > >
> > > I use UML all the time. It's great, but it doesn't work for SMP debugging.
> >
> > That should not be something fundamental, though. It should be perfectly
> > doable with threading. "SMOP".
>
> I run into problems fairly often that I can't reproduce in UML - timing
> sensitive, hardware sensitive, etc. Some of them KGDB perturbs too
> much to be useful, but most of the time I can get it to work. UML also
> doesn't use a lot of the code under arch/i386/ (or didn't at least)
> which makes debugging that code under UML a bit futile.

True, however, it's really amazing what a wide range of kernel problems
UML does faithfully reproduce. It is, after all, just another port. You
could make the same argument about MIPS distorting timings vis a vis i386.

> > Yeah, and gdb (not to mention all the grapical nice stuff) sucks in a
> > threaded environment. At least it used to.
>
> Well, yeah. It's getting a little bit better - a lot better for some
> cases - but no one's really sure where it needs to go to keep
> improving. I'm making a little progress.

Oh, and there is another big suckage in UML in the area of modules, you
have to load the symbols by hand, which is just enough pain to make it
not worth compiling things as modules in UML. Jeff has reasons why its
hard to get gdb to load the module symbols automatically, but hopefully,
that is another problem we can dig into now that UML is officially part
of the tree.

--
Daniel

2002-09-15 23:19:22

by Larry McVoy

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 03:04:35PM -0400, Daniel Jacobowitz wrote:
> Tracking this bug down took me about six hours. Someone more familiar
> with that particular segment of code could, I assume, have done it more
> quickly. One advantage of a debugger is that it's easier to look at

I'm not speaking for Linus, but I wouldn't be surprised we share the
same view on this one. As someone who maintains a fairly large source
base I get nervous when people tell me they need a debugger to work on
the code. Why? Because if you really need that it is EXTREMELY likely
that you don't understand the code. If you don't understand the code
then YOU SHOULDN'T BE CHANGING IT. It is infuriating to have a section
of tricky code that used to work, you turn your back, only to find that
someone made a "simple change" which seems to work but actually makes
things worse and invariably seems to break the code in a far more
subtle way.

My position is that you either understand the code or you don't. Code
that you don't understand is read only. Having a debugger show you some
variables isn't going to make you understand the code at the level which
is required in order to be making changes.

Does this mean I'm against debuggers? Not at all. But in 15 years of
doing kernel work and 5 years of doing BK work the only thing I've ever
used one for was to get a few variables printed out. And I've written
a substantial chunk of a debugger years ago, it's not a question of lack
of debugger knowledge. I just rarely find them useful.

> Plus, in my experience the work model that BitKeeper encourages puts a
> significant penalty on including unrelated patches in the tree while
> you're debugging. It can be gotten around but it's exceptionally
> awkward. Adding in KGDB means time spent merging it into my tree and
> time spend merging it cleanly out when I'm done with it.

Create a throwaway clone, merge in kdb, tag the tree with "baseline".
Now hack away until you have a fix. If you never checked anything in
after the baseline then "bk -r diffs -u" creates the patch for your
bugfix. If you did, then diff against the baseline.

If BK is awkward by comparison to diff and patch, something is wrong, it
definitely has the ability to make things far more pleasant than you
seem to be experiencing.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-09-15 23:35:56

by Daniel Jacobowitz

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 04:24:12PM -0700, Larry McVoy wrote:
> On Sun, Sep 15, 2002 at 03:04:35PM -0400, Daniel Jacobowitz wrote:
> > Tracking this bug down took me about six hours. Someone more familiar
> > with that particular segment of code could, I assume, have done it more
> > quickly. One advantage of a debugger is that it's easier to look at
>
> I'm not speaking for Linus, but I wouldn't be surprised we share the
> same view on this one. As someone who maintains a fairly large source
> base I get nervous when people tell me they need a debugger to work on
> the code. Why? Because if you really need that it is EXTREMELY likely
> that you don't understand the code. If you don't understand the code
> then YOU SHOULDN'T BE CHANGING IT. It is infuriating to have a section
> of tricky code that used to work, you turn your back, only to find that
> someone made a "simple change" which seems to work but actually makes
> things worse and invariably seems to break the code in a far more
> subtle way.
>
> My position is that you either understand the code or you don't. Code
> that you don't understand is read only. Having a debugger show you some
> variables isn't going to make you understand the code at the level which
> is required in order to be making changes.

That's a circular argument. I use debuggers in order to understand
code _better_. And to understand how code that I do know interacts
with code that I don't know. I also rarely have the luxury of working in
source bases that I've got long intimate association with - about the
only code I can claim that for is GDB itself and it still surprises me.

Using a debugger I can pick up greatly improved understanding of what's
going on - both what is and what should be. And what has changed in it
recently, since I was last familiar with it, which was the problem
here.

> Does this mean I'm against debuggers? Not at all. But in 15 years of
> doing kernel work and 5 years of doing BK work the only thing I've ever
> used one for was to get a few variables printed out. And I've written
> a substantial chunk of a debugger years ago, it's not a question of lack
> of debugger knowledge. I just rarely find them useful.

Good for you; as I said, everyone operates differently. The number of
printing/thinking/rebooting cycles involved for me to work with a
debugger is much shorter than without.

> > Plus, in my experience the work model that BitKeeper encourages puts a
> > significant penalty on including unrelated patches in the tree while
> > you're debugging. It can be gotten around but it's exceptionally
> > awkward. Adding in KGDB means time spent merging it into my tree and
> > time spend merging it cleanly out when I'm done with it.
>
> Create a throwaway clone, merge in kdb, tag the tree with "baseline".
> Now hack away until you have a fix. If you never checked anything in
> after the baseline then "bk -r diffs -u" creates the patch for your
> bugfix. If you did, then diff against the baseline.
>
> If BK is awkward by comparison to diff and patch, something is wrong, it
> definitely has the ability to make things far more pleasant than you
> seem to be experiencing.

Perhaps I (yet again) need to spend a day learning more about the
depths of BitKeeper. The last time I did it all I found were BitKeeper
bugs, because the way I try to work with it appears to be contrary to
the way other people want to work with it.

I guess I just need to get more used to doing all of my development in
throwaway clones, and when it's absolutely perfect checking it into a
longer-lived tree via exporting a GNU patch and importing that.

The above model gets much more complicated when the development lasts
longer than a couple of hours, and you have to pull from another tree;
already, to upgrade my working trees I need to cascade three pulls -
with two sets of resolves if Ingo's been changing CLONE_ flags on me
again :)

--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer

2002-09-15 23:47:43

by Larry McVoy

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 07:41:08PM -0400, Daniel Jacobowitz wrote:
> I also rarely have the luxury of working in
> source bases that I've got long intimate association with

We clearly have different definitions of what is acceptable. If your
attitude is that understanding the code is a luxury all that means is
that I want you nowhere near any code I maintain. Non-trivial code
requires non-trivial understanding and that understanding is not a
"luxury" in my book.

If your company has such a poor business model that they can't afford to
pay you enough to take the time to do a good job then find a different
place to work. No amount of debugger "help" is going to make up for a
lack of understanding.

Yeah, I'm up on my high horse, but this sort of stuff just infuriates me.
If you aren't going to spend the time to do the code right, which means
spending the time to *completely* understand what it is that the code is
doing, why the problem is occurring, and why the fix is a real fix and
not a bandaid, then all you are doing is creating more work for somebody
else in the future. You are *guessing*. Maybe your company tolerates
that but I don't have to and I sure as hell don't and I'd question
anyone or any company that does. If you like working like that maybe
you should consider Microsoft, they seem to really value that approach.
Otherwise realize that you are straying and get back on the correct path.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-09-15 23:56:16

by Robert Love

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, 2002-09-15 at 19:52, Larry McVoy wrote:

> We clearly have different definitions of what is acceptable. If your
> attitude is that understanding the code is a luxury all that means is
> that I want you nowhere near any code I maintain. Non-trivial code
> requires non-trivial understanding and that understanding is not a
> "luxury" in my book.

Well, since the code Daniel was referring to having debugged (ptrace and
thread debacle earlier) was broken and he successfully fixed it -- he is
welcome to come near code as far as I am concerned.

In short, he and Ingo fixed some messed up code and that is all that
matters. I do not care if he uses a debugger or a magic ball, so long
as he fixes it.

Personally, I do not use a debugger and it is partly because of the
reasons you list. But if Daniel can fix a problem (which he most likely
ran into firsthand with his work on gcc and gdb) then he is welcome in
my eyes.

> If your company has such a poor business model that they can't afford to
> pay you enough to take the time to do a good job then find a different
> place to work. No amount of debugger "help" is going to make up for a
> lack of understanding.

Please, give it up.

Robert Love

2002-09-16 00:41:31

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 01:24, Larry McVoy wrote:
> Does this mean I'm against debuggers? Not at all. But in 15 years of
> doing kernel work and 5 years of doing BK work the only thing I've ever
> used one for was to get a few variables printed out. And I've written
> a substantial chunk of a debugger years ago, it's not a question of lack
> of debugger knowledge. I just rarely find them useful.

You are hereby invited to debug the DAC960 driver. Making the needed
changes only took a few hours to get it to the point where it initializes,
now it oopses in the interrupt handler, because I did the bio part wrong.
Of course, you might not make any mistakes, but you probably will, so get
ready with your printks. It's going to take you a while, I can assure
you.

I flat out refuse to do this without kgdb.

--
Daniel

2002-09-16 01:17:26

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Mon, 2002-09-16 at 00:24, Larry McVoy wrote:
> My position is that you either understand the code or you don't. Code
> that you don't understand is read only. Having a debugger show you some
> variables isn't going to make you understand the code at the level which
> is required in order to be making changes.

It isnt about understanding the code. You issue a perfectly valid
command to your scsi driver. Your ethernet crashes. All your code is
perfect.

Thats a real scenario. In fact thats a horribly common scenario, and the
kind of thing I have to deal with every day of the week because I debug
driver code.

Understanding your own code by using a debugger (or printk) to see what
is does is not good practice. Understanding misdocumented or crap code
often needs a debugger to see what is going on, and to be able to
understand how to fix it. The moment you hit the hardware layer the fun
really begins, and you need the debugger, not to understand your code
flow but to snoop around the machine to see what the device did or
didn't do.

2002-09-16 01:22:07

by Alan Cox

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Mon, 2002-09-16 at 00:52, Larry McVoy wrote:
> If your company has such a poor business model that they can't afford to
> pay you enough to take the time to do a good job then find a different
> place to work. No amount of debugger "help" is going to make up for a
> lack of understanding.

Maybe he works at a company with a good enough business model to realise
that people who can't or won't use tools for pet high horse reasons are
ineffeciencies that are best "downsized" either in ego or personnel
count.

"If you aren't going to spend the time to do the code right"

Well DAC960 isn't his code. DAC960 hardware is also opaque. Its
unfortunate you can't seperate the right and wrong way to use a debugger
and what you learn from it. It's a tool nothing more. Its what you do
with the data you get from it that matters. A debugger is a great way to
make pigs apparently fly, but that isnt its only use


Alan



2002-09-16 02:08:27

by Larry McVoy

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Mon, Sep 16, 2002 at 02:29:10AM +0100, Alan Cox wrote:
> On Mon, 2002-09-16 at 00:52, Larry McVoy wrote:
> > If your company has such a poor business model that they can't afford to
> > pay you enough to take the time to do a good job then find a different
> > place to work. No amount of debugger "help" is going to make up for a
> > lack of understanding.
>
> Maybe he works at a company with a good enough business model to realise
> that people who can't or won't use tools for pet high horse reasons are
> ineffeciencies that are best "downsized" either in ego or personnel
> count.

You are completely missing the point. I explictly stated that debuggers
are just fine with me, I don't really care one way or the other. I also
stated that I use debuggers and I'm OK with other people using them as
well. Where's the "won't use tools for pet high horse reasons"? Nowhere.

My comments, which I stand behind now and will stand behind 10 years
from now, are based on the fact that people who don't understand the
code shouldn't be modifying the code. If a debugger helps you understand
the code, go for it. However, my experience is that what he was saying
resonates with "I'm going in to fix this problem so I can get back to
work on my real project". And that is almost always wrong. If the
problem was that bloody simple don't you think the original author of
the code would have fixed it already? It's almost never as simple as a
naive point of view thinks it is and that's exactly why you don't want
people hacking about in that code. Either understand it and really fix
it, own it, maintain it, live with it, or leave it alone.

You may have a different opinion, Alan, and that's fine. All that
means is that you won't ever work here. One of the nice things about
being the guy who runs the company is that you get to insist on a certain
level of competence and professionalism or you're fired. It's one of
the reasons I don't work for someone else, I like being able to say
"do it right or don't do it, I pay the bills, that's what I want".
I learned that at Sun, over my every objection because I was an idiot,
but I learned it. After you learn the benefits of doing things right
you have nothing but pity for people who do it wrong.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-09-16 02:31:14

by Rob Landley

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sunday 15 September 2002 02:07 am, Pete Zaitcev wrote:
> > From: Daniel Phillips <[email protected]>
> > Date: Sun, 15 Sep 2002 07:10:00 +0200
> >
> >[...]
> > Let's try a different show of hands: How many users would be happier if
> > they knew that kernel developers are using modern techniques to improve
> > the quality of the kernel?
>
> I do not see how using a debugger improves a quality of the kernel.

I believe we agree that you do not see it. :)

> Good thinking and coding does improve kernel quality. Debugger
> certainly does not help if someone cannot code.

And reading glasses won't help someone who can't read. Is your above set of
statements somehow meant to imply that a debugger cannot help someone who CAN
code? (Logic. Logic is good here.)

A similar argument would be "Nobody should own an oven. If you can't cook
you'll just make a mess."

> A debugger can do some good things. Some people argue that it
> improves productivity, which I think may be true under some
> circomstances.

It's a tool. Does anybody really disagree about its nature?

The main purpose of tools is to save time. No tool is a substitute for
skill. it's possible to do everything a debugger can do via printks out to
the printer and having a stack of source code printouts, and doing all your
debugging in pencil. It's also possible to pound in a nail with a rock, if
you think hammers are for sissys.

If you stop and think about it, it's sort of like amish kernel development,
isn't it? (Hey, we've already got the community barn raising aspect. Now
let's outlaw certain construction techniques! :)

> If you work with something like Linux, and compile on something
> better than a 333MHz x86, it probably does not help your
> productivity.

I had the source code to the contractor-produce pile of cruft I worked with
under OS/2. I could even read some of it, given enough time. Trust me, a
source debugger REALLY helped find the differences between what it was
supposed to be doing and what it was actually doing. (Or more accurately,
the few places where what it was supposed to be doing and what it was doing
matched up...)

> This is all wonderful, but has nothing to do
> with the code quality.

I think we agree. So why recommend against debuggers if they save time and
effort while producing patches of equal quality?

> And to think that your users would be happier with a crap produced
> by a debugger touting Windows graduate

Sure, blame the tool. :)

> than with a quality code
> debugged with observation simply defies any reason.

So the state of the running kernel is more observablee without a kernel
debugger than with one?

Debugging means, basically, "this sucker is doing something I didn't expect
it to, and I have to find out what". A debugger is a tool with which to
examine running code. It doesn't help you write the stuff, just examine it.
The proper tool for the proper job...

The unix philosophy has always been to give people enough rope to shoot
themselves in the foot with. I'm just wondering why the exception. (Yes I
read the "bastard" speech. It seemed to boil down to a crude sturgeon's law
filter because Linus was once again being overwhelmed by crap patches (an
annual event around here). Making less than perfect coders code blind won't
increase the quality of anything, but might decrease the quanity. I suppose
that's one way of addressing the problem, but doesn't seem to me to be an
optimal solution...)

> -- Pete

Rob

(P.S. Don't mind me: I have a cold. If the above comes out as anything even
resembling english, I'm ahead of the game...)

2002-09-16 02:55:09

by Larry McVoy

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 05:35:59PM -0400, Rob Landley wrote:
> And reading glasses won't help someone who can't read. Is your above set of
> statements somehow meant to imply that a debugger cannot help someone who CAN
> code? (Logic. Logic is good here.)
>
> A similar argument would be "Nobody should own an oven. If you can't cook
> you'll just make a mess."
>
> > A debugger can do some good things. Some people argue that it
> > improves productivity, which I think may be true under some
> > circomstances.
>
> It's a tool. Does anybody really disagree about its nature?

I can't speak for others, but my guess is that the people who don't like
debuggers don't like them for pretty much the same reasons they don't like
C++. The tool makes bad behaviour too seductive.

It's true that one can write supportable perl but noone but a naive person
would base a multiple platform, multi-year lifespan product on perl.

It's true that one can write good systems code in C++ but experience has
shown that noone but a naive person would argue for C++ for a kernel.

Debuggers are sort of in this camp. Yup, useful tool. The problem is
that the real answer is that you should read and understand the code.
It's a sign of a naive programmer when you hear "this code is all shit"
and it's useful code. That means the programmer would rather rewrite
working code than understand it enough to fix it. Extremely common.
And extremely wrong in almost all cases. It's *hard* to understand code.
Get over it. Read the code, think, read again, think some more, keep
it up. Always always always assume the guy who came before you *did*
know what they were doing. Otherwise all you do is replace mostly working
code with brand new code that works for the *one* case in front of the
new programmer and none of the 100's of cases that the old code handled.

I don't think anyone is against debuggers. I'm not. I'm against people
not thinking. I'm for people who think, who are careful, who have some
respect for code that works.

It's so much more fun to say "this code is shit, I can do better", but
whenever I've said that I've been wrong about 90% of the time. And I'm
a pretty good programmer, I know that I shouldn't think like that. All
I'm saying is that thinking is greater than debuggers. Much greater.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-09-16 03:04:18

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 05:00, Larry McVoy wrote:
> It's so much more fun to say "this code is shit, I can do better", but
> whenever I've said that I've been wrong about 90% of the time. And I'm
> a pretty good programmer, I know that I shouldn't think like that. All
> I'm saying is that thinking is greater than debuggers. Much greater.

Second post in my "Understanding the Principles of Argumentation" series.

You have just provided us with a fine example of a common logical falacy
called "bifurcation":

"Also referred to as the "black and white" fallacy. Bifurcation is
the presentation of a situation or condition with only two
alternatives, whereas in fact other alternatives exist or can exist."
(http://www.theology.edu/logic/logic23.htm)

--
Daniel

2002-09-16 03:48:16

by Jeff Dike

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

[email protected] said:
> It's great, but it doesn't work for SMP debugging.

Not yet, but it's on my short list now.

When it happens, you can send Peter Braam a nice note for enticing me to
get it done :-)

Jeff

2002-09-16 03:52:53

by Jeff Dike

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

[email protected] said:
> I run into problems fairly often that I can't reproduce in UML -
> timing sensitive,

Timing works both ways. Maybe you run into a bug on native i386 that UML
won't reproduce. Maybe UML will reproduce bugs you won't see on hardware.
That doesn't help you track down a particular bug you're seeing on hardware,
but as far as overall bug smashing goes, it looks like a wash to me.

> hardware sensitive, etc.

That's potentially fixable. UML has SCSI support now, with a USB driver
in the works. Other hardware access is possible, too.

> UML
> also doesn't use a lot of the code under arch/i386/ (or didn't at
> least) which makes debugging that code under UML a bit futile.

Then we need to push code out from arch into the generic kernel. That's
happening slowly, but there's a bunch more to go.

Jeff

2002-09-16 03:56:39

by Jeff Dike

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

[email protected] said:
> Oh, and there is another big suckage in UML in the area of modules,
> you have to load the symbols by hand, which is just enough pain to
> make it not worth compiling things as modules in UML.

There is a nice expect script available with automates that for you. See
http://user-mode-linux.sf.net/debugging.html, towards the bottom.

That's from Chandan Kudige, who followed that up by making UML sort of work
on Windows :-)

Jeff

2002-09-16 04:03:36

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 06:59, Jeff Dike wrote:
> [email protected] said:
> > Oh, and there is another big suckage in UML in the area of modules,
> > you have to load the symbols by hand, which is just enough pain to
> > make it not worth compiling things as modules in UML.
>
> There is a nice expect script available with automates that for you. See
> http://user-mode-linux.sf.net/debugging.html, towards the bottom.
>
> That's from Chandan Kudige, who followed that up by making UML sort of work
> on Windows :-)

In addition, there is a gdb hacker the thread who has been thinking a
little about nonsucky ways of doing it. You might want to kick around
some ideas with Daniel, some cute way of passing events.

--
Daniel

2002-09-16 08:32:46

by Ian molton

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, 15 Sep 2002 17:35:59 -0400
Rob Landley <[email protected]> wrote:

> I think we agree. So why recommend against debuggers if they save
> time and effort while producing patches of equal quality?

I think the point here is that debuggers are not for everyone. They are
a tool, in much the same way as an adjustable wrench is. I prefer a set
of spanners.

Is there anything wrong with NOT wanting to use a debugger? really?

2002-09-16 09:02:04

by Jens Axboe

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15 2002, Daniel Phillips wrote:
> > I did. Matt probably did. But I didn't see you fixing it with your
> > debugger.
>
> I am setting up the debugger to work on the DAC960.

See, even though I'm not fundamentally against using kernel debuggers, I
think this is very wrong. Where are you now? You are just learning about
the bio interface and the changes needed to make it run. And this is
definitely the most wrong point to start using a debugger, and can only
result in a dac960 that works by trial and error.

--
Jens Axboe

2002-09-16 14:00:28

by Rogier Wolff

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 07:13:18PM -0700, Larry McVoy wrote:
> the code would have fixed it already? It's almost never as simple as a
> naive point of view thinks it is and that's exactly why you don't want
> people hacking about in that code. Either understand it and really fix
> it, own it, maintain it, live with it, or leave it alone.

Sometimes people who maintain the code move on to other things.

So, maintenance is sometimes required by people who do not have
"maintenance of this code" in their job description. When I'm paying
their salary, they get to fix other peoples code, even if it's none of
their business.

Now preferably such a fix would be passed through a maintainer who can
say (with his intimate knowledge of the code): "Oops, you're right
your patch is indeed an improvement".

If that's not possible, we'll have to make do with some testing.

Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currenly in such an *
* excursion: The stable situation does not include humans. ***************

2002-09-16 14:09:59

by David Woodhouse

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


[email protected] said:
> See, even though I'm not fundamentally against using kernel
> debuggers, I think this is very wrong. Where are you now? You are just
> learning about the bio interface and the changes needed to make it
> run. And this is definitely the most wrong point to start using a
> debugger, and can only result in a dac960 that works by trial and
> error.

Nevertheless, the existence of a case where it's not sensible to use a
debugger does not prove the non-existence of cases where it _is_ sensible
to use a debugger.

A case that happened to me recently -- tail-call optimisations screwed up
the return value of a function somewhere deep in the routing code. Adding a
printk made the problem go away. Staring at the C code was also of no use --
the C code was just fine.

Now, are you seriously suggesting that instead of using GDB to work out WTF
was going on I should have spent three weeks starting at the output of
'objdump -d vmlinux' ?

While my balls are big enough and hairy enough that I don't need to use GDB
to debug my own code, I feel no shame in admitting that I'm so much of a
pussy I can't deal with the above case without GDB.

--
dwmw2


2002-09-16 14:32:58

by Rob Landley

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 04:50 am, Ian Molton wrote:
> On Sun, 15 Sep 2002 17:35:59 -0400
>
> Rob Landley <[email protected]> wrote:
> > I think we agree. So why recommend against debuggers if they save
> > time and effort while producing patches of equal quality?
>
> I think the point here is that debuggers are not for everyone. They are
> a tool, in much the same way as an adjustable wrench is. I prefer a set
> of spanners.
>
> Is there anything wrong with NOT wanting to use a debugger? really?

I don't want to use reiserfs. I'm not lobbying to keep it out of the kernel
for the significant number of people who do want it.

It's Linus's final call, and now we can fake 95% of the same functionality by
attaching userspace debuggers to UML, so...

Rob

2002-09-16 14:49:32

by Jens Axboe

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Mon, Sep 16 2002, David Woodhouse wrote:
>
> [email protected] said:
> > See, even though I'm not fundamentally against using kernel
> > debuggers, I think this is very wrong. Where are you now? You are just
> > learning about the bio interface and the changes needed to make it
> > run. And this is definitely the most wrong point to start using a
> > debugger, and can only result in a dac960 that works by trial and
> > error.
>
> Nevertheless, the existence of a case where it's not sensible to use a
> debugger does not prove the non-existence of cases where it _is_ sensible
> to use a debugger.
>
> A case that happened to me recently -- tail-call optimisations screwed up
> the return value of a function somewhere deep in the routing code. Adding a
> printk made the problem go away. Staring at the C code was also of no use --
> the C code was just fine.
>
> Now, are you seriously suggesting that instead of using GDB to work out WTF
> was going on I should have spent three weeks starting at the output of
> 'objdump -d vmlinux' ?

I'm not suggesting anything for your case, and I'm not arguing against
kernel debuggers. Please re-read what I wrote: using a debugger for what
Daniel is attempting right now is stupid. Are you seriously suggesting
that you would trust your data to a driver that had been ported to 2.5,
not by studying the interface changes but by 'code blow up, gdb
inspection' method? I hope not.

I've used a kernel debugger a few times, for the things I tend to do
it's not very helpful. Or let me rephrase that into saying that it's not
more helpful than simply having the kernel compiled with -g and using
gdb on the resulting vmlinux just for backtraces and code inspection. So
I'm fine, and so I don't care very much about the typical pro/con
integrated debugger debates.

--
Jens Axboe

2002-09-16 15:12:18

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 11:06, Jens Axboe wrote:
> On Sun, Sep 15 2002, Daniel Phillips wrote:
> > > I did. Matt probably did. But I didn't see you fixing it with your
> > > debugger.
> >
> > I am setting up the debugger to work on the DAC960.
>
> See, even though I'm not fundamentally against using kernel debuggers, I
> think this is very wrong. Where are you now? You are just learning about
> the bio interface and the changes needed to make it run. And this is
> definitely the most wrong point to start using a debugger, and can only
> result in a dac960 that works by trial and error.

I am not using the debugger to learn about bio, I use LXR for that. I
am using the debugger because of the nature of the bugs I expect and
already seen. Jens, I have 27 years of professional programming
experience, you do not have to tell me when to use a debugger and when
not to.

--
Daniel

2002-09-16 15:25:14

by Oliver Xymoron

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, Sep 15, 2002 at 12:08:23PM -0700, Linus Torvalds wrote:
>
> On Sun, 15 Sep 2002, Linus Torvalds wrote:
>
> > ... or are at least capable to add a patch on their own.
>
> Actually, in BK-current you don't even need to do that, as UML has been
> integrated into 2.5.x for a few days now.
>
> I'll make a 2.5.35 later today, assuming no new threading bugs crop up.
> And then you can use a debugger on a bog-standard kernel, and maybe this
> issue won't come up ever again.

Given that most of the codebase is in drivers (never mind non-UML
supported arches), that's a little unlikely...

--
"Love the dolphins," she advised him. "Write by W.A.S.T.E.."

2002-09-16 15:58:19

by Soewono Effendi

[permalink] [raw]
Subject: kernel debuggers was [Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34]

Hi all,

just my .02 $

what I mostly admire in linux and it's development till now,
is it's flexibility to adapt the dynamic complex changing requirements
from server to desktop even embedded system and real-time system.
>From high throughput to low latency, you just name it!

And still though no one feels disadvantaged,
since features not needed are "removable".

I'm just wondering "if it's worth the effort" to provide
"removable customizable kernel debugging tools (entry/break points)",
which one can replaces with tools of his/her choice.
I'm talking about some unified cleaver reasonable MACROS,
which one can easy insert/use them where needed.

It's still wrong to force development tools to be persist in running system
where they are not needed.

There are lots of "nice things" that can be implemented in kernel,
but are they essential?
Tools should not become burdens!

Best regards,
S. Effendi

2002-09-16 16:19:43

by Marco Colombo

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, 15 Sep 2002, Larry McVoy wrote:

[...]
> people hacking about in that code. Either understand it and really fix
> it, own it, maintain it, live with it, or leave it alone.
>
> You may have a different opinion, Alan, and that's fine. All that
> means is that you won't ever work here. One of the nice things about
> being the guy who runs the company is that you get to insist on a certain
> level of competence and professionalism or you're fired. It's one of
> the reasons I don't work for someone else, I like being able to say
> "do it right or don't do it, I pay the bills, that's what I want".
> I learned that at Sun, over my every objection because I was an idiot,
> but I learned it. After you learn the benefits of doing things right
> you have nothing but pity for people who do it wrong.

Larry, you tend to use words like 'company' and 'business' too much in
your messages... B-)

Suppose that I (definitely not a guru - I deal with the Linux kernel just
for fun, and I don't *need* any *professional* knowledge on it) find some
obscure bug (maybe I'm using some uncommon HW). I understand *nothing*
about the piece of kernel that actually BUG()s - probably it's the
first time I see it. Nevertheless, I *like* kernel hacking. So I start
chasing the bug (and a debugger helps me a lot). It takes hours/days but
who cares? I'm having fun. Later I come up with a patch, which seems to
fix it. It may be complete crap. But I send a bug report *and* my patch
to Alan (or J. Kernel Mantainer). In a hour I get a reply back, saying:
"your patch is wrong, but I see the bug and I've fixed it in my tree"
(that's because *he* knows the code).

Don't you think that even random and clueless hacking can be (at times)
a valuable source of information for real hackers? That's very open
source specific, of course you don't run a *company* like that, but who
cares? Many of the lessons you learned in your business career
apply here, but some simply don't. The Linux kernel development isn't
run by someone that pays the bills...

.TM.

2002-09-16 18:30:00

by Thunder from the hill

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

Hi,

On Sun, 15 Sep 2002, Larry McVoy wrote:
> It's true that one can write supportable perl but noone but a naive person
> would base a multiple platform, multi-year lifespan product on perl.

Perl is pretty useful for code generation and/or parsing, e.g. in order to
find mistakes you did or to build the thing you've created. It can also be
used to convert code. Thus I wouldn't say it's useless in an
multiple-platform multi-year environment.

> All I'm saying is that thinking is greater than debuggers. Much
> greater.

That's true, but no reason to eat people for breakfast coz they do use
debuggers to get beyond where code does what wrong.

Thunder
--
!assert(typeof((fool)->next) == typeof(fool));

2002-09-16 18:40:23

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 20:35, Thunder from the hill wrote:
> !assert(typeof((fool)->next) == typeof(fool));

You meant:

assert(typeof((fool)->next) != typeof(fool));

--
Daniel

2002-09-16 19:31:01

by Thunder from the hill

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

Hi,

On Mon, 16 Sep 2002, Daniel Phillips wrote:
> On Monday 16 September 2002 20:35, Thunder from the hill wrote:
> > !assert(typeof((fool)->next) == typeof(fool));
>
> You meant:
>
> assert(typeof((fool)->next) != typeof(fool));

No, I mean "Never assert that the one next to a fool must be a fool,
either. You might be wrong."

Thunder
--
!assert(typeof((fool)->next) == typeof(fool));

2002-09-16 19:35:10

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Monday 16 September 2002 21:36, Thunder from the hill wrote:
> Hi,
>
> On Mon, 16 Sep 2002, Daniel Phillips wrote:
> > On Monday 16 September 2002 20:35, Thunder from the hill wrote:
> > > !assert(typeof((fool)->next) == typeof(fool));
> >
> > You meant:
> >
> > assert(typeof((fool)->next) != typeof(fool));
>
> No, I mean "Never assert that the one next to a fool must be a fool,
> either. You might be wrong."

A proper assert does not return a value, by definition. ?It relies purely
on side effects, that is, it causes a screeching halt if the logical
expression was false. ?It's always wrong to use an asssert in an expression,
and since BUG_ON is just assert(!expression) this applies to BUG_ON as
well. ?The compiler should prevent you from making this mistake.

What I *thought* you meant was: "if the next fool that comes along is
exactly the same as the last fool, run away screaming". ?There's some wisdom
in that.

--
Daniel

2002-09-18 00:31:32

by Rusty Russell

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Sun, 15 Sep 2002 11:06:09 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:

>
> On Sun, 15 Sep 2002, Daniel Phillips wrote:
> >
> > Let's try a different show of hands: How many users would be happier if
> > they knew that kernel developers are using modern techniques to improve
> > the quality of the kernel?
>
> You're all talk and no action.

That's a little out of line. While Daniel seems to enjoy the sound of his
own typing even more than most of us, he *does* do useful work.

Rusty.
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy

2002-09-18 00:41:33

by Linus Torvalds

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34


On Wed, 18 Sep 2002, Rusty Russell wrote:
>
> That's a little out of line. While Daniel seems to enjoy the sound of his
> own typing even more than most of us, he *does* do useful work.

Yeah, I'm sorry Daniel. I just don't appreciate people who demand me to
support built-in kernel debuggers, despite me having made my position
quite clear over the years...

Linus

2002-09-18 00:46:52

by Daniel Phillips

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Wednesday 18 September 2002 02:46, Linus Torvalds wrote:
> On Wed, 18 Sep 2002, Rusty Russell wrote:
> >
> > That's a little out of line. While Daniel seems to enjoy the sound of his
> > own typing even more than most of us, he *does* do useful work.
>
> Yeah, I'm sorry Daniel. I just don't appreciate people who demand me to
> support built-in kernel debuggers, despite me having made my position
> quite clear over the years...

Thanks. Well, it comes under the category of "managing Linus", and
everybody does, or tries to, up to a point. Not that it often works.

--
Daniel

2002-09-18 01:12:27

by Rik van Riel

[permalink] [raw]
Subject: Re: [linux-usb-devel] Re: [BK PATCH] USB changes for 2.5.34

On Wed, 18 Sep 2002, Daniel Phillips wrote:

> Thanks. Well, it comes under the category of "managing Linus", and
> everybody does, or tries to, up to a point. Not that it often works.

Well, how much work would you manage to do if you had 350 managers ? ;)

Rik
--
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/ http://distro.conectiva.com/

Spamtraps of the month: [email protected] [email protected]