Is there a source level remote kernel debugger that
communicates over an ethernet interface? The debugger
kgdb from kgdb.sourceforge.net works only with serial port.
On Fri, Nov 15, 2002 at 12:29:18PM -0800, Kallol Biswas wrote:
> Is there a source level remote kernel debugger that
> communicates over an ethernet interface? The debugger
> kgdb from kgdb.sourceforge.net works only with serial port.
With some hacking to produce a polling network driver, it should very
well be possible to provide a communication back-end for a gdb stub
that uses an ethernet NIC.
gdb itself is already perfectly capable of communicating with
such targets, using the specifier
target remote otherhost:1234
where 1234 is the port number the gdb stub on the target reserves for
itself to communicate with the host. It's probably best to use a
dedicated network interface, though it's not absolutely required. (With
a dedicated interface otherhost will then be the interface's hostname.)
A serious issue is that the communication protocol is somewhat more
complex, so there is a possibility the debug runtime will not be robust.
Bill
> Is there a source level remote kernel debugger that
> communicates over an ethernet interface? The debugger
> kgdb from kgdb.sourceforge.net works only with serial port.
A cheap terminal server might work ...
M.
In article <334960000.1037397999@flay>,
Martin J. Bligh <[email protected]> wrote:
>> Is there a source level remote kernel debugger that
>> communicates over an ethernet interface? The debugger
>> kgdb from kgdb.sourceforge.net works only with serial port.
>
>A cheap terminal server might work ...
Well, apart from the fact that a lot of machines don't even _have_
serial ports..
I dunno. I might even be willing to apply kgdb patches to my tree if it
just could use the regular network card I already have connected on all
my machines. None of my laptops have a serial line, for example, but
they all have networking.
Soon even _desktops_ probably won't have serial lines any more, only USB.
Linus
Hi,
Anything that can help with source level debugging of custom drivers
would be great for bringing up new hardware.
One of my frustrations with debugging a driver on a StrongARM based platform
has been that I don't know of a way to step through the code other than a
post-mortem review of the printk's.
An ethernet interface would be great, like the existing gdb
interface for cross-platform development.
Edwin Bland,
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Friday, November 15, 2002 1:26 PM
To: [email protected]
Subject: Re: lan based kgdb
In article <334960000.1037397999@flay>,
Martin J. Bligh <[email protected]> wrote:
>> Is there a source level remote kernel debugger that
>> communicates over an ethernet interface? The debugger
>> kgdb from kgdb.sourceforge.net works only with serial port.
>
>A cheap terminal server might work ...
Well, apart from the fact that a lot of machines don't even _have_
serial ports..
I dunno. I might even be willing to apply kgdb patches to my tree if it
just could use the regular network card I already have connected on all
my machines. None of my laptops have a serial line, for example, but
they all have networking.
Soon even _desktops_ probably won't have serial lines any more, only USB.
Linus
Linus Torvalds wrote:
>
> In article <334960000.1037397999@flay>,
> Martin J. Bligh <[email protected]> wrote:
> >> Is there a source level remote kernel debugger that
> >> communicates over an ethernet interface? The debugger
> >> kgdb from kgdb.sourceforge.net works only with serial port.
> >
> >A cheap terminal server might work ...
>
> Well, apart from the fact that a lot of machines don't even _have_
> serial ports..
>
> I dunno. I might even be willing to apply kgdb patches to my tree if it
> just could use the regular network card I already have connected on all
> my machines. None of my laptops have a serial line, for example, but
> they all have networking.
>
> Soon even _desktops_ probably won't have serial lines any more, only USB.
>
The only real work which has ever been done on this was by San
Mehat earlier this year. When he had the advantage of sharing
an office with me and being repeatedly harangued to do it ;)
He did have it working - it was basically the same idea as Ingo's
netconsole code, using a little polling stub in each driver. He
extended the concept to support Rx as well as Tx. You provide
a whole bunch of parameters to the kernel and to the debugger, right
down to the MAC address.
But San became quite unwell some months ago and vanished. As far as
I know, the code is lost.
He may have sent a copy to Amit?
But as far as integrating the stub goes: IMO it would need quite
some cleanup work first. Great tool, but the implementation is
quite straggly.
An ethernet version could never be as robust as something which
spins on a uart port though. Anyone who seriously wants to use the
facility would just need to get themselves a 16550.
Many of our embedded motherboards do not have
a serial port. May be I should consider adding ethernet
interface support to kgdb.
Linus Torvalds wrote:
> In article <334960000.1037397999@flay>,
> Martin J. Bligh <[email protected]> wrote:
> >> Is there a source level remote kernel debugger that
> >> communicates over an ethernet interface? The debugger
> >> kgdb from kgdb.sourceforge.net works only with serial port.
> >
> >A cheap terminal server might work ...
>
> Well, apart from the fact that a lot of machines don't even _have_
> serial ports..
>
> I dunno. I might even be willing to apply kgdb patches to my tree if it
> just could use the regular network card I already have connected on all
> my machines. None of my laptops have a serial line, for example, but
> they all have networking.
>
> Soon even _desktops_ probably won't have serial lines any more, only
> USB.
>
> Linus
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On Fri, Nov 15, 2002 at 09:26:00PM +0000, Linus Torvalds wrote:
> I dunno. I might even be willing to apply kgdb patches to my tree if it
> just could use the regular network card I already have connected on all
> my machines. None of my laptops have a serial line, for example, but
> they all have networking.
>
> Soon even _desktops_ probably won't have serial lines any more, only USB.
Using USB instead of the serial line or the network card would be
the best IMHO, because:
* many machines have network cards, but all machines have USB
(and it's gonna stay this way for some time)
* the USB stack seems simpler than the net stack +
(eventualy) pcmcia + network card driver.
Maybe the 'simpler' USB protocols (usbkbd and usbmouse) could be
used for this, I don't know...
Stelian, which has a Vaio Picturebook without a serial port.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
Stelian Pop wrote:
>
> On Fri, Nov 15, 2002 at 09:26:00PM +0000, Linus Torvalds wrote:
>
> > I dunno. I might even be willing to apply kgdb patches to my tree if it
> > just could use the regular network card I already have connected on all
> > my machines. None of my laptops have a serial line, for example, but
> > they all have networking.
> >
> > Soon even _desktops_ probably won't have serial lines any more, only USB.
>
> Using USB instead of the serial line or the network card would be
> the best IMHO, because:
>
Here is the kgdb stub's "send a byte" function:
static void
write_char(int chr)
{
while (!(inb(gdb_port + UART_LSR) & UART_LSR_THRE)) ;
outb(chr, gdb_port + UART_TX);
}
Need I say more?
On Fri, Nov 15, 2002 at 02:51:11PM -0800, Andrew Morton wrote:
> > Using USB instead of the serial line or the network card would be
> > the best IMHO, because:
>
> Here is the kgdb stub's "send a byte" function:
>
> static void
> write_char(int chr)
> {
> while (!(inb(gdb_port + UART_LSR) & UART_LSR_THRE)) ;
>
> outb(chr, gdb_port + UART_TX);
> }
>
> Need I say more?
I already know that, but this is not the point. The point is that
more and more boxes have no serial (or paralel) ports.
But even on those boxes, sometimes I'd just love to be able to use
kgdb. And I can't.
Ok, it will have to be at a higher level than the inb/outb serial
transport implementation (with possible bad effects on what can
and what cannot be debugged), but still, I feel there is a need
for that.
USB (with USB-to-serial adapter), network, ieee1394 would be
acceptable replacements for me.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
On Fri, 2002-11-15 at 14:24, Stelian Pop wrote:
> Using USB instead of the serial line or the network card would be
> the best IMHO, because:
>
> * many machines have network cards, but all machines have USB
> (and it's gonna stay this way for some time)
>
> * the USB stack seems simpler than the net stack +
> (eventualy) pcmcia + network card driver.
>
> Maybe the 'simpler' USB protocols (usbkbd and usbmouse) could be
> used for this, I don't know...
USB hardware and protocols are master-slave, meaning that you can not
connect another computer to this one directly. What USB *device* would
you want to see connected?
Of course, a USB-Serial adapter would work, and you can connect any
serial terminal, but then we are back to using serial ports; it's just
you will need a different driver for that.
Dmitri
On Fri, Nov 15, 2002 at 02:47:35PM -0800, Dmitri wrote:
> > Using USB instead of the serial line or the network card would be
> > the best IMHO, because:
[...]
>
> USB hardware and protocols are master-slave, meaning that you can not
> connect another computer to this one directly.
Of course, you are correct :(
> What USB *device* would
> you want to see connected?
Well, I could say 'iPAQ' here but I guess it's getting out of
topic.
> Of course, a USB-Serial adapter would work, and you can connect any
> serial terminal, but then we are back to using serial ports; it's just
> you will need a different driver for that.
Well, I have one of those, and it could be a good replacement because
it needs serial only on the development box, not on the target. It
would be acceptable for me in any case (because I already have the
adapter), but it probably won't be in the general case, given the
price ratio between a network card and the USB-Serial adapter.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
On Fri, 2002-11-15 at 22:51, Andrew Morton wrote:
> Stelian Pop wrote:
> >
> > On Fri, Nov 15, 2002 at 09:26:00PM +0000, Linus Torvalds wrote:
> >
> > > I dunno. I might even be willing to apply kgdb patches to my tree if it
> > > just could use the regular network card I already have connected on all
> > > my machines. None of my laptops have a serial line, for example, but
> > > they all have networking.
> > >
> > > Soon even _desktops_ probably won't have serial lines any more, only USB.
> >
> > Using USB instead of the serial line or the network card would be
> > the best IMHO, because:
> >
>
> Here is the kgdb stub's "send a byte" function:
>
> static void
> write_char(int chr)
> {
> while (!(inb(gdb_port + UART_LSR) & UART_LSR_THRE)) ;
>
> outb(chr, gdb_port + UART_TX);
> }
>
> Need I say more?
netdump has polled eepro100 handlers that will plug nicely into this. Of
course you still want a protocol on top of it, but there are some tiny
tcp implementations that are GPL (eg the Linux 8086 TCP by Harry K)
In article <[email protected]>,
Stelian Pop <[email protected]> wrote:
>
> * the USB stack seems simpler than the net stack +
> (eventualy) pcmcia + network card driver.
What drugs are you on? The USB stack is extremely complex and fragile,
and depends on a lot more working than just about any network driver out
there. We're still debugging basic USB functionality.
Yes, if you're comparing to a full TCP implementation, plain USB serial
lines may be simpler (ignoring for the moment the fact that there isn't
even a standard USB serial line protocol, and they may be going the same
way as the hardware serial lines - the way of the dodo).
But it should be possible to do a really simple UDP-packets-only thing
for kgdb. Sure, it may lose packets. Tough. Don't debug over a WAN,
and try to keep a clean direct network connection if you are worried
about it. But we want kernel printk's to be synchronous anyway, without
timeouts etc.
And I suspect you're better off losing packets (very rarely over any
normal local network) if that means that your debugger needs only
minimal support. You can always re-type.
Linus
>>>>> On Sat, 16 Nov 2002 05:30:05 +0100, [email protected] (Linus Torvalds) said:
Linus> Yes, if you're comparing to a full TCP implementation, plain
Linus> USB serial lines may be simpler (ignoring for the moment the
Linus> fact that there isn't even a standard USB serial line
Linus> protocol, and they may be going the same way as the hardware
Linus> serial lines - the way of the dodo).
Linus> But it should be possible to do a really simple
Linus> UDP-packets-only thing for kgdb. Sure, it may lose packets.
Linus> Tough. Don't debug over a WAN, and try to keep a clean
Linus> direct network connection if you are worried about it. But
Linus> we want kernel printk's to be synchronous anyway, without
Linus> timeouts etc.
Linus> And I suspect you're better off losing packets (very rarely
Linus> over any normal local network) if that means that your
Linus> debugger needs only minimal support. You can always re-type.
I did this a couple of years ago for my research OS (Scout) and it
worked great. It did UDP over Ethernet and was about 300 lines of
code. Rather than using the normal UDP stack, the kernel gdb I/O ran
directly on top of the network driver to reduce the risk of getting
hit by a breakpoint that is in the way of the gdb I/O. The code is
almost too trivial to mention, but for anyone interested, it can still
be found at:
http://www.cs.arizona.edu/scout/software.html
The relevant files are scout/sys/ai/kgdb_net.c and
scout/router/tulip/tulip.c (the latter contains the Ethernet driver
portion).
--david
PS: IIRC, the kgdb protocol has a simple checksumming protocol so it
can deal with packet losses (perhaps not very gracefully, but on a
LAN it's not going to be a problem anyhow).
On Fri, Nov 15, 2002 at 11:59:32PM +0100, Stelian Pop wrote:
> On Fri, Nov 15, 2002 at 02:51:11PM -0800, Andrew Morton wrote:
>
> > > Using USB instead of the serial line or the network card would be
> > > the best IMHO, because:
> >
> > Here is the kgdb stub's "send a byte" function:
> >
> > static void
> > write_char(int chr)
> > {
> > while (!(inb(gdb_port + UART_LSR) & UART_LSR_THRE)) ;
> >
> > outb(chr, gdb_port + UART_TX);
> > }
> >
> > Need I say more?
>
> I already know that, but this is not the point. The point is that
> more and more boxes have no serial (or paralel) ports.
>
> But even on those boxes, sometimes I'd just love to be able to use
> kgdb. And I can't.
>
> Ok, it will have to be at a higher level than the inb/outb serial
> transport implementation (with possible bad effects on what can
> and what cannot be debugged), but still, I feel there is a need
> for that.
> USB (with USB-to-serial adapter), network, ieee1394 would be
> acceptable replacements for me.
Have you ever looked at a USB or 1394 driver? The nice thing about
serial is that the software to make it work is trivial. A debugger that
relies on a 5000 line driver is quite suspect.
>
>
> Stelian.
> --
> Stelian Pop <[email protected]>
> Alcove - http://www.alcove.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
---------------------------------------------------------
Victor Yodaiken
Finite State Machine Labs: The RTLinux Company.
http://www.fsmlabs.com http://www.rtlinux.com
1+ 505 838 9109
On Sat, Nov 16, 2002 at 09:23:41AM -0700, [email protected] wrote:
> > > Here is the kgdb stub's "send a byte" function:
[...]
>
> > USB (with USB-to-serial adapter), network, ieee1394 would be
> > acceptable replacements for me.
>
> Have you ever looked at a USB or 1394 driver?
As a matter of fact, I did.
> The nice thing about
> serial is that the software to make it work is trivial. A debugger that
> relies on a 5000 line driver is quite suspect.
Agreed. But even a suspect debugger is preferable to no debugger at all.
Look, serial ports are becoming obsolete. We (not everybody but many
people) need kgdb.
The general oppinion seems to be that a mini network stack + a simple
network card driver is the easiest thing to implement (mostly because
already done). That's fine for me.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
On Fri, Nov 15, 2002 at 11:24:49PM -0800, David Mosberger-Tang wrote:
> >>>>> On Sat, 16 Nov 2002 05:30:05 +0100, [email protected] (Linus Torvalds) said:
>
> Linus> Yes, if you're comparing to a full TCP implementation, plain
> Linus> USB serial lines may be simpler (ignoring for the moment the
> Linus> fact that there isn't even a standard USB serial line
> Linus> protocol, and they may be going the same way as the hardware
> Linus> serial lines - the way of the dodo).
>
> Linus> But it should be possible to do a really simple
> Linus> UDP-packets-only thing for kgdb. Sure, it may lose packets.
> Linus> Tough. Don't debug over a WAN, and try to keep a clean
> Linus> direct network connection if you are worried about it. But
> Linus> we want kernel printk's to be synchronous anyway, without
> Linus> timeouts etc.
>
> Linus> And I suspect you're better off losing packets (very rarely
> Linus> over any normal local network) if that means that your
> Linus> debugger needs only minimal support. You can always re-type.
>
> I did this a couple of years ago for my research OS (Scout) and it
> worked great. It did UDP over Ethernet and was about 300 lines of
> code. Rather than using the normal UDP stack, the kernel gdb I/O ran
> directly on top of the network driver to reduce the risk of getting
> hit by a breakpoint that is in the way of the gdb I/O. The code is
> almost too trivial to mention, but for anyone interested, it can still
> be found at:
>
> http://www.cs.arizona.edu/scout/software.html
>
> The relevant files are scout/sys/ai/kgdb_net.c and
> scout/router/tulip/tulip.c (the latter contains the Ethernet driver
> portion).
And I'm 95% sure that someone here (MontaVista) has done the same thing
for Linux... ARM if my memory serves me correctly. You'll also notice
that UDP debugging support is now in FSF GDB 5.3 snapshots. Sure, it's
a little lossy, but it works in the useful cases.
I'm not quite sure what happened to the kernel side... but it sounds
like this could be a nice candidate for the network dumping stack, too.
> PS: IIRC, the kgdb protocol has a simple checksumming protocol so it
> can deal with packet losses (perhaps not very gracefully, but on a
> LAN it's not going to be a problem anyhow).
Yes. The protocol also has explicit ACKs and has the basis for
retransmission; it's not extensive but it's enough to get by over even
a lossy serial cable.
--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer
On Sat, Nov 16, 2002 at 02:35:03AM +0000, Alan Cox wrote:
>
> netdump has polled eepro100 handlers that will plug nicely into this. Of
> course you still want a protocol on top of it, but there are some tiny
> tcp implementations that are GPL (eg the Linux 8086 TCP by Harry K)
TCP is complete overkill. You don't want to run a debugging protocol
over a WAN in any case. Let's assume LAN, 1ms or less round-trip,
let's assume polling ACKs per packet so we don't have to keep a
window, UDP with hand-specified src/dst/gateway/MACs/ports.
--
"Love the dolphins," she advised him. "Write by W.A.S.T.E.."
On Sat, Nov 16, 2002 at 04:19:45AM +0000, Linus Torvalds wrote:
> But it should be possible to do a really simple UDP-packets-only thing
> for kgdb. Sure, it may lose packets. Tough. Don't debug over a WAN,
> and try to keep a clean direct network connection if you are worried
> about it. But we want kernel printk's to be synchronous anyway, without
> timeouts etc.
LAN latencies should be low enough that waiting on an ACK for each
packet will do just fine for error correction. If someone wants to do
remote debugging, they can ssh into a debugging machine on the same LAN.
--
"Love the dolphins," she advised him. "Write by W.A.S.T.E.."
On Sat, 16 Nov 2002, Oliver Xymoron wrote:
>
> LAN latencies should be low enough that waiting on an ACK for each
> packet will do just fine for error correction. If someone wants to do
> remote debugging, they can ssh into a debugging machine on the same LAN.
I agree in theory on a technical level, yet at the same time it's clearly
advantageous _not_ to wait, since it would allow you to just universally
enable the LAN as the console on all your machines when you maintain them,
and then not have that LAN console be a maintenance problem.
Basically, I don't personally care too much for kgdb itself, but I see a
asynchronous LAN console as a more generic tool for just doing not just
kernel debugging, but management in general. syslogd is fine for when the
machines work, but it tends to lose important information just when you
need it the most, ie when a machine starts having serious problems.
So if you see the LAN interface more as a simple console (that _also_
gives you access to kgdb), then it's a whole lot more usable for much more
than just kernel hacking.
A synchronous interface would make logging basically useless - it would
basically crash the machine if there are logger problems, and it would
make log output slow things down a lot.
Linus
On Sat, Nov 16, 2002 at 10:33:57AM -0800, Linus Torvalds wrote:
>
> On Sat, 16 Nov 2002, Oliver Xymoron wrote:
> >
> > LAN latencies should be low enough that waiting on an ACK for each
> > packet will do just fine for error correction. If someone wants to do
> > remote debugging, they can ssh into a debugging machine on the same LAN.
>
> I agree in theory on a technical level, yet at the same time it's clearly
> advantageous _not_ to wait, since it would allow you to just universally
> enable the LAN as the console on all your machines when you maintain them,
> and then not have that LAN console be a maintenance problem.
Definitely agreed on the usefulness of LAN console. Being able to just
run netcat for all the boxes in your datacenter is a huge win. Cuts
your cable count by a third to a half and eliminates a bunch of term
servers and KVMs.
Other folks have pointed out that the GDB stub protocol includes
checksumming and retries up at the 'application' layer so we don't
actually have to worry about it.
--
"Love the dolphins," she advised him. "Write by W.A.S.T.E.."
Linus Torvalds wrote:
> And I suspect you're better off losing packets (very rarely over any
> normal local network) if that means that your debugger needs only
> minimal support. You can always re-type.
Of course, if your virtual LAN has several machines that all think
that congestion control only makes sense in a WAN setting, loss may
get quite common :-) I still have fond memories of shared Ethernet
melting down under the load of dozens of diskless clients rebooting
after some problem hit the cluster ...
Loss recovery doesn't have to be painful, and it can still be real
time. When sending, include the packet sequence number, and the
sequence number of the earliest packet buffered. Buffer up to N
packets after sending. Packets older than current_sequence-N drop
off the buffer and cannot be recovered. Receiver can request
retransmission of specific buffered packets. Retransmitted packets
carry an up to date earliest packet buffered sequence number.
This way, it's up to the receiver to decide what to do with lost
packets, e.g. request retransmission, and wait up to a certain
deadline, just flag the loss and proceed, of maybe flag the loss,
request retransmission, and when receiving the missing packet,
redraw the output.
Congestion control would be very good to have, though. As soon as
you put something on IP, it's almost guaranteed to eventually end
up crossing a WAN.
Crazy idea: maybe one could combine the ideas from MCORE and RT
Linux: use a dispatcher for basic system resources (interrupts,
etc.), and run two kernels. One of them would just run the
debugger and serve its communication needs. That would leave
things still relatively close to hardware (unlike just using
UML).
- Werner
--
_________________________________________________________________________
/ Werner Almesberger, Buenos Aires, Argentina [email protected] /
/_http://www.almesberger.net/____________________________________________/
On Sat, 2002-11-16 at 09:21, Stelian Pop wrote:
> Agreed. But even a suspect debugger is preferable to no debugger at all.
>
> Look, serial ports are becoming obsolete. We (not everybody but many
> people) need kgdb.
>
Machines may not ship with serial ports anymore, but theoretically they
should have debug ports.
(A Debug Port being a 16550 with an arbitrary IO port, an ACPI table
entry, and a connector on the motherboard. There's a spec on the
Microsoft web site with the details, but there's a click-through license
agreement and IANAL, so read at your own risk.)
--
Nicholas Miell <[email protected]>
On Sat, 2002-11-16 at 04:19, Linus Torvalds wrote:
> But it should be possible to do a really simple UDP-packets-only thing
> for kgdb. Sure, it may lose packets. Tough. Don't debug over a WAN,
> and try to keep a clean direct network connection if you are worried
> about it. But we want kernel printk's to be synchronous anyway, without
> timeouts etc.
And in the real world you end up back with TCP. Been there, done that
with network debugger tools before.
Alan
On Sat, 2002-11-16 at 07:24, David Mosberger-Tang wrote:
> hit by a breakpoint that is in the way of the gdb I/O. The code is
> almost too trivial to mention, but for anyone interested, it can still
> be found at:
>
> http://www.cs.arizona.edu/scout/software.html
>
> The relevant files are scout/sys/ai/kgdb_net.c and
> scout/router/tulip/tulip.c (the latter contains the Ethernet driver
> portion).
Very nice. That solves the debugger question elegantly. I'd still rather
have tcp for a console control though 8)
On 16 Nov 2002, Alan Cox wrote:
>
> And in the real world you end up back with TCP. Been there, done that
> with network debugger tools before.
.. and we have another "been there, done that" that says UDP works fine.
I also dislike overdesign with a passion. I'll believe we have to go to
TCP when I see it - and even if that happens, I think we should do the UDP
case first just to avoid having a monster ("start off simple, and evolve"
as opposed to "try to get it right the first time").
Linus
On Sat, Nov 16, 2002 at 07:19:10PM -0800, Linus Torvalds wrote:
> On 16 Nov 2002, Alan Cox wrote:
> > And in the real world you end up back with TCP. Been there, done that
> > with network debugger tools before.
>
> .. and we have another "been there, done that" that says UDP works fine.
>
> I also dislike overdesign with a passion. I'll believe we have to go to
> TCP when I see it - and even if that happens, I think we should do the UDP
> case first just to avoid having a monster ("start off simple, and evolve"
> as opposed to "try to get it right the first time").
I've done this sort of "reliable UDP" on local networks before, it works
just fine, no TCP is needed. TCP is a really well designed protocol, it
handles an amazing range of problems gracefully. It's overkill for a
lan remote gdb.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
* Dmitri <[email protected]> on Fri, Nov 15, 2002:
> On Fri, 2002-11-15 at 14:24, Stelian Pop wrote:
>
> > Using USB instead of the serial line or the network card would be
> > the best IMHO, because:
> >
> > * many machines have network cards, but all machines have USB
> > (and it's gonna stay this way for some time)
> >
> > * the USB stack seems simpler than the net stack +
> > (eventualy) pcmcia + network card driver.
> >
> > Maybe the 'simpler' USB protocols (usbkbd and usbmouse) could be
> > used for this, I don't know...
>
> USB hardware and protocols are master-slave, meaning that you can not
> connect another computer to this one directly. What USB *device* would
> you want to see connected?
>
Not true. Just grab any USB master<->master cable. There's P2P network
drivers already in the kernel that support most brands of cables.
M. R.
On Sat, 2002-11-16 10:33:57 -0800, Linus Torvalds <[email protected]>
wrote in message <[email protected]>:
> On Sat, 16 Nov 2002, Oliver Xymoron wrote:
> >
> > LAN latencies should be low enough that waiting on an ACK for each
> > packet will do just fine for error correction. If someone wants to do
> > remote debugging, they can ssh into a debugging machine on the same LAN.
>
> Basically, I don't personally care too much for kgdb itself, but I see a
> asynchronous LAN console as a more generic tool for just doing not just
> kernel debugging, but management in general. syslogd is fine for when the
...which reminds me to DEC's MOP (Maintainence and Operator's Protocol),
which is ethernet (but not IP) based remote console and a mixture of
bootp/tftp. Sure, we won't (yet) go as far as sending the next kernel to
boot via our new console protocol to kexec(), but wait for the very
first S-Records to arrive:-p
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur
fuer einen Freien Staat voll Freier B?rger" | im Internet!
Shell Script APT-Proxy: http://lug-owl.de/~jbglaw/software/ap2/
As long as the network console/debug interface includes basic a basic
check to verify that the packets it accepts are from the local network.
And it's outgoing packets have a ttl of one. I don't have a problem.
Otherwise the concept gives me security nightmares.
Eric
Eric W. Biederman wrote:
> As long as the network console/debug interface includes basic a basic
> check to verify that the packets it accepts are from the local network.
> And it's outgoing packets have a ttl of one. I don't have a problem.
Is there a working network console? It would be _great_ to have a
network console to my _remote_ server, far far away on the internet.
-- Jamie
Jamie Lokier <[email protected]> writes:
> Eric W. Biederman wrote:
> > As long as the network console/debug interface includes basic a basic
> > check to verify that the packets it accepts are from the local network.
> > And it's outgoing packets have a ttl of one. I don't have a problem.
>
> Is there a working network console? It would be _great_ to have a
> network console to my _remote_ server, far far away on the internet.
There are bits and pieces, and a lan based kgdb is basically the same
problem security wise. When you allow any kind of control the security
cannot be in the console protocol. Therefore it can only be used on
a trusted lan and be only talk to local addresses. At the same time
if you have two remote machines on that trusted lan you can use one
to control the other. Knowing that a root exploit on one likely
becomes a root exploit on both.
And weather or not we have one at the moment, it is an active area of
research. We just need to get a useable security model. And I think
enforcing that the console be on a secure lan where every connected
machine is trusted is a good first draft, at the latter.
So I do not think this is the kind of thing that will help if you
only have one _remote_ server, far far away on the internet, but when
you start getting a collection of them it may help.
Eric
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 18 Nov 2002 06:42, Eric W. Biederman wrote:
> As long as the network console/debug interface includes basic a basic
> check to verify that the packets it accepts are from the local network.
This is pretty hard to do in some configurations. You essentially have to do
this at the router, not at destination.
> And it's outgoing packets have a ttl of one. I don't have a problem.
Recent IETF work on link-local has used TTL=255 outgoing, and it has to be 255
at the receive end too. That is a reasonable way to ensure that is is
link-local, since even the most brain-dead routers will at least decrement
TTL.
> Otherwise the concept gives me security nightmares.
Computing should give you security nightmares :)
Brad
- --
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE91/s3W6pHgIdAuOMRAsCNAJ42Fc7CCclgD+zMbraiHYFydMcKJACfegyr
JcZ8T8+1nIe2f8G3eerNwVs=
=uGdb
-----END PGP SIGNATURE-----
Brad Hards <[email protected]> writes:
> On Mon, 18 Nov 2002 06:42, Eric W. Biederman wrote:
> > As long as the network console/debug interface includes basic a basic
> > check to verify that the packets it accepts are from the local network.
> This is pretty hard to do in some configurations. You essentially have to do
> this at the router, not at destination.
I agree that you cannot do a perfect job. The goal is to get something
that is good enough so that it can be enabled and not give an automatic root
exploit if someone accidentally leaves it on at the wrong time.
> > And it's outgoing packets have a ttl of one. I don't have a problem.
> Recent IETF work on link-local has used TTL=255 outgoing, and it has to be 255
> at the receive end too. That is a reasonable way to ensure that is is
> link-local, since even the most brain-dead routers will at least decrement
> TTL.
Nice. And then on the transmit end I would still use a TTL=1 so that
routers will refuse to route the packets. A bit asymmetric but I only
care about security in one direction.
But in what kind of configurations is checking the ip against the
current netmask insufficient? Checking for a TTL of 255 will
trivially make the check stronger.
Having a network console for various debugging tasks could be very
useful. The question is how to implement it simply and reliably.
Eric
a couple quick questions from an end-user
1. will an interface be dedicated to this use, or will it share an
interface with other traffic.
2. if it is dedicated how do you inform the rest of the system that this
card is off limits? (and can you use one port on a quad card, or do you
require a dedicated card?)
3. if we really want to make this limited to the local wire why not use
something other then UDP? either another IP protocol number (more likly to
be blocked by routers) or somethign not IP compatable (so that routers
couldn't forward it if they wanted to). especially if you are talking
about useing a special (aka stripped down, simplified) stack for this
interface instead of the full-blown version
normally I would agree that standards are good, becouse they let you
interoperate with other equipment, but in this case I'm not sure that's
really what we want. All communications is not IP :-)
as someone managing 60 or so remote boxes, this sounds really nice, if it
can be made to work securely.
David Lang
On 17 Nov 2002 [email protected] wrote:
> Date: 17 Nov 2002 14:30:00 -0700
> From: [email protected]
> To: Brad Hards <[email protected]>
> Cc: Linux Kernel Mailing List <[email protected]>
> Subject: Re: lan based kgdb
>
> Brad Hards <[email protected]> writes:
>
> > On Mon, 18 Nov 2002 06:42, Eric W. Biederman wrote:
> > > As long as the network console/debug interface includes basic a basic
> > > check to verify that the packets it accepts are from the local network.
> > This is pretty hard to do in some configurations. You essentially have to do
> > this at the router, not at destination.
>
> I agree that you cannot do a perfect job. The goal is to get something
> that is good enough so that it can be enabled and not give an automatic root
> exploit if someone accidentally leaves it on at the wrong time.
>
> > > And it's outgoing packets have a ttl of one. I don't have a problem.
> > Recent IETF work on link-local has used TTL=255 outgoing, and it has to be 255
> > at the receive end too. That is a reasonable way to ensure that is is
> > link-local, since even the most brain-dead routers will at least decrement
> > TTL.
>
> Nice. And then on the transmit end I would still use a TTL=1 so that
> routers will refuse to route the packets. A bit asymmetric but I only
> care about security in one direction.
>
> But in what kind of configurations is checking the ip against the
> current netmask insufficient? Checking for a TTL of 255 will
> trivially make the check stronger.
>
> Having a network console for various debugging tasks could be very
> useful. The question is how to implement it simply and reliably.
>
> Eric
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 18 Nov 2002 08:30, Eric W. Biederman wrote:
> Brad Hards <[email protected]> writes:
> > On Mon, 18 Nov 2002 06:42, Eric W. Biederman wrote:
> > > As long as the network console/debug interface includes basic a basic
> > > check to verify that the packets it accepts are from the local network.
> >
> > This is pretty hard to do in some configurations. You essentially have to
> > do this at the router, not at destination.
>
> I agree that you cannot do a perfect job. The goal is to get something
> that is good enough so that it can be enabled and not give an automatic
> root exploit if someone accidentally leaves it on at the wrong time.
if you put it on the network, the best you can do is restrict the problem to
being link-local. If you exclude wireless, then the problem is basically then
a matter of physical security, which isn't much different to having a console
running on the box itself.
> > > And it's outgoing packets have a ttl of one. I don't have a problem.
> >
> > Recent IETF work on link-local has used TTL=255 outgoing, and it has to
> > be 255 at the receive end too. That is a reasonable way to ensure that is
> > is link-local, since even the most brain-dead routers will at least
> > decrement TTL.
>
> Nice. And then on the transmit end I would still use a TTL=1 so that
> routers will refuse to route the packets. A bit asymmetric but I only
> care about security in one direction.
I was completely confused when I first read this.
I take it that you mean:
On machine under test (the target in the debug context) you make TTL=1 on
outgoing packets, and drop incoming packets that don't have TTL=255
On machine being used to view console (the host in the debug context) you make
TTL=255 on outgoing packets.
That doesn't ensure that the machine you are monitoring (or debugging) is
really the machine you think it is, but it might solve enough of the problem.
> But in what kind of configurations is checking the ip against the
> current netmask insufficient? Checking for a TTL of 255 will
> trivially make the check stronger.
It prevents source field spoofing. I don't know how much defence you get from
this on fielded routers, but faking the source field in a UDP packet doesn't
seem too hard.
TTL=255 ensures that the machine you are talking to is _really_ on the same
link. Any other number is subject to an attacker knowing how many hops it is,
and setting TTL appropriately.
Brad
- --
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE92A07W6pHgIdAuOMRAhmBAJ9O//ObT/e2pq6lrbiKUDhkNNJslwCdHYEN
mXWWIAkZW7SDUSRhY7AvN0s=
=EA7y
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 18 Nov 2002 08:32, David Lang wrote:
> a couple quick questions from an end-user
>
> 1. will an interface be dedicated to this use, or will it share an
> interface with other traffic.
I imagined that it would have to be shared. The world is not a PC, and you
can't trivially add extra connectivtity to that embedded ARM board...
<snip>
> 3. if we really want to make this limited to the local wire why not use
> something other then UDP? either another IP protocol number (more likly to
> be blocked by routers) or somethign not IP compatable (so that routers
> couldn't forward it if they wanted to). especially if you are talking
> about useing a special (aka stripped down, simplified) stack for this
> interface instead of the full-blown version
If you really want link-local, use the link-local IP addresses (169.254/16).
See http://files.zeroconf.org/draft-ietf-zeroconf-ipv4-linklocal.txt
(I actually have an implementation of this for linux, alas it is only in
userspace :)
> normally I would agree that standards are good, becouse they let you
> interoperate with other equipment, but in this case I'm not sure that's
> really what we want. All communications is not IP :-)
All _good_ communication is IP :)
> as someone managing 60 or so remote boxes, this sounds really nice, if it
> can be made to work securely.
OK, I'm confused again. Do you want remote, or to you want link-local?
Brad
- --
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE92A64W6pHgIdAuOMRArrEAJ9Yp6w85APLudklNBVxfY6nAF066ACfeLc7
z00tKrs79Ri3k7UbFNCQzJo=
=VXrF
-----END PGP SIGNATURE-----
On Mon, 18 Nov 2002, Brad Hards wrote:
> On Mon, 18 Nov 2002 08:32, David Lang wrote:
> > a couple quick questions from an end-user
> >
> > 1. will an interface be dedicated to this use, or will it share an
> > interface with other traffic.
> I imagined that it would have to be shared. The world is not a PC, and you
> can't trivially add extra connectivtity to that embedded ARM board...
I can see that, it will add more complexity thought
> <snip>
> > as someone managing 60 or so remote boxes, this sounds really nice, if it
> > can be made to work securely.
> OK, I'm confused again. Do you want remote, or to you want link-local?
I want link-local. as has been discused once you have two boxes in one
location link-local is good enough and I always deploy boxes in HA pairs
David Lang
On Sun, 2002-11-17 at 09:56, Jan-Benedict Glaw wrote:
> ...which reminds me to DEC's MOP (Maintainence and Operator's Protocol),
> which is ethernet (but not IP) based remote console and a mixture of
> bootp/tftp. Sure, we won't (yet) go as far as sending the next kernel to
> boot via our new console protocol to kexec(), but wait for the very
> first S-Records to arrive:-p
We have working DECnet and MOP server protocol support. That doesn't
mean its a good idea
David Lang <[email protected]> writes:
> On Mon, 18 Nov 2002, Brad Hards wrote:
>
> > On Mon, 18 Nov 2002 08:32, David Lang wrote:
> > > a couple quick questions from an end-user
> > >
> > > 1. will an interface be dedicated to this use, or will it share an
> > > interface with other traffic.
> > I imagined that it would have to be shared. The world is not a PC, and you
> > can't trivially add extra connectivtity to that embedded ARM board...
>
> I can see that, it will add more complexity thought
There are only three cases I can see supporting.
- Transmit a kernel message
- Process a MAGIC-SYSRQ packet
- Remote GDB.
For transmitting a kernel message, sharing the nic is really not more
complicated. It simply preempts the normal driver transmits it's
packet and then goes away.
For processing a MAGIC-SYSRQ that is almost certainly going to be
interrupt driven just a special debug path that can safely execute
inside of the interrupt handler. For conditions that make the box
unpingable this likely will not help, but very will at that point.
For a remote GDB transmitting a packet is the same as for kernel
messages. If a packet is expected it just waits on the nic throwing
out the rest of the packets until the one it is expecting comes in.
And if it isn't particularly expecting a packet it can work just like
the MAGIC-SYSRQ case.
So while there is some small additional complexity to sharing the link
I do not think it is significant. But just because you can share the
link does not mean you have to.
> > <snip>
> > > as someone managing 60 or so remote boxes, this sounds really nice, if it
> > > can be made to work securely.
> > OK, I'm confused again. Do you want remote, or to you want link-local?
>
> I want link-local. as has been discused once you have two boxes in one
> location link-local is good enough and I always deploy boxes in HA
> pairs
I am still trying to hammer down the protocol enough to suit me.
IPv4 Link local IP addresses are not generally appropriate, because
claim-and-defend is potentially a pain. IPv6 link-local addresses
look better as everything can be configured statically. Ideally
everything can be kept simple enough that retransmissions do not need
to at all. Nor extraneous traffic processing like arp.
Configuration needed:
Interface: (This should default to none)
Local-ip address: (For ipv6 we can default to the link-local address
derived from the MAC address)
Remote-ip address: (By default this should probably be a multi-cast IP
address, derived from the local-ip address)
The important things:
- The ability to have everything hard coded, so you can find the
network console/network gdb session.
- A trivial implementation that can fire and forget packets. No
retransmit should be needed on the machine being debugged. This is
over a mostly reliable interface so that should not be a problem.
- A setup that does not loose a connection when you reboot a machine.
This rules out TCP.
- The ability to have good defaults so you can setup a cluster without
manual configuration.
The way I envision it, on the debugged machine:
- Transmit:
TTL=1 to some address (ideally this would be a multicast ip)
- Receive:
verify TTL=255 ip on the same subnet. Packet is not fragmented.
Dest IP is my local ip (Ipv6 link-local ip?)
It will be a little bit before I scratch this itch but I have a good
feeling about how to do it now.
Eric
[email protected] (Eric W. Biederman) writes:
> I agree that you cannot do a perfect job. The goal is to get something
> that is good enough so that it can be enabled and not give an automatic root
> exploit if someone accidentally leaves it on at the wrong time.
You can always use a simple mac / shared secret scheme. Just use HMAC on each
packet. The kernel has all the needed code in crypto/ now.
May not be military grade, but should be good enough to stop most attacks.
-Andi
Eric W. Biederman wrote:
> Otherwise the concept gives me security nightmares.
Bah, there are a few Fundamental Truths of Networking Simplification
that we can absolutely rely on:
- it is perfectly safe to require that a given port be connected only
to a secure and trusted network
- if you design a protocol not to work over a WAN, everybody will
respect this, and deploy it only on LANs
- if any of the above constraints is no longer tenable, people will
carefully redesign the protocol in question, and replace the
installed base
- all LANs have simple, well-understood characteristics - now and
forever
After all, SNMP, FSP, LANE, NAT, WEP, etc. can't be wrong (-:C
And yes, I can vividly imagine users of dedicated hosts rush to switch
on that remote console the very moment it becomes available, and use
it across half of the planet.
- Werner
--
_________________________________________________________________________
/ Werner Almesberger, Buenos Aires, Argentina [email protected] /
/_http://www.almesberger.net/____________________________________________/
Stelian Pop <[email protected]> writes:
> Using USB instead of the serial line or the network card would be
> the best IMHO, because:
>
> * many machines have network cards, but all machines have USB
> (and it's gonna stay this way for some time)
Um, no; _many_ machines have USB.
[The boards I work with all have a UART _on the CPU chip_, so it's
pretty cheap to give them a serial connection, and many of them have a
network connection because it's needed -- but none of them have USB...
Even if you're talking about just PCs, most of the PCs I use are
slightly old, and don't have USB; maybe I'm just from mars or something,
but it's always kind of annoying to see people talking as if USB were
universal...]
-Miles
--
80% of success is just showing up. --Woody Allen
On Sun, 2002-11-17 14:50:32 +0000, Alan Cox <[email protected]>
wrote in message <[email protected]>:
> On Sun, 2002-11-17 at 09:56, Jan-Benedict Glaw wrote:
> > ...which reminds me to DEC's MOP (Maintainence and Operator's Protocol),
> > which is ethernet (but not IP) based remote console and a mixture of
> > bootp/tftp. Sure, we won't (yet) go as far as sending the next kernel to
> > boot via our new console protocol to kexec(), but wait for the very
> > first S-Records to arrive:-p
>
> We have working DECnet and MOP server protocol support. That doesn't
> mean its a good idea
Really? Do we? Well, sure for the server part (-> booting. Half a dozend
implementations from which only one ot possibly two are really useable),
but the client side (remote console) was never implemented IIRC. And for
sure there's no MOP console server on linux:-)
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur
fuer einen Freien Staat voll Freier B?rger" | im Internet!
Shell Script APT-Proxy: http://lug-owl.de/~jbglaw/software/ap2/
Looks like there is a lot of interset in lan based kgdb.
I've got the kgdb patch with lan written by San, thanks to Samuel Flory.
I'll integrate the patch into mainline kgdb
(http://kgdb.sourceforge.net/) and release it asap.
--
Amit Kale
Veritas Software India ( http://www.veritasindia.com/ )