2009-01-23 17:42:16

by Andrew Walrond

[permalink] [raw]
Subject: serial8250 madness

Is anyone active in the serial code?

I thought I would make some in-house servers send output via ttyS0 from
grub, kernel boot and console, like our rented remote servers do. I set
it all up and basically nothing worked properly, so I started
investigating with a null modem cable, setserial and cu to see what was
happening.

Machine A

ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0
as I wanted.
ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc
but just doesn't work at all.

Machine B

ttyS0 and ttyS1 will both receive, but neither will send. Go figure.
Tested using a null modem cable, setserial and cu against the known good
machine above.

Machine C

Grub and kernel outputs fine during boot, right up til the serial8250
output appears in the kernel log, at which point it then prints 'LSR
safety check engaged' and reboots.
Tried again without the console=ttyS0,57600 kernel parameter and it
boots OK but starts printing the 'LSR...' message again as soon as
agetty tries to attach to ttyS0. Doesn't reboot though...

The Machines are all amd64, various motherboards, identical distros
using 2.6.28.1 kernel.
Either I have a particularly motley collection of kit, or the serial
code is in a pretty bad way.

I can provide further info/testing if anyone is active/interested.

Andrew Walrond


2009-01-23 17:52:18

by David Daney

[permalink] [raw]
Subject: Re: serial8250 madness

Andrew Walrond wrote:
> Is anyone active in the serial code?
>

Yes.

> [...]
>
> The Machines are all amd64, various motherboards, identical distros
> using 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial
> code is in a pretty bad way.
>

I use serial consoles on a variety of embedded boards every day (albeit
mostly mips based) ,so I think it works.

You do have to have your kernel configured properly for a serial console
for it to work right.

I would test it using some other device as the console. Only once you
have it working try for the serial console.

David Daney

2009-01-23 18:00:25

by Alan

[permalink] [raw]
Subject: Re: serial8250 madness

> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0
> as I wanted.
> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc
> but just doesn't work at all.

You don't give much information but if it seems to be non working then I
would look hard at the IRQ/IO reported

> Machine B
>
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure.
> Tested using a null modem cable, setserial and cu against the known good
> machine above.

Boggle - what flow control wiring have you done ?

> Machine C
>
> Grub and kernel outputs fine during boot, right up til the serial8250
> output appears in the kernel log, at which point it then prints 'LSR
> safety check engaged' and reboots.
> Tried again without the console=ttyS0,57600 kernel parameter and it
> boots OK but starts printing the 'LSR...' message again as soon as
> agetty tries to attach to ttyS0. Doesn't reboot though...
>
> The Machines are all amd64, various motherboards, identical distros
> using 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial
> code is in a pretty bad way.

It is maintained and should be pretty solid. Seria console is one of the
less used bits but when it gets broken people do jump up and down and
shout about it.

The big outstanding bug I know about is the incompatibility between the
PnP 8250 drivers and serial console - that may be your machine C case.
That one is rather hard to fix so on the long rather than short term hit
list.

Alan

2009-01-23 18:05:20

by Andrew Walrond

[permalink] [raw]
Subject: Re: serial8250 madness

David Daney wrote:
> You do have to have your kernel configured properly for a serial
> console for it to work right.
>
Sorry to be blunt, but I don't think you read my message very carefully!
I am reporting kernel bug(s) in the serial subsystem, not amateur
configuration problems. I know what I'm about, but on a random selection
of 3 servers with identical kernels/software, the serial subsystem works
fully on none of them. All boot just fine to the normal console.

Andrew Walrond

2009-01-23 18:13:57

by David Daney

[permalink] [raw]
Subject: Re: serial8250 madness

Andrew Walrond wrote:
> David Daney wrote:
>> You do have to have your kernel configured properly for a serial
>> console for it to work right.
>>
> Sorry to be blunt, but I don't think you read my message very carefully!

I did, I read every word of it. I am sorry I was not specific enough
for you. I really meant to ask:

Do you have: CONFIG_SERIAL_8250_CONSOLE=y


David Daney.

2009-01-23 18:24:58

by Bruce Ashfield

[permalink] [raw]
Subject: Re: serial8250 madness

On Fri, Jan 23, 2009 at 12:41 PM, Andrew Walrond <[email protected]> wrote:
> Is anyone active in the serial code?
>

Definitely. Collectively, I see boards from 4 major archs
boot and use a serial console on a daily basis. Including
a pretty decent set of x86 boards.

> I thought I would make some in-house servers send output via ttyS0 from
> grub, kernel boot and console, like our rented remote servers do. I set it
> all up and basically nothing worked properly, so I started investigating
> with a null modem cable, setserial and cu to see what was happening.
>
> Machine A
>
> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0 as
> I wanted.
> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc but
> just doesn't work at all.

Interesting. In the past the BIOS has normally given us
problems when trying to get grub to output via serial,
first contact often appears once the kernel is booting.

>
> Machine B
>
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure. Tested
> using a null modem cable, setserial and cu against the known good machine
> above.
>
> Machine C
>
> Grub and kernel outputs fine during boot, right up til the serial8250 output
> appears in the kernel log, at which point it then prints 'LSR safety check
> engaged' and reboots.
> Tried again without the console=ttyS0,57600 kernel parameter and it boots OK
> but starts printing the 'LSR...' message again as soon as agetty tries to
> attach to ttyS0. Doesn't reboot though...

Haven't witnessed B or C, but have had plenty of flow control
problems with infrastructure that prevented serial from properly
displaying.

I've also seen a few minor issues in the kernel itself, but it's
normally some slightly different chip and/or polling mode changes
interfering with normal operation. None of which seem useful
here.

>
> The Machines are all amd64, various motherboards, identical distros using
> 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial code is
> in a pretty bad way.
>
> I can provide further info/testing if anyone is active/interested.

I'm not much help, but I thought some extra data
points *might* be of use.

Bruce

>
> Andrew Walrond
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>



--
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

2009-01-23 18:39:24

by Andrew Walrond

[permalink] [raw]
Subject: Re: serial8250 madness

David Daney wrote:
> I did, I read every word of it. I am sorry I was not specific enough
> for you. I really meant to ask:
>
> Do you have: CONFIG_SERIAL_8250_CONSOLE=y
>
"Machine A - ttyS0 works fine. Can get grub output, kernel boot and
console via ttyS0 as I wanted."

So the whole chain works fine on this machine. This couldn't work
without CONFIG_SERIAL_8250_CONSOLE or if I didn't know _how_ to do it.

But to prove it conclusively
$ zcat /proc/config.gz | grep CONFIG_SERIAL_8250_CONSOLE
CONFIG_SERIAL_8250_CONSOLE=y

Sorry for being snappy. Long day ;)

Andrew Walrond

2009-01-23 19:05:32

by Andrew Walrond

[permalink] [raw]
Subject: Re: serial8250 madness

Alan Cox wrote:
>> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0
>> as I wanted.
>> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc
>> but just doesn't work at all.
>>
>
> You don't give much information but if it seems to be non working then I
> would look hard at the IRQ/IO reported
>
>
On Machine A (ttyS0 works fine. ttyS1 dead as dodo):
$ dmesg | grep ttyS

Command line: console=tty0 console=ttyS0,57600
Kernel command line: console=tty0 console=ttyS0,57600
console [ttyS0] enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

$ sudo ./setserial -a /dev/ttyS1

/dev/ttyS1, Line 1, UART: 16550A, Port: 0x02f8, IRQ: 3
Baud_base: 115200, close_delay: 50, divisor: 0
closing_wait: 3000
Flags: spd_normal skip_test

user@orac ~ $ cat /proc/interrupts
CPU0 CPU1
0: 35 20 IO-APIC-edge timer
1: 0 2 IO-APIC-edge i8042
3: 0 1 IO-APIC-edge
4: 0 179 IO-APIC-edge serial
[snip]

>> Machine B
>>
>> ttyS0 and ttyS1 will both receive, but neither will send. Go figure.
>> Tested using a null modem cable, setserial and cu against the known good
>> machine above.
>>
>
> Boggle - what flow control wiring have you done ?
>
>
:) Same null modem cable as used in all other tests. Setup with
setserial then connect either ends with the likes of
cu -s 9600 -l /dev/ttyS?
and this machine can receive and display characters typed on the other
end, but nothing gets sent the other way.
Have tried with hardware/software flow on/off at both ends. Clueless...

I havent tried yet but will see if grub will push anything out (ie
independent of the kernel) which will rule out a hardware problem and
implicate the kernel. But its a relatively new m/b with onboard serial
so I doubt it's broken.
Will report back.
>
> The big outstanding bug I know about is the incompatibility between the
> PnP 8250 drivers and serial console - that may be your machine C case.
> That one is rather hard to fix so on the long rather than short term hit
> list.
>
Sounds likely. Any way I can know for sure?
The kernel log looks a bit different on this machine:

Serial: 8250/16550 driver4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
serial 00:09: activated
00:09: ttyS2 at I/O 0x3e8 (irq = 3) is a 16550A
serial 00:0a: activated
00:0a: ttyS3 at I/O 0x2e8 (irq = 5) is a 16550A

compared with Machine A:

Serial: 8250/16550 driver4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

Thanks for your input

Andrew Walrond
> Alan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>

2009-01-23 23:12:53

by Alan

[permalink] [raw]
Subject: Re: serial8250 madness

> Sounds likely. Any way I can know for sure?

Turn off the 8250 PnP support and retest is the best way to test that.

2009-01-24 12:06:54

by Andrew Walrond

[permalink] [raw]
Subject: Re: serial8250 madness

Andrew Walrond wrote:
>
> Machine B
>
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure.
> Tested using a null modem cable, setserial and cu against the known
> good machine above.
>
It's not just the linux kernel; grub also refuses to send anything using
this board's serial devices, so I think the hardware/bios must be
stuffed somehow.

And since this is an MSI board with 'Microsoft only Live Bios Update
software (tm)' as the only bios upgrade option, it will be visiting the
skip in short order.

I haven't seen this before; I hope it's not becoming a new fashion.

Andrew Walrond

2009-01-24 15:13:20

by Clark Williams

[permalink] [raw]
Subject: Re: serial8250 madness

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 23 Jan 2009 18:00:33 +0000
Alan Cox <[email protected]> wrote:
>
> The big outstanding bug I know about is the incompatibility between the
> PnP 8250 drivers and serial console - that may be your machine C case.
> That one is rather hard to fix so on the long rather than short term hit
> list.
>
> Alan


I spend a few days trying to figure out a panic in the uart driver that
turned out to be this issue. Would it make sense to add the following?

diff --git a/drivers/serial/Kconfig b/drivers/serial/Kconfig
index 579d63a..35acc6b 100644
- --- a/drivers/serial/Kconfig
+++ b/drivers/serial/Kconfig
@@ -91,7 +91,7 @@ config SERIAL_8250_PCI

config SERIAL_8250_PNP
tristate "8250/16550 PNP device support" if EMBEDDED
- - depends on SERIAL_8250 && PNP
+ depends on SERIAL_8250 && PNP && ! SERIAL_8250_CONSOLE
default SERIAL_8250
help
This builds standard PNP serial support. You may be able to

Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAkl7L/MACgkQHyuj/+TTEp21NACgpQI89LMX5ZtJbVlMcpa9A/I0
6JkAoIa5Nwt4mvm6UJGzGkoUGKOCzn5V
=YUtB
-----END PGP SIGNATURE-----
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay?ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?