I'm forwarding this post by the author of a great little program for
digital amateur radio on Linux, because I'm curious whether or not the
problem he is seeing can be resolved outside the kernel.
All comments welcome on/off list.
Thanks,
Joe Barr
K1GPL
--
It's a strange world when proprietary software is not worth stealing,
but free software is.
Hi,
On Fri, Jan 19, 2007 at 03:37:34PM -0600, Joe Barr wrote:
>
> I'm forwarding this post by the author of a great little program for
> digital amateur radio on Linux, because I'm curious whether or not the
> problem he is seeing can be resolved outside the kernel.
At least, I see one wrong claim and one unexplored track in his report.
The wrong claim : the serial port can only be controled by the kernel.
It is totally wrong for true serial ports. If he does not want to use
ioctl(), then he can directly program the I/O port.
The unexplored track : he talked about nice -20. He did not seem to try
playing with sched_setscheduler(). I've been using this with a few programs
to get (close to) real-time responsiveness and it gives very good results.
Not sure whether it will work for his case, though, but it's easy to try,
basically, he just has to add this to the top of his program :
#include <sched.h>
...
main() {
struct sched_param sch;
/* see man sched_setscheduler for other options */
sch.sched_priority = 1;
if (sched_setscheduler(getpid(), SCHED_FIFO, &sch) == -1)
perror("failed. Got root ?");
/* rest of the program now running with real-time prio */
}
Now he must be careful about avoiding busy loops in the rest of the
program, or he will have to use the reset button.
> All comments welcome on/off list.
>
> Thanks,
> Joe Barr
> K1GPL
[ rest stripped ]
Regards,
Willy
From: On Behalf Of Joe Barr
> I'm forwarding this post by the author of a great little program for
> digital amateur radio on Linux, because I'm curious whether or not the
> problem he is seeing can be resolved outside the kernel.
From: w1hkj [mailto:[email protected]]
> I am now convinced that the current serial port drivers
> available to us
> on the Linux platform WILL NOT support CW and/or RTTY that is
> software
> generated in a satisfactory manner.
I don't know what FSK/CW/RTTY/BAUDOT is.
> Direct access to the serial port(s) is a kernel perogative in Linux.
> Only kernel level drivers are allowed such port access.
Not true.
> Until such time as new information becomes available I am going to
> comment out all references to CW and / or FSK via RTS/DTR. I also
> question how useful the FSK on TxD (UART derived) might be to
> most users
> since the 45.45 baudrate is not available in the serial port driver.
> That function will also be commented out.
You may be confusing the old-style baud rate lookup table (B9600 et
al) with the actual capabilities of the serial port. The lookup table
has a limited number of baud rates. You can get more rates than that
using a custom divisor. Most motherboard-based serial ports are
clocked at 1.8432 Mhz. The UART does 16 samples per bit time, yielding
a max baud rate of 115200.
115200 / 25 yields 4608, which is a 1.4% error rate from 4545. This
may or may not be acceptable. 115200 / 2535 yields 45.44, which is a
0.01% error rate, which is likely acceptable.
> Sorry folks, but we win some and lose some.
We make serial port boards with very flexible UARTS. 4545 exactly is
achievable. 45.45 too. Linux supported.
..Stu
--
We make multiport serial products.
http://www.connecttech.com/
(800) 426-8979
On Sat, Jan 20, 2007 at 06:36:44PM +0100, Willy Tarreau wrote:
> On Fri, Jan 19, 2007 at 03:37:34PM -0600, Joe Barr wrote:
> >
> > I'm forwarding this post by the author of a great little program for
> > digital amateur radio on Linux, because I'm curious whether or not the
> > problem he is seeing can be resolved outside the kernel.
>
> At least, I see one wrong claim and one unexplored track in his report.
> The wrong claim : the serial port can only be controled by the kernel.
> It is totally wrong for true serial ports. If he does not want to use
> ioctl(), then he can directly program the I/O port.
There's more wrong with his claim than just that. Another wrong claim
is that it's caused by the Linux kernel not treating ioctl requests
with high priority. Of course that's nonsense. It might be the case
if we were using brain-damaged messaging-passing approach like what
Andrew Tenenbaum is proposing with Minix 3.1, but in Linux, the serial
port DTR/CTS lines are toggled as soon as the userspace executes the
ioctl.
The real issue is when does the userspace program get a chance to run.
He's using the select() system call, which only guarantees accuracy up
to the granularity of the system clock. Given that he's reporting a
jitter of between 0 and 4ms, I'm guessing that he's running with a
system clock tick of 250HZ (since 1/250 == 4ms ).
So if he wants accuracy greater than that, there are a couple of
things he can do. One is to recompile his kernel with HZ=1000. That
will give him accuracy up to 1ms or so. If he needs better than 1ms
granularity, there are two options. One is use sched_setscheduler()
to enable posix soft-realtime, and then calibrate a busy loop. This
will of course burn power and completely busy out one CPU, so if he
needs to run CW continuously this probably isn't a great solution. On
an SMP system it might work, although it is obviously a huge kludge.
The other choice would be to install Ingo's -rt patches (see
http://rt.wiki.kernel.org for more information), and then use the
Posix high-resolution timer API's (i.e., timer_create, et. al). Make
sure you enable CONFIG_HIGH_RES_TIMERS after you apply the patch. It
would also be a good idea to set a real-time scheduling priority for
the application, to make sure that when the timer goes off, the
process doesn't get preempted by some background cron job.
> Now he must be careful about avoiding busy loops in the rest of the
> program, or he will have to use the reset button.
An easy way of dealing with this is to have an sshd running
an alternative port running at a nice high priority (say, prio 95 or
so). That way, if you screw up, you can always login remotely and
kill the offending program.
There is also a RT Watchdog program which can be found on
rt.wiki.kernel.org which can be used to recover from runaway real-time
processes without needing to hit the reset button.
Finally, please feel free to direct your amateur radio friend to the
[email protected]. There are plenty of folks there who
would be very happy to help him out.
73 de Ted, N1ZSU
On Sun, Jan 21, 2007 at 12:54:56AM -0500, Theodore Tso wrote:
> On Sat, Jan 20, 2007 at 06:36:44PM +0100, Willy Tarreau wrote:
> > On Fri, Jan 19, 2007 at 03:37:34PM -0600, Joe Barr wrote:
> > >
> > > I'm forwarding this post by the author of a great little program for
> > > digital amateur radio on Linux, because I'm curious whether or not the
> > > problem he is seeing can be resolved outside the kernel.
> >
> > At least, I see one wrong claim and one unexplored track in his report.
> > The wrong claim : the serial port can only be controled by the kernel.
> > It is totally wrong for true serial ports. If he does not want to use
> > ioctl(), then he can directly program the I/O port.
>
> There's more wrong with his claim than just that. Another wrong claim
> is that it's caused by the Linux kernel not treating ioctl requests
> with high priority. Of course that's nonsense. It might be the case
> if we were using brain-damaged messaging-passing approach like what
> Andrew Tenenbaum is proposing with Minix 3.1, but in Linux, the serial
> port DTR/CTS lines are toggled as soon as the userspace executes the
> ioctl.
Damn you're right. It shocked me too and I know I was missing something
when replying but I did not remember what.
> The real issue is when does the userspace program get a chance to run.
> He's using the select() system call, which only guarantees accuracy up
> to the granularity of the system clock. Given that he's reporting a
> jitter of between 0 and 4ms, I'm guessing that he's running with a
> system clock tick of 250HZ (since 1/250 == 4ms ).
Yes, that's what I thought too. In the past, I've been having better
resolution with select() and real-time scheduling, but I cannot reliably
reproduce it, even on SMP. I remember nothing was running at all on the
machine (not even X) and that can make an important difference. But as
you say, there will be no guarantee of better accuracy anyway.
> So if he wants accuracy greater than that, there are a couple of
> things he can do. One is to recompile his kernel with HZ=1000. That
> will give him accuracy up to 1ms or so. If he needs better than 1ms
> granularity, there are two options. One is use sched_setscheduler()
> to enable posix soft-realtime, and then calibrate a busy loop. This
> will of course burn power and completely busy out one CPU, so if he
> needs to run CW continuously this probably isn't a great solution. On
> an SMP system it might work, although it is obviously a huge kludge.
Hmmm the busy loop is dirty as hell, even on SMP, but it works ;-)
I remember is was possible to reprogram the RTC to interrupt at 8192 Hz.
If the task is running with real time prio, it should get this accuracy,
or am I mistaken ?
> The other choice would be to install Ingo's -rt patches (see
> http://rt.wiki.kernel.org for more information), and then use the
> Posix high-resolution timer API's (i.e., timer_create, et. al). Make
> sure you enable CONFIG_HIGH_RES_TIMERS after you apply the patch. It
> would also be a good idea to set a real-time scheduling priority for
> the application, to make sure that when the timer goes off, the
> process doesn't get preempted by some background cron job.
>
> > Now he must be careful about avoiding busy loops in the rest of the
> > program, or he will have to use the reset button.
>
> An easy way of dealing with this is to have an sshd running
> an alternative port running at a nice high priority (say, prio 95 or
> so). That way, if you screw up, you can always login remotely and
> kill the offending program.
>
> There is also a RT Watchdog program which can be found on
> rt.wiki.kernel.org which can be used to recover from runaway real-time
> processes without needing to hit the reset button.
Thanks for those hints, I've been used to play with the reset button,
at least it has forced me to double check my code before running it :-)
> Finally, please feel free to direct your amateur radio friend to the
> [email protected]. There are plenty of folks there who
> would be very happy to help him out.
>
> 73 de Ted, N1ZSU
Regards,
Willy
Joe Barr wrote:
> I'm forwarding this post by the author of a great little program for
> digital amateur radio on Linux, because I'm curious whether or not the
> problem he is seeing can be resolved outside the kernel.
>
> All comments welcome on/off list.
>
> Thanks,
> Joe Barr
> K1GPL
[...]
>
> I've spent the last day staring at the oscilloscope and pins RTS and DTR
> on the serial output for 4 different computers running 4 different
> versions of Linux. Also have exhausted the search on the internet for
> information regarding both the latency and jitter associated with ioctl
> calls to the serial driver (both ttyS and ttyUSB). I'm sure it is out
> there somewhere, I just cannot find it.
>
> I am now convinced that the current serial port drivers available to us
> on the Linux platform WILL NOT support CW and/or RTTY that is software
> generated in a satisfactory manner.
>
> To test the latency and jitter of the ioctl calls to set or clear RTS
> and / or DTR I built a basic square wave generator with microsecond
> timing precision. The timing could be derived either from the select
> system call or by controlled i/o to the sound card. Both provide very
> precise timing of the program loop. Each time through the loop either
> the RTS/DTR was set or cleared. The timing jitter for each 1/2 cycle
> was from 0 to +4 msec. This varied between systems as each had
> different cpu clock rates. The jitter is caused by the asynchronous
> response of the kernel to the request to control the port. ioctl
> requests apparantly do not have a very high priority for the kernel.
> They are probably just serviced by a first-in first-out interrupt
> service request loop. That type of jitter is tolerable up to about 20
> wpm CW. It totally wipes out the ability to generate an FSK signal on
> the DTR or RTS pin.
Okay, here he's using bit-banging of the DTR and RTS pins to generate a
fairly high precision output wave. This is not really the
> Direct access to the serial port(s) is a kernel perogative in Linux.
> Only kernel level drivers are allowed such port access.
So write a kernel driver. It's not like we're locking anybody out.
There is certainly enough Amateur Radio/Linux crossover that a kernel
enhancement to support Amateur Radio is going to get frowned upon.
> So ... bottom line is that all of my attempts over the past couple of
> months to provide CW and / or FSK output signal have been to fraught
> with pitfalls. The CW seems OK for slow speed keying, but the FSK seems
> impossible to achieve.
>
> The FSK using the UART is also limited by the Linux operating system and
> the current drivers. That limitation excludes the use of 45 or 56 baud
> BAUDOT.
That is true at the moment (due to unfortunate design choices made early
on), but this is already in the process of being changed:
http://lkml.org/lkml/2006/10/18/280
> Until such time as new information becomes available I am going to
> comment out all references to CW and / or FSK via RTS/DTR. I also
> question how useful the FSK on TxD (UART derived) might be to most users
> since the 45.45 baudrate is not available in the serial port driver.
> That function will also be commented out.
>
> All this should not really come as a surprise since Linux is not a
> real-time operating system. By the way, I did try the tests with the
> test program running with nice -20. Not much difference.
See again how he should be using real-time priority rather than nice -20.
> Sorry folks, but we win some and lose some.
>
> 73, Dave, W1HKJ
-hpa
Joe Barr wrote:
> I'm forwarding this post by the author of a great little program for
> digital amateur radio on Linux, because I'm curious whether or not the
> problem he is seeing can be resolved outside the kernel.
>
> All comments welcome on/off list.
>
> Thanks,
> Joe Barr
> K1GPL
[...]
>
> I've spent the last day staring at the oscilloscope and pins RTS and DTR
> on the serial output for 4 different computers running 4 different
> versions of Linux. Also have exhausted the search on the internet for
> information regarding both the latency and jitter associated with ioctl
> calls to the serial driver (both ttyS and ttyUSB). I'm sure it is out
> there somewhere, I just cannot find it.
>
> I am now convinced that the current serial port drivers available to us
> on the Linux platform WILL NOT support CW and/or RTTY that is software
> generated in a satisfactory manner.
>
> To test the latency and jitter of the ioctl calls to set or clear RTS
> and / or DTR I built a basic square wave generator with microsecond
> timing precision. The timing could be derived either from the select
> system call or by controlled i/o to the sound card. Both provide very
> precise timing of the program loop. Each time through the loop either
> the RTS/DTR was set or cleared. The timing jitter for each 1/2 cycle
> was from 0 to +4 msec. This varied between systems as each had
> different cpu clock rates. The jitter is caused by the asynchronous
> response of the kernel to the request to control the port. ioctl
> requests apparantly do not have a very high priority for the kernel.
> They are probably just serviced by a first-in first-out interrupt
> service request loop. That type of jitter is tolerable up to about 20
> wpm CW. It totally wipes out the ability to generate an FSK signal on
> the DTR or RTS pin.
Okay, here he's using bit-banging of the DTR and RTS pins to generate a
fairly high precision output wave. These bits are being used as GPIOs,
and would need very precise control. This is much worse for USB serial
ports than for ordinary serial ports.
> Direct access to the serial port(s) is a kernel perogative in Linux.
> Only kernel level drivers are allowed such port access.
So write a kernel driver. It's not like we're locking anybody out.
There is certainly enough Amateur Radio/Linux crossover that a kernel
enhancement to support Amateur Radio is going to get frowned upon.
> So ... bottom line is that all of my attempts over the past couple of
> months to provide CW and / or FSK output signal have been to fraught
> with pitfalls. The CW seems OK for slow speed keying, but the FSK seems
> impossible to achieve.
>
> The FSK using the UART is also limited by the Linux operating system and
> the current drivers. That limitation excludes the use of 45 or 56 baud
> BAUDOT.
That is true at the moment (due to unfortunate design choices made early
on), but this is already in the process of being changed:
http://lkml.org/lkml/2006/10/18/280
> Until such time as new information becomes available I am going to
> comment out all references to CW and / or FSK via RTS/DTR. I also
> question how useful the FSK on TxD (UART derived) might be to most users
> since the 45.45 baudrate is not available in the serial port driver.
> That function will also be commented out.
>
> All this should not really come as a surprise since Linux is not a
> real-time operating system. By the way, I did try the tests with the
> test program running with nice -20. Not much difference.
See again how he should be using real-time priority rather than nice -20.
> Sorry folks, but we win some and lose some.
>
> 73, Dave, W1HKJ
-hpa
On Sun, Jan 21, 2007 at 08:05:57AM +0100, Willy Tarreau wrote:
> On Sun, Jan 21, 2007 at 12:54:56AM -0500, Theodore Tso wrote:
> > On Sat, Jan 20, 2007 at 06:36:44PM +0100, Willy Tarreau wrote:
> >
> > > Now he must be careful about avoiding busy loops in the rest of the
> > > program, or he will have to use the reset button.
> >
> > An easy way of dealing with this is to have an sshd running
> > an alternative port running at a nice high priority (say, prio 95 or
> > so). That way, if you screw up, you can always login remotely and
> > kill the offending program.
> >
> > There is also a RT Watchdog program which can be found on
> > rt.wiki.kernel.org which can be used to recover from runaway real-time
> > processes without needing to hit the reset button.
>
> Thanks for those hints, I've been used to play with the reset button,
> at least it has forced me to double check my code before running it :-)
I think SysRq-N sets all processes with RT-priority to non-RT.
Johannes
On Sun, Jan 21, 2007 at 03:50:32PM +0100, Grzegorz Ja?kiewicz wrote:
> funny, how even very fast box can be useless for such activities with modern
> kernels, such as linux. After all, serial ports don't require too much
> horsepower to be highly accurate.
> >From designer perspective, I would rather go for uC solution in his case.
> That is, design and build fairly simple board - based on atmega AVRs. 16mhz
> 8bit chip can do so much better compared to N gigs mhz box running "modern"
> operating system, these days....
It is true, but the point is not "modern" vs "ancient" OS, it's "multitasking"
vs "monotasking" kernel. If your program does the busy loop, you'll get a
very high accuracy at the expense of burning watts polling one bit one
billion times a second waiting for a change every millisecond.
18 years ago, I wrote a soft-only 8250 emulator to connect my second 8088
to the first one's serial port at 19200 bauds. At this time, waiting for
a change only took a few hundred cycles and the busy loop was the default
mode of the OS anyway. Things have changed since.
Willy
On Sun, Jan 21, 2007 at 04:10:58PM +0100, Grzegorz Jaśkiewicz wrote:
> you're right, I used wrong term to describe.
> But the problem still exists. Nowadays it should be possible to run many
> serial ports fully accurate "at the same time". It is also true that the
> same problem exists in windows, and bsd worlds. So it is not only Linux
> problem.
It's not so much a "problem", but rather, most people aren't
particularly interested in using the millions and millions of
transisters of a modern CPU to generate a square wave. So OS's aren't
optimized to do that. Linux has no problems running many serial
ports; but in their normal designed function, which is to send and
receive characters, using the UART's FIFO's and interrupts to do so.
The question of manually toggling DTR/CTS RS-232 lines to generate a
tone is something that you *can* do, but you won't be doing anything
else --- and that's simply because fundamentally that's a very silly
thing to do. Most people will use a tiny amount of dedicated hardware
--- like the sound care that is built into every single modern PC ---
rather than manually waggling the RS-232 lines in order to generate a
tone.
> Like I said previously, 30$ board (usb+avr+max232) would do it accurately +
> over 300$ PC to control it :D funny...
You *can* do it, and we've described how to do it. It won't be
efficient (you won't be doing much else), but that's because a PC and
Linux is optimized for different set of tasks. Sometimes dedicated
hardware is the far superior option to a general purpose OS and a
general-purpose hardware. This should not at all be surprising.
- Ted
On Sun, Jan 21, 2007 at 08:05:57AM +0100, Willy Tarreau wrote:
> Hmmm the busy loop is dirty as hell, even on SMP, but it works ;-)
> I remember is was possible to reprogram the RTC to interrupt at 8192 Hz.
> If the task is running with real time prio, it should get this accuracy,
> or am I mistaken ?
You can do in a kernel module. The problem with a userspace program
is that without the -rt patches, you can't guarantee that userspace
process will get a chance to run in time.
- Ted
On Sun, Jan 21, 2007 at 08:05:52PM +0100, Grzegorz Jaśkiewicz wrote:
>
> But on the other hand, PC nowadays are capable of handling RT tasks +
> running multiple programs in background, but OS has to be build from ground
> up to handle such conditions (I guess most RTOSes are justa about that, but
> I never had a chance/need to use one - I tend to use loads of AVRs just for
> those small tasks).
Check out http://rt.wiki.kernel.org
- Ted
H. Peter Anvin wrote:
>
> So write a kernel driver. It's not like we're locking anybody out.
> There is certainly enough Amateur Radio/Linux crossover that a kernel
> enhancement to support Amateur Radio is going to get frowned upon.
>
Argh! Not only did the message go out twice, but that should have been
"is *not* going to get frowned upon..."
-hpa
Serial port latency is heavily dependant on the HZ rate for data bits
and input side stuff and you can set the low latency flag to improve upon
that. Beyond that if you are using the modem control ioctls then it
depends a lot on the hardware. USB has some implicit queuing on the bus
but generic uarts have very little on the whole.
You should be able to get much better results by using
mlockall(MCL_FUTURE) on the actual test process and setting the priority
into the real time range, in combination with turning on low latency on
the motherboard ports.
The current -mm kernels also support arbitary baud rate (well 45 or 50
rather than 45.5), although this hasn't yet been enabled for all
platforms or pushed into the base kernel for i386 yet. It will be soon
however and then glibc can be tweaked to use it by default.
Alan