2005-11-17 16:30:49

by Dag Nygren

[permalink] [raw]
Subject: nanosleep with small value


Hi,

seeing a strange thing happening here:
using nanosleep() with a smallish value gives me a very long sleeptime?

Is this because of a context switch being forced?
Shouldn't the scheduler change affect that?

The test program:
===================================
#include <time.h>
#include <sched.h>
#include <stdio.h>

void delay_ns(unsigned long dly)
{
static struct timespec time;
int err;
{
time.tv_sec = 0;
time.tv_nsec = dly;
err = nanosleep(&time, NULL);
if (err) {
perror( "nanosleep failed" );
}
}
}


main()
{
int i;

struct sched_param mysched;
int err;

if ( sched_getparam( 0, &mysched ) != 0 )
perror( "" );
else {
mysched.sched_priority = sched_get_priority_max(SCHED_FIFO);
err = sched_setscheduler(0, SCHED_FIFO, &mysched);
if( err != 0 ) {
fprintf (stderr,"sched_setscheduler returned: %d\n",
err );
perror( "" );
}
}

for (i=0; i < 1000; i++)
delay_ns(1000UL);
}
==================================
The result running this is:
% time ./tst

real 0m8.000s
user 0m0.000s
sys 0m0.000s

I would have expected about 1000 * 1 us + overhead,
but 8 seconds ????

Noticed this when trying to debug a PIC-programming
software where the delay_ns() routine is used.

Increasing the nanosleep() argument to something more than HZ
will give me the expected sleep times.

Best

--
Dag Nygren email: [email protected]
Oy Espoon NewTech Ab phone: +358 9 8024910
Tr?sktorpet 3 fax: +358 9 8024916
02360 ESBO Mobile: +358 400 426312
FINLAND



2005-11-17 16:55:18

by linux-os (Dick Johnson)

[permalink] [raw]
Subject: Re: nanosleep with small value


On Thu, 17 Nov 2005, Dag Nygren wrote:

>
> Hi,
>
> seeing a strange thing happening here:
> using nanosleep() with a smallish value gives me a very long sleeptime?
>
> Is this because of a context switch being forced?
> Shouldn't the scheduler change affect that?
>
> The test program:
> ===================================
> #include <time.h>
> #include <sched.h>
> #include <stdio.h>
>
> void delay_ns(unsigned long dly)
> {
> static struct timespec time;
> int err;
> {
> time.tv_sec = 0;
> time.tv_nsec = dly;
> err = nanosleep(&time, NULL);
> if (err) {
> perror( "nanosleep failed" );
> }
> }
> }
>
>
> main()
> {
> int i;
>
> struct sched_param mysched;
> int err;
>
> if ( sched_getparam( 0, &mysched ) != 0 )
> perror( "" );
> else {
> mysched.sched_priority = sched_get_priority_max(SCHED_FIFO);
> err = sched_setscheduler(0, SCHED_FIFO, &mysched);
> if( err != 0 ) {
> fprintf (stderr,"sched_setscheduler returned: %d\n",
> err );
> perror( "" );
> }
> }
>
> for (i=0; i < 1000; i++)
> delay_ns(1000UL);
> }
> ==================================
> The result running this is:
> % time ./tst
>
> real 0m8.000s
> user 0m0.000s
> sys 0m0.000s

On an unprivilged account, I get this with
version 2.6.13.4

Script started on Thu 17 Nov 2005 11:44:47 AM EST
LINUX> time ./xxx
sched_setscheduler returned: -1
Operation not permitted

real 0m2.000s
user 0m0.000s
sys 0m0.001s
LINUX> uname -r
2.6.13.4
LINUX> exit
Script done on Thu 17 Nov 2005 11:45:07 AM EST

>From the root account where the scheduler could be set:

Script started on Thu 17 Nov 2005 11:45:29 AM EST
[root@chaos root]# time /tmp/xxx

real 0m2.001s
user 0m0.000s
sys 0m0.001s
[root@chaos root]# exit

Script done on Thu 17 Nov 2005 11:45:53 AM EST

... essentially the same thing. And 2 seconds, not 8.

The HZ value for my kernel is 1000. It you are at 100 HZ,
that might explain it.

Note that nanosleep() doesn't claim to be able to sleep
less than the resolution of some kernel timer. It just takes
parameters in seconds and nanoseconds.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.44 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

2005-11-17 17:12:59

by Nish Aravamudan

[permalink] [raw]
Subject: Re: nanosleep with small value

On 11/17/05, Dag Nygren <[email protected]> wrote:
>
> Hi,
>
> seeing a strange thing happening here:
> using nanosleep() with a smallish value gives me a very long sleeptime?
>
> Is this because of a context switch being forced?
> Shouldn't the scheduler change affect that?
>
> The test program:
> ===================================
> #include <time.h>
> #include <sched.h>
> #include <stdio.h>
>
> void delay_ns(unsigned long dly)
> {
> static struct timespec time;
> int err;
> {
> time.tv_sec = 0;
> time.tv_nsec = dly;
> err = nanosleep(&time, NULL);
> if (err) {
> perror( "nanosleep failed" );
> }
> }
> }
>
>
> main()
> {
> int i;
>
> struct sched_param mysched;
> int err;
>
> if ( sched_getparam( 0, &mysched ) != 0 )
> perror( "" );
> else {
> mysched.sched_priority = sched_get_priority_max(SCHED_FIFO);
> err = sched_setscheduler(0, SCHED_FIFO, &mysched);
> if( err != 0 ) {
> fprintf (stderr,"sched_setscheduler returned: %d\n",
> err );
> perror( "" );
> }
> }
>
> for (i=0; i < 1000; i++)
> delay_ns(1000UL);
> }
> ==================================
> The result running this is:
> % time ./tst
>
> real 0m8.000s
> user 0m0.000s
> sys 0m0.000s
>
> I would have expected about 1000 * 1 us + overhead,
> but 8 seconds ????

Which kernel, what value of HZ? In either case, it's absurd to assume
that the kernel is going to provide you 1 microsecond resolution in
2.6 mainline, as the best HZ value is 1000 (1 millisecond). And we
don't busy-wait ever in nanosleep(). So the fastest your loop can run
is 1000 * 1 ms = 1 second. That's assuming the only time-consuming
thing is sleeping (minimal overhead). But, in sys_nanosleep(), we
convert nanoseconds to jiffies and add 1 if you requested any sleep
time. So,

HZ = 100
1000 * (10 + 1 ms) = 11 s
HZ = 250
1000 * (4 + 1 ms) = 5 s
HZ = 1000
1000 * (1 + 1 ms) = 2 s (which is what Dick Johnson reported).

Note, that with HZ=250, there might be some extra rounding occurring
timespec_to_jiffies() that I've forgotten.

So 8 s may not be terribly unreasonable. I don't know, though, what's
add the 3 seconds if you're using 250.

Thanks,
Nish

2005-11-17 17:32:38

by Eric Piel

[permalink] [raw]
Subject: Re: nanosleep with small value

11/17/2005 05:55 PM, linux-os (Dick Johnson) wrote/a écrit:
> On Thu, 17 Nov 2005, Dag Nygren wrote:
:
>>
>>real 0m8.000s
>>user 0m0.000s
>>sys 0m0.000s
>
>
> On an unprivilged account, I get this with
> version 2.6.13.4
>
> Script started on Thu 17 Nov 2005 11:44:47 AM EST
> LINUX> time ./xxx
> sched_setscheduler returned: -1
> Operation not permitted
>
> real 0m2.000s
> user 0m0.000s
> sys 0m0.001s
> LINUX> uname -r
> 2.6.13.4
> LINUX> exit
> Script done on Thu 17 Nov 2005 11:45:07 AM EST
>
>>From the root account where the scheduler could be set:
>
> Script started on Thu 17 Nov 2005 11:45:29 AM EST
> [root@chaos root]# time /tmp/xxx
>
> real 0m2.001s
> user 0m0.000s
> sys 0m0.001s
> [root@chaos root]# exit
>
> Script done on Thu 17 Nov 2005 11:45:53 AM EST
>
> ... essentially the same thing. And 2 seconds, not 8.
>
> The HZ value for my kernel is 1000. It you are at 100 HZ,
> that might explain it.
No, it means he is at 250 HZ.


>
> Note that nanosleep() doesn't claim to be able to sleep
> less than the resolution of some kernel timer. It just takes
> parameters in seconds and nanoseconds.
>
In general small nanosleep() returns between 1/HZ and 2/HZ. In this
code, the sleep is (indirectly) synchronised on a tick so it's always
2/HZ, the worse!

1000 small nanosleeps take therefore 1000 * 2/HZ s = 8s :-)

Eric

2005-11-17 18:47:30

by Dag Nygren

[permalink] [raw]
Subject: Re: nanosleep with small value

> On 11/17/05, Dag Nygren <[email protected]> wrote:

>
> Which kernel, what value of HZ?

Sorry, the kernel is 2.6.13 and HZ is 250.

> In either case, it's absurd to assume
> that the kernel is going to provide you 1 microsecond resolution in
> 2.6 mainline, as the best HZ value is 1000 (1 millisecond). And we
> don't busy-wait ever in nanosleep().

Not?
The man page for nanosleep saya that times under 2 us are implemented
by a busywait and this is why I expected it to work.

> So the fastest your loop can run
> is 1000 * 1 ms = 1 second. That's assuming the only time-consuming
> thing is sleeping (minimal overhead). But, in sys_nanosleep(), we
> convert nanoseconds to jiffies and add 1 if you requested any sleep
> time. So,
>
> HZ = 100
> 1000 * (10 + 1 ms) = 11 s
> HZ = 250
> 1000 * (4 + 1 ms) = 5 s
> HZ = 1000
> 1000 * (1 + 1 ms) = 2 s (which is what Dick Johnson reported).
>
> Note, that with HZ=250, there might be some extra rounding occurring
> timespec_to_jiffies() that I've forgotten.
>
> So 8 s may not be terribly unreasonable. I don't know, though, what's
> add the 3 seconds if you're using 250.

OK, in that case the manpage should be changed. And an alternative
has to be worked out by me ;-).

Thankyou
Dag

2005-11-17 19:11:22

by Dag Nygren

[permalink] [raw]
Subject: Re: nanosleep with small value

> On 11/17/05, Dag Nygren <[email protected]> wrote:

> > The man page for nanosleep saya that times under 2 us are implemented
> > by a busywait and this is why I expected it to work.
>
> Update your manpages. You're depending on 2.4 behavior in a 2.6 kernel.

You are right. The system is one I have upgraded piece by piece and the
manpages
weren't upgraded.

But what is the point of having a nanosleep() in that case when you could do
just fine with usleep() ?

> > OK, in that case the manpage should be changed. And an alternative
> > has to be worked out by me ;-).
>
> My man-pages are quite clear on what nanosleep() does. Nothing needs
> to be changed there.
>
> Alternative wise, I'm not sure, but you might want to look into the
> HRT stuff that's going on in Ingo's -RT tree. I don't know if / what
> changes have been made to sys_nanosleep(), but low-latency is most
> likely to occur there.

I will look into that.
Quite annoying that software that worked just fine in 2.4 doesn't
work in 2.6.

What does POSIX say about nanosleep()?

Dag

2005-11-17 19:15:32

by Randy Dunlap

[permalink] [raw]
Subject: Re: nanosleep with small value

On Thu, 17 Nov 2005, Dag Nygren wrote:

> > On 11/17/05, Dag Nygren <[email protected]> wrote:
>
> > > The man page for nanosleep saya that times under 2 us are implemented
> > > by a busywait and this is why I expected it to work.
> >
> > Update your manpages. You're depending on 2.4 behavior in a 2.6 kernel.
>
> You are right. The system is one I have upgraded piece by piece and the
> manpages
> weren't upgraded.
>
> But what is the point of having a nanosleep() in that case when you could do
> just fine with usleep() ?
>
> > > OK, in that case the manpage should be changed. And an alternative
> > > has to be worked out by me ;-).
> >
> > My man-pages are quite clear on what nanosleep() does. Nothing needs
> > to be changed there.
> >
> > Alternative wise, I'm not sure, but you might want to look into the
> > HRT stuff that's going on in Ingo's -RT tree. I don't know if / what
> > changes have been made to sys_nanosleep(), but low-latency is most
> > likely to occur there.
>
> I will look into that.
> Quite annoying that software that worked just fine in 2.4 doesn't
> work in 2.6.
>
> What does POSIX say about nanosleep()?

Maybe you want to add/patch your kernel with the high-res-timer
patch from
http://sourceforge.net/projects/high-res-timers/

--
~Randy

2005-11-17 19:17:51

by Nish Aravamudan

[permalink] [raw]
Subject: Re: nanosleep with small value

On 11/17/05, Dag Nygren <[email protected]> wrote:
> > On 11/17/05, Dag Nygren <[email protected]> wrote:
>
> > > The man page for nanosleep saya that times under 2 us are implemented
> > > by a busywait and this is why I expected it to work.
> >
> > Update your manpages. You're depending on 2.4 behavior in a 2.6 kernel.
>
> You are right. The system is one I have upgraded piece by piece and the
> manpages
> weren't upgraded.

No problem.

> But what is the point of having a nanosleep() in that case when you could do
> just fine with usleep() ?

Check the usleep() manpage:

This function is obsolete. Use nanosleep(2) or setitimer(2) instead.

And in any case, I think usleep() just ends up calling nanosleep()?
It's not a sys-call in an of itself, like sys_nanosleep().

> > > OK, in that case the manpage should be changed. And an alternative
> > > has to be worked out by me ;-).
> >
> > My man-pages are quite clear on what nanosleep() does. Nothing needs
> > to be changed there.
> >
> > Alternative wise, I'm not sure, but you might want to look into the
> > HRT stuff that's going on in Ingo's -RT tree. I don't know if / what
> > changes have been made to sys_nanosleep(), but low-latency is most
> > likely to occur there.
>
> I will look into that.
> Quite annoying that software that worked just fine in 2.4 doesn't
> work in 2.6.

Well, your general resolution also was improved quite a bit in 2.6
(HZ=1000 vs. HZ=100 is a 10-fold improvement). But I agree, is a big
difference if you depend on that udelay() functionality -- but it was
a delay of up to 2 milliseconds, which is generally frowned upon in
the kernel.

> What does POSIX say about nanosleep()?

Not sure, but I think the only requirement is that we don't return
early (i.e. request 2 milliseconds, return in 1 millisecond).

Thanks,
Nish

2005-11-17 19:47:18

by Frank Sorenson

[permalink] [raw]
Subject: Re: nanosleep with small value

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dag Nygren wrote:
> But what is the point of having a nanosleep() in that case when you could do
> just fine with usleep() ?

I'd suggest looking into the kthrt patches (which incorporates ktimers
and John Stultz's timeofday patches): http://www.tglx.de/projects/ktimers/

Running your program, here are some results (latest git tree with the
latest kthrt and timeofday patches):

shortest of 10 runs as non-root:
real 0m0.418s
user 0m0.000s
sys 0m0.003s

longest of 10 runs as non-root:
real 0m0.794s
user 0m0.000s
sys 0m0.002s

shortest of 10 runs as root:
real 0m0.066s
user 0m0.001s
sys 0m0.007s

longest of 10 runs as root:
real 0m0.325s
user 0m0.000s
sys 0m0.004s

Frank
- --
Frank Sorenson - KD7TZK
Systems Manager, Computer Science Department
Brigham Young University
[email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFDfN49aI0dwg4A47wRAoHxAKDTeMGGnv21qem2Ll+SG8x5q+pV7ACgpUiT
ru0P0KXOet7eNJhLYNRJvpk=
=7IFW
-----END PGP SIGNATURE-----

2005-11-17 20:25:06

by Dag Nygren

[permalink] [raw]
Subject: Re: nanosleep with small value


OK,

thanks to all that sorted this out for me. What mislead me
big time was my old manpage for nanosleep() which claimed
to busywait for short nanosleeps().

2.6 doesn't seem to do that.

I will look into the suggested patches. Just hate not being able
to just grab the latest kernel from kernel.org and throw it in there.
Already have to recompile a Matrox display module. Now this would
be an additional problem nuisance.

But once again thanks to all
Dag

2005-11-21 07:12:16

by Dag Nygren

[permalink] [raw]
Subject: Re: nanosleep with small value


> Dag Nygren wrote:
> > But what is the point of having a nanosleep() in that case when you could do
> > just fine with usleep() ?
>
> I'd suggest looking into the kthrt patches (which incorporates ktimers
> and John Stultz's timeofday patches): http://www.tglx.de/projects/ktimers/

Thanks for that tip. This completely solved my problem; The program works
again.

There seems to be two "competing" patches out there, but this seems to be the
way
to go as the kernel bunch is behind it.

Best

Dag