With the current scheduler we can prioritize the CPU usage for each process.
What I think would be extremely useful (as I have needed it many times) is the
scheduling of disk I/O and net I/O traffic. 2 examples showing the importance
(the numbers are estimations just to explain whati I mean):
1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of
bandwidth available. What I want to do is listen to icecast radio via xmms (at
22 kbits), download the kernel sources with wget, and browse the web at the same
time. Currently I think that this is *impossible* (correct me if I'm wrong) as
the radio will be full of pauses and the browsing experience painfully slow.
What I would like to be able to do (let's suppose nice has the --net option to
set net I/O priority):
$ nice --net -1 xmms
$ nice --net 1 wget ftp://.../KernelSources.tar.bz2
$ mozilla
This way, xmms which has top priority whould always get the 22kbits it needs.
What remains should go to the browser when I ask for a web page, and when the
browser doesn't request anything (let's say I'm reading a big doc in tldp) what
remains should go to wget. Wget has lower priority and won't irritate the
browsing experience, though the file will be downloaded when there is free
bandwidth.
2) Normally mozilla starts in 5 seconds after intense disk I/O to load all
needed libraries. If I run in the background a long disk intense process
(like find / -name 'whatever' -xdev) loading mozilla could need 20 boring
seconds, or doing other simple tasks might be irritating slow. What I would like
to be able to do is (once again let's suppose nice has the --disk option to set
disk I/O priority):
$ nice --disk 1 find / -name 'whatever' -xdev
$ mozilla
and load mozilla ,which has the default disk priority 0, fast. The scheduler
should give to mozilla most disk troughput when it needs it.
Notes:
1) PLEASE CC REPLIES BACK TO ME since I 'm not subscribed to the list (I can't
stand the traffic). However I 'll be checking periodically the list via NNTP.
2) As I have no idea of kernel programming I hope what I propose is aplicable
and relevant to the kernel, as I believe. Sorry if not.
3) I hope what I propose is implementable using the existing scheduler. It would
be nice to have one scheduler to handle them all.
4) I believe that these features don't need a lot of CPU power because the disk
and net I/O troughput are relatively slow.
5) If you think that UNIX tradition forbits what I propose I must say that these
features could be invisible to programs, setting the new priorities to the
default 0 or maybe to the same number as CPU priority.
Thank you very much for your time,
Dimitris
On Wed, Jul 23, 2003 at 01:57:41PM +0300, [email protected] wrote:
> 1)I 'm connected to the internet via dial-up, therefore I only have 40
> kbits of bandwidth available. What I want to do is listen to icecast radio
> via xmms (at 22 kbits), download the kernel sources with wget, and browse
> the web at the same time. Currently I think that this is *impossible*
> (correct me if I'm wrong) as the radio will be full of pauses and the
> browsing experience painfully slow. What I would like to be able to do
> (let's suppose nice has the --net option to set net I/O priority):
> $ nice --net -1 xmms
> $ nice --net 1 wget ftp://.../KernelSources.tar.bz2
> $ mozilla
I think what you need to look at is network-level QoS, this is far too
intrusive for what you are trying to do. Try tldp.org.
David.
[email protected] wrote:
> With the current scheduler we can prioritize the CPU usage for each
> process. What I think would be extremely useful (as I have needed it
> many times) is the scheduling of disk I/O and net I/O traffic. 2
> examples showing the importance (the numbers are estimations just to
> explain whati I mean):
>
can address only your first problem.
> 1)I 'm connected to the internet via dial-up, therefore I only have 40
> kbits of bandwidth available. What I want to do is listen to icecast
> radio via xmms (at 22 kbits), download the kernel sources with wget, and
> browse the web at the same time. Currently I think that this is
> *impossible* (correct me if I'm wrong) as the radio will be full of
> pauses and the browsing experience painfully slow. What I would like to
> be able to do (let's suppose nice has the --net option to set net I/O
> priority):
> $ nice --net -1 xmms
> $ nice --net 1 wget ftp://.../KernelSources.tar.bz2
> $ mozilla
> This way, xmms which has top priority whould always get the 22kbits it
> needs. What remains should go to the browser when I ask for a web page,
> and when the browser doesn't request anything (let's say I'm reading a
> big doc in tldp) what remains should go to wget. Wget has lower priority
> and won't irritate the browsing experience, though the file will be
> downloaded when there is free bandwidth.
>
this is already in kernel and called Quality of Service (QoS) or
traffic shaping.
command is called /sbin/tc. (located in iproute.rpm in RH)
instead of man page (because there is no man page) I can recommend
to look in Internet.
e.g. http://users.belgacom.net/staf/ -> QoS intro
also you can try to "rpm -ql $(which tc)" - probably your distor has
some docs.
Jimis,
I have been thinking about some /etc/nice file where kernel could
pick up process nice
when doing the sys_execve stuff.Maybe it could help ...
Regards,
Fabian
-----Message d'origine-----
De : [email protected] [mailto:[email protected]]
Envoy? : mercredi 23 juillet 2003 12:58
? : [email protected]
Objet : Feature proposal (scheduling related)
With the current scheduler we can prioritize the CPU usage for each process.
What I think would be extremely useful (as I have needed it many times) is
the
scheduling of disk I/O and net I/O traffic. 2 examples showing the
importance
(the numbers are estimations just to explain whati I mean):
1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits
of
bandwidth available. What I want to do is listen to icecast radio via xmms
(at
22 kbits), download the kernel sources with wget, and browse the web at the
same
time. Currently I think that this is *impossible* (correct me if I'm wrong)
as
the radio will be full of pauses and the browsing experience painfully slow.
What I would like to be able to do (let's suppose nice has the --net option
to
set net I/O priority):
$ nice --net -1 xmms
$ nice --net 1 wget ftp://.../KernelSources.tar.bz2
$ mozilla
This way, xmms which has top priority whould always get the 22kbits it
needs.
What remains should go to the browser when I ask for a web page, and when
the
browser doesn't request anything (let's say I'm reading a big doc in tldp)
what
remains should go to wget. Wget has lower priority and won't irritate the
browsing experience, though the file will be downloaded when there is free
bandwidth.
2) Normally mozilla starts in 5 seconds after intense disk I/O to load all
needed libraries. If I run in the background a long disk intense process
(like find / -name 'whatever' -xdev) loading mozilla could need 20 boring
seconds, or doing other simple tasks might be irritating slow. What I would
like
to be able to do is (once again let's suppose nice has the --disk option to
set
disk I/O priority):
$ nice --disk 1 find / -name 'whatever' -xdev
$ mozilla
and load mozilla ,which has the default disk priority 0, fast. The scheduler
should give to mozilla most disk troughput when it needs it.
Notes:
1) PLEASE CC REPLIES BACK TO ME since I 'm not subscribed to the list (I
can't
stand the traffic). However I 'll be checking periodically the list via
NNTP.
2) As I have no idea of kernel programming I hope what I propose is
aplicable
and relevant to the kernel, as I believe. Sorry if not.
3) I hope what I propose is implementable using the existing scheduler. It
would
be nice to have one scheduler to handle them all.
4) I believe that these features don't need a lot of CPU power because the
disk
and net I/O troughput are relatively slow.
5) If you think that UNIX tradition forbits what I propose I must say that
these
features could be invisible to programs, setting the new priorities to the
default 0 or maybe to the same number as CPU priority.
Thank you very much for your time,
Dimitris
On Wed, 23 Jul 2003 13:57:41 +0300, [email protected] said:
> 1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of
> bandwidth available. What I want to do is listen to icecast radio via xmms (at
> 22 kbits), download the kernel sources with wget, and browse the web at the same
> time. Currently I think that this is *impossible* (correct me if I'm wrong) as
> the radio will be full of pauses and the browsing experience painfully slow.
Basically, you're stuck. The biggest part of the problem is that although you
can certainly control the outbound packets, you have no real control over when
inbound packets arrive at the other end of your dial-up. One person suggested
using QoS to help things along - but that needs to be implemented at the OTHER
end of the dial-up - which means unless your provider does QoS on the terminal
server, you're basically stuck. Packets will probably just get queued up in
order of arrival.
And even if the packets do get prioritized so your xmms traffic "goes first",
that's still not going to be perfect, due to a phenomenon canned 'jitter',
which is a measure of how "bursty" the traffic is, and how much *variability*
there is.
Let's say you have xmms going, and a very slow wget. The last 5 packets of
xmms traffic arrived when the modem was idle, and immediately start coming
across, so the jitter is near zero. Now a 6th xmms packet arrives - but the
modem is 3 bytes into transmitting a 1500 byte wget packet. Wham! that packet
gets to wait 300ms for its turn, and you just got 300ms of 'jitter'. If the
xmms didn't have 300ms of sound already buffered, you're screwed and about to
have a drop-out. QoS doesn't help here - the best it can do is guarantee that
you only have to wait for the currently transmitting packet and not for any
already-queued packets. (As an aside - TCP windowing will probably guarantee
that any given connection will only have 1 or 2 packets queued up - but if
you're going to run 5 concurrent data-transfer connections, that's still 5-10
packets. If you don't have QoS, any given packet *could* end up with several
seconds of latency here, and the exact order of arrival could make your jitter
anywhere from 0 to several seconds...
You *might* be able to improve the jitter characteristics by dropping the MTU
on the dialup connection from the 1500 byte range down to 576, but this has its
own problems. First, you're going to be sending 3x as many packets, so the
overhead of packet headers is going to triple, cutting into your effective
bandwidth. Second, there are often issues with windowing, fragmented packets,
and PMTU Discovery when you do this.
> 2) Normally mozilla starts in 5 seconds after intense disk I/O to load all
> needed libraries. If I run in the background a long disk intense process
> (like find / -name 'whatever' -xdev) loading mozilla could need 20 boring
> seconds, or doing other simple tasks might be irritating slow. What I would like
> to be able to do is (once again let's suppose nice has the --disk option to set
> disk I/O priority):
> $ nice --disk 1 find / -name 'whatever' -xdev
> $ mozilla
> and load mozilla ,which has the default disk priority 0, fast. The scheduler
> should give to mozilla most disk troughput when it needs it.
Again, here there be serious and nasty dragons. The problem is that you have
to balance several competing issues:
1) Trying to maintain good overall disk throughput - if you keep pushing
Mozilla's disk requests *right now*, regardless of what the disk is doing, the
disk ends up spending a LOT of time just moving the heads back and forth. Even
at 5ms a shot, seeks are *expensive*, so you often want to re-order the
requests to minimize the distance travelled. Think of it as the difference
between planning your trip to the grocery store, the gas station, the post
office, and the bank to make it shortest, as opposed to going back home after
each one because there's something at home that needs doing Right Now...
2) There's a phenomenon called "starvation". See that 'find' command in your
example? If mozilla is disk-hungry enough, bad I/O scheduling can mean that
'find' command will sit there *forever*, tying up resources the whole time.
This can cause issues. For instance - if you've flagged 'mozilla' as the
process that gets first shot at the disk, what do you do if you start paging to
the swap area, and some OTHER process has to read a page in? What if that
"other process" is the X server or your window manager? Think REALLY hard here
- just saying "I'll renice them too" is NOT the right answer.. .;)
On Mer, 2003-07-23 at 15:17, [email protected] wrote:
> Basically, you're stuck. The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up. One person suggested
> using QoS to help things along - but that needs to be implemented at the OTHER
> end of the dial-up - which means unless your provider does QoS on the terminal
> server, you're basically stuck. Packets will probably just get queued up in
> order of arrival.
There are a few things that help in the general real world but not
mathematical sense. Use an ftp client like gnome-ftp which can set the
rate it accepts data and window sizes. It'll still jam the modem a
little when it starts a transfer but then it'll generally be ok if you
have a bit of buffering for your icecast stream.
[email protected] writes:
> 2) There's a phenomenon called "starvation". See that 'find' command in your
> example? If mozilla is disk-hungry enough, bad I/O scheduling can mean that
> 'find' command will sit there *forever*, tying up resources the whole time.
> This can cause issues. For instance - if you've flagged 'mozilla' as the
> process that gets first shot at the disk, what do you do if you start paging to
> the swap area, and some OTHER process has to read a page in? What if that
> "other process" is the X server or your window manager? Think REALLY hard here
> - just saying "I'll renice them too" is NOT the right answer.. .;)
I'm sure it's a serious issue, and yet my network has QoS set up and the low
priority flows still eventually get through just fine even though it has much
lower bandwidth available than my disk controller.
I think it would be really cool to be able to control disk i/o with the same
level of flexibility as network i/o. I could see setting up cbq trees that can
key off things like whether it's paging, a blocking/nonblocking i/o, or a
nonblocking i/o. They could also see what user owns the process, and what
inode the process's executable image is.
I would just wonder about the overhead.
--
greg
On Wed, 23 Jul 2003, Alan Cox wrote:
> On Mer, 2003-07-23 at 15:17, [email protected] wrote:
> > Basically, you're stuck. The biggest part of the problem is that although you
> > can certainly control the outbound packets, you have no real control over when
> > inbound packets arrive at the other end of your dial-up. One person suggested
> > using QoS to help things along - but that needs to be implemented at the OTHER
> > end of the dial-up - which means unless your provider does QoS on the terminal
> > server, you're basically stuck. Packets will probably just get queued up in
> > order of arrival.
>
> There are a few things that help in the general real world but not
> mathematical sense. Use an ftp client like gnome-ftp which can set the
> rate it accepts data and window sizes. It'll still jam the modem a
> little when it starts a transfer but then it'll generally be ok if you
> have a bit of buffering for your icecast stream.
>
More info. I use a 56k dialup link and PPD to essentially become
my own ISP for my network at home. The machines at home are nodes
on the company's LAN. I have two dedicated machines, one at
work and one at home who's only purpose is to forward IP packets.
These machines are, otherwise, idle.
When I am using ftp to download some work to one of my other
home computers, it is functional impossible to do any useful
work on any other connection. The connections persist, but
no data will get through when a FTP transfer is in progress
except for the FTP data. Somehow the FTP data stream is able
to hog the entire transmission bandwidth. Attempts to "fix" it
be tuning Nagle off, Van Jacobson, etc., all off doesn't do
anything useful.
I have from time-to-time reported as far back as Linux 2.0, the
fact that there are TCP stalls when using PPP. These stalls
still persist with 2.4.20 and I've just had to accept the so-
called "fact" that 1 to 10 second stalls when using PPP are
"normal".
At one time I thought it was just that the modems had lost
sync so I set up a wire-to-wire PPP link here at work. The
stalls still exist when using the RS-232C link with a null-
modem cable. Evidence is that packets are lost, even with
a direct connection. It takes <too much> time for Linux to
discover the missing datagram and have it re-sent. When a
stream of data (FTP) is being sent, nothing seems to get
lost except the ACKs from the receiving end. It seems as
though RS-232C is not full duplex. If something is being
sent, nothing will be received and vice versa.
But... If you write to RS-232C with a test program, with
pins 2,3 shorted (loop-back), you can always read what
you wrote with no errors IFF hardware CRTCTS is turned OFF.
If I enable hardware handshaking, necessary for a modem,
with RTS jumpered to CTS, all bets are off. There are
many missing bytes, even at 9600 baud. However, when
connected through modems, this affect goes away. So,
it seems as though the PPP problem is related to RS-232C
problems, but I'll be damned if I can find it.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.
On Wed, Jul 23, 2003 at 11:10:08AM -0400, Richard B. Johnson wrote:
> On Wed, 23 Jul 2003, Alan Cox wrote:
>
> > On Mer, 2003-07-23 at 15:17, [email protected] wrote:
> > > Basically, you're stuck. The biggest part of the problem is that although you
> > > can certainly control the outbound packets, you have no real control over when
> > > inbound packets arrive at the other end of your dial-up. One person suggested
> > > using QoS to help things along - but that needs to be implemented at the OTHER
> > > end of the dial-up - which means unless your provider does QoS on the terminal
> > > server, you're basically stuck. Packets will probably just get queued up in
> > > order of arrival.
> >
> > There are a few things that help in the general real world but not
> > mathematical sense. Use an ftp client like gnome-ftp which can set the
> > rate it accepts data and window sizes. It'll still jam the modem a
> > little when it starts a transfer but then it'll generally be ok if you
> > have a bit of buffering for your icecast stream.
> >
>
> More info. I use a 56k dialup link and PPD to essentially become
> my own ISP for my network at home. The machines at home are nodes
> on the company's LAN. I have two dedicated machines, one at
> work and one at home who's only purpose is to forward IP packets.
> These machines are, otherwise, idle.
>
> When I am using ftp to download some work to one of my other
> home computers, it is functional impossible to do any useful
> work on any other connection. The connections persist, but
> no data will get through when a FTP transfer is in progress
> except for the FTP data. Somehow the FTP data stream is able
> to hog the entire transmission bandwidth. Attempts to "fix" it
> be tuning Nagle off, Van Jacobson, etc., all off doesn't do
> anything useful.
You need QoS at the router level to resolve this. Since you are
running your own routers to connect your ethernet segments, QoS
should be done at both ends of the connection. If it's available
on your distro, try wondershaper, it's a nice script which you tell
your upstream and downstream rates and then it adjusts QoS parameters
to provide great response. The most important thing is that it prioritises
ACK packets above everything else. This helps a lot when there is heavy
traffic (FTP for example) in both directions at the same time.
> I have from time-to-time reported as far back as Linux 2.0, the
> fact that there are TCP stalls when using PPP. These stalls
> still persist with 2.4.20 and I've just had to accept the so-
> called "fact" that 1 to 10 second stalls when using PPP are
> "normal".
>
> At one time I thought it was just that the modems had lost
> sync so I set up a wire-to-wire PPP link here at work. The
> stalls still exist when using the RS-232C link with a null-
> modem cable. Evidence is that packets are lost, even with
> a direct connection. It takes <too much> time for Linux to
> discover the missing datagram and have it re-sent. When a
> stream of data (FTP) is being sent, nothing seems to get
> lost except the ACKs from the receiving end. It seems as
> though RS-232C is not full duplex. If something is being
> sent, nothing will be received and vice versa.
>
> But... If you write to RS-232C with a test program, with
> pins 2,3 shorted (loop-back), you can always read what
> you wrote with no errors IFF hardware CRTCTS is turned OFF.
>
> If I enable hardware handshaking, necessary for a modem,
> with RTS jumpered to CTS, all bets are off. There are
> many missing bytes, even at 9600 baud. However, when
> connected through modems, this affect goes away. So,
> it seems as though the PPP problem is related to RS-232C
> problems, but I'll be damned if I can find it.
>
Greets, Antonio.
Alternately, openbsd can do a similar thing with great results (although
I haven't tried it over serial/pppd.) And it works reasonably well even
when done on only one end.
http://www.benzedrine.cx/ackpri.html
On Wed, 2003-07-23 at 11:13, Antonio Vargas wrote:
> You need QoS at the router level to resolve this. Since you are
> running your own routers to connect your ethernet segments, QoS
> should be done at both ends of the connection. If it's available
> on your distro, try wondershaper, it's a nice script which you tell
> your upstream and downstream rates and then it adjusts QoS parameters
> to provide great response. The most important thing is that it prioritises
> ACK packets above everything else. This helps a lot when there is heavy
> traffic (FTP for example) in both directions at the same time.
--
Disconnect <[email protected]>
On 23 Jul 2003 12:55:35 -0400
Disconnect <[email protected]> wrote:
> Alternately, openbsd can do a similar thing with great results (although
> I haven't tried it over serial/pppd.) And it works reasonably well even
> when done on only one end.
Wondershaper usually does too.
It's all the same stuff.
Hi!
> With the current scheduler we can prioritize the CPU usage for each
> process. What I think would be extremely useful (as I have needed it
> many times) is the scheduling of disk I/O and net I/O traffic. 2
> examples showing the importance (the numbers are estimations just to
> explain whati I mean):
Yes that would be nice, and in 2.5 timeframe
there was patch doing that. Port it to 2.6 an test it!
--
Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...
[email protected] wrote:
> On Wed, 23 Jul 2003 13:57:41 +0300, [email protected] said:
>
>>1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of
>>bandwidth available. What I want to do is listen to icecast radio via xmms (at
>>22 kbits), download the kernel sources with wget, and browse the web at the same
>>time. Currently I think that this is *impossible* (correct me if I'm wrong) as
>>the radio will be full of pauses and the browsing experience painfully slow.
>
>
> Basically, you're stuck. The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up. One person suggested
But the incoming packets are those that I request to be sent to me (most times).
When I know the packet size I will accept I may not request all packets now, but
a bit later, to leave bandwidth for other requested packets of higher priority.
> using QoS to help things along - but that needs to be implemented at the OTHER
> end of the dial-up - which means unless your provider does QoS on the terminal
> server, you're basically stuck. Packets will probably just get queued up in
> order of arrival.
This is what I don't like, but I bet there must be a way to slow down the other
end, even if it has not QoS.
What might help, and needs only be implemented on my side of the connection, is
requesting the packets of higher priorities first. And if I know my total
bandwidth, perhaps I can request as many packets needed to fill it (and not
flood it like it happens now).
Thank you all for your attention,
Dimitris
On Thu, Jul 24, 2003 at 01:17:57AM +0300, [email protected] wrote:
> What might help, and needs only be implemented on my side of the
> connection, is requesting the packets of higher priorities first. And if I
> know my total bandwidth, perhaps I can request as many packets needed to
> fill it (and not flood it like it happens now).
Yes, this is exactly what wondershaper does now.
Please give it a try.
I use it on a dsl line, and it works wonders. (pardon the pun).
On ons, 2003-07-23 at 16:17, [email protected] wrote:
> On Wed, 23 Jul 2003 13:57:41 +0300, [email protected] said:
> > 1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of
> > bandwidth available. What I want to do is listen to icecast radio via xmms (at
> > 22 kbits), download the kernel sources with wget, and browse the web at the same
> > time. Currently I think that this is *impossible* (correct me if I'm wrong) as
> > the radio will be full of pauses and the browsing experience painfully slow.
>
> Basically, you're stuck. The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up.
Take a look at http://trash.net/~kaber/imq/
Does wonders here, both local and forwarded traffic.
--
Cheers,
Andr? Tomt
[email protected]
On Wed, 23 Jul 2003, Pavel Machek wrote:
> Hi!
>
> > With the current scheduler we can prioritize the CPU usage for each
> > process. What I think would be extremely useful (as I have needed it
> > many times) is the scheduling of disk I/O and net I/O traffic. 2
> > examples showing the importance (the numbers are estimations just to
> > explain whati I mean):
>
> Yes that would be nice, and in 2.5 timeframe
> there was patch doing that. Port it to 2.6 an test it!
Pavel,
Do you remember who wrote those or where one can find it?
Hi!
> > > With the current scheduler we can prioritize the CPU usage for each
> > > process. What I think would be extremely useful (as I have needed it
> > > many times) is the scheduling of disk I/O and net I/O traffic. 2
> > > examples showing the importance (the numbers are estimations just to
> > > explain whati I mean):
> >
> > Yes that would be nice, and in 2.5 timeframe
> > there was patch doing that. Port it to 2.6 an test it!
>
> Do you remember who wrote those or where one can find it?
After a bit of searching I found this. I'm not 100% sure this is the
same one I am remembering, but it looks like that.
http://marc.theaimsgroup.com/?l=linux-kernel&m=103962160319984&w=2
[No, I'm probably remembering some other patch, this looks way too
simple, but may be good point to start...]
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]