Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server
to a crawl no matter how much "nice" you put on them. Is there any way
to make "nice" smarter in that it limits io as well as processor usage?
If cp and rsyne ran a little slower IO wise then everything else could
run too.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Sun, 2 Oct 2005 12:26, Marc Perkel wrote:
> Just a thought -----
>
> Programs like cp -a /bigdir /backup and rsync usually bring the server
> to a crawl no matter how much "nice" you put on them. Is there any way
> to make "nice" smarter in that it limits io as well as processor usage?
> If cp and rsyne ran a little slower IO wise then everything else could
> run too.
The latest cfq io scheduler supports io nice levels. By default it links the
io nice levels to the cpu nice levels so if you use cfq and set your file
commands nice 19 they will use as little io priority as possible. Note this
only works on the read side but that makes a dramatic difference already.
Cheers
Con
Con Kolivas wrote:
>On Sun, 2 Oct 2005 12:26, Marc Perkel wrote:
>
>
>>Just a thought -----
>>
>>Programs like cp -a /bigdir /backup and rsync usually bring the server
>>to a crawl no matter how much "nice" you put on them. Is there any way
>>to make "nice" smarter in that it limits io as well as processor usage?
>>If cp and rsyne ran a little slower IO wise then everything else could
>>run too.
>>
>>
>
>The latest cfq io scheduler supports io nice levels. By default it links the
>io nice levels to the cpu nice levels so if you use cfq and set your file
>commands nice 19 they will use as little io priority as possible. Note this
>only works on the read side but that makes a dramatic difference already.
>
>Cheers
>Con
>
>
Kewl - so - what version is it in?
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Sun, 2 Oct 2005 13:18, Marc Perkel wrote:
> Con Kolivas wrote:
> >On Sun, 2 Oct 2005 12:26, Marc Perkel wrote:
> >>Just a thought -----
> >>
> >>Programs like cp -a /bigdir /backup and rsync usually bring the server
> >>to a crawl no matter how much "nice" you put on them. Is there any way
> >>to make "nice" smarter in that it limits io as well as processor usage?
> >>If cp and rsyne ran a little slower IO wise then everything else could
> >>run too.
> >
> >The latest cfq io scheduler supports io nice levels. By default it links
> > the io nice levels to the cpu nice levels so if you use cfq and set your
> > file commands nice 19 they will use as little io priority as possible.
> > Note this only works on the read side but that makes a dramatic
> > difference already.
>
> Kewl - so - what version is it in?
2.6.13 already has it. Note that the io priority is only inherited when a
process first starts so doing renice to something already running will only
change the cpu nice, not the ionice. You can do that on the fly using the
ionice application. If you look in the kernel source in
Documentation/block/ioprio.txt you'll find the source to ionice.c
Cheers,
Con
P.S. It is considered routine to reply-to-all when posting to lkml and not
stripping anybody on the cc list.
On Sun, 2005-10-02 at 13:07 +1000, Con Kolivas wrote:
> On Sun, 2 Oct 2005 12:26, Marc Perkel wrote:
> > Just a thought -----
> >
> > Programs like cp -a /bigdir /backup and rsync usually bring the server
> > to a crawl no matter how much "nice" you put on them. Is there any way
> > to make "nice" smarter in that it limits io as well as processor usage?
> > If cp and rsyne ran a little slower IO wise then everything else could
> > run too.
>
> The latest cfq io scheduler supports io nice levels. By default it links the
> io nice levels to the cpu nice levels so if you use cfq and set your file
> commands nice 19 they will use as little io priority as possible. Note this
> only works on the read side but that makes a dramatic difference already.
>
> Cheers
> Con
If you want a way to assign io priorities without relying on process
inheritance and (re)nice you might find CKRM, with it's cfq-based IO
controller, useful.
Basically you create a set of classes that group tasks and give an
appropriate share of IO performance to tasks in that class. As processes
get created CKRM will assign tasks to the IO classes based on a set of
rules. You can run commands like:
mkdir /rcfs/taskclass/makin_copies
echo 'res=io,guarantee=20' > /rcfs/taskclass/makin_copies/shares
echo 'path=/usr/bin/rsync,class=makin_copies' > /rcfs/ce/rsync_rule
echo 'path=/bin/cp,class=makin_copies' > /rcfs/ce/cp_rule
to set this up. This should make CKRM useful for those unwilling to
become full-time administrators just to run their own desktops.
Cheers,
-Matt Helsley
On Tue, Oct 04 2005, Matt Helsley wrote:
> On Sun, 2005-10-02 at 13:07 +1000, Con Kolivas wrote:
> > On Sun, 2 Oct 2005 12:26, Marc Perkel wrote:
> > > Just a thought -----
> > >
> > > Programs like cp -a /bigdir /backup and rsync usually bring the server
> > > to a crawl no matter how much "nice" you put on them. Is there any way
> > > to make "nice" smarter in that it limits io as well as processor usage?
> > > If cp and rsyne ran a little slower IO wise then everything else could
> > > run too.
> >
> > The latest cfq io scheduler supports io nice levels. By default it links the
> > io nice levels to the cpu nice levels so if you use cfq and set your file
> > commands nice 19 they will use as little io priority as possible. Note this
> > only works on the read side but that makes a dramatic difference already.
> >
> > Cheers
> > Con
>
> If you want a way to assign io priorities without relying on process
> inheritance and (re)nice you might find CKRM, with it's cfq-based IO
> controller, useful.
>
> Basically you create a set of classes that group tasks and give an
> appropriate share of IO performance to tasks in that class. As processes
> get created CKRM will assign tasks to the IO classes based on a set of
> rules. You can run commands like:
>
> mkdir /rcfs/taskclass/makin_copies
> echo 'res=io,guarantee=20' > /rcfs/taskclass/makin_copies/shares
> echo 'path=/usr/bin/rsync,class=makin_copies' > /rcfs/ce/rsync_rule
> echo 'path=/bin/cp,class=makin_copies' > /rcfs/ce/cp_rule
>
> to set this up. This should make CKRM useful for those unwilling to
> become full-time administrators just to run their own desktops.
Has the CKRM io controller been updated to 2.6.13 level CFQ, or is it
still using the very basic per-request based io sharing?
--
Jens Axboe