From: "Lever, Charles" <Charles.Lever@netapp.com>
Subject: RE: [PATCH 2.6.3] Add write throttling to NFS client
Date: Thu, 26 Feb 2004 09:22:18 -0800
Sender: nfs-admin@lists.sourceforge.net
Message-ID: <482A3FA0050D21419C269D13989C611302B07B91@lavender-fe.eng.netapp.com>
Mime-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Cc: <trond.myklebust@fys.uio.no>, "Greg Banks" <gnb@melbourne.sgi.com>,
        "ShantanuGoel" <sgoel01@yahoo.com>, <nfs@lists.sourceforge.net>
To: "Bogdan Costescu" <bogdan.costescu@iwr.uni-heidelberg.de>,
        "Olaf Kirch" <okir@suse.de>
Errors-To: nfs-admin@lists.sourceforge.net

seems like we have three different kinds of writes.

1.  writes that need to go now because the VM needs to reclaim
    the dirty pages

2.  async writebehind writes

3.  writes generated by the synchronous path because of the
    "sync" mount option or O_SYNC open flag.

1 and 3 are usually a bounded number of writes and fairly
urgent, but 2 can be effectively unbounded, but not terribly
urgent at all.

likewise for reads, you have reads that are due to readahead,
and reads that are pushed out because of a sync_page (someone
is waiting for that very page).

rather than using something like the RPC_SWAPPER_TASK flag
to change queuing behavior in the RPC client, perhaps the
RPC client should have a two- (or more) level scheduler
queue.  place the requests that have to go now on the top
queue, and the readahead/writebehind tasks on the bottom.
have a mechanism for boosting the priority of bottom
queue requests that suddenly become important.

then the RPC client can visit requests on the top queue
first, and when that is empty, visit requests on the
bottom queue.  there is still the potential for some
unfairness here, so, after every 'n' top queue requests,
service a bottom queue request so that eventually all
bottom queue requests are completed even if there is
a steady flow of top queue requests.

this is a mechanism that has worked in the past, and
requires very little tuning (ie low maintenance).


> -----Original Message-----
> From: Bogdan Costescu [mailto:bogdan.costescu@iwr.uni-heidelberg.de]=20
> Sent: Thursday, February 26, 2004 8:25 AM
> To: Olaf Kirch
> Cc: trond.myklebust@fys.uio.no; Greg Banks; ShantanuGoel;=20
> nfs@lists.sourceforge.net
> Subject: Re: [NFS] [PATCH 2.6.3] Add write throttling to NFS client
>=20
>=20
> On Thu, 26 Feb 2004, Olaf Kirch wrote:
>=20
> > The general "unfairness" observed in writes is due to the=20
> fact that we=20
> > allow a writing process to dirty a large number of pages without=20
> > blocking for the actual IO.
>=20
> Yes, and I think that this is pretty similar to the situation=20
> encountered with block IO, where queued writes had to be serviced=20
> before later queued reads, giving an impression of unresponsivness.=20
> It's easier to accept a delayed write than a delayed read.
>=20
> > Applications doing lots of stats etc are punished so=20
> severely because=20
> > all these operations are synchronous.
>=20
> That's exactly the case for block IO reads.
>=20
> > BTW I think reading is much less of a problem. The number of=20
> > readaheads scheduled by generic_file_read is limited,
>=20
> On the other hand, based on the block IO experience, reads=20
> should have=20
> higher priority than writes :-)
>=20
> --=20
> Bogdan Costescu
>=20
> IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches=20
> Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
> Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
> E-mail: Bogdan.Costescu@IWR.Uni-Heidelberg.De
>=20
>=20
>=20
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!=20
> http://ads.osdn.com/?ad_id=3D1356&alloc_id=3D3438> &op=3Dclick
>=20
> _______________________________________________
>=20
> NFS maillist  -  NFS@lists.sourceforge.net=20
> https://lists.sourceforge.net/lists/listinfo/n> fs
>=20


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs