saeed bishara wrote:
>> (linux-nfs added to cc)
>>
>> I cannot speak for the NFS server code specifically, but 32kb sounds
>> like a network read (or write) data size limit.
> yes
>> Are you using TCP? Are you using NFSv4, or an older version?
> I'm using NFSv3/UDP.
IMO, you definitely want TCP and NFSv4. Much better network behavior,
with some of the silly UDP limits (plus greatly improved caching
behavior, due to v4 delegations).
> I found that the actual requests size was 16KB, after doing some hacks
> in server&client I managed to make it 60KB, now I see better
> performance, and I see that the average request size is ~130KB which
> means that there is actually read-ahead. but why it's only 130KB? how
> can I make it larger?
> when I run local dd with bs=4K, I can see that the average IO size is
> more than 300KB.
Read-ahead is easier in NFSv4, because the client probably has the file
delegated locally, and has far less need to constantly revalidate file
mapping(s).
Jeff
> >> Are you using TCP? Are you using NFSv4, or an older version?
> > I'm using NFSv3/UDP.
>
> IMO, you definitely want TCP and NFSv4. Much better network behavior,
> with some of the silly UDP limits (plus greatly improved caching
> behavior, due to v4 delegations).
the clients of my system going to be embedded system with low
performance cpus and I need UDP as it needs less cpu power.
> > when I run local dd with bs=4K, I can see that the average IO size is
> > more than 300KB.
>
> Read-ahead is easier in NFSv4, because the client probably has the file
> delegated locally, and has far less need to constantly revalidate file
> mapping(s).
I'll check that.
but what about the server side? why the issued IO's are only as twice
as the size of the NFS requests?
On Thu, Dec 27, 2007 at 05:00:12PM +0200, saeed bishara wrote:
> > >> Are you using TCP? Are you using NFSv4, or an older version?
> > > I'm using NFSv3/UDP.
> >
> > IMO, you definitely want TCP and NFSv4. Much better network behavior,
> > with some of the silly UDP limits (plus greatly improved caching
> > behavior, due to v4 delegations).
> the clients of my system going to be embedded system with low
> performance cpus and I need UDP as it needs less cpu power.
You can try the attached adaptive readahead patch.
Apply it on your server and compile kernel with CONFIG_ADAPTIVE_READAHEAD.
Use large 1MB readahead on server and small readahead on clients.
> > > when I run local dd with bs=4K, I can see that the average IO size is
> > > more than 300KB.
> >
> > Read-ahead is easier in NFSv4, because the client probably has the file
> > delegated locally, and has far less need to constantly revalidate file
> > mapping(s).
> I'll check that.
> but what about the server side? why the issued IO's are only as twice
> as the size of the NFS requests?
The readahead code is helpless in NFSv3 :-(
Use NFS over TCP and rsize=readahead=1MB on client side could help.
But if you prefer UDP, the above patch may help you :-)
Fengguang