Return-Path: linux-nfs-owner@vger.kernel.org
Received: from mail-ig0-f181.google.com ([209.85.213.181]:57129 "EHLO
	mail-ig0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751269AbaDWWOH (ORCPT
	<rfc822;linux-nfs@vger.kernel.org>); Wed, 23 Apr 2014 18:14:07 -0400
Received: by mail-ig0-f181.google.com with SMTP id h18so167665igc.2
        for <linux-nfs@vger.kernel.org>; Wed, 23 Apr 2014 15:14:07 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <20140423211518.GA16506@us.ibm.com>
References: <CALXu0UcdfrvCajbAagGLV=dP=Kk3VWeR4VS5Wna44UroxM4JAQ@mail.gmail.com>
	<20140423202401.GA16168@us.ibm.com>
	<CALXu0UdVYtbKQqskETk9jEK3+=nFphUtFSP4i_ey=7WwZg3-XQ@mail.gmail.com>
	<20140423204405.GB16168@us.ibm.com>
	<CALXu0UdbSHfFM+tq2_zssMhiwZ1qMJdieNdvwtxz6k31CMWAUA@mail.gmail.com>
	<20140423211518.GA16506@us.ibm.com>
Date: Thu, 24 Apr 2014 00:14:06 +0200
Message-ID: <CALXu0UfjY288HZ_NHH_njs0iLhcY5Xn7NUJNYa9kL45m2YHfAw@mail.gmail.com>
Subject: Re: Tuning Linux NFSv4 for high latency connections?
From: Cedric Blancher <cedric.blancher@gmail.com>
To: Cedric Blancher <cedric.blancher@gmail.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-nfs-owner@vger.kernel.org
List-ID: <linux-nfs.vger.kernel.org>

On 23 April 2014 23:15, Malahal Naineni <malahal@us.ibm.com> wrote:
> Cedric Blancher [cedric.blancher@gmail.com] wrote:
>> On 23 April 2014 22:44, Malahal Naineni <malahal@us.ibm.com> wrote:
>> > Cedric Blancher [cedric.blancher@gmail.com] wrote:
>> >> On 23 April 2014 22:24, Malahal Naineni <malahal@us.ibm.com> wrote:
>> >> > Cedric Blancher [cedric.blancher@gmail.com] wrote:
>> >> >> Are there any options to improve the Linux NFSv4 performance over a
>> >> >> high latency connection?
>> >> >>
>> >> >> We currently use Solaris/Illumos as NFSv4 server and client over a
>> >> >> cross continental Internet connection. Latency is terrible (~220ms)
>> >> >> but the counter this by running work in parallel so the latency is
>> >> >> mostly mitigated.
>> >> >>
>> >> >> We now wish to migrate (short: Away from Oracle because support is
>> >> >> basically unbearable) to Linux (tested SuSE 13.1 and current Fedora)
>> >> >> and build times are 17 times (!!!) SLOWER than on Solaris/Illumos.
>> >> >>
>> >> >> Are there any tunables besides actimeo=300?
>> >> >
>> >> > rsize and wsize may help! You need to figure out if the read is the
>> >> > issue or the write before you dig further.
>> >>
>> >> I already tried to tune rsize/wsize, making them both smaller or the
>> >> maximum of 1048576 bytes, with no effect.
>> >>
>> >> One possible theory is that maybe something in Linux doesn't allow
>> >> multiple requests to be issued in parallel and waits for each request
>> >> to be completed before issuing the next one?
>> >
>> > Linux NFS client can issue I/Os in parallel. Should be limited by number
>> > of RPC slots though.
>>
>> What controls the number of RPC slots? is there a tunable? Is there
>> something to monitor the usage?
>
> sysctl sunrpc.tcp_slot_table_entries (if you are using tcp)

Its 16
NFSv4 is tcp only

I tried to bump the value to 128 - without effect - but the change is
not persistent across reboots. Is there something like Solaris
/etc/system which the kernel reads to set these values?

> Also, mountstats <mount-point> would be very helpful.

I don't have that command. likely my test machine is too old

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
Institute Pasteur