Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Benjamin Coddington" <bcodding@redhat.com>
To:     hedrick@rutgers.edu
Cc:     "Patrick Goetz" <pgoetz@math.utexas.edu>, linux-nfs@vger.kernel.org
Subject: Re: safe versions of NFS
Date:   Tue, 13 Apr 2021 12:23:30 -0400
Message-ID: <08DD4F45-E1CB-4FCB-BA0C-A6B8BD86FEDE@redhat.com>
In-Reply-To: <22DE8966-253D-49A7-936D-F0A0B5246BE6@rutgers.edu>
References: <D8F59140-83D4-49F8-A858-D163910F0CA1@rutgers.edu>
 <e6501675-7cb4-6f5b-78f7-abb1be332a34@math.utexas.edu>
 <506B20E8-CE9C-499B-BF53-6026BA132D30@rutgers.edu>
 <1EA88CB0-7639-479C-AB1D-CAF5C67AA5C5@redhat.com>
 <22DE8966-253D-49A7-936D-F0A0B5246BE6@rutgers.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

(resending this as it bounced off the list - I accidentally embedded 
HTML)

Yes, if you're pretty sure your hostnames are all different, the 
client_ids
should be different.  For v4.0 you can turn on debugging (rpcdebug -m 
nfs -s
proc) and see the client_id in the kernel log in lines that look like: 
"NFS
call setclientid auth=%s, '%s'\n", which will happen at mount time, but 
it
doesn't look like we have any debugging for v4.1 and v4.2 for 
EXCHANGE_ID.

You can extract it via the crash utility, or via systemtap, or by doing 
a
wire capture, but nothing that's easily translated to running across a 
large
number of machines.  There's probably other ways, perhaps we should tack
that string into the tracepoints for exchange_id and setclientid.

If you're interested in troubleshooting, wire capture's usually the most
informative.  If the lockup events all happen at the same time, there
might be some network event that is triggering the issue.

You should expect NFSv4.1 to be rock-solid.  Its rare we have reports
that it isn't, and I'd love to know why you're having these problems.

Ben

On 13 Apr 2021, at 11:38, hedrick@rutgers.edu wrote:

> The server is ubuntu 20, with a ZFS file system.
>
> I don’t set the unique ID. Documentation claims that it is set from 
> the hostname. They will surely be unique, or the whole world would 
> blow up. How can I check the actual unique ID being used? The kernel 
> reports a blank one, but I think that just means to use the hostname. 
> We could obviously set a unique one if that would be useful.
>
>> On Apr 13, 2021, at 11:35 AM, Benjamin Coddington 
>> <bcodding@redhat.com> wrote:
>>
>> It would be interesting to know why your clients are failing to 
>> reclaim their locks.  Something is misconfigured.  What server are 
>> you using, and is there anything fancy on the server-side (like HA)?  
>> Is it possible that you have clients with the same nfs4_unique_id?
>>
>> Ben
>>
>> On 13 Apr 2021, at 11:17, hedrick@rutgers.edu wrote:
>>
>>> many, though not all, of the problems are “lock reclaim failed”.
>>>
>>>> On Apr 13, 2021, at 10:52 AM, Patrick Goetz 
>>>> <pgoetz@math.utexas.edu> wrote:
>>>>
>>>> I use NFS 4.2 with Ubuntu 18/20 workstations and Ubuntu 18/20 
>>>> servers and haven't had any problems.
>>>>
>>>> Check your configuration files; the last time I experienced 
>>>> something like this it's because I inadvertently used the same fsid 
>>>> on two different exports. Also recommend exporting top level 
>>>> directories only.  Bind mount everything you want to export into 
>>>> /srv/nfs and only export those directories. According to Bruce F. 
>>>> this doesn't buy you any security (I still don't understand why), 
>>>> but it makes for a cleaner system configuration.
>>>>
>>>> On 4/13/21 9:33 AM, hedrick@rutgers.edu wrote:
>>>>> I am in charge of a large computer science dept computing 
>>>>> infrastructure. We have a variety of student and develo9pment 
>>>>> users. If there are problems we’ll see them.
>>>>> We use an Ubuntu 20 server, with NVMe storage.
>>>>> I’ve just had to move Centos 7 and Ubuntu 18 to use NFS 4.0. We 
>>>>> had hangs with NFS 4.1 and 4.2. Files would appear to be locked, 
>>>>> although eventually the lock would time out. It’s too soon to be 
>>>>> sure that moving back to NFS 4.0 will fix it. Next is either NFS 3 
>>>>> or disabling delegations on the server.
>>>>> Are there known versions of NFS that are safe to use in production 
>>>>> for various kernel versions? The one we’re most interested in is 
>>>>> Ubuntu 20, which can be anything from 5.4 to 5.8.
>>