Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <3447df77-1b2f-6d36-0516-3ae7267ab509@genome.arizona.edu>
 <20191023171523.GA18802@fieldses.org> <b6248a82-1f1a-7329-5ee0-6e026f6db697@genome.arizona.edu>
 <YTBPR01MB2845B12E9C59F837FE35F40DDD650@YTBPR01MB2845.CANPRD01.PROD.OUTLOOK.COM>
 <82ee292f-f126-9e9f-d023-deb72d1a3971@genome.arizona.edu> <1079a074-7580-e257-8b52-6e48f8822176@genome.arizona.edu>
 <CAN-5tyExtO_HRGYrqq7UTObrHNsTp7UqwW=Kg4CFM3q-OnaUiQ@mail.gmail.com> <49472814-8dcd-4594-d48e-be4c1d9a8d8f@genome.arizona.edu>
In-Reply-To: <49472814-8dcd-4594-d48e-be4c1d9a8d8f@genome.arizona.edu>
From:   Olga Kornievskaia <aglo@umich.edu>
Date:   Mon, 11 Nov 2019 10:46:05 -0500
Message-ID: <CAN-5tyGD3hNcG7=+xdB4Bs1OCCUoLzZ5ocsJusuxd3n3+x=7gw@mail.gmail.com>
Subject: Re: NFS hangs on one interface
To:     Chandler <admin@genome.arizona.edu>
Cc:     "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hi Chandler,

Given what you say, it sounds to me more like a generic networking
issue between this particular problem and the server.

debug messages are logging that client can't reach the server:
Nov  8 17:58:21 NFSclient kernel: nfs: server x.2 not responding, still try=
ing

I'd recommend making sure that your network works alright between
those interfaces. Perhaps running an iperf for a few minutes to make
sure you are seeing expected, consistent performance between those two
nodes. Another thing to check if you for some reason have duplicate
IPs in the system that can show up as weird hangs.

On Fri, Nov 8, 2019 at 8:22 PM Chandler <admin@genome.arizona.edu> wrote:
>
> Hi Olga, thanks so much for your help.
>
> I tried to reboot and still having weird issues.  If I mount over the loc=
al network (10.x address) then there are no issues.  As soon as I try to mo=
unt over the 10G network, weird things happen.  For example, I can perform =
the mount just fine and do "ll /mount" but as soon as I try another directo=
ry like "ll /mount/users" then it hangs.  Also this only happens between th=
ese two machines with 10G interfaces.  The server with the 10G interface ha=
s several other 1G clients that outside the local 10.x network that connect=
 to it on the 10G interface, and those clients all work fine as well, so se=
ems to be an issue specific to this client on the 10G interface.
>
> In my earlier post, I did try troubleshooting with vers=3D3 and vers=3D2 =
just to see if that was the issue, but since then have been using the defau=
lts (so vers=3D4).
>
> I turned on the rpcdebug on both client and server with "rpcdebug -m nfs(=
d) all"  but it seemed to lock up the server and i had to reboot it, so wil=
l keep that off for now.  I attached a log of the debug messages from the c=
lient showing what commands I executed (snoopy) and the resulting kernel de=
bug entries, hope this helps.  The hangup happens at the end when I ls -l o=
n the users directory.  Let me know if there's anything else I can provide.
>