Received: by 2002:a25:ad19:0:0:0:0:0 with SMTP id y25csp9079644ybi; Wed, 10 Jul 2019 04:21:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqxJHH4rWLvIUew7CItwVT04T1zJMPbeD+lk/VUnjSxbD5K9HODAibkiVyGlm8bA815E2Rad X-Received: by 2002:a63:e53:: with SMTP id 19mr36260915pgo.137.1562757699453; Wed, 10 Jul 2019 04:21:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562757699; cv=none; d=google.com; s=arc-20160816; b=Kjf55EU65LdnRVnZ4eam5MB3OaAOoFDUm4lremAg/SbhP+ivcRsiD0+vFnJDEeBFH+ tbjljZHRDuRgUxB4p4W7doANMKqhiiHB9isJ3rJ5g3YrTNJjYgks+xmMmXC6DlGDLL2t Sa+JzmMIym6APlbNxvINHvhZqJ33BioxqllKH5IbbX35iKP11IUk+4DuTZlPyim3VKrb E0sce6fURjNUWpkYu72UsboKSJyzkhChBKkmJkE2d85blPWdOfstAwUgLY//rkCW2dZS KMa8QybiRBfJEIlWYzoLEGh+E5NArFMbdc7JKvaRT/xM5g/oYwOAxo/V4BI3ZBTd0d1J dnPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:thread-index:thread-topic :content-transfer-encoding:mime-version:subject:references :in-reply-to:message-id:cc:to:from:date:dkim-signature:dkim-filter; bh=t4feWG50La/zeR/aO+pQ2pQ6kyErloQ2Pafx5IuybUY=; b=V4XzXrlgsp5DXYg1bkY3G8ylEvnuCddOA751eFWduHvuQ7PKM2HjHzUqlT90F72Pxg wUaRUiz7vo8SRgVpb2HEXEfSzHFxRa22NGOrb7QphOjXhTH7X8HfMchNlVHdkQT25L7S v0gsMg/jYOkbkvDMamIAutrhHkgrJpA9ggKiAAJWoWocAvJD918z8LbykyiIEjjX5g9y oTwzfRJqRdpnfWSLr1S1rV11WXVrXSfvkFQ2osDSECp3vpQSUF39ej55ilYs/FthxXTO OY17aJ/BSNm1LEp/xqi+PYEQwTWlLM1lG84VxlgNzEkGcwyKGgUu1/sc5E4ynMVMKqVH 7LHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@desy.de header.s=default header.b=pjHaIXeF; spf=pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q24si2069053pff.62.2019.07.10.04.21.09; Wed, 10 Jul 2019 04:21:39 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@desy.de header.s=default header.b=pjHaIXeF; spf=pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726281AbfGJLUw (ORCPT + 99 others); Wed, 10 Jul 2019 07:20:52 -0400 Received: from smtp-o-2.desy.de ([131.169.56.155]:51796 "EHLO smtp-o-2.desy.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725956AbfGJLUw (ORCPT ); Wed, 10 Jul 2019 07:20:52 -0400 Received: from smtp-buf-3.desy.de (smtp-buf-3.desy.de [IPv6:2001:638:700:1038::1:a6]) by smtp-o-2.desy.de (Postfix) with ESMTP id 2C5361608ED for ; Wed, 10 Jul 2019 13:20:50 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp-o-2.desy.de 2C5361608ED DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=desy.de; s=default; t=1562757650; bh=t4feWG50La/zeR/aO+pQ2pQ6kyErloQ2Pafx5IuybUY=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From; b=pjHaIXeFoFHUS8eK52B6N+f0Sx4WIFIJ2R/h4Fu/AFarH7o17uSpVKSieqaTN8DcX 0rKCKb7Ip2fjW13xuQdjKvg36sA67+7d2hE4lt16f8ofE7oJveBmpRtvLKq1xq64p4 6kIvCgPwWQiyICphoU3Dmwi+5cm5KZVUhd97ZYas= Received: from smtp-m-3.desy.de (smtp-m-3.desy.de [131.169.56.131]) by smtp-buf-3.desy.de (Postfix) with ESMTP id 27C2FA0040; Wed, 10 Jul 2019 13:20:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at desy.de Received: from z-mbx-2.desy.de (z-mbx-2.desy.de [131.169.55.140]) by smtp-intra-1.desy.de (Postfix) with ESMTP id EE62AC003B; Wed, 10 Jul 2019 13:20:49 +0200 (CEST) Date: Wed, 10 Jul 2019 13:20:49 +0200 (CEST) From: "Mkrtchyan, Tigran" To: John Dorminy Cc: linux-nfs Message-ID: <676688269.16845251.1562757649858.JavaMail.zimbra@desy.de> In-Reply-To: References: Subject: Re: Request for help debugging readdirplus malfunction on NFS v3 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Mailer: Zimbra 8.8.10_GA_3781 (ZimbraWebClient - FF67 (Linux)/8.8.10_GA_3786) Thread-Topic: Request for help debugging readdirplus malfunction on NFS v3 Thread-Index: qO4xER9Em8npJi12+ReEqiSvozW5Xg== Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Hi Dorminy, there are fixes form Trond that possibly address your issues. Did you have tried them? https://www.spinics.net/lists/linux-nfs/msg73754.html Regards, Tigran. ----- Original Message ----- > From: "John Dorminy" > To: "linux-nfs" > Sent: Wednesday, July 10, 2019 12:11:46 PM > Subject: Request for help debugging readdirplus malfunction on NFS v3 > Greetings; > > In the lab for the group I'm in, we have three NFS servers each > serving different parts of our shared filesystem. However, as of > kernel 5.1 or so on the clients, the clients have ceased working: a > 'ls' on any directory within one mountpoint (the only one hosted on > one server) fails to show any files. > > The mount is: > nfs-02:/nbu1 on /p/not-backed-up type nfs > (rw,noatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.19.119.4,mountvers=3,mountport=4048,mountproto=udp,local_lock=none,addr=10.19.119.4) > > Bisection on the client side indicates > be4c2d4723a4a637f0d1b4f7c66447141a4b3564 is the commit at which this > mountpoint ceases to work. `rpcdebug -m nfs -s all` results in the > following going to dmesg: > [10146.723030] NFS call readdirplus 162 > [10146.724908] NFS reply readdirplus: -2 > [10146.725429] NFS: readdir(/) returns -2 > > I'm somewhat out of ideas; are there other tools I should be using to > hunt this down, short of adding print statements? and is this a known > bug already? > > Thanks in advance! > > John Dorminy