Received: by 2002:a25:8b12:0:0:0:0:0 with SMTP id i18csp303228ybl; Fri, 23 Aug 2019 00:50:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqye7xuiRZf0aAvYp1U7Ivt22k5tH2waUpM/HljIlMHgZWa8JGJKQkUJGcBf5uN2MJIJ+wHg X-Received: by 2002:aa7:991a:: with SMTP id z26mr3659237pff.43.1566546626165; Fri, 23 Aug 2019 00:50:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1566546626; cv=none; d=google.com; s=arc-20160816; b=I/4Ef0ioFZlS2GcSYaqL5X6KBBqgj6Wf0XXwGmNCl5EUIc9zZGJD18O5ErUcWP7ba+ X1IfLGcSDiyLNNZmLLuipESeNtigkza9ccCoDRuHJ1lcFQ80tey3kcDDmrtH3q41f09K QG0uq/cBvV9GwTYoCTeHFiLl4dvrwpmRyo9rWVI1f+XkCXW6xaKixIq5TgDnAXqmtldP SD7mutXh8iNxJ9LzmL0B7UDeE503skTw9E6nmJkSet610RYTWOvPnPMPxlzPBc28eXSI adrG0372P1w/w3hsoSx5/6xBxoCEIf8a6P0F34nSxmjBeR9XQrIcvNDstPNooBrPzWbL xBWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from; bh=0CZ04zm42POpwiyHKKv7b336SGYgzBiVD6DhCf7CPx8=; b=ionuUCxTZyOU0KOYn4Rjh0SU5WX2Vt4xgVJJGaXKrRG2mu+BEDRfFHIr8oIZWJj6Rd T6vQgdrVVIvkcfUJ3uzuXbQHBXH7NTw6pimvw4DOBlh2V5AGrKElR2/MuPCUTZYQrFU3 02IsqnCSp5zzmzC9A0IlYSw5i7V4YeyOIzV05wlow9JlKPuSI72pu5hcr0owy9DvpgYr aEHnQNqcxx3qvwfCbjTOoS38mAdKwIAF5ddrXYrS5KzsjWh2n60sOIYaIuuP7QXhoF0M g3T6dCCAELmgBJDkbVZ1sM7DMZrk7PR4bGUPgFsrb0J0a4Ed7LOm28M38SZhwt7iS8b8 V63A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l5si1641305pjt.106.2019.08.23.00.50.02; Fri, 23 Aug 2019 00:50:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732838AbfHVTja (ORCPT + 99 others); Thu, 22 Aug 2019 15:39:30 -0400 Received: from mx2.math.uh.edu ([129.7.128.33]:51826 "EHLO mx2.math.uh.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726319AbfHVTj3 (ORCPT ); Thu, 22 Aug 2019 15:39:29 -0400 Received: from epithumia.math.uh.edu ([129.7.128.2]) by mx2.math.uh.edu with esmtp (Exim 4.92) (envelope-from ) id 1i0sve-0004Tw-Mp; Thu, 22 Aug 2019 14:39:28 -0500 Received: by epithumia.math.uh.edu (Postfix, from userid 7225) id 9C2EF801554; Thu, 22 Aug 2019 14:39:26 -0500 (CDT) From: Jason L Tibbitts III To: linux-nfs@vger.kernel.org Cc: km@cm4all.com, linux-kernel@vger.kernel.org Subject: Re: Regression in 5.1.20: Reading long directory fails References: Date: Thu, 22 Aug 2019 14:39:26 -0500 In-Reply-To: (Jason L. Tibbitts, III's message of "Tue, 13 Aug 2019 10:08:55 -0500") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.9 (--) Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org I now have another user reporting the same failure of readdir on a long directory which showed up in 5.1.20 and was traced to 3536b79ba75ba44b9ac1a9f1634f2e833bbb735c. I'm not sure what to do to get more traction besides reposting and adding some addresses to the CC list. If there is any information I can provide which might help to get to the bottom of this, please let me know. To recap: 5.1.20 introduced a regression reading some large directories. In this case, the directory should have 7800 files or so in it: [root@ld00 ~]# ls -l ~dblecher|wc -l ls: reading directory '/home/dblecher': Input/output error 1844 [root@ld00 ~]# cat /proc/version Linux version 5.1.20-300.fc30.x86_64 (mockbuild@bkernel04.phx2.fedoraproject.org) (gcc version 9.1.1 20190503 (Red Hat 9.1.1-1) (GCC)) #1 SMP Fri Jul 26 15:03:11 UTC 2019 (The server is a Centos 7 machine running kernel 3.10.0-957.12.2.el7.x86_64.) Building a kernel which reverts commit 3536b79ba75ba44b9ac1a9f1634f2e833bbb735c: Revert "NFS: readdirplus optimization by cache mechanism" (memleak) fixes the issue, but of course that revert was fixing a real issue so I'm not sure what to do. I can trivially reproduce this by simply trying to list the problematic directories but I'm not sure how to construct such a directory; simply creating 10000 files doesn't cause the problem for me. I am willing to test patches and can build my own kernels, and I'm happy to provide any debugging information you might require. Unfortunately I don't know enough to dig in and figure out for myself what's going wrong. I did file https://bugzilla.redhat.com/show_bug.cgi?id=1740954 just to have this in a bug tracker somewhere. I'm happy to file one somewhere else if that would help. - J<