Return-Path: Received: from mail-ua1-f65.google.com ([209.85.222.65]:42532 "EHLO mail-ua1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726496AbeI2CVR (ORCPT ); Fri, 28 Sep 2018 22:21:17 -0400 Received: by mail-ua1-f65.google.com with SMTP id w7-v6so2729941uan.9 for ; Fri, 28 Sep 2018 12:55:58 -0700 (PDT) MIME-Version: 1.0 References: <20180905192400.107485-1-trond.myklebust@hammerspace.com> <20180905192400.107485-2-trond.myklebust@hammerspace.com> <20180905192400.107485-3-trond.myklebust@hammerspace.com> <20180905192400.107485-4-trond.myklebust@hammerspace.com> <20180905192400.107485-5-trond.myklebust@hammerspace.com> <20180905192400.107485-6-trond.myklebust@hammerspace.com> <9dbc1442dd1e7bba1903a5ecf2855054ffcd0ee4.camel@gmail.com> In-Reply-To: From: Olga Kornievskaia Date: Fri, 28 Sep 2018 15:55:46 -0400 Message-ID: Subject: Re: [PATCH 5/7] NFS: Convert lookups of the open context to RCU To: Trond Myklebust Cc: linux-nfs Content-Type: text/plain; charset="UTF-8" Sender: linux-nfs-owner@vger.kernel.org List-ID: On Fri, Sep 28, 2018 at 3:10 PM Olga Kornievskaia wrote: > > On Fri, Sep 28, 2018 at 2:54 PM Trond Myklebust wrote: > > > > On Fri, 2018-09-28 at 14:31 -0400, Olga Kornievskaia wrote: > > > On Fri, Sep 28, 2018 at 1:50 PM Trond Myklebust > > > wrote: > > > > > > > > On Fri, 2018-09-28 at 12:54 -0400, Olga Kornievskaia wrote: > > > > > Also shouldn't this be a bug fix for 4.19 and also go to stable? > > > > > > > > It wasn't really intended as a bugfix because I wasn't aware that > > > > there > > > > was a bug to fix in the first place. I assume the modification to > > > > nfs4_state_find_open_context() to cause it to look for an open > > > > context > > > > with a READ/WRITE open mode first is what fixes the bug. Is that > > > > the > > > > case? > > > > > > I don't think so. nfs4_state_find_open_context() never gets calls > > > during the run of this test case. This function is called during the > > > reclaim on an open. In delegation recall the opens are not reclaimed, > > > it is just reclaiming the locks. > > > > > > Actually, when I undo this piece of the patch, I can make it fail > > > again. > > > > > > @@ -1027,10 +1027,7 @@ void nfs_inode_attach_open_context(struct > > > nfs_open_context *ctx) > > > struct nfs_inode *nfsi = NFS_I(inode); > > > > > > spin_lock(&inode->i_lock); > > > - if (ctx->mode & FMODE_WRITE) > > > - list_add(&ctx->list, &nfsi->open_files); > > > - else > > > - list_add_tail(&ctx->list, &nfsi->open_files); > > > + list_add_tail_rcu(&ctx->list, &nfsi->open_files); > > > spin_unlock(&inode->i_lock); > > > } > > > EXPORT_SYMBOL_GPL(nfs_inode_attach_open_context); > > > > > > It looks like the placement in the list matters? Any ideas why? > > > > Currently, the list is ordered so that writeable open contexts are > > always found first. The reason is that when traversing the list during > > server reboot recovery, we want to ensure that we reclaim any write > > delegations on the file first so that we can cache all subsequent opens > > and locks of that file. > > > > A delegation return requires us to recover all cached state, so it must > > reclaim both OPEN and LOCK state. It is not, however, expected to > > depend on the open context list ordering, since there can be no further > > caching. > > IOW: no, I don't see why your bug would depend on the list order unless > > some part of the recovery is actually failing. > > Ok thanks. Need to fix the lack of OPEN reclaim. Wait, why are we suppose to reclaim the open state when we have a valid open stateid? We don't have any cached opens that server doesn't know about. RFC7530 says "if the file has other open reference", I think the emphasis is on "other". I don't believe we need to be sending anything besides the locks to the server. Then I'm back to square one. > > > -- > > Trond Myklebust > > Linux NFS client maintainer, Hammerspace > > trond.myklebust@hammerspace.com > > > >