Received: by 2002:a25:868d:0:0:0:0:0 with SMTP id z13csp1893483ybk; Mon, 11 May 2020 06:59:33 -0700 (PDT) X-Google-Smtp-Source: APiQypI8K9Z3UtTI4QjotMCHzhKZnKTsU3+bm/3KoCuad2T2diLUXI3D4vDcUDcdFFHQTt+XBRQj X-Received: by 2002:a17:906:7fd7:: with SMTP id r23mr12992443ejs.32.1589205573233; Mon, 11 May 2020 06:59:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589205573; cv=none; d=google.com; s=arc-20160816; b=Y+PwiMzwlroDb1G3iP8Rpqr936uOtSwJeyaGeWRTSuLrszI//5Hs815RTlRscuWpKp ILywxWWmSMW4OZum9vlZCUgFWDG4GCcJ6GP2G0LJ+NRqtlNzUW/sUguiPBFYOOYQOVDg qfIs1aMiKSlUaq9hcGMqSAcuuhWLFUkIJP/+uu+rSoz6Xw3NeiAuXKjYv4UGQCHPeQvg YMFtzxnD6MOF6/DWQqegibUQv1u/0shHt5t4knRyZfhJ9BFKuA4wkZLAbN5SIcpDqYeg 8WX1ppq9bqgo5NyWR5G3ogUydtdYp+c4XOG4NS2ioZoGM+wLNQmX+RyyAqIxDIs/Obds eTCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=PN8atOAasm7otGniDLCLF5cpkLmU25aatR8vPQrhMlg=; b=BFWXyql/lFuehNeazzXPgib+Q61CTpvSuHX4H18AjB1IiTe4o0oT1SAAJdBNlu8Ow0 rHfTIxGHc3ox7rvi1BjeYrbDxkO+eqU+oE2WuWNGb3ghJi00OyJwmpSueEjWJLJhIBwX jAoy8+iuUwij6YZoc1PCtXPGb2Zc2oC4h+xUGzRp6ePSTxhv4+UzBhKvfqledH3W/laF fjhcSbN8JWh7AkvUXqPtdguDs0eRhiAqmO9FJJScXvc8dWZFE4LoxlH6bgwR1+8J6e6y Z+WfrNymE7GkwF1YrbCub+rQ2R/EDSCF5cxgV1EuEXqE3UhboA4oAnhjx1T1uqJMcYeb 2sNA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p24si5425136ejz.82.2020.05.11.06.59.02; Mon, 11 May 2020 06:59:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730260AbgEKN5q (ORCPT + 99 others); Mon, 11 May 2020 09:57:46 -0400 Received: from fieldses.org ([173.255.197.46]:54464 "EHLO fieldses.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729641AbgEKN5q (ORCPT ); Mon, 11 May 2020 09:57:46 -0400 Received: by fieldses.org (Postfix, from userid 2815) id A9AA38A6; Mon, 11 May 2020 09:57:45 -0400 (EDT) Date: Mon, 11 May 2020 09:57:45 -0400 From: "bfields@fieldses.org" To: Trond Myklebust Cc: "msys.mizuma@gmail.com" , "schumakeranna@gmail.com" , "linux-nfs@vger.kernel.org" Subject: Re: [PATCH] nfs: fix NULL deference in nfs4_get_valid_delegation Message-ID: <20200511135745.GB8629@fieldses.org> References: <20200508221935.GA11225@fieldses.org> <20200511121054.l2j34vnwqxhvd2ao@gabell> <20200511131637.GA8629@fieldses.org> <8f9f84f11df6f5caf054d1eada2d91ea158a6882.camel@hammerspace.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8f9f84f11df6f5caf054d1eada2d91ea158a6882.camel@hammerspace.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org On Mon, May 11, 2020 at 01:43:30PM +0000, Trond Myklebust wrote: > On Mon, 2020-05-11 at 09:16 -0400, J. Bruce Fields wrote: > > Thanks, applying.--b. > > > > You're applying? So should I remove it from the NFS client bugfixes? No. Sorry, I responded to the wrong email! --b. > > > On Mon, May 11, 2020 at 08:10:54AM -0400, Masayoshi Mizuma wrote: > > > On Fri, May 08, 2020 at 06:19:35PM -0400, J. Bruce Fields wrote: > > > > From: "J. Bruce Fields" > > > > > > > > We add the new state to the nfsi->open_states list, making it > > > > potentially visible to other threads, before we've finished > > > > initializing > > > > it. > > > > > > > > That wasn't a problem when all the readers were also taking the > > > > i_lock > > > > (as we do here), but since we switched to RCU, there's now a > > > > possibility > > > > that a reader could see the partially initialized state. > > > > > > > > Symptoms observed were a crash when another thread called > > > > nfs4_get_valid_delegation() on a NULL inode. > > > > > > > > Fixes: 9ae075fdd190 "NFSv4: Convert open state lookup to use RCU" > > > > Signed-off-by: J. Bruce Fields > > > > --- > > > > fs/nfs/nfs4state.c | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c > > > > index ac93715c05a4..a8dc25ce48bb 100644 > > > > --- a/fs/nfs/nfs4state.c > > > > +++ b/fs/nfs/nfs4state.c > > > > @@ -734,9 +734,9 @@ nfs4_get_open_state(struct inode *inode, > > > > struct nfs4_state_owner *owner) > > > > state = new; > > > > state->owner = owner; > > > > atomic_inc(&owner->so_count); > > > > - list_add_rcu(&state->inode_states, &nfsi->open_states); > > > > ihold(inode); > > > > state->inode = inode; > > > > + list_add_rcu(&state->inode_states, &nfsi->open_states); > > > > spin_unlock(&inode->i_lock); > > > > /* Note: The reclaim code dictates that we add > > > > stateless > > > > * and read-only stateids to the end of the list */ > > > > -- > > > > > > Thank you for posting the patch! It works for our box. > > > Please feel free to add: > > > > > > Reviewed-by: Seiichi Ikarashi > > > Tested-by: Daisuke Matsuda > > > Tested-by: Masayoshi Mizuma > > > > > > Without the patch, the system which is a NFSv4 client has been > > > crashed randomly. The panic log is such as: > > > > > > BUG: unable to handle page fault for address: ffffffffffffffb0 > > > ... > > > RIP: 0010:nfs4_get_valid_delegation+0x6/0x30 [nfsv4] > > > ... > > > Call Trace: > > > nfs4_open_prepare+0x80/0x1c0 [nfsv4] > > > __rpc_execute+0x75/0x390 [sunrpc] > > > ? finish_task_switch+0x75/0x260 > > > rpc_async_schedule+0x29/0x40 [sunrpc] > > > process_one_work+0x1ad/0x370 > > > worker_thread+0x30/0x390 > > > ? create_worker+0x1a0/0x1a0 > > > kthread+0x10c/0x130 > > > ? kthread_park+0x80/0x80 > > > ret_from_fork+0x22/0x30 > > > > > > After applied the patch, the panic is gone. > > > > > > Thanks! > > > Masa > -- > Trond Myklebust > Linux NFS client maintainer, Hammerspace > trond.myklebust@hammerspace.com > >