Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp442988pxj; Tue, 18 May 2021 06:55:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwxuo5pjbyZeqV+E4/phd0Oo7FzTRaNZYGPMWwLWOJVTxJk/mzF/k9/YCQXmHrfCa5JGCs8 X-Received: by 2002:a05:6402:896:: with SMTP id e22mr7362072edy.256.1621346155316; Tue, 18 May 2021 06:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1621346155; cv=none; d=google.com; s=arc-20160816; b=Rjtw4uSJYF5/T7f7rl2BFQ/+m09jpNGFEdzogUg0thWJl3Ji0P1WVo3YNg23to2fOv H9uSoJ7X83zJmvuDjEHL/ewWP0CH8r1MAubT12LsOc7FdPQ4EqguuhA0ARjU9zqONM/s rNqu/TwzX+9rtlsjtG7wKABHsiJFcMwO3hUPlNk2uabsnQ/iCLCzDgnpNmQghPfsOvZI IdmC6SRuB+bE6N9ItN/uelF6RPf2ilvU+/m5XZzytgYajb+RvNhRjBfcE/uP+pv0m+8s Ykm7pgvhFNi8zKG5IGmU9P8PiLryDmR+1uL575aDNKWyxVEfpUZlBOdLd/aipiGy47r9 JTcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=nVj/BqKjPmFeKt4P9AulYGb2D3HwB4AbOR1raBWLnZc=; b=z2czNylCcSqGWw96hcyDke39HgOa/5CmpcEQAbVCLJVBg9NPwFdUj3eHuetVBNZwtT tg2YLl7ELnIFSWVAfiUAHaRAHgkTSKHcN5Ey/+SXlqhI0VscTfPMVaNHGyA1EaqwZu7s uTI5FVJY1gdZ74+CADhNcYlQym04MYuv9VbyH+g2N/cI1GaXr/qeLuFxuZtmTYQAn8ur Z+nrhuUnlbYnj85w3ctIeIdotM3BM4NlOZblAIrIVHhByGraZzW63jzKPmco6dPKc7Fm ZSaqZsnREsMbfUh8YuUFfcCZDPwfKH1qdDJZVV9LcgFN64CRQWv1ZQZBbZQe1wvr2FHR oseg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=yHD5BcwM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id z26si7122637ejr.452.2021.05.18.06.55.31; Tue, 18 May 2021 06:55:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=yHD5BcwM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344055AbhEQPkN (ORCPT + 99 others); Mon, 17 May 2021 11:40:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:41894 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245168AbhEQPY6 (ORCPT ); Mon, 17 May 2021 11:24:58 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B2D7361C96; Mon, 17 May 2021 14:35:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1621262158; bh=sPOblJm98xVOOlqb9zUovcLgdc/ASxL/tDJssDeQBBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=yHD5BcwMjHxbmCuYe6/r/wwv4qHZ7V0HOZMJdQShupnS0zu7Ka0vxXRCdVpFvvPPH D2lznfq4113wd/7Djhzp/gAyxd1dcDwcppny6K4owN7V9ZEOoi70iwyV59EBOQRanN Wos/i29KOQKJo/FRRP3GPDDSxFkWzMVeyrxrWfFA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Chuck Lever , Trond Myklebust , Sasha Levin Subject: [PATCH 5.10 123/289] SUNRPC: Move fault injection call sites Date: Mon, 17 May 2021 16:00:48 +0200 Message-Id: <20210517140309.303881237@linuxfoundation.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210517140305.140529752@linuxfoundation.org> References: <20210517140305.140529752@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chuck Lever [ Upstream commit 7638e0bfaed1b653d3ca663e560e9ffb44bb1030 ] I've hit some crashes that occur in the xprt_rdma_inject_disconnect path. It appears that, for some provides, rdma_disconnect() can take so long that the transport can disconnect and release its hardware resources while rdma_disconnect() is still running, resulting in a UAF in the provider. The transport's fault injection method may depend on the stability of transport data structures. That means it needs to be invoked only from contexts that hold the transport write lock. Fixes: 4a0682583988 ("SUNRPC: Transport fault injection") Signed-off-by: Chuck Lever Signed-off-by: Trond Myklebust Signed-off-by: Sasha Levin --- net/sunrpc/clnt.c | 1 - net/sunrpc/xprt.c | 6 ++++-- net/sunrpc/xprtrdma/transport.c | 6 ++++-- 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 3259120462ed..69d8843a26e0 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1802,7 +1802,6 @@ call_allocate(struct rpc_task *task) status = xprt->ops->buf_alloc(task); trace_rpc_buf_alloc(task, status); - xprt_inject_disconnect(xprt); if (status == 0) return; if (status != -ENOMEM) { diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 57f09ea3ef2a..99d07513237e 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1455,7 +1455,10 @@ bool xprt_prepare_transmit(struct rpc_task *task) void xprt_end_transmit(struct rpc_task *task) { - xprt_release_write(task->tk_rqstp->rq_xprt, task); + struct rpc_xprt *xprt = task->tk_rqstp->rq_xprt; + + xprt_inject_disconnect(xprt); + xprt_release_write(xprt, task); } /** @@ -1857,7 +1860,6 @@ void xprt_release(struct rpc_task *task) spin_unlock(&xprt->transport_lock); if (req->rq_buffer) xprt->ops->buf_free(task); - xprt_inject_disconnect(xprt); xdr_free_bvec(&req->rq_rcv_buf); xdr_free_bvec(&req->rq_snd_buf); if (req->rq_cred != NULL) diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 035060c05fd5..f93ff4282bf4 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -262,8 +262,10 @@ xprt_rdma_connect_worker(struct work_struct *work) * xprt_rdma_inject_disconnect - inject a connection fault * @xprt: transport context * - * If @xprt is connected, disconnect it to simulate spurious connection - * loss. + * If @xprt is connected, disconnect it to simulate spurious + * connection loss. Caller must hold @xprt's send lock to + * ensure that data structures and hardware resources are + * stable during the rdma_disconnect() call. */ static void xprt_rdma_inject_disconnect(struct rpc_xprt *xprt) -- 2.30.2