Received: by 2002:a25:c593:0:0:0:0:0 with SMTP id v141csp790660ybe; Fri, 13 Sep 2019 06:26:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqyF/QR8A7e7tVaC4IgaTUQMEPlI+h2M5gV7j4+IvCMlC4w14SsJn9HyPePXlfRWnMeHBkEH X-Received: by 2002:a17:906:6bd0:: with SMTP id t16mr40307455ejs.189.1568381197232; Fri, 13 Sep 2019 06:26:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568381197; cv=none; d=google.com; s=arc-20160816; b=wlTgekR29xxGnmeh5qUuzaeW20NsqhnclcF3+1uzngAhLI4HsadR+DM/jwUJ9sDfn+ TLaHeYewSX2xI6mlJ5Z5nzhe4uRKJ9bkboH2W6njD+SkckEbBziV4UYHL0w33BSGcXm8 ZwHIHNlXVobikqPv5CctX8cUTTUGYdt4U9wdAOA1k3uwQtPzVhWMU+Kc4YYKmZFsVGqk si+r7U++8eO0YGm5sqDyFrS4L/mnYSOnbQIOmznpZQGzZKHD5/8YTi9F0faslJVQOO8e V5PUjMfRrEuJRUorDbhcPaM2ECWEeXy7AC37O3tpCCspKrMM2BCcm+q8ClTrujVKm9f4 xR1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=va82r55Oy/1yXkZqIDkzIK4hy21MrTgjLGmsLXo3xmE=; b=uCH7NGqDQiRFqLZF8i6OxVbSSTewois2C1LTjf/LM4LXLU3O/RdSbqL+txsI8g4/Xu ursoay29gyZdUC6yGiB5r0w+ErK61vSz3u/Kn+IqwgDV/ZD88KaL0UPb8U1Y3TGVNFJl aLZQrSblOFm8lHLzXC16mdOatsluVK5wXsAXEDyOfBjxAlPk7DDNuMP0gSaQS26uFhLe iMYGB9mzqPGzayoSRswzXKmKM9qk3fVin1YgXZAeN7BwchNsxZavPFU04F0h9ahzLtZG aQetgBhoudHG2x4pX+PI2nsP+mBVQ568oAZcQvhJMUWOyvb2czET7tiSOJiXKpXZBoQY BXUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=zepvc19J; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id ba7si11933892edb.76.2019.09.13.06.26.14; Fri, 13 Sep 2019 06:26:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=zepvc19J; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390193AbfIMNVX (ORCPT + 99 others); Fri, 13 Sep 2019 09:21:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:50846 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388155AbfIMNVT (ORCPT ); Fri, 13 Sep 2019 09:21:19 -0400 Received: from localhost (unknown [104.132.45.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 81886206A5; Fri, 13 Sep 2019 13:21:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1568380879; bh=upUGwm/msuem5sT6UoS7SFgRcUJ9ZfVcRt9ovRejGh0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zepvc19JQyh51SB3EWTh82LZ07vs7x+TlGowziSgSiBYQwwpH1BSNU+44ZxLxhGyx B/7gL07zTaCnoPevgbmULpxyn4NyLbyAf6U1Hnm/1Qba5pvr7ZfrlzCVqpbJyAUTek CsH5xU9NnIHhf6VpQqOEsFiMfPT+2UjrSJ0HsTl4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn , Dennis Dalessandro , Doug Ledford Subject: [PATCH 5.2 21/37] IB/rdmavt: Add new completion inline Date: Fri, 13 Sep 2019 14:07:26 +0100 Message-Id: <20190913130519.201854592@linuxfoundation.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190913130510.727515099@linuxfoundation.org> References: <20190913130510.727515099@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org There is opencoded send completion logic all over all the drivers. We need to convert to this routine to enforce ordering issues for completions. This routine fixes an ordering issue where the read of the SWQE fields necessary for creating the completion can race with a post send if the post send catches a send queue at the edge of being full. Is is possible in that situation to read SWQE fields that are being written. This new routine insures that SWQE fields are read prior to advancing the index that post send uses to determine queue fullness. Reviewed-by: Michael J. Ruhl Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro Signed-off-by: Doug Ledford --- include/rdma/rdmavt_qp.h | 72 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h index 68e38c20afc04..6014f17669071 100644 --- a/include/rdma/rdmavt_qp.h +++ b/include/rdma/rdmavt_qp.h @@ -737,6 +737,78 @@ static inline void rvt_put_qp_swqe(struct rvt_qp *qp, struct rvt_swqe *wqe) atomic_dec(&ibah_to_rvtah(wqe->ud_wr.ah)->refcount); } +/** + * rvt_qp_sqwe_incr - increment ring index + * @qp: the qp + * @val: the starting value + * + * Return: the new value wrapping as appropriate + */ +static inline u32 +rvt_qp_swqe_incr(struct rvt_qp *qp, u32 val) +{ + if (++val >= qp->s_size) + val = 0; + return val; +} + +/** + * rvt_qp_complete_swqe - insert send completion + * @qp - the qp + * @wqe - the send wqe + * @opcode - wc operation (driver dependent) + * @status - completion status + * + * Update the s_last information, and then insert a send + * completion into the completion + * queue if the qp indicates it should be done. + * + * See IBTA 10.7.3.1 for info on completion + * control. + * + * Return: new last + */ +static inline u32 +rvt_qp_complete_swqe(struct rvt_qp *qp, + struct rvt_swqe *wqe, + enum ib_wc_opcode opcode, + enum ib_wc_status status) +{ + bool need_completion; + u64 wr_id; + u32 byte_len, last; + int flags = wqe->wr.send_flags; + + rvt_put_qp_swqe(qp, wqe); + + need_completion = + !(flags & RVT_SEND_RESERVE_USED) && + (!(qp->s_flags & RVT_S_SIGNAL_REQ_WR) || + (flags & IB_SEND_SIGNALED) || + status != IB_WC_SUCCESS); + if (need_completion) { + wr_id = wqe->wr.wr_id; + byte_len = wqe->length; + /* above fields required before writing s_last */ + } + last = rvt_qp_swqe_incr(qp, qp->s_last); + /* see rvt_qp_is_avail() */ + smp_store_release(&qp->s_last, last); + if (need_completion) { + struct ib_wc w = { + .wr_id = wr_id, + .status = status, + .opcode = opcode, + .qp = &qp->ibqp, + .byte_len = byte_len, + }; + + rvt_cq_enter(ibcq_to_rvtcq(qp->ibqp.send_cq), &w, + status != IB_WC_SUCCESS); + } + return last; +} + extern const int ib_rvt_state_ops[]; struct rvt_dev_info; -- 2.20.1