Received: by 2002:a05:7412:d1aa:b0:fc:a2b0:25d7 with SMTP id ba42csp432254rdb; Mon, 29 Jan 2024 06:50:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IGGXSkA8oDh6AvAoBYIE1xcgnUCOSWcA08KQdsjBlWocpjPzIvM6VIy8nx2atu7aRprCwqt X-Received: by 2002:a05:622a:30f:b0:42a:62fe:e13e with SMTP id q15-20020a05622a030f00b0042a62fee13emr5347582qtw.10.1706539839761; Mon, 29 Jan 2024 06:50:39 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706539839; cv=pass; d=google.com; s=arc-20160816; b=Bnz1YABYR/5KM5Wdlyoxd0Rvzbh9uUmvh+jJP56TOlqOBPGVJ/YisSKjIH1Sf2+K2B 2jpE5xBySoo+pOFiGY3mvUw6oLDU0g1T8WXReqHKN2y9v1qAeRacPcQ4nl9lKhsSh3P0 uu0Ov8oTITIMdPaf2d6KjsHRjwYi1DLfW4QHAAmJP18DMPrmAbWIIX7hQmiwFQupkVBX vtQ4cFP/MvxPCRlPJ3XtA4emcqKcwjxXonTaBt3qW0ukjbo1xz0H4MLys9n/vh8/89Cy LswOW8b/quEqo6J+xvxxYBjl68Lhm/FH/A7eY2OCBMkyedaf6wv4IubfY8/o1W/17JL/ r0+A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:user-agent:message-id:date:to :from:subject:dkim-signature; bh=GLukOLsWXdZQyzmsiKIzGtABlY3f0ia5Two8S2ZVnRk=; fh=LKxEe41Gr2gTDki9xCc9CAGhSERUe+JHYnWi9W0utDU=; b=oE6uPxLgePI0BUyjno/d6Fu70br2twnZSq43NxHY9fEkItLXW4dJr4/9HpRTSzIyjz anhF/RtRnsMbhcE4s1PM0wYmhIPnS1bY+cgmSyrHZG8YvpOhUwJ+ahzyZGjYegdAOxpV aygfTohJMibkrxedG7RaotuCMUvrwsXT4YCoLkvOiIAgUotgNr7W3mZupEHwKTkK9WZ4 QQzQ9JwE7pgHtxZxXAuwF4Em+KMR69+hLDVwvJiqxS1l8jNj/X59kIeYs+t9H35AYQwJ lsHwVVVVGuXW1+IrNa7K+hc64GIzzxXlW+WgaC/XPBHt27U7cvL6K1xNLcfzLKWCJEeS IgvQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YFRmNkMd; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-nfs+bounces-1551-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-nfs+bounces-1551-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id h11-20020ac8584b000000b0042a9cfcbdbdsi3350118qth.191.2024.01.29.06.50.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 06:50:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-nfs+bounces-1551-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YFRmNkMd; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-nfs+bounces-1551-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-nfs+bounces-1551-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 6B90C1C23290 for ; Mon, 29 Jan 2024 14:50:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 60C011487DA; Mon, 29 Jan 2024 14:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YFRmNkMd" X-Original-To: linux-nfs@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 398F4657DB; Mon, 29 Jan 2024 14:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706539832; cv=none; b=AJ73yaaeGwpdz6ARlu85oUcuGOUB6yY3d5rAnR1XEwpiMInBOHUmtlHjL+Ot/+f1K3/pJMWS7Lr8iIKT1jaTWSTDXyHVnEdeLvCAblLVUt01K+vNgXQK9iacj1B1bwMdKMQbD9dTVVtao5eXFJipsSv3vGQSO7FcnjEDb1j3PVI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706539832; c=relaxed/simple; bh=WR7CaSkWa+MpjiMPr9rAAIPRit8J5SbzTCIEV1+UMZs=; h=Subject:From:To:Date:Message-ID:MIME-Version:Content-Type; b=grjRAtGNEU/H9slG3BrAfJz7ujujpgxUvq5CrH/x9o4P/QHUJId9T4Bw5RpNHTgiwfLtMUHCr/nId2CwVObGW9KyZBnRKYPoxfvJZbHKAYlPPF8ZYsfEXaMdcpRjgUKsEi9Jodbna/FvQug2/6RbQ7M7S/5PrE/Eh07p3mEasTc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YFRmNkMd; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id D6142C433C7; Mon, 29 Jan 2024 14:50:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706539832; bh=WR7CaSkWa+MpjiMPr9rAAIPRit8J5SbzTCIEV1+UMZs=; h=Subject:From:To:Date:From; b=YFRmNkMd26rAlCknJmrhcwXi7HP0cgPUkgQ+yumV4QQOZzNv6nS96hTVRuejN/Ak/ 4F1EBKZXO5iHSBzz/DQ3kDEn+tO5FqWNukcOfrLucqUZqPkn7+uwF1o1JPdQk0iBV+ +/x4OVaaz+dcBeHIfVDld2EUoYnMiiYgKig/OVpu6leeRZ8vVy+IyaZVuGtbwXM0BC OvJiM/oK4rIqEPjaUs8H0k0noI4BC/attOyAPnWj8irGS5+46FXJFs6/qKqAcnam9e 88i5gpf0988hbuYYf2CS/OvPZc7RyBJ7HxZ91WMW06ais1ch6dz3YY6eLiuPTf6bGt Wg1Mu6cddUecw== Subject: [PATCH v1 00/11] NFSD RDMA transport improvements From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 29 Jan 2024 09:50:30 -0500 Message-ID: <170653967395.24162.4661804176845293777.stgit@manet.1015granger.net> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit These were left over from the last series (for 6.8). The idea here is to post all work needed for sending one Reply with just a single ib_post_send() -- the Send WR and all Write WRs are chained together. The purpose of that is to reduce the number of doorbells and completions per RPC, which will hopefully improve transport scalability. --- Chuck Lever (11): svcrdma: Reserve an extra WQE for ib_drain_rq() svcrdma: Use all allocated Send Queue entries svcrdma: Increase the per-transport rw_ctx count svcrdma: Fix SQ wake-ups svcrdma: Prevent a UAF in svc_rdma_send() svcrdma: Fix retry loop in svc_rdma_send() svcrdma: Post Send WR chain svcrdma: Move write_info for Reply chunks into struct svc_rdma_send_ctxt svcrdma: Post the Reply chunk and Send WR together svcrdma: Post WRs for Write chunks in svc_rdma_sendto() svcrdma: Add Write chunk WRs to the RPC's Send WR chain include/linux/sunrpc/svc_rdma.h | 55 ++++- include/trace/events/rpcrdma.h | 4 + net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 245 ++++++++++++++------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 151 ++++++++----- net/sunrpc/xprtrdma/svc_rdma_transport.c | 46 ++-- 6 files changed, 338 insertions(+), 165 deletions(-) -- Chuck Lever