Received: by 2002:a05:6358:11c7:b0:104:8066:f915 with SMTP id i7csp4092659rwl; Mon, 10 Apr 2023 06:09:35 -0700 (PDT) X-Google-Smtp-Source: AKy350ahbSkdyYIfUEvp7aUXrUFvl0q7C+kUm5/I5l87mwx4yupfYtNdRpqLN7IorF9uOHXdwDfs X-Received: by 2002:aa7:d855:0:b0:504:88fb:8841 with SMTP id f21-20020aa7d855000000b0050488fb8841mr6490354eds.21.1681132175294; Mon, 10 Apr 2023 06:09:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681132175; cv=none; d=google.com; s=arc-20160816; b=bdVrxwtAXNSBacdxo61mHyiG4Ajr48zXl2lxic0aR/Jvi0ew1HZM3cGWVEso4rbvoR 0M2Qr1XleFG2nk6s7jQfSHCdB/xXG0vUR3wyPzn6Xiunxvn9pJyVDUthmA1kKqMLdTDi icRiCXgiYexIGhghUO4poyQdVrNoLPrbgJiuofgOk8Dt163QjHQGn8pnnSkdG1Oo2en8 AFgNQToc1olPCKPMbmttUBR13o19xSrJY2nEn3oIDQuRqsuchpZmR1Z03ja9f9opZ99+ t8pcCW7Om4DuQi8GLjcc+x9+i7DiEb52m0ulkDQSKZqoBdbi8tG24Vn2yg86yQhJxj8d kOtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=7aI4YEqZyyAkkZ1bKbVfp8oeMzbXbEwBC+CLfGrEUnQ=; b=yy4PcWecqw5W2ia3I7W55hTPUYU4+pwjN7qwYkQWaRjQTAkEG13DuEIEGyxN1jiwIC BbXd27aWvHDVe+G7Ilr0XgGz8QMLcfF/w3e1SSy7bjrBQZ3rLmwDidVZrdmX3f67kony WPTnwOCWCCJuP+euMxgiUn/I0oonVKffrwNkh5kqPUtEnnXx74xbLW8ALXKhwObuYGal seFI8abwTz5++JFK/cBLuOJ1ZWhLV51+ZVAgyhdxXINpZIfYrZ6zuRLDu3XGlthG/sIX 9fT8kwhUvqP3RdhmXHynXY6aeUMDO3SLyGnqaX7apiFnrVLeKNo9NZmini7y+ghKDZJX EHew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uYs88a7c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j18-20020aa7c0d2000000b005002e947c3fsi6158656edp.119.2023.04.10.06.09.09; Mon, 10 Apr 2023 06:09:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uYs88a7c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229795AbjDJNIC (ORCPT + 99 others); Mon, 10 Apr 2023 09:08:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229614AbjDJNIB (ORCPT ); Mon, 10 Apr 2023 09:08:01 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97AB0212F; Mon, 10 Apr 2023 06:07:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 32DC260F55; Mon, 10 Apr 2023 13:07:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 110A5C433D2; Mon, 10 Apr 2023 13:07:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681132078; bh=3aM8xzi3Ejc9lw0KWj0iPFmTjjhZiJ/W+M8ER7GMoLE=; h=From:To:Cc:Subject:Date:From; b=uYs88a7cywe+vk7/CcSmdMTe1aljB6GFFqTWSGvZ7drx6m8BpaW+9afYDCXMw/sxq usn7CgaZBdEEhTpIePVtUo67enAq/bKpz5yahqfo8M9rDwHtdmWiqkk3mydBRr1jKw Nk9EqgZdK4Qqwcahzpx0PTeCG4ggpFbg2EW4/UAV4IBS8XhGk5NY8zGrGL5rr8T7pi i66sDu5JEYVjY/l4j6UL1U/foOZouVfGYMuXmAtrTqqVeVT8CPLw/kdEz3uIyG4QeE Gno4vrMOn3spYcZ4uFK/d9lSe2EiMtUgMzNRxOdGdSXAWXtBj9lHQw0KqH7CPygwz6 iA/Pi1NHHzU8w== From: Leon Romanovsky To: Jason Gunthorpe Cc: Leon Romanovsky , Avihai Horon , Aya Levin , Eric Dumazet , Jakub Kicinski , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, Meir Lichtinger , Michael Guralnik , netdev@vger.kernel.org, Paolo Abeni , Saeed Mahameed , Shay Drory Subject: [PATCH rdma-next 0/4] Allow relaxed ordering read in VFs and VMs Date: Mon, 10 Apr 2023 16:07:49 +0300 Message-Id: X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Leon Romanovsky From Avihai, Currently, Relaxed Ordering (RO) can't be used in VFs directly and in VFs assigned to QEMU, even if the PF supports RO. This is due to issues in reporting/emulation of PCI config space RO bit and due to current HCA capability behavior. This series fixes it by using a new HCA capability and by relying on FW to do the "right thing" according to the PF's PCI config space RO value. Allowing RO in VFs and VMs is valuable since it can greatly improve performance on some setups. For example, testing throughput of a VF on an AMD EPYC 7763 and ConnectX-6 Dx setup showed roughly 60% performance improvement. Thanks Avihai Horon (4): RDMA/mlx5: Remove pcie_relaxed_ordering_enabled() check for RO write RDMA/mlx5: Check pcie_relaxed_ordering_enabled() in UMR net/mlx5: Update relaxed ordering read HCA capabilities RDMA/mlx5: Allow relaxed ordering read in VFs and VMs drivers/infiniband/hw/mlx5/mr.c | 12 ++++++++---- drivers/infiniband/hw/mlx5/umr.c | 7 +++++-- drivers/infiniband/hw/mlx5/umr.h | 3 ++- drivers/net/ethernet/mellanox/mlx5/core/en/params.c | 3 +-- drivers/net/ethernet/mellanox/mlx5/core/en_common.c | 9 +++++---- include/linux/mlx5/mlx5_ifc.h | 5 +++-- 6 files changed, 24 insertions(+), 15 deletions(-) -- 2.39.2