Received: by 2002:a05:7412:2a8c:b0:e2:908c:2ebd with SMTP id u12csp4158909rdh; Fri, 29 Sep 2023 13:02:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEQPg2aUzCPWGDzN8puQ4kIsP6qUFitbQ2j44As3oNIIF3/RLH0jEG8eLRaywKxe1hlbk34 X-Received: by 2002:a05:6870:3281:b0:1d5:9b34:94e1 with SMTP id q1-20020a056870328100b001d59b3494e1mr5377959oac.59.1696017767067; Fri, 29 Sep 2023 13:02:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696017767; cv=none; d=google.com; s=arc-20160816; b=WMl1XxD9Upru38cZLMUdjMo1hLKBQr9KFdUBD/n2q5VflZz5F42ni3SmnpzcptgvOj VMlNQYctRSHfW0xP7mBl1YC8/v5lQF4iwJhAQlBrX5i1RjNJKz99JS/hH53uNtk/6u0j YIqMYi8qsmofgOv9Y93Iun/qTyB5LmNUGRMtcIUuTScPU0DzgnVlBxJWePyapbcJsvz+ wkUMsFLphAUyNtQ8wVge2on2QkbpHbb4LOmSeMrkbwCv2Ieb7hq7omATxBlx0rhhJb7V cIZE9574r6xmVMxlyDvkxlonod2IvaTl6jklwPTIaWdDjuXisTfy2mv+ltbtwr05Iy6u EDlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=PT/gxRoshkKTmcPZcWfZDHHm18qFEqlVOXhm8WI1FfY=; fh=TrSvJtkCL2r+xwmHkjLtKiVT6lX2BXt9y5z9/DQVde4=; b=Rh4I0Kd5d6rzqqj4t+YkxFtpdNfOJtbMQa+E3sDFsoPyVvXdSHaIIvueYmNI8A7NW1 VaTZbB/IApWKp1We0e9DyptTw0aOYRlyS43oizVtUU71BD0sZUdFG657d+NMYaLBSzLM FXdT02G+GzWKtt2r7HmwYPqqI0OvwksURdP8CWOw4sC8mvnIHTi3Ucbh3uTiOWSq4ReQ HPXmO1AShHVgqdrb86hXLLE4u6Sc8/5UAvOlWhEvLDUcYBNhiX6HFyuzRZOGPvPshig4 N1vVd7x7xsW0hw9Gf2DT+v4wTQbQujElQYOn+6LQ7LDRVFGBFBrWcrWX/cGieQlT6tQP 7HOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mAaTDsqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id b19-20020a637153000000b00578a66db83fsi10688158pgn.141.2023.09.29.13.02.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 13:02:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mAaTDsqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 612FF82A95D5; Fri, 29 Sep 2023 04:58:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232925AbjI2L5p (ORCPT + 99 others); Fri, 29 Sep 2023 07:57:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229508AbjI2L5o (ORCPT ); Fri, 29 Sep 2023 07:57:44 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A830193; Fri, 29 Sep 2023 04:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695988661; x=1727524661; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=O9aEWMt/4svyxRYZ/LYDGVClZj5FenXgVAAtjploJC8=; b=mAaTDsqR9Vd1YgUXYrqv8GzYHjxJnn3Ft89F4ebzvLbAV/1vf4kPVIhz e9Lbh3dVnhsdn8HXoi3LhXpyy4vN95YsyxAbEnJNJrlHJmCQihZMdijTu 7afZrCavWuqh3fwQiPW6kr82k4LwdbfZUgT8rGkbK/Z2uX0WmB55kxREI awj66U9N/rkpr7eDHU+SdHQQDkqjkoSeXP3M8Owj+zT2fE0RttwhWoYR/ REYsQ8dA0tbMN4KK1n7tl7tmGmfqIpsLOn+MGSA5SoseVFdPiAYpWBB/3 1svPJxGK1MDJS4q3rhPt9dk/xLK+HhbvUyAreiTAMNwDBGOSv6N4L2L5S A==; X-IronPort-AV: E=McAfee;i="6600,9927,10847"; a="361658664" X-IronPort-AV: E=Sophos;i="6.03,187,1694761200"; d="scan'208";a="361658664" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 04:57:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10847"; a="873668379" X-IronPort-AV: E=Sophos;i="6.03,187,1694761200"; d="scan'208";a="873668379" Received: from valeks2x-mobl.ger.corp.intel.com (HELO localhost) ([10.252.53.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 04:57:32 -0700 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Lukas Wunner , Alexandru Gagniuc , Krishna chaitanya chundru , Srinivas Pandruvada , "Rafael J . Wysocki" , linux-pm@vger.kernel.org Cc: LKML , Alex Deucher , Daniel Lezcano , Amit Kucheria , Zhang Rui , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= Subject: [PATCH v3 00/10] Add PCIe Bandwidth Controller Date: Fri, 29 Sep 2023 14:57:13 +0300 Message-Id: <20230929115723.7864-1-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 29 Sep 2023 04:58:10 -0700 (PDT) Hi all, This series adds PCIe bandwidth controller (bwctrl) and associated PCIe cooling driver to the thermal core side for limiting PCIe Link Speed due to thermal reasons. PCIe bandwidth controller is a PCI express bus port service driver. A cooling device is created for each port the service driver finds if they support changing speeds. This series only adds support for controlling PCIe Link Speed. Controlling PCIe Link Width might also be useful but AFAIK, there is no mechanism for that until PCIe 6.0 (L0p). Based on feedback for v1, the thermal/cooling device side prefers Link Speed and Link Width to be separate cooling devices [1] which is taken into account in naming the cooling device for Link Speed but the Link Width one is not added yet as it would not be able to control anything at the moment. bwctrl is built on top of BW notifications revert. I'm just not sure what is the best practice when re-adding some old functionality in a modified form so please let me know if I need to somehow alter that patch. [1] https://lore.kernel.org/linux-pci/f35db90cd67adf4b0f48cd6f2a6ad8fbd0c1a679.camel@linux.intel.com/ v3: - Correct hfi1 shortlog prefix - Improve error prints in hfi1 - Add L: linux-pci to the MAINTAINERS entry v2: - Adds LNKCTL2 to RMW safe list in Documentation/PCI/pciebus-howto.rst - Renamed cooling devices from PCIe_Port_* to PCIe_Port_Link_Speed_* in order to plan for possibility of adding Link Width cooling devices later on - Moved struct thermal_cooling_device declaration to the correct patch - Small tweaks to Kconfig texts - Series rebased to resolve conflict (in the selftest list) Ilpo Järvinen (10): PCI: Protect Link Control 2 Register with RMW locking drm/radeon: Use RMW accessors for changing LNKCTL2 drm/amdgpu: Use RMW accessors for changing LNKCTL2 RDMA/hfi1: Use RMW accessors for changing LNKCTL2 PCI: Store all PCIe Supported Link Speeds PCI: Cache PCIe device's Supported Speed Vector PCI/LINK: Re-add BW notification portdrv as PCIe BW controller PCI/bwctrl: Add "controller" part into PCIe bwctrl thermal: Add PCIe cooling driver selftests/pcie_bwctrl: Create selftests Documentation/PCI/pciebus-howto.rst | 8 +- MAINTAINERS | 9 + drivers/gpu/drm/amd/amdgpu/cik.c | 41 +-- drivers/gpu/drm/amd/amdgpu/si.c | 41 +-- drivers/gpu/drm/radeon/cik.c | 40 +-- drivers/gpu/drm/radeon/si.c | 40 +-- drivers/infiniband/hw/hfi1/pcie.c | 30 +- drivers/pci/pcie/Kconfig | 9 + drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/bwctrl.c | 309 ++++++++++++++++++ drivers/pci/pcie/portdrv.c | 9 +- drivers/pci/pcie/portdrv.h | 10 +- drivers/pci/probe.c | 38 ++- drivers/pci/remove.c | 2 + drivers/thermal/Kconfig | 10 + drivers/thermal/Makefile | 2 + drivers/thermal/pcie_cooling.c | 107 ++++++ include/linux/pci-bwctrl.h | 33 ++ include/linux/pci.h | 3 + include/uapi/linux/pci_regs.h | 1 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/pcie_bwctrl/Makefile | 2 + .../pcie_bwctrl/set_pcie_cooling_state.sh | 122 +++++++ .../selftests/pcie_bwctrl/set_pcie_speed.sh | 67 ++++ 24 files changed, 800 insertions(+), 135 deletions(-) create mode 100644 drivers/pci/pcie/bwctrl.c create mode 100644 drivers/thermal/pcie_cooling.c create mode 100644 include/linux/pci-bwctrl.h create mode 100644 tools/testing/selftests/pcie_bwctrl/Makefile create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_cooling_state.sh create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_speed.sh -- 2.30.2