Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
From:   Ajay Kaher <akaher@vmware.com>
To:     bhelgaas@google.com, helgaas@kernel.org, gregkh@linuxfoundation.org
Cc:     ymcavikas@gmail.com, akaher@vmware.com, amakhalov@vmware.com,
        anishs@vmware.com, linux-kernel@vger.kernel.org,
        linux-pci@vger.kernel.org, srinidhir@vmware.com,
        srivatsa@csail.mit.edu, srivatsab@vmware.com,
        vsirnapalli@vmware.com, Vikash Bansal <bvikas@vmware.com>
Subject: [PATCH v3] PCI: Speed up device init by parsing capabilities all at once
Date:   Wed, 23 Feb 2022 17:04:09 +0530
Message-Id: <1645616049-19712-1-git-send-email-akaher@vmware.com>
In-Reply-To: <6827AF44-4412-4ABA-A66B-562D7E86C847@vmware.com>
References: <6827AF44-4412-4ABA-A66B-562D7E86C847@vmware.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?83ammAkQQzD4LgmC/nFOYJiRS9gxLSsClB2UzVHtrdIxZBSVNGY5Ye4NaTzd?=
 =?us-ascii?Q?OzqIt8h0EvaqSMFQtw1KoIkaFsHUn93BQzgkwv+fQ+bmx3USpwiKnqRQiJfl?=
 =?us-ascii?Q?9ezDhBO3cNePQK//OQBlgkSNtogucSq/3vNVnYcgySSMH8+oNs0XVoJILWi3?=
 =?us-ascii?Q?JYe9RQ3o5bhzz27cuWYvj9sKohZ2p2RNRyzw1QsdoVJ85nnhJKE1C+pAlpW3?=
 =?us-ascii?Q?kv6cwHJblEWsj0Ds4PhOEvNo6y39jCn8VqRlxrQCZjWZ+gwY8eFv9/iM78D3?=
 =?us-ascii?Q?/gFAWCKPdAbky//3lGblXKCjHyHydnQA5gzfn5Es2e+vGSnSDpERclVY6apA?=
 =?us-ascii?Q?E+aZZEZlWifwyvpz8Ybc6SbMdUd5s+0zoPKIEEluHS6LnnQ+NW2tAwq6GuTJ?=
 =?us-ascii?Q?Ui/JnPNCZoAUSnaKBPGG1CVC9bocEiZe3ZNMqq+kUl+fn0zD3YN1GdoGo85y?=
 =?us-ascii?Q?srul0rO8iyb8ozdAgukTgMvcjPAaW2+jCzzXh8XGP8LOJbxbLAi3FNO+EWVR?=
 =?us-ascii?Q?cORZGyznGnCGnqTLIAFgtaCZw0HdCa5uJ26LhQgEATnWQW86XXQqWqTqFXhy?=
 =?us-ascii?Q?vANs4z3nQjZNFIb2muREcnt+iinZVB01wx+0jSti4nQDa+NML6z8+tHvXP2t?=
 =?us-ascii?Q?TyUI10iltYF4fxu1olWoUlz73mFCnrucmcPOjEFGx9neR/ADbd9NlQxNo90Y?=
 =?us-ascii?Q?pTUV6sRE2Bi3lwL2MSblOS9u8VFPBpKGIz5bAbBn1RyWnqisB3TGfn/lW/C3?=
 =?us-ascii?Q?HD4R4hbtPhhN0oCK7QTFa+KFz+Y9vfyEdvyPzYUranARAoww7tVOu2Xu3MWA?=
 =?us-ascii?Q?zHHhFk5CsZEaHOq1Tfrf2ahWlmNrunWaQWv3sBB2I/Z1H7AH6U7vN9pIV+7L?=
 =?us-ascii?Q?KrOaORpPc+BJedkp4M6rLVHPzWwz5SU3qYTm0IbB3nb315yAYszHW6nOYk+a?=
 =?us-ascii?Q?qP1psNt9aJhe4nkqat2colaQsc9s1POliA+1jF+6qBHXBnGTP7aOp9/211BQ?=
 =?us-ascii?Q?n2hvhfDPGkeJONMraNv6z87jtRL++rqeEqiw7CahQdD/X8G44HSwJB1Y9OQH?=
 =?us-ascii?Q?IPzdbG23rTwJkK2hH9UCW98bbc1EA6+aBQXIZkP3mt2qDe9dd+5o1DdFvjmW?=
 =?us-ascii?Q?sKsf/9CqJh75Zpt7pWIojYF1yCuIqwnzj2Ako5LzSaqT1+hjPoBTnDHN4uqr?=
 =?us-ascii?Q?nCH0oe33vyIxQXpcbqLtRU9Zi0E7k9pJMVdxwxF/X3VvIkmu3Gd+m2TwHLcd?=
 =?us-ascii?Q?1tPo/hHk/R6BTkXOhO+neU05Y0PnwM4fuGaltbonJAQkQBBf2yMNQB7tH+Bl?=
 =?us-ascii?Q?scFgDLf39bLcjWDoAapjl15O84vZLXaq2JEy+Erkhz/lqbbGA3vbzzsiFI1D?=
 =?us-ascii?Q?kOw2swSWLJ6JrJ6omMHtIcPJps9lYygmclXD4Up//U9eCiMoPD0XCt0C2jp8?=
 =?us-ascii?Q?+hn6VEEvDuWnuhvfLicXNlqa8wk1IlRsP2Dfoqn5mKCBszzJHfGsZFHewi42?=
 =?us-ascii?Q?DT6z6AR6YJt+Jq32hFwKvT+07NE+d6w8AjMBmN/beV/Zj/V+rkHEmACgYi4b?=
 =?us-ascii?Q?GZfOBbC9LeVxdNSOJx1VSbL1VqCC2wDeRhbOM3Ynq+tX5UTwmVTjjkfa9Jzh?=
 =?us-ascii?Q?Xg=3D=3D?=
X-OriginatorOrg: vmware.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 763720c7-54ba-4c96-6a5e-08d9f6c2b33a
X-MS-Exchange-CrossTenant-AuthSource: PH0PR05MB8703.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2022 11:50:34.2392
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A3zZi3iOT/iSsd023DCJG+tPM8HGo7O4S+UAB3vAtBypbX8KtWDI+QqNSUH0Po7p3jjqje7s1/oR2xKQFeNYow==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR05MB7334
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Vikash Bansal <bvikas@vmware.com>

In the current implementation, the PCI capability list is parsed from
the beginning to find each capability, which results in a large number
of redundant PCI reads.

Instead, we can parse the complete list just once, store it in the
pci_dev structure, and get the offset of each capability directly from
the pci_dev structure.

This implementation improves pci devices initialization time  by ~2-3%
(from 270ms to 261ms) in case of bare metal and 7-8% (From 201ms to 184ms)
in case of VM running on ESXi.

It also adds a memory overhead of 20bytes (value of PCI_CAP_ID_MAX) per
PCI device.

Ran pahole for pci_dev structure. This patch is not adding any padding
bytes.

Signed-off-by: Vikash Bansal <bvikas@vmware.com>
Signed-off-by: Ajay Kaher <akaher@vmware.com>

---

Changes in v3:
  - Added check before access cap_off from pci_find_capability().
  - Moved declaration of pci_find_all_capabilities() to drivers/pci/pci.h.

Changes in v2:
  - Ran pahole tool.
  - Modified comments to add "clock time".
  - Removed comments before call to pci_find_all_capabilities.

---
 drivers/pci/pci.c   | 44 +++++++++++++++++++++++++++++++++++++-------
 drivers/pci/pci.h   |  1 +
 drivers/pci/probe.c |  1 +
 include/linux/pci.h |  1 +
 4 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 3d2fb39..cf54811 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -469,6 +469,40 @@ static u8 __pci_bus_find_cap_start(struct pci_bus *bus,
 }
 
 /**
+ * pci_find_all_capabilities - Read all capabilities
+ * @dev: the PCI device
+ *
+ * Read all capabilities and store offsets in cap_off
+ * array in pci_dev structure.
+ */
+void pci_find_all_capabilities(struct pci_dev *dev)
+{
+	int ttl = PCI_FIND_CAP_TTL;
+	u16 ent;
+	u8 pos;
+	u8 id;
+
+	pos = __pci_bus_find_cap_start(dev->bus, dev->devfn, dev->hdr_type);
+	if (!pos)
+		return;
+	pci_bus_read_config_byte(dev->bus, dev->devfn, pos, &pos);
+	while (ttl--) {
+		if (pos < 0x40)
+			break;
+		pos &= ~3;
+		pci_bus_read_config_word(dev->bus, dev->devfn, pos, &ent);
+		id = ent & 0xff;
+		if (id == 0xff)
+			break;
+
+		/* Read first instance of capability */
+		if (!(dev->cap_off[id]))
+			dev->cap_off[id] = pos;
+		pos = (ent >> 8);
+	}
+}
+
+/**
  * pci_find_capability - query for devices' capabilities
  * @dev: PCI device to query
  * @cap: capability code
@@ -489,13 +523,9 @@ static u8 __pci_bus_find_cap_start(struct pci_bus *bus,
  */
 u8 pci_find_capability(struct pci_dev *dev, int cap)
 {
-	u8 pos;
-
-	pos = __pci_bus_find_cap_start(dev->bus, dev->devfn, dev->hdr_type);
-	if (pos)
-		pos = __pci_find_next_cap(dev->bus, dev->devfn, pos, cap);
-
-	return pos;
+	if(cap >= PCI_CAP_ID_MAX)
+		return 0;
+	return dev->cap_off[cap];
 }
 EXPORT_SYMBOL(pci_find_capability);
 
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 3d60cab..3cb70d5 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -87,6 +87,7 @@ bool pci_bridge_d3_possible(struct pci_dev *dev);
 void pci_bridge_d3_update(struct pci_dev *dev);
 void pci_bridge_wait_for_secondary_bus(struct pci_dev *dev);
 void pci_bridge_reconfigure_ltr(struct pci_dev *dev);
+void pci_find_all_capabilities(struct pci_dev *dev);
 
 static inline void pci_wakeup_event(struct pci_dev *dev)
 {
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 087d365..d75e1fa 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1839,6 +1839,7 @@ int pci_setup_device(struct pci_dev *dev)
 	dev->hdr_type = hdr_type & 0x7f;
 	dev->multifunction = !!(hdr_type & 0x80);
 	dev->error_state = pci_channel_io_normal;
+	pci_find_all_capabilities(dev);
 	set_pcie_port_type(dev);
 
 	pci_set_of_node(dev);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 18a75c8..bd61cee 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -326,6 +326,7 @@ struct pci_dev {
 	unsigned int	class;		/* 3 bytes: (base,sub,prog-if) */
 	u8		revision;	/* PCI revision, low byte of class word */
 	u8		hdr_type;	/* PCI header type (`multi' flag masked out) */
+	u8		cap_off[PCI_CAP_ID_MAX]; /* Offsets of all pci capabilities */
 #ifdef CONFIG_PCIEAER
 	u16		aer_cap;	/* AER capability offset */
 	struct aer_stats *aer_stats;	/* AER stats for this device */
-- 
2.7.4