Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751474AbeAELFA (ORCPT + 1 other); Fri, 5 Jan 2018 06:05:00 -0500 Received: from mail-by2nam01on0084.outbound.protection.outlook.com ([104.47.34.84]:58944 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751008AbeAELEg (ORCPT ); Fri, 5 Jan 2018 06:04:36 -0500 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com; From: Ioana Radulescu To: CC: , , , , , Subject: [PATCH 2/2] staging: fsl-dpaa2/eth: Use affine DPIO services Date: Fri, 5 Jan 2018 05:04:32 -0600 Message-ID: <20180105110432.26813-3-ruxandra.radulescu@nxp.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180105110432.26813-1-ruxandra.radulescu@nxp.com> References: <20180105110432.26813-1-ruxandra.radulescu@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131596238473195305;(91ab9b29-cfa4-454e-5278-08d120cd25b8);() X-Forefront-Antispam-Report: CIP:192.88.168.50;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(376002)(346002)(396003)(39380400002)(39860400002)(2980300002)(1110001)(1109001)(339900001)(199004)(189003)(39060400002)(68736007)(36756003)(97736004)(54906003)(5660300001)(6916009)(316002)(2906002)(4326008)(2950100002)(16586007)(48376002)(53936002)(50466002)(105606002)(86362001)(2351001)(106466001)(1076002)(104016004)(85426001)(356003)(81156014)(8676002)(498600001)(305945005)(47776003)(51416003)(8936002)(76176011)(59450400001)(50226002)(77096006)(81166006)(8656006);DIR:OUT;SFP:1101;SCL:1;SRVR:MWHPR03MB2703;H:tx30smr01.am.freescale.net;FPR:;SPF:Fail;PTR:InfoDomainNonexistent;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BY2FFO11FD039;1:nn/2CTsPhchkvqWth+MmkzYjlwecJ+EB6VRQCWMxbnAI+xeQsFiD5VsX9AoFQJb5XKoPb9P80tE/l0J7F0WsngWt6fI7dqG5nm+NjYAW5Myo25ByCGVCr5RG4uraLugd MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8b9f50c7-ae34-46db-ef29-08d5542c0a6f X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(4534020)(4628075)(201703131517081)(5600026)(4604075)(2017052603307);SRVR:MWHPR03MB2703; X-Microsoft-Exchange-Diagnostics: 1;MWHPR03MB2703;3:P6V/PgORGyVIKOMtl7k6vCnuAMAfPAfrLub/vZ161EJ21Hw7HeZcz//haI4i+C+MEQqM8odUKxNhZPMHEnwjTztgWxOm4iZ3ruovO6QqNPy+2m+sqhILvCJa7pnL4XrVq4D0WbIXXMHBU16GU5aD+ANIbk7LA+PZK21KJGcteEIdUqQ/+5FeDeNnKdTyRgTF239skoXrBWSYMoGfowiVD1UjGtMe6IzF8pcXJjckbHhOaLuFbB1yqhZk9OjZxNbKUjarm1xgafQhmnyO1rUPgs7rVmLWkBn2UhSNVLZsnRviIkIYHjWhq7RR/roiLGFRLvVgHJFrFwySFQv07GQTbED0nIIyOOaGQeXH5Y0+UbA=;25:vIoqLBxcr2HaXeH/D/hg49lHTLkCE6B6Gpv5UJaCGee3gLg1tb5k4MHrHBGVduIi1ngzIcEZefaBHbaO8Ilh3yS1pf2L4ReBnLjLk9DqLdLXwe1pErQzCtNY+3zw6xQh9h8pUNYKh/zzjrlagRWlwJGApLmJmgPuvpJlSjenGWlEXdYV/G9Mzo8UhHN+ayom+llx5g1nAb2cmyt5/rzjBJPkcg7RL5K6tqWohH6DO7xngLt3RR9cfTPRTuPwDWRKpgneWa70cDlQX7bvCdP9LGUMSMX3r9TdG/us4Uz80kiKEoSR42lU+rNutv/jbebe8JT5gFXVbQErKLpKwZ/6vA== X-MS-TrafficTypeDiagnostic: MWHPR03MB2703: X-Microsoft-Exchange-Diagnostics: 1;MWHPR03MB2703;31:FZCcs3+2RcHxs0RxKaGjRyx4Oz0itZ650Jh8TNjrO23y59v3lUOMxUrAvxpUitOTJs/J8raJDEO9poS3wJXCmacK23L4/b17NQ7/oDeZO37stmFjaAhlZ/IpjeSZMkJvCIeDEGm+p3Hy/8cbwOfXDuOUnt9wG/IE5Dlu8mElUfv3X6PPcetsHjZdoJvFuuIvC3HjM9iHxWNChGnB8VcQjZmY66EGiC9aN3SCWbe9vm8=;4:BNrHoEsJzoItScp1AO2gdnvyxj7EXpr4kKt7IN6CoooBuCX3BILBFAmx/T+joycMRp02xzOro+4yXRY5fbIDZKBovDsE1EAP2QbvVlIKXxolDCMGsFlQEeQEBluAOP8AYxxGvWBi3ufYTVrKgjfcI14DIncX5SdmzNEW8vlil+F5ziRRyDenoHWMl/bOzp6m0xTTtknhhoLKUvYKzkQjk/04G9tLHJKyp8arjYl0waN6dU/oLYZOzqggDaOX0E4hku+/dTA5shjFQ/rbqibwoNht/mWtMKKsSTj2rx+IVxPGtUig5oQ3j05dxi/sZmlmHqAqNmZng6urmxT4Uo3p2m66wOSio3NTTQHAh5IugaA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6095135)(2401047)(8121501046)(5005006)(3231023)(944510075)(944921075)(946801075)(946901075)(3002001)(10201501046)(93006095)(93001095)(6055026)(6096035)(20161123559100)(20161123563025)(20161123561025)(20161123556025)(20161123565025)(201703131430075)(201703131448075)(201703131433075)(201703161259150)(201703151042153)(201708071742011);SRVR:MWHPR03MB2703;BCL:0;PCL:0;RULEID:(100000803101)(100110400095)(400006);SRVR:MWHPR03MB2703; X-Forefront-PRVS: 05437568AA X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;MWHPR03MB2703;23:TQf8OiSM2lFEbxnVLcdSspi0742gDJ3W/OzNkRyVj?= =?us-ascii?Q?a45xeefhyMOytjgECNBnNE7UDLNjuouQM8oYBhFQdt7AoJhNf/8Mfv5POiFM?= =?us-ascii?Q?axJi4y5HFtjgTwdyAGZDW3W0T6WzxLU1qCu372ViHH4tAjuzQxwiV9AJWZXk?= =?us-ascii?Q?txuGSzwT3Z93YHoHWvT6WmGFoFog4q6VfPzCNm6JGy/0rdo4Qqx86OMkCO6C?= =?us-ascii?Q?H2KkO9pwxvkujkf0FXmR/WX7Kn/+yUuui3xXdzZ0Z8ugbkxxSJgkmm/rtQfu?= =?us-ascii?Q?paH0cVrWF0Gs/7kTokKwT9fkJ8/PrcNdbVXmUwu9s8TsSBKe5MbBTMRO+j+M?= =?us-ascii?Q?nOOiQEnukOS1439d7YYn6uEcJrA8lcQ3bpIAwQq7who5rGr7hCH3/t52xjsL?= =?us-ascii?Q?tV8XWU1qhrMeX8HFlxtcpNGQKQjgGT+2EEOzSyTBahdPCcrIOcURFrmJS4Hh?= =?us-ascii?Q?/uBQZIPuz5Qek8BNBzZcWyTCETFYVSIKkiV5+B1D59JWensNAFlFcGDsI4Mw?= =?us-ascii?Q?AzQ/VNIj8kDYBsp/j2Ut5fg3YK9gmyzDvwHHldzTbeF9tJnMWbf79rJ7Uc3I?= =?us-ascii?Q?aEpBgYTy9FgeZdGBfatAx0S+FnFNLO0KdrQOXFZGt1rM7fYyrLKy5HE346T1?= =?us-ascii?Q?htu8ade4eQV6IfQAduMU8otGcMvjfLTyCP23cF8tuSSt04dzk8cOGqAfV+PL?= =?us-ascii?Q?cMh4fE+638EPH7EL16hKXJfsBOm/gIRmRq8HAwYQxtW09dvV1qpj08POwSDU?= =?us-ascii?Q?rI6USRrPnJhLcNMajHk9GtlXJYOW5NyHxDCSz0q18JC3GYR7KXec976EH7J+?= =?us-ascii?Q?C8J7wiGpfViF9Szgk7EmsNtqsgBT8CVSsTSSjRvnYGfM5kEd063KGncGAAXc?= =?us-ascii?Q?FCtiG45ssjH0VBcR3Jd29UZGqhZJop8xKTMLNoupdoamoQYn2G5ablHsImEG?= =?us-ascii?Q?bDYwMvTB/Am5m71gFTlzvmT6oMBAVE5QFDN2g3B9FHk7O/gr6AUOzFXfIXyL?= =?us-ascii?Q?5JbdDCUK/K45qDpKBr/szAAx47Pta7xHwyIvtgU/naLps/NOaFc7dnfOZLor?= =?us-ascii?Q?E7px2Wr1CvdBy6ed04qMEbWI1lwClnOYxjzbk8R0uBJyE8nmOMp8YqN0jzNE?= =?us-ascii?Q?lOd91aWU/s=3D?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR03MB2703;6:6U+V//9cI91D6wtWofPg+gSq/dQrse6j9fqa5trbiEb5tZ4UgcmOQINslbXFme2fmi7mT2VOYfRlYtLDdh/MgAJIRoM5L1YSyXUdr/1zWsgVNPH7ZyX5bC5DgMIMVjzHP8wbVBs/TMP0tuV+o//DiccOgImIsav/HkYv5U7RyhpbEGBHRxYfmn9QOWQ+BhdMMkItgjO8go8PMZhmMP70gstpURgJvuMkT6rP/l/IYq8GYAnKs2VlZ9zfSJAsLLm7aS5QLHONnfK/aVxViUuG84ptac2zvesvz3vhDOTGWQyKKdqzY5be5r9/7q7J+4E4LCR25JnCVGN/L7NNrwLIILgOgHdVJNxsxvHabXE5MR4=;5:x71AoiYDOhnqaLOqGW4UwpWSYYe5oG0Bdq1mcsxUFpXNXOwbYPQEB6wrITjVLQVhay30msp1SoPMlkIXIk60cPIZn5LGqfv0xZm6X0KgKgbWnMTFi6GGjB7h7ynEhws+Vo59U9+UWQNTBa61ElkQTFXj0HDtueIYUgONiwjAVyA=;24:6AWNMF8YOXN83IL/H+W2KbAH8HcC36uBQ+4J5W6Q39I6mz2VgAEffo9WxhsVoZOrtV4P72Q9LjTauNkeCKTuTEXx5F5J29456CLVwgpb6FQ=;7:1z4/NcXQIVooxxulVVpIUQwEuTNPVPmoNoYUcg3xoUOaeuL7hhD+Bztra6BghBSJ98bblYzGrlII4JJE+0NchkME3MfHpdSFJog8d3AEB6cW2eNG399iZ+hQFSnb6A4aHJebssgV5hkpe+ye0tfHUXf/GgaTOnhtfEY7DXk0DQXDUolYCrh7RthOfeu+WAcHBKGaHyUowTzD7a88pTImoTJw7nBidc0SnuS0T/gNLWiWnhG5HhAWtKx8pNzmzisP SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2018 11:04:07.0543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8b9f50c7-ae34-46db-ef29-08d5542c0a6f X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e;Ip=[192.88.168.50];Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2703 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: Use the newly added DPIO service API to map cpu-affine DPIO services to channels. The DPAA2 Ethernet driver already had mappings of frame queues and channels to cpus, but had no control over the DPIOs used. We can now ensure full affinity of hotpath hardware resources to cores, which improves performance and almost eliminates some resource contentions (e.g. enqueue/dequeue busy counters should be close to zero from now on). Making the pull channel operation core affine brings the most significant benefits. This ensures the same DPIO service will be used for all dequeue commands issued for a certain frame queue, which is in line with the way hardware is optimized. Additionally, we also use affine DPIOs for the frame enqueue and buffer release operations in order to avoid resource contention. dpaa2_io_service_register() and dpaa2_io_service_rearm() functions receive an affine DPIO as argument mostly for uniformity, but this doesn't change the previous functionality. Signed-off-by: Ioana Radulescu --- drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c | 24 ++++++++++++++---------- drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h | 1 + 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c index 824c4ad..2817e67 100644 --- a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c +++ b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c @@ -597,7 +597,8 @@ static netdev_tx_t dpaa2_eth_tx(struct sk_buff *skb, struct net_device *net_dev) queue_mapping = skb_get_queue_mapping(skb); fq = &priv->fq[queue_mapping]; for (i = 0; i < DPAA2_ETH_ENQUEUE_RETRIES; i++) { - err = dpaa2_io_service_enqueue_qd(NULL, priv->tx_qdid, 0, + err = dpaa2_io_service_enqueue_qd(fq->channel->dpio, + priv->tx_qdid, 0, fq->tx_qdbin, &fd); if (err != -EBUSY) break; @@ -719,7 +720,8 @@ static void free_bufs(struct dpaa2_eth_priv *priv, u64 *buf_array, int count) /* Perform a single release command to add buffers * to the specified buffer pool */ -static int add_bufs(struct dpaa2_eth_priv *priv, u16 bpid) +static int add_bufs(struct dpaa2_eth_priv *priv, + struct dpaa2_eth_channel *ch, u16 bpid) { struct device *dev = priv->net_dev->dev.parent; u64 buf_array[DPAA2_ETH_BUFS_PER_CMD]; @@ -753,7 +755,7 @@ static int add_bufs(struct dpaa2_eth_priv *priv, u16 bpid) release_bufs: /* In case the portal is busy, retry until successful */ - while ((err = dpaa2_io_service_release(NULL, bpid, + while ((err = dpaa2_io_service_release(ch->dpio, bpid, buf_array, i)) == -EBUSY) cpu_relax(); @@ -794,7 +796,7 @@ static int seed_pool(struct dpaa2_eth_priv *priv, u16 bpid) for (j = 0; j < priv->num_channels; j++) { for (i = 0; i < DPAA2_ETH_NUM_BUFS; i += DPAA2_ETH_BUFS_PER_CMD) { - new_count = add_bufs(priv, bpid); + new_count = add_bufs(priv, priv->channel[j], bpid); priv->channel[j]->buf_count += new_count; if (new_count < DPAA2_ETH_BUFS_PER_CMD) { @@ -852,7 +854,7 @@ static int refill_pool(struct dpaa2_eth_priv *priv, return 0; do { - new_count = add_bufs(priv, bpid); + new_count = add_bufs(priv, ch, bpid); if (unlikely(!new_count)) { /* Out of memory; abort for now, we'll try later on */ break; @@ -873,7 +875,8 @@ static int pull_channel(struct dpaa2_eth_channel *ch) /* Retry while portal is busy */ do { - err = dpaa2_io_service_pull_channel(NULL, ch->ch_id, ch->store); + err = dpaa2_io_service_pull_channel(ch->dpio, ch->ch_id, + ch->store); dequeues++; cpu_relax(); } while (err == -EBUSY); @@ -923,7 +926,7 @@ static int dpaa2_eth_poll(struct napi_struct *napi, int budget) if (cleaned < budget && napi_complete_done(napi, cleaned)) { /* Re-enable data available notifications */ do { - err = dpaa2_io_service_rearm(NULL, &ch->nctx); + err = dpaa2_io_service_rearm(ch->dpio, &ch->nctx); cpu_relax(); } while (err == -EBUSY); WARN_ONCE(err, "CDAN notifications rearm failed on core %d", @@ -1531,7 +1534,8 @@ static int setup_dpio(struct dpaa2_eth_priv *priv) nctx->desired_cpu = i; /* Register the new context */ - err = dpaa2_io_service_register(NULL, nctx); + channel->dpio = dpaa2_io_service_select(i); + err = dpaa2_io_service_register(channel->dpio, nctx); if (err) { dev_dbg(dev, "No affine DPIO for cpu %d\n", i); /* If no affine DPIO for this core, there's probably @@ -1571,7 +1575,7 @@ static int setup_dpio(struct dpaa2_eth_priv *priv) return 0; err_set_cdan: - dpaa2_io_service_deregister(NULL, nctx); + dpaa2_io_service_deregister(channel->dpio, nctx); err_service_reg: free_channel(priv, channel); err_alloc_ch: @@ -1594,7 +1598,7 @@ static void free_dpio(struct dpaa2_eth_priv *priv) /* deregister CDAN notifications and free channels */ for (i = 0; i < priv->num_channels; i++) { ch = priv->channel[i]; - dpaa2_io_service_deregister(NULL, &ch->nctx); + dpaa2_io_service_deregister(ch->dpio, &ch->nctx); free_channel(priv, ch); } } diff --git a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h index fb8fb5c..e577410 100644 --- a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h +++ b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h @@ -288,6 +288,7 @@ struct dpaa2_eth_channel { int ch_id; int dpio_id; struct napi_struct napi; + struct dpaa2_io *dpio; struct dpaa2_io_store *store; struct dpaa2_eth_priv *priv; int buf_count; -- 2.7.4