Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753444AbdLUMd3 (ORCPT ); Thu, 21 Dec 2017 07:33:29 -0500 Received: from mail-bn3nam01on0044.outbound.protection.outlook.com ([104.47.33.44]:28971 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752553AbdLUMdY (ORCPT ); Thu, 21 Dec 2017 07:33:24 -0500 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com; From: Ioana Radulescu To: CC: , , Subject: [PATCH 1/2] staging: fsl-dpaa2/eth: Flow affinity for IP forwarding Date: Thu, 21 Dec 2017 06:33:20 -0600 Message-ID: <20171221123321.22064-1-ruxandra.radulescu@nxp.com> X-Mailer: git-send-email 2.11.0 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131583331883111132;(91ab9b29-cfa4-454e-5278-08d120cd25b8);() X-Forefront-Antispam-Report: CIP:192.88.168.50;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(336005)(346002)(39860400002)(376002)(396003)(39380400002)(2980300002)(1109001)(1110001)(339900001)(199004)(189003)(5660300001)(316002)(498600001)(50226002)(2351001)(97736004)(6916009)(105606002)(53936002)(68736007)(8936002)(106466001)(86362001)(36756003)(77096006)(47776003)(59450400001)(51416003)(4326008)(48376002)(50466002)(2906002)(81166006)(356003)(81156014)(104016004)(54906003)(8656006)(305945005)(8676002)(1076002)(85426001)(16586007);DIR:OUT;SFP:1101;SCL:1;SRVR:CO2PR03MB2358;H:tx30smr01.am.freescale.net;FPR:;SPF:Fail;PTR:InfoDomainNonexistent;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BN1BFFO11FD019;1:zi6JuSWnrGIemeR4UQeVIrepCY5YR/+Tptt5aoqumLwqRhixKb73XW+Wt77WhXrxT8AuwyWYwBf3miW1AMGYFn8/fbNtTpT4RNJqAHWLPpsVnOHuAgaGnOH48FYpwhEu MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e71db6b7-3015-4b7a-52dc-08d5486efdb8 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(5600026)(4604075)(4534020)(4628075)(201703131517081)(2017052603307);SRVR:CO2PR03MB2358; X-Microsoft-Exchange-Diagnostics: 1;CO2PR03MB2358;3:XPqPSzRIweM99kU5afzYq+bR/nDnyBDqoUCCxaAg6++rt5DbjGanW4TbER8RpT+It8Rg+5jLIbHHGozWuXgkwrnHqRf7S9FIxSX3o84AOnShui0Cxtz6qP9Liq9AQ+jZ1KDnkiMFOfWXlE/FLuNlP2qEmpLuk3Tft2VIzWiNYUrRrD4s25/weXMyC4zf7mSAPkHB0U0JRt9aqLUpWMKN+yZqtK1OvF+KPlz7u4o2fmt5sYR5u63ayTpeS9TG5Y0u9w0Yiizw5+yDJWIoCcFxfxKDVtcTvYqG4ZUYyz1EJhMmwz9wEM2D/DMmIxTKRYvNzUYKX9vN+/KjYsDhb1NvvYkMw1SNDzsNc5wHCwW2fKs=;25:LcULLYTZeZmm83K7uww/ItsCmIKanDyWn4OkhVV/khv+qAmrr6WwtXjVNuo9Nb5CM9/b1zupI7vDkiHLBndkTpWzQS4uHNZAkEtU8t3PmKS+FKSUbd4AEgROVTRGfEN8vKAgmcI7Pi7S/f4B511IbVfd3Yx569fjDI3mlvQIWXnhobzxrNrbm/qbg4YiU/wJnGAiYcpz0PqZ/+4awQl9bW7EZDSaOiIO/U8Hmwbo1j7hFqrGt+SZ60Jz5DmTIi+WfpTt15wNe3UkSaJAUj8aKRTUSEMk+69s2pVrBOzFIydblC60P4YolLoezs28YGw00L1/9PuI1Jgcv8RR9GUvEYrdjVYCT0+7RodRN8ZNASE= X-MS-TrafficTypeDiagnostic: CO2PR03MB2358: X-Microsoft-Exchange-Diagnostics: 1;CO2PR03MB2358;31:2F3pVvzRYk+LNiMnWN02LE8b6e4hawh01s10kRxxV24vbw5UBui25I3HtSKKESbv4j3PTejLsRkfs64TtZbUfBymNrGFirnWX1B0iscy1MHQLjvnt4fvhK4IsZqWJeTeeUX0m/bOUG0OE4HiFkts2a3D0IEDL1E46CEf2jylP8m4Ql6p/+fUunVyLW2/oMBiU2vOHeCg+M5BmyrgTw3S9KkQkNFtRKEfkdxN8dugfVw=;4:qBVF+z5FGmhSeL6ZRIRaSDbJ2L73s8b41hftRVVrKBGtoBzZt3wpJSVlCJEjMV7BX5sWRzVJpjuXfXVnR4m1abobq77ZanuwmqgzcmCx4+mkXl1+0qoJPI2CB2JnANUZtGIjaBzu0zWfHSoFEYOdgXMzG5mrUorUumYTJnNfnMacW0dmRx3OarDTEEnvAL/OIXZyXHpVBDhNNTi64mHDGd1LmrX/scbEDAve26tNMJFc0DrDtvCO5nuEKPRt3LSw4W1xV8QhiOsc9PGcLpa0L+2dEtivikwVQpB+KW0N5OWVYvsT075tUu/BXIv0Uuqw X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6095135)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231023)(6055026)(6096035)(20161123559100)(20161123561025)(201703131430075)(201703131433075)(201703131448075)(201703161259150)(201703151042153)(20161123563025)(20161123556025)(20161123565025)(201708071742011);SRVR:CO2PR03MB2358;BCL:0;PCL:0;RULEID:(100000803101)(100110400095)(400006);SRVR:CO2PR03MB2358; X-Forefront-PRVS: 0528942FD8 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CO2PR03MB2358;23:A55ruzpkgIwL2QylmEZuL8A7SnZXEDFmdtNgzdUKi?= =?us-ascii?Q?1mgNJ/2SjFlWglovAKroUMOIfN/TnjZFw9E3H+N1F13fxEueYsn231eD3Gtp?= =?us-ascii?Q?WZ+oVidFE0yLserIyDRFmVjmV0ZDjzZb6cHk3fYdpuTmUYrLnqmIoj/jVc+S?= =?us-ascii?Q?Wa7d5oUs5NrBGuL776icNCDh7y+6jf3yc+LgoiSU9SGGUxV7YEE0Y3w1EK8W?= =?us-ascii?Q?NDfQk9wtPk3Fe13Be0oWlla6O/tAkYNqTPCxxWidbZq361aXaZR5+mSnsGZT?= =?us-ascii?Q?fxFRTx+0+Mnh9fVbje/QJa2GPW/rz0qToQz0vo1QtU0bzQJikmHhYKmAekY0?= =?us-ascii?Q?hgmzfyvoJlE+Yb7uwaRlYCtTr9aLC4NwMhAp6AVAfXyHMl5BovqCAutuMCxn?= =?us-ascii?Q?amq5U4AQzYVjfsbCQlSdYhAfIZWzgwkT8rA3AmKcjfUTBNVeW8lK5feBdfuy?= =?us-ascii?Q?W3iQZJS/N4Y6RRXkDFLd4DOlTKyTHbU7w5Kujj3oSZGARuRQBMCwIMVbX02Z?= =?us-ascii?Q?3qSD5SjJvYAxbGaNEdgI/dwaKercLkcVM2TakW7Vps6zJegVhdMdUAhGl8tE?= =?us-ascii?Q?QcGs+Xm17JLjvtPvPhDS6BKJYgPbKsMIWwpr9cNynYKb5U6LTDVyG3IBm5+/?= =?us-ascii?Q?78YjD/ELykLRvVe0VOmQ92BBzFUFtRsQYZUbbgXhV9VGZEHVZmsyMnORdpnu?= =?us-ascii?Q?LiX1AfyaLNRvNfEwuc6uh3e2rG8YKDaa92xB1k1sdnOUMHl7jnE7w1Dy7fUh?= =?us-ascii?Q?eVCso7jT6pwqYu+LkHcCwv1FSXqRDHJQuTGVOgZFm46ffp0W+kBP6LvG1Jwa?= =?us-ascii?Q?DYlNuZyKMTWVeftXvjwhD1XyGghhJmfmHqN7qmFgJC1iPkN3UFGWQ4B2EiYr?= =?us-ascii?Q?BN8GpUOe36IqJf5TQNHXCZ5EMPolK5VrLcQDxwE5t0mdk/g2GGDWCwQwsaA0?= =?us-ascii?Q?P/R+/6gu1kqPtMXcyhueeNxs1fpO9Fo7BdfB9KioZfQkTAkmkCX8YXhod2dr?= =?us-ascii?Q?rp71Ca7s2HhjJl+Vru/O6vTXJY1ZeiMvVEuf0o25vowzJFYFX0DmSD1rCNVZ?= =?us-ascii?Q?uIasfs=3D?= X-Microsoft-Exchange-Diagnostics: 1;CO2PR03MB2358;6:luC7yeyXNUltR2R5BzZFKtV6UjszLd48PDrG/F1PoWCLKCKiDNC61bhfualv//kLWnFADjecy1ueZrDJMapcsmLQ6r4o2JS1DkRNlsSrtSNca3Mv9F56GceqC01TDjSqZBFSw5k8vrZ+GbhIsnpEM7kLulltseUCYR6PPDlsnFPFTp0ZKJDoS6Jj1qwuR8plX2F04r5+XRFpoJG2h+dwI8PDpVobXDtrdmSQeoK1BqJBIOuBB3scANk0mVTfxhANoU2yL810W5OZKYo++1d2WhlZpKXEQhBm/FJ5NBXnChY13vu6FmWrw2DmG3gNE6Y2QVDbTEX24dAdKw6vvK4mBIYWq6YpRQ8a/owKZdz1Mcc=;5:bj+A+Onmmf1QncaeESJA6kg8UMFXwsRqHS0BSYI9t82tQZwJpK9snR8pZJL65N7qQ0sOhIYQW8OUYfwJLemESVnbON1Tu7guPcCGJuzUYrM0aeoif0NjCnF4Y1xa6zCtBKFSQdmQkkMiJQP7dbeA0wQxtxb0j3uUa02Llym8o1M=;24:hRFmzSx4sd4KXIGNUtteY2AkZW7gAX1rYEFbxkVBIIp5+lvNbngQYweFjyPVdlhMnnKplD/6vEkhsR4l2lJUWQyWb2GkM9p4Z13I8Z0nO0c=;7:P5HXN+Ogo92nmTAQ88GmG4SV+nJjUObhRUEFZ1bdGSCeHzB1emwQ9ZOYrLvcGU99wN8RfFxkcct9pino0iWkiMHyiuGR3YHLweNjirtiQslQUhKrF767n0fGx3YJTR4CDjgT/Wn6BSnocXOfjxrjTpJydmPj5OQ4jStYmyAZT599BNVAO6Xp9ocCIv3BhzshX89Ecldznv8f4Vk/uRILocrjv4wOndouurn8v5kC0mf2I9cfeE333BryDisxXba2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Dec 2017 12:33:08.1239 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e71db6b7-3015-4b7a-52dc-08d5486efdb8 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e;Ip=[192.88.168.50];Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR03MB2358 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3707 Lines: 89 The driver xmit function chooses an egress FQ based on the current core id. The network stack itself sets a mapping field in the skb based on many things - the default one being a hash on packet fields, which the current driver ignores. This patch saves the ingress frame flow affinity information in the skb. In case of forwarded frames, this info will then be used for Tx and Tx confirmation hardware queue selection, ensuring all processing of the given frame is done on a single core. Signed-off-by: Bogdan Purcareata Signed-off-by: Ioana Radulescu --- drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c | 16 ++++++++++------ drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h | 3 ++- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c index 7f3e4fa..b63ae09 100644 --- a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c +++ b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.c @@ -230,7 +230,8 @@ static struct sk_buff *build_frag_skb(struct dpaa2_eth_priv *priv, static void dpaa2_eth_rx(struct dpaa2_eth_priv *priv, struct dpaa2_eth_channel *ch, const struct dpaa2_fd *fd, - struct napi_struct *napi) + struct napi_struct *napi, + u16 queue_id) { dma_addr_t addr = dpaa2_fd_get_addr(fd); u8 fd_format = dpaa2_fd_get_format(fd); @@ -281,6 +282,7 @@ static void dpaa2_eth_rx(struct dpaa2_eth_priv *priv, } skb->protocol = eth_type_trans(skb, priv->net_dev); + skb_record_rx_queue(skb, queue_id); percpu_stats->rx_packets++; percpu_stats->rx_bytes += dpaa2_fd_get_len(fd); @@ -325,7 +327,7 @@ static int consume_frames(struct dpaa2_eth_channel *ch) fq = (struct dpaa2_eth_fq *)dpaa2_dq_fqd_ctx(dq); fq->stats.frames++; - fq->consume(priv, ch, fd, &ch->napi); + fq->consume(priv, ch, fd, &ch->napi, fq->flowid); cleaned++; } while (!is_last); @@ -588,10 +590,11 @@ static netdev_tx_t dpaa2_eth_tx(struct sk_buff *skb, struct net_device *net_dev) /* Tracing point */ trace_dpaa2_tx_fd(net_dev, &fd); - /* TxConf FQ selection primarily based on cpu affinity; this is - * non-migratable context, so it's safe to call smp_processor_id(). + /* TxConf FQ selection relies on queue id from the stack. + * In case of a forwarded frame from another DPNI interface, we choose + * a queue affined to the same core that processed the Rx frame */ - queue_mapping = smp_processor_id() % dpaa2_eth_queue_count(priv); + queue_mapping = skb_get_queue_mapping(skb); fq = &priv->fq[queue_mapping]; for (i = 0; i < DPAA2_ETH_ENQUEUE_RETRIES; i++) { err = dpaa2_io_service_enqueue_qd(NULL, priv->tx_qdid, 0, @@ -622,7 +625,8 @@ static netdev_tx_t dpaa2_eth_tx(struct sk_buff *skb, struct net_device *net_dev) static void dpaa2_eth_tx_conf(struct dpaa2_eth_priv *priv, struct dpaa2_eth_channel *ch, const struct dpaa2_fd *fd, - struct napi_struct *napi __always_unused) + struct napi_struct *napi __always_unused, + u16 queue_id __always_unused) { struct rtnl_link_stats64 *percpu_stats; struct dpaa2_eth_drv_stats *percpu_extras; diff --git a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h index d68ac38..fb8fb5c 100644 --- a/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h +++ b/drivers/staging/fsl-dpaa2/ethernet/dpaa2-eth.h @@ -276,7 +276,8 @@ struct dpaa2_eth_fq { void (*consume)(struct dpaa2_eth_priv *, struct dpaa2_eth_channel *, const struct dpaa2_fd *, - struct napi_struct *); + struct napi_struct *, + u16 queue_id); struct dpaa2_eth_fq_stats stats; }; -- 2.7.4