Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756547AbbLAOkQ (ORCPT ); Tue, 1 Dec 2015 09:40:16 -0500 Received: from mailout2.w1.samsung.com ([210.118.77.12]:52766 "EHLO mailout2.w1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756338AbbLAOkN (ORCPT ); Tue, 1 Dec 2015 09:40:13 -0500 X-AuditID: cbfec7f5-f79b16d000005389-a8-565db14aa582 From: Pavel Fedin To: "'Sunil Goutham'" , netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Sunil.Goutham@caviumnetworks.com, "'Sunil Goutham'" References: <1448961223-41888-4-git-send-email-sunil.kovvuri@gmail.com> In-reply-to: <1448961223-41888-4-git-send-email-sunil.kovvuri@gmail.com> Subject: RE: [PATCH 3/6] net: thunderx: Increase transmit queue length Date: Tue, 01 Dec 2015 17:40:09 +0300 Message-id: <01d101d12c46$2de43b40$89acb1c0$@samsung.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-index: AQGScnHROc58U50eD5J6ppH35rF/UQG2SVlqnyX0nyA= Content-language: ru X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrOLMWRmVeSWpSXmKPExsVy+t/xK7peG2PDDM728FpsenyN1eLyrjls FscWiFlcvvyD2eLAx70sFoua3rE4sHnM+L2IxWPDiX5Wj52z7rJ7bF5S7/F5k1wAaxSXTUpq TmZZapG+XQJXxs3en4wFb+QqFtxPb2BcLdXFyMkhIWAiMef1FHYIW0ziwr31bCC2kMBSRolr nbEQ9ndGiU2NniA2m4C6xOmvH1hAbBEBV4k129YA1XNxMAtMZJT4tvAfK0RDpMT/dweYQGxO AQ+JnRNWgzUIC7hJzNm6GmwBi4CqxMrJD8BqeAUsJe7Pu8gOYQtK/Jh8D6yeWUBLYv3O40wQ trzE5jVvmSEOVZDYcfY1I8QRVhLrl21mhKgRkZj27x7zBEahWUhGzUIyahaSUbOQtCxgZFnF KJpamlxQnJSea6RXnJhbXJqXrpecn7uJERIdX3cwLj1mdYhRgINRiYdXYm1MmBBrYllxZe4h RgkOZiUR3qdrY8OEeFMSK6tSi/Lji0pzUosPMUpzsCiJ887c9T5ESCA9sSQ1OzW1ILUIJsvE wSnVwLhCbY2YQuJs4/0vS6YFV4QlvDkltUnIZLE9h5jY1KcJjQu3LLK9ptlqbqv3p7+TVUJ7 a1ZVfPTZDxs0M52zgubt/pG/+g/rvAVpW3QyL/5cxfBoQvUqOeVvl+o/npx9clbDgq7Nmpyf 5h699fbcgmfSHXw8KrMcJvwKZgoN+/5+zWSbL9MrmX8psRRnJBpqMRcVJwIAu57SEIoCAAA= Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4097 Lines: 102 Hello! This one breaks networking on my machine: --- cut --- swiotlb: coherent allocation failed for device 0002:01:08.4 size=4198400 CPU: 2 PID: 3655 Comm: NetworkManager Tainted: G W O 4.2.6+ #201 Hardware name: Cavium ThunderX CN88XX board (DT) Call trace: [] dump_backtrace+0x0/0x124 [] show_stack+0x14/0x1c [] dump_stack+0x88/0xc8 [] swiotlb_alloc_coherent+0x150/0x168 [] __dma_alloc+0x58/0x1e8 [] nicvf_alloc_q_desc_mem.isra.14+0xc0/0xdc [] nicvf_config_data_transfer+0x448/0x728 [] nicvf_open+0x184/0x994 [] __dev_open+0xb4/0x120 [] __dev_change_flags+0x8c/0x150 [] dev_change_flags+0x20/0x5c [] do_setlink+0x27c/0x8e0 [] rtnl_newlink+0x4b4/0x6c4 [] rtnetlink_rcv_msg+0x90/0x22c [] netlink_rcv_skb+0xd8/0x100 [] rtnetlink_rcv+0x30/0x40 [] netlink_unicast+0x158/0x1f8 [] netlink_sendmsg+0x324/0x380 [] sock_sendmsg+0x18/0x58 [] ___sys_sendmsg+0x254/0x264 [] __sys_sendmsg+0x40/0x84 [] SyS_sendmsg+0x10/0x20 thunder-nicvf 0002:01:08.4 enP2p1s8f4: Failed to alloc/config VF's QSet resources --- cut --- And none of interfaces work after this. Reverting this patch helps. Kind regards, Pavel Fedin Expert Engineer Samsung Electronics Research center Russia > -----Original Message----- > From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On Behalf Of Sunil > Goutham > Sent: Tuesday, December 01, 2015 12:14 PM > To: netdev@vger.kernel.org > Cc: linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > Sunil.Goutham@caviumnetworks.com; Sunil Goutham > Subject: [PATCH 3/6] net: thunderx: Increase transmit queue length > > From: Sunil Goutham > > Under high transmit rates and with TSO enabled observing fluctuations > in TX performance. Seen especially with iperf3 application. > > Since TSO is taken care at driver level, with 64KB of TSO packets > and when window size is also high the rate at which CPU fills in > transmit descriptors is much higher than what HW is able to process. > Each 64KB TSO packet occupies gets segmented to ~43 1500 byte MTU packets > and occupies ~130 descriptors. Hence increasing transmit queue size. > > Signed-off-by: Sunil Goutham > --- > drivers/net/ethernet/cavium/thunder/nicvf_queues.h | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.h > b/drivers/net/ethernet/cavium/thunder/nicvf_queues.h > index fb4957d..b1e93a9 100644 > --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.h > +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.h > @@ -62,7 +62,7 @@ > #define SND_QUEUE_CNT 8 > #define CMP_QUEUE_CNT 8 /* Max of RCV and SND qcount */ > > -#define SND_QSIZE SND_QUEUE_SIZE2 > +#define SND_QSIZE SND_QUEUE_SIZE3 > #define SND_QUEUE_LEN (1ULL << (SND_QSIZE + 10)) > #define MAX_SND_QUEUE_LEN (1ULL << (SND_QUEUE_SIZE6 + 10)) > #define SND_QUEUE_THRESH 2ULL > @@ -73,7 +73,7 @@ > /* Keep CQ and SQ sizes same, if timestamping > * is enabled this equation will change. > */ > -#define CMP_QSIZE CMP_QUEUE_SIZE2 > +#define CMP_QSIZE CMP_QUEUE_SIZE3 > #define CMP_QUEUE_LEN (1ULL << (CMP_QSIZE + 10)) > #define CMP_QUEUE_CQE_THRESH 0 > #define CMP_QUEUE_TIMER_THRESH 220 /* 10usec */ > -- > 1.7.1 > > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/