From: Dexuan Cui <decui@microsoft.com>
To: Eric Dumazet <eric.dumazet@gmail.com>,
        David Ahern <dsa@cumulusnetworks.com>,
        Simon Xiao <sixiao@microsoft.com>
CC: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "devel@linuxdriverproject.org" <devel@linuxdriverproject.org>,
        David Miller <davem@davemloft.net>
Subject: RE: linux-next network throughput performance regression
Thread-Topic: linux-next network throughput performance regression
Thread-Index: AQHRGZOP0PW5JYF2Rw2lkRwdWFynkZ6S+F3w
Date: Mon, 9 Nov 2015 02:39:24 +0000
Message-ID: <6c0fed5fc63a4c3488f5f08409f508ee@HKXPR30MB0039.064d.mgd.msft.net>
References: <CY1PR03MB13760C9F1AABC4EBC455B85CC4280@CY1PR03MB1376.namprd03.prod.outlook.com>
 <563D1180.60206@cumulusnetworks.com>
 <CY1PR03MB13763CB0C0A9C822660E248AC4280@CY1PR03MB1376.namprd03.prod.outlook.com>
 <563D1BE0.2070107@cumulusnetworks.com>
 <1446924959.17135.23.camel@edumazet-glaptop2.roam.corp.google.com>
In-Reply-To: <1446924959.17135.23.camel@edumazet-glaptop2.roam.corp.google.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8BIT
MIME-Version: 1.0
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2015 02:39:29.7904
 (UTC)
X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47;Ip=[206.191.230.228];Helo=[064-smtp-out.microsoft.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR03MB127
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 2014
Lines: 50

> From: devel [mailto:driverdev-devel-bounces@linuxdriverproject.org] On Behalf
> Of Eric Dumazet
> Sent: Sunday, November 8, 2015 3:36
> To: David Ahern <dsa@cumulusnetworks.com>
> Cc: netdev@vger.kernel.org; Haiyang Zhang <haiyangz@microsoft.com>; linux-
> kernel@vger.kernel.org; devel@linuxdriverproject.org; David Miller
> <davem@davemloft.net>
> Subject: Re: linux-next network throughput performance regression
> 
> On Fri, 2015-11-06 at 14:30 -0700, David Ahern wrote:
> > On 11/6/15 2:18 PM, Simon Xiao wrote:
> > > The .config file used to build linux-next kernel is attached to this mail.
> >
> > Thanks.
> >
> > Failed to notice this on the first response; my brain filled in. Why
> > linux-next tree? Can you try net-next which is more relevant for this
> > mailing list, post the top commit id and config file used?
> 
> Throughput on a single TCP flow for a 40G NIC can be tricky to tune.
Why is a single TCP flow trickier than multiple TCP flows?
IMO it should be easier to analyze the issue of a single TCP flow?

Here the perf drop in Simon's test is very obvious -- 50%, but it looks Eric
can't reproduce it, so I suppose some net-related kernel config options may
do the magic?

Maybe Simon can narrow the regression down by bisecting. :-)
 
> Make sure IRQ are properly setup/balanced, as I know that IRQ names were
> changed recently and your scripts might have not noticed...
> 
> Also "ethtool -c eth0" might show very different interrupt coalescing
> params ?
> 
> I too have a Mellanox 40Gb in my lab and saw no difference in
> performance with recent kernels.
> 
> Of course, a simple "perf record -a -g sleep 4 ; perf report" might
> point to some obvious issue. Like unexpected segmentation in case of
> forwarding...
> 

Thanks,
-- Dexuan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/