Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755857AbbDIQiU (ORCPT ); Thu, 9 Apr 2015 12:38:20 -0400 Received: from smtp.citrix.com ([66.165.176.89]:60084 "EHLO SMTP.CITRIX.COM" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753716AbbDIQiR (ORCPT ); Thu, 9 Apr 2015 12:38:17 -0400 X-IronPort-AV: E=Sophos;i="5.11,550,1422921600"; d="scan'208";a="251760702" Date: Thu, 9 Apr 2015 17:36:47 +0100 From: Stefano Stabellini X-X-Sender: sstabellini@kaball.uk.xensource.com To: Eric Dumazet CC: Stefano Stabellini , , , , , Ian Campbell , Wei Liu , David Vrabel , , Konrad Rzeszutek Wilk , netdev Subject: Re: "tcp: refine TSO autosizing" causes performance regression on Xen In-Reply-To: <1428596218.25985.263.camel@edumazet-glaptop2.roam.corp.google.com> Message-ID: References: <1428596218.25985.263.camel@edumazet-glaptop2.roam.corp.google.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-DLP: MIA2 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2100 Lines: 62 On Thu, 9 Apr 2015, Eric Dumazet wrote: > On Thu, 2015-04-09 at 16:46 +0100, Stefano Stabellini wrote: > > Hi all, > > > > I found a performance regression when running netperf -t TCP_MAERTS from > > an external host to a Xen VM on ARM64: v3.19 and v4.0-rc4 running in the > > virtual machine are 30% slower than v3.18. > > > > Through bisection I found that the perf regression is caused by the > > prensence of the following commit in the guest kernel: > > > > > > commit 605ad7f184b60cfaacbc038aa6c55ee68dee3c89 > > Author: Eric Dumazet > > Date: Sun Dec 7 12:22:18 2014 -0800 > > > > tcp: refine TSO autosizing > > > > > > A simple revert would fix the issue. > > > > Does anybody have any ideas on what could be the cause of the problem? > > Suggestions on what to do to fix it? > > You sent this to lkml while networking discussions are on netdev. > > This topic had been discussed on netdev multiple times. Sorry, and many thanks for the quick reply! > This commit restored original TCP Small Queue behavior, which is the > first step to fight bufferbloat. > > Some network drivers are known to be problematic because of a delayed TX > completion. > > So far this commit did not impact max single flow throughput on 40Gb > mlx4 NIC. (ie : line rate is possible) > > Try to tweak /proc/sys/net/ipv4/tcp_limit_output_bytes to see if it > makes a difference ? A very big difference: echo 262144 > /proc/sys/net/ipv4/tcp_limit_output_bytes brings us much closer to the original performance, the slowdown is just 8% echo 1048576 > /proc/sys/net/ipv4/tcp_limit_output_bytes fills the gap entirely, same performance as before "refine TSO autosizing" What would be the next step for here? Should I just document this as an important performance tweaking step for Xen, or is there something else we can do? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/