Received: by 10.223.185.116 with SMTP id b49csp1055987wrg; Fri, 16 Feb 2018 11:36:25 -0800 (PST) X-Google-Smtp-Source: AH8x225fOJkk65ocwLoRWMLlGSRaX7p1fXzlNdHJhDn6zeq8kjWmHPhuTcTXZfpI/ZnT5z/bQ1SS X-Received: by 10.101.78.200 with SMTP id w8mr6041607pgq.43.1518809784925; Fri, 16 Feb 2018 11:36:24 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1518809784; cv=pass; d=google.com; s=arc-20160816; b=r7jNtJ4Lou2iw9kFudYOvir2/Lj7Sf9K1cVjP7LHTqQVdzSbR4kyinm+LjmYPaMF1J sjYej7k8ZVYnkYIrgOkMC2vbVS/5j0AOzSJQi3KVdSZnWQipP99PaGs96wRealq96WV0 /S7CwUH8VbxitCAzus8eXRvVgvVDgkgKSuQgajdQlbRAjpAfvoO59A//09QmxgNeUI+j fFTGWF7GWqjdZOriku4Ut51DkSh0RTKSH9Otlw6mKz86ycrRrzpi7SX7Lj05Of/egygW pvXgC5za3DY9mbh/KQVE3DICVmfqlxl60/JF6sqhvMUu/uncDTMbyI9kBCM8xaM/QyhR OX/g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:arc-authentication-results :arc-message-signature:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dmarc-filter:dkim-signature:arc-authentication-results; bh=Wbvz+gb0Oj9m6JnBK1J1Tppkz2AtsxrdiW5uW2Sp3kk=; b=XFf0iXpQFXFijXE/+iuiQWCbPt7idB+Tvj4HYcHqVnJKYdU2XWSaaSetnlnKLaa31g 6Q/8Eq5vYaCxoc7tOJ/5K2fMadaNeYlwlHZomXDZJzmW7ZgJUgez/CI/0p29vdiXDVHG EuzHUO8u7F7XRBCwjQXZSdqxLSuWkC+WSS240m3wOXI2+kYodreyW93rXmEPtD3okM/C 0vImGVir3bD/rfZtbVj1K5r/JoWhNHpbH+zJFASUu4k7WgsuQMImU0TnudnyG5x0iMCR qCSvAeiv9iBJNyeTKfT87bZYkCfBa8JtaIxmt3LAvRYB+fePK5yYU6LTotAOiwX9ENrE +QqQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@natalenko.name header.s=dkim-20170712 header.b=jS0sl0V1; arc=pass (i=1); spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=natalenko.name Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z2si2824434pfe.350.2018.02.16.11.36.10; Fri, 16 Feb 2018 11:36:24 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@natalenko.name header.s=dkim-20170712 header.b=jS0sl0V1; arc=pass (i=1); spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=natalenko.name Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162082AbeBPRfH (ORCPT + 99 others); Fri, 16 Feb 2018 12:35:07 -0500 Received: from vulcan.natalenko.name ([104.207.131.136]:43324 "EHLO vulcan.natalenko.name" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161197AbeBPRfF (ORCPT ); Fri, 16 Feb 2018 12:35:05 -0500 Received: from spock.localnet (unknown [IPv6:2001:470:5b39:28:d9be:599a:83a5:fae4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by vulcan.natalenko.name (Postfix) with ESMTPSA id A829B2F90A1; Fri, 16 Feb 2018 18:35:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=natalenko.name; s=dkim-20170712; t=1518802503; h=from:subject:date:message-id:to:cc:mime-version:content-type:content-transfer-encoding:in-reply-to:references; bh=Wbvz+gb0Oj9m6JnBK1J1Tppkz2AtsxrdiW5uW2Sp3kk=; b=jS0sl0V1M4t6lyKU+hLt+bbemEJxgOfCl/nZEfhkxzfDcfNvLdVSQ27A2B10/8PmFLkFwm ZDPBB1HAeIkw+g/1aipRuOzmXOF8/c2Logbsotai+0smtSjKZMpUf/mx30deYNTuzkLoOj AkyV+auRLqSUJM4vWJE2B93f9C21hU8= DMARC-Filter: OpenDMARC Filter v1.3.2 vulcan.natalenko.name A829B2F90A1 Authentication-Results: vulcan.natalenko.name; dmarc=fail (p=none dis=none) header.from=natalenko.name From: Oleksandr Natalenko To: Holger =?ISO-8859-1?Q?Hoffst=E4tte?= Cc: "David S. Miller" , Alexey Kuznetsov , Hideaki YOSHIFUJI , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Eric Dumazet , Soheil Hassas Yeganeh , Neal Cardwell , Yuchung Cheng , Van Jacobson , Jerry Chu Subject: Re: TCP and BBR: reproducibly low cwnd and bandwidth Date: Fri, 16 Feb 2018 18:35:03 +0100 Message-ID: <2938690.9Cepv1nWrF@natalenko.name> In-Reply-To: <061740d0-9876-c905-7466-ef225ec3cdc5@applied-asynchrony.com> References: <1697118.nv5eASg0nx@natalenko.name> <2189487.nPhU5NAnbi@natalenko.name> <061740d0-9876-c905-7466-ef225ec3cdc5@applied-asynchrony.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=natalenko.name; s=arc-20170712; t=1518802503; h=from:subject:date:message-id:to:cc:mime-version:content-type:content-transfer-encoding:in-reply-to:references; bh=Wbvz+gb0Oj9m6JnBK1J1Tppkz2AtsxrdiW5uW2Sp3kk=; b=A33eoSZcR2ctcyQOCUsa2i9wLffuvpiX5ifzYzKWRN9lpMBBQeoctFjklyhgXreMT09Wp9 yFRRxsLfjAOuE3Z6j95Ipc4AslXbkoqYbJaEbbSAXgZ1BMEcOeXpPKuxRCeZ9lfZGqd+Q6 gnamAcPy3whPIB67beEK8+tfZUcrzMM= ARC-Seal: i=1; s=arc-20170712; d=natalenko.name; t=1518802503; a=rsa-sha256; cv=none; b=pKqD2gcWjBCpmsB6SvAkDSzAsjubfOkBnO4CoTDL5CJqRTWrNsZd9ERBQ5X6WzZmeKxgQOyXYVhg0FVyk0PrcNQisenW0hZIjuCRM5wArhEphTkFNhjuLX2Ss7zNMKd20y/jtK5Od68wOsTRjNunOLximqRKC4N6RfN3R+urFOk= ARC-Authentication-Results: i=1; auth=pass smtp.auth=oleksandr@natalenko.name smtp.mailfrom=oleksandr@natalenko.name Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi. On p=E1tek 16. =FAnora 2018 17:26:11 CET Holger Hoffst=E4tte wrote: > These are very odd configurations. :) > Non-preempt/100 might well be too slow, whereas PREEMPT/1000 might simply > have too much overhead. Since the pacing is based on hrtimers, should HZ matter at all? Even if so,= =20 poor 1 Gbps link shouldn't drop to below 100 Mbps, for sure. > BBR in general will run with lower cwnd than e.g. Cubic or others. > That's a feature and necessary for WAN transfers. Okay, got it. > Something seems really wrong with your setup. I get completely > expected throughput on wired 1Gb between two hosts: > /* snip */ Yes, and that's strange :/. And that's why I'm wondering what I am missing= =20 since things cannot be *that* bad. > /* snip */ > Please note that BBR was developed to address the case of WAN transfers > (or more precisely high BDP paths) which often suffer from TCP throughput > collapse due to single packet loss events. While it might "work" in other > scenarios as well, strictly speaking delay-based anything is increasingly > less likely to work when there is no meaningful notion of delay - such > as on a LAN. (yes, this is very simplified..) >=20 > The BBR mailing list has several nice reports why the current BBR > implementation (dubbed v1) has a few - sometimes severe - problems. > These are being addressed as we speak. >=20 > (let me know if you want some of those tech reports by email. :) Well, yes, please, why not :). > /* snip */ > I'm not sure testing the old version without builtin pacing is going to h= elp > matters in finding the actual problem. :) > Several people have reported severe performance regressions with 4.15.x, > maybe that's related. Can you test latest 4.14.x? Observed this on v4.14 too but didn't pay much attention until realised tha= t=20 things look definitely wrong. > Out of curiosity, what is the expected use case for BBR here? Nothing special, just assumed it could be set as a default for both WAN and= =20 LAN usage. Regards, Oleksandr