Received: by 2002:a25:8b91:0:0:0:0:0 with SMTP id j17csp1913918ybl; Thu, 5 Dec 2019 08:58:26 -0800 (PST) X-Google-Smtp-Source: APXvYqypLNTye6iXvvXitJTkbfFFyzUBQPLA5eY9Ywnn9pd6ce5SF3RyB6ZwhC9VMl3szFPBtg4l X-Received: by 2002:aca:111a:: with SMTP id 26mr7639382oir.145.1575565105936; Thu, 05 Dec 2019 08:58:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1575565105; cv=none; d=google.com; s=arc-20160816; b=Raj2oBvlC4UMDG+8rYX9o6b6d6WTNczfKjV8rxLQe5Jc4KHio6ls2dHCHh7haDcbDm HflNVLWkBWZULi2EVXuK3W7IvkKFPeTt000H4aLWR/KKBDuGgyxbkistD8qo4/gzChyW Zrx8UojhqgNQu06SQNtYGOzXLnqXLIp60jZQNlJT78dORkerRn4YyodazlcaREIXtitg aw0sQjrm0FFr/Av2J0TTCvXXHL/N+ipvh+ZIcl4sSCsGW6cfTNtqK0a6gZE13Iq3IcfN PVvFVG7YKuepztepp380gIHpFphcirVpfQYWqY5D3ukEyks81R72aNzdofGzZpqYhEaA NB0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:organization:from:references:cc:to:subject :dkim-signature:dkim-filter; bh=6EH0JVvdj3zRs4Qkm+MUOXzLrlXRHdw9QQW3nHEB/y4=; b=m0LpRbb/areDFc3wyQSgl+nzcRsb04eVV/z4kyBm8cqUnpIT1zj2m50h3hOgvAUmT6 4CQghAtPOiEuXzvqzlYdhbl6xZsEC6hdsKutySz3rC2lZaKSxOJCUTWXD8PAJsi82Xys NZ/vfNo0Ab8IXe8f8M+SbVZOg3NnRIiLXVLncE1TZcVl3XtnrWQK8Tla9MDDDiMvB0Q8 QwzBn5Lcnidf+b+UBeNC+TxjyCBbCcRai6+TMbv5d/Jr0liAqGAKG6DE/GJpaTnNKS0z C+ddwbkmZDSwwni32lV7jKmJJiiH4nxNDKCspnlAyU1iI3GdU0B+kn/ZXVPuDvqCjyWC i7ww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@candelatech.com header.s=default header.b=h97OGLcK; spf=pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-wireless-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=candelatech.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t2si4980374oth.32.2019.12.05.08.58.13; Thu, 05 Dec 2019 08:58:25 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@candelatech.com header.s=default header.b=h97OGLcK; spf=pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-wireless-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=candelatech.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729047AbfLEQ5i (ORCPT + 99 others); Thu, 5 Dec 2019 11:57:38 -0500 Received: from mail2.candelatech.com ([208.74.158.173]:38130 "EHLO mail3.candelatech.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726028AbfLEQ5i (ORCPT ); Thu, 5 Dec 2019 11:57:38 -0500 Received: from [192.168.100.195] (50-251-239-81-static.hfc.comcastbusiness.net [50.251.239.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail3.candelatech.com (Postfix) with ESMTPSA id A5EF3137531; Thu, 5 Dec 2019 08:57:37 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 mail3.candelatech.com A5EF3137531 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=candelatech.com; s=default; t=1575565057; bh=GT3Kfq28O8/jytxcmj3IIENh+6oLIrfoYN9hxhL1ZHs=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=h97OGLcKYPCRqG/zq1UgY6wVsSt79+lvv11iRcMkNGBKe5JE8Mo16t7PP6+4ju/CB zbzHK7xO2pn8/Fk5fHLBw2Tky8zB4vKkpWbTC6Z8QrqX49zLF1v95SmDFqXzJB3B7C V7bzjlhVhYyyW6SOTHTM+NwnpGId4tuSr3A2wSMo= Subject: Re: debugging TXQs being empty To: Johannes Berg , =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= Cc: linux-wireless@vger.kernel.org References: <49288397-55b3-f49a-e277-5197d7c1bea3@candelatech.com> <2e01fb1d5b8ff3335220231251f893c01c6c22d3.camel@sipsolutions.net> From: Ben Greear Organization: Candela Technologies Message-ID: <64a6ae15-b7be-8c27-4e60-4cb3d253f3dc@candelatech.com> Date: Thu, 5 Dec 2019 08:57:37 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: <2e01fb1d5b8ff3335220231251f893c01c6c22d3.camel@sipsolutions.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org On 12/5/19 8:49 AM, Johannes Berg wrote: > On Thu, 2019-12-05 at 08:37 -0800, Ben Greear wrote: > >>> All this seems to mean that the TCP stack isn't feeding us fast enough, >>> but is that really possible? >> >> Does UDP work better? > > Somewhat, I get about 1020-1030 Mbps. But still a TON of "TXQ of STA ... > is now empty" messages. Say this run got about 15 per second of those. It would seem that it is not some issue with TCP stack then? In general, UDP uses more CPU to send from user-space than TCP because of TSO, etc. Sendmmsg can help a bit, but it is a bit painful to code against so things like iperf do not use it, at least ones I've looked at. Can you provide some details on how you are generating this load? For what it's worth, we've seen about 1.9Gbps download goodput when using ax200 as a station receiving traffic from 160Mhz AP. I don't have any reports of > 1Gbps of upload performance though, not sure our user with the fast AP has done much upload testing... >> or pktgen? > > I haven't really tried, the setup is a bit complicated ... and it's > nowhere near me either :) Yeah, it will likely crash your system unless you apply years-old patches I posted too :) But, at least with pktgen, you can be quite sure it is not some slowdown farther up the stack that is causing the problem. Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com