Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758506AbcLBImG (ORCPT ); Fri, 2 Dec 2016 03:42:06 -0500 Received: from mx08-00178001.pphosted.com ([91.207.212.93]:24742 "EHLO mx07-00178001.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754166AbcLBImE (ORCPT ); Fri, 2 Dec 2016 03:42:04 -0500 Subject: Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses. To: Pavel Machek , David Miller References: <20161123105125.GA26394@amd> <20161124085506.GA25007@amd> <20161124.110416.198867271899443489.davem@davemloft.net> <20161124212540.GA24796@amd> CC: , , From: Giuseppe CAVALLARO Message-ID: Date: Fri, 2 Dec 2016 09:41:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.52.139.54] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-12-02_04:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2619 Lines: 77 + Lino On 12/2/2016 9:24 AM, Giuseppe CAVALLARO wrote: > > Hello > > > On 11/24/2016 10:25 PM, Pavel Machek wrote: >> Hi! >> >>>>> I'm debugging strange delays during transmit in stmmac driver. They >>>>> seem to be present in 4.4 kernel (and older kernels, too). Workload is >>>>> burst of udp packets being sent, pause, burst of udp packets, ... >> ... >>>> 4.9-rc6 still has the delays. With the >>>> >>>> #define STMMAC_COAL_TX_TIMER 1000 >>>> #define STMMAC_TX_MAX_FRAMES 2 >>>> >>>> settings, delays go away, and driver still works. (It fails fairly >>>> fast in 4.4). Good news. But the question still is: what is going on >>>> there? >>> >>> 256 packets looks way too large for being a trigger for aborting the >>> TX coalescing timer. >>> >>> Looking more deeply into this, the driver is using non-highres timers >>> to implement the TX coalescing. This simply cannot work. >>> >>> 1 HZ, which is the lowest granularity of non-highres timers in the >>> kernel, is variable as well as already too large of a delay for >>> effective TX coalescing. >>> >>> I seriously think that the TX coalescing support should be ripped out >>> or disabled entirely until it is implemented properly in this >>> driver. >> >> Ok, I'd disable coalescing, but could not figure it out till. What is >> generic way to do that? >> >> It seems only thing stmmac_tx_timer() does is calling >> stmmac_tx_clean(), which reclaims tx_skbuff[] entries. It should be >> possible to do that explicitely, without delay, but it stops working >> completely if I attempt to do that. >> >> On a side note, stmmac_poll() does stmmac_enable_dma_irq() while >> stmmac_dma_interrupt() disables interrupts. But I don't see any >> protection between the two, so IMO it could race and we'd end up >> without polling or interrupts... > > > the idea behind the TX mitigation is to mix the interrupt and > timer and this approach gave us real benefit in terms > of performances and CPU usage (especially on SH4-200/SH4-300 platforms > based). > In the ring, some descriptors can raise the irq (according to a > threshold) and set the IC bit. In this path, the NAPI poll will be > scheduled. > But there is a timer that can run (and we experimented that no high > resolution is needed) to clear the tx resources. > Concerning the lock protection, we had reviewed long time ago and > IIRC, no raise condition should be present. Open to review it, again! > > So, welcome any other schema and testing on platforms supported. > > > Hoping this summary can help. > > Peppe > > >> >> Thanks and best regards, >> Pavel >> > >