Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751716Ab3CNUXt (ORCPT ); Thu, 14 Mar 2013 16:23:49 -0400 Received: from mout.web.de ([212.227.15.4]:56380 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751081Ab3CNUXr (ORCPT ); Thu, 14 Mar 2013 16:23:47 -0400 Message-ID: <51423196.6090907@web.de> Date: Thu, 14 Mar 2013 21:22:46 +0100 From: Soeren Moch User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130308 Thunderbird/17.0.4 MIME-Version: 1.0 To: Arnd Bergmann CC: Alan Stern , USB list , Jason Cooper , Andrew Lunn , Sebastian Hesselbarth , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH] USB: EHCI: fix for leaking isochronous data References: <51421B89.6020308@web.de> In-Reply-To: <51421B89.6020308@web.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:JpG3N3GQeKAYbcUNEC0V8g0tUGWNIORipkX1PoHjl2F bi49YPEpk9/nwKaB6epx10L11FQa4lNLfID2ncX8iUbeQzLn4I O0x8H9OAs8Yu2PlNi9yfi2BhiJ4/a1F1388mYSlDZecj+Afb+m 1O5I9kIXtyMFX2CT7RLuCoByye06M9ByVrlDM4sCm3WvkImBhS ++ns3gT5PGfX2U2O5Pe1g== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4316 Lines: 102 On 14.03.2013 19:48, Soeren Moch wrote: > On 10.03.2013 21:59, Alan Stern wrote: >> On Sun, 10 Mar 2013, Soeren Moch wrote: >>>> On Wed, 20 Feb 2013, Soeren Moch wrote: >>>> >>>>> Ok. I use 2 em2840-based usb sticks (em28xx driver) attached to a >>>>> Marvell Kirkwood-SoC with a orion-ehci usb controller. These usb >>>>> sticks >>>>> stream dvb data (digital TV) employing isochronous usb transfers (user >>>>> application is vdr). >>>>> >>>>> Starting from linux-3.6 I see >>>>> ERROR: 1024 KiB atomic DMA coherent pool is too small! >>>>> in the syslog after several 10 minutes (sometimes hours) of streaming >>>>> and then streaming stops. >>>>> >>>>> In linux-3.6 the memory management for the arm architecture was >>>>> changed, >>>>> so that atomic coherent dma allocations are served from a special >>>>> pool. >>>>> This pool gets exhausted. The only user of this pool (in my test) is >>>>> orion-ehci. Although I have only 10 URBs in flight (5 for each stick, >>>>> resubmitted in the completion handler), I have 256 atomic coherent >>>>> allocations (memory from the pool is allocated in pages) from >>>>> orion-ehci >>>>> when I see this error. So I think there must be a memory leak (memory >>>>> allocated atomic somewhere below the usb_submit_urb call in >>>>> em28xx-core.c). >>>>> >>>>> With other dvb sticks using usb bulk transfers I never see this error. >>>>> >>>>> Since you already found a memory leak in the ehci driver for isoc >>>>> transfers, I hoped you can help to solve this problem. If there are >>>>> additional questions, please ask. If there is something I can test, I >>>>> would be glad to do so. >>>> >>>> I guess the first thing is to get a dmesg log showing the problem. You >>>> should build a kernel with CONFIG_USB_DEBUG enabled and post the part >>>> of the dmesg output starting from when you plug in the troublesome DVB >>>> stick. >>> >>> Sorry for my late response. Now I built a kernel 3.8.0 with usb_debug >>> enabled. See below for the syslog of device plug-in. >>> >>>> It also might help to have a record of all the isochronous-related >>>> coherent allocations and deallocations done by the ehci-hcd driver. >>>> Are you comfortable making your own debugging changes? The allocations >>>> are done by a call to dma_pool_alloc() in >>>> drivers/usb/host/ehci-sched.c:itd_urb_transaction() if the device runs >>>> at high speed and sitd_urb_transaction() if the device runs at full >>>> speed. The deallocations are done by calls to dma_pool_free() in >>>> ehci-timer.c:end_free_itds(). >>>> >>> >>> I added a debug message to >>> drivers/usb/host/ehci-sched.c:itd_urb_transaction() to log the >>> allocation flags, see log below. >> >> But it looks like you didn't add a message to end_free_itds(), so we >> don't know when the memory gets deallocated. And you didn't print out >> the values of urb, num_itds, and i, or the value of itd (so we can >> match up allocations against deallocations). > > OK, I will implement this more detailed logging. But with several > allocations per second and runtime of several hours this will result in > a very long logfile. > >>> For me this looks like nothing is >>> allocated atomic here, so this function should not be the root cause of >>> the dma coherent pool exhaustion. >> >> I don't understand. If non-atomic allocations can't exhaust the pool, >> why do we see these allocations fail? > > Good point. Unfortunately I'm not familiar with the memory management > details. > > Arnd, can memory allocated with dma_pool_alloc() and gfp_flags > 0x20000093 or 0x80000093 come from the atomic dma coherent pool? Sorry, I logged the wrong flags. All allocations are GFP_ATOMIC (0x20) and therefore coming from the pool. Soeren >>> Are there other allocation functions >>> which I could track? >> >> Yes, but they wouldn't be used for isochronous transfers. See >> ehci_qtd_alloc(), ehci_qtd_free(), ehci_qh_alloc(), and qh_destroy() in >> ehci-mem.c, as well as some other one-time-only coherent allocations in >> that file. >> >> Alan Stern >> > Soeren Moch -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/