Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756044AbXKYCCk (ORCPT ); Sat, 24 Nov 2007 21:02:40 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753195AbXKYCCa (ORCPT ); Sat, 24 Nov 2007 21:02:30 -0500 Received: from smtp1.betherenow.co.uk ([87.194.0.68]:35109 "EHLO smtp1.bethere.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753120AbXKYCC3 (ORCPT ); Sat, 24 Nov 2007 21:02:29 -0500 From: Alistair John Strachan To: Francois Romieu Subject: Re: 2.6.24-rc3, 4GB RAM, swiotlb, r8169, out of space Date: Sun, 25 Nov 2007 02:02:27 +0000 User-Agent: KMail/1.9.7 Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Alan Cox References: <200711242344.49958.alistair@devzero.co.uk> <20071125002510.GA30759@electric-eye.fr.zoreil.com> <20071125012754.GC30759@electric-eye.fr.zoreil.com> In-Reply-To: <20071125012754.GC30759@electric-eye.fr.zoreil.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200711250202.27660.alistair@devzero.co.uk> Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4107 Lines: 121 On Sunday 25 November 2007 01:27:54 Francois Romieu wrote: > Francois Romieu : > > Alistair John Strachan : > > [...] > > > > > The "choke" affects other devices on the system too, notably libata, > > > which does not recover gracefully. In my logs, I see a stream of: > > > > > > DMA: Out of SW-IOMMU space for 7222 bytes at device 0000:04:00.0 > > > DMA: Out of SW-IOMMU space for 7222 bytes at device 0000:04:00.0 > > > > You are using jumbo frames, aren't you ? > > See below for my late night crap. At least it should avoid the driver > issuing Rx/Tx DMA with the single static buffer of lib/swiotlb.c > (io_tlb_overflow_buffer). Ghee. No improvement. It might be possible to reproduce the problem on your end if you add iommu support and force enable the swiotlb (which should be possible even with <4GB RAM). > diff --git a/drivers/net/r8169.c b/drivers/net/r8169.c > index 1f647b9..72a7370 100644 > --- a/drivers/net/r8169.c > +++ b/drivers/net/r8169.c > @@ -2262,10 +2262,16 @@ static struct sk_buff *rtl8169_alloc_rx_skb(struct > pci_dev *pdev, mapping = pci_map_single(pdev, skb->data, rx_buf_sz, > PCI_DMA_FROMDEVICE); > > + if (pci_dma_mapping_error(mapping)) > + goto err_kfree_skb; > + > rtl8169_map_to_asic(desc, mapping, rx_buf_sz); > out: > return skb; > > +err_kfree_skb: > + dev_kfree_skb(skb); > + skb = NULL; > err_out: > rtl8169_make_unusable_by_asic(desc); > goto out; > @@ -2486,6 +2492,7 @@ static int rtl8169_xmit_frags(struct rtl8169_private > *tp, struct sk_buff *skb, dma_addr_t mapping; > u32 status, len; > void *addr; > + int rc; > > entry = (entry + 1) % NUM_TX_DESC; > > @@ -2493,6 +2500,22 @@ static int rtl8169_xmit_frags(struct rtl8169_private > *tp, struct sk_buff *skb, len = frag->size; > addr = ((void *) page_address(frag->page)) + frag->page_offset; > mapping = pci_map_single(tp->pci_dev, addr, len, PCI_DMA_TODEVICE); > + rc = pci_dma_mapping_error(mapping); > + if (unlikely(rc < 0)) { > + while (cur_frag-- > 0) { > + frag = info->frags + cur_frag; > + entry = (entry - 1) % NUM_TX_DESC; > + txd = tp->TxDescArray + entry; > + len = frag->size; > + mapping = le64_to_cpu(txd->addr); > + pci_unmap_single(tp->pci_dev, mapping, len, > + PCI_DMA_TODEVICE); > + txd->opts1 = 0x00; > + txd->opts2 = 0x00; > + txd->addr = 0x00; > + } > + return rc; > + } > > /* anti gcc 2.95.3 bugware (sic) */ > status = opts1 | len | (RingEnd * !((entry + 1) % NUM_TX_DESC)); > @@ -2534,13 +2557,13 @@ static inline u32 rtl8169_tso_csum(struct sk_buff > *skb, struct net_device *dev) static int rtl8169_start_xmit(struct sk_buff > *skb, struct net_device *dev) { > struct rtl8169_private *tp = netdev_priv(dev); > - unsigned int frags, entry = tp->cur_tx % NUM_TX_DESC; > + unsigned int entry = tp->cur_tx % NUM_TX_DESC; > struct TxDesc *txd = tp->TxDescArray + entry; > void __iomem *ioaddr = tp->mmio_addr; > dma_addr_t mapping; > u32 status, len; > u32 opts1; > - int ret = NETDEV_TX_OK; > + int frags, ret = NETDEV_TX_OK; > > if (unlikely(TX_BUFFS_AVAIL(tp) < skb_shinfo(skb)->nr_frags)) { > if (netif_msg_drv(tp)) { > @@ -2557,7 +2580,11 @@ static int rtl8169_start_xmit(struct sk_buff *skb, > struct net_device *dev) opts1 = DescOwn | rtl8169_tso_csum(skb, dev); > > frags = rtl8169_xmit_frags(tp, skb, opts1); > - if (frags) { > + if (frags < 0) { > + printk(KERN_ERR "%s: PCI mapping failure (%d).\n", dev->name, > + frags); > + goto err_busy; > + } else if (frags > 0) { > len = skb_headlen(skb); > opts1 |= FirstFrag; > } else { > @@ -2605,6 +2632,7 @@ out: > > err_stop: > netif_stop_queue(dev); > +err_busy: > ret = NETDEV_TX_BUSY; > err_update_stats: > dev->stats.tx_dropped++; -- Cheers, Alistair. 137/1 Warrender Park Road, Edinburgh, UK. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/