Received: by 2002:a25:b794:0:0:0:0:0 with SMTP id n20csp5663887ybh; Wed, 7 Aug 2019 09:23:50 -0700 (PDT) X-Google-Smtp-Source: APXvYqycfjZ9p+gHTojUNcSGRqCCToIsnkBA0yABlhRkpDd/lK4kcPxsZUVHKXtCeleI0aDJcopc X-Received: by 2002:a17:902:b08a:: with SMTP id p10mr9053277plr.83.1565195029917; Wed, 07 Aug 2019 09:23:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565195029; cv=none; d=google.com; s=arc-20160816; b=tp8qB3We/4fAq+mKVo2BiNM9ab55PnxEQirL6UTlMbOzqbVfz9VqpB23rDw7u8RapL vr9NB1mjZ+ORkcvdAW8KD5d8FDrE5Vdbzz6F/wNEr8dCHgpRlOxrS1y5UcR0d0Re91Pe 9+VbrzEiR1QRowTufaRzZrt+3JkgNaJVcJ2uEdTKXgLkJPltiDkc6fiFHR2YOvwl8VuT Ysx8e+fGKwW8ff3Y09j7qkOX7xJjluf6KawKkrmaHqXQXShXvNRphsMoc6jI9GaTz4Ek NgjlzS4WGQfu5GKqnOLhI3IomnnJDKlJ/2a2+iWXeYFlv4zdSCZxgk/6nYL5lxbAq0Cx iBQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=AOfNJoVis4tu5FW/x4K5fd3Kb4BI9lC9c+PcHuGbSJI=; b=rjTxEfGN1qDYM+QWb/XhGT64ny/8c6pkKpGKt4ieVN/LGzODnP2huEZuTmmF8QsJF0 6AOZi24Lt8B/HZ9dzbRyIcfvp53TxFlGqVNfYHLW1JxxfkGqLtn/reYvZ2kZqFlQidhW BDttt1oUNpnssP6lo/xgCBCmXy26NS1SLyfPxZ3xvMX2gFcP8BSAe4mkpLmxZgsK4tfc HOlThrODDh/ZmqJNdjQr7rlU92/YY9Ufb9Kjr6HVQdGcWb/zwJuJNI7vMwB82/zIVHYs f0DRzep2ipYiATrJiT7dxDiYiFtOKlWQRCUoQKTCX88zDhFHn5qig8r8+ceEH4H+HW3T eiCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=rHZRSP4M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v41si50672870pgn.481.2019.08.07.09.23.34; Wed, 07 Aug 2019 09:23:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=rHZRSP4M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388890AbfHGQGZ (ORCPT + 99 others); Wed, 7 Aug 2019 12:06:25 -0400 Received: from mail-ot1-f66.google.com ([209.85.210.66]:45306 "EHLO mail-ot1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387626AbfHGQGY (ORCPT ); Wed, 7 Aug 2019 12:06:24 -0400 Received: by mail-ot1-f66.google.com with SMTP id x21so13099080otq.12; Wed, 07 Aug 2019 09:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AOfNJoVis4tu5FW/x4K5fd3Kb4BI9lC9c+PcHuGbSJI=; b=rHZRSP4Mj7U/HkytF9Zva56bn+5Sg4odmEri/N7n+ITRLCyiUj2eMmCJlHy4XlLYsc 7gJIN/GF3DAsUHa4BH7V0W2QaOPem6U3rgj0srz/w+BOfWwZ+8s7urdEe+Z3nXeAP9o6 6+oCplvH6V5cloglseO3mDXC06ioD25CV2idliDu8whCh8Py2B8HSijwVJ4l/U3wzT/e 2PI/Sf1UElctkc8X8QP9CZGa6RbSuvH81uctJMqTFtQhvbRtItLQ5e4wLnXl8N3jSmuI E0FWZ/m5t803vlXqBH1l4W/l02psddI0zSf8Vq/2EzBEDcA8/sbkzk7IYRug/6obhO5Q MI3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AOfNJoVis4tu5FW/x4K5fd3Kb4BI9lC9c+PcHuGbSJI=; b=f8r64OV/fk/WlgkfqMqSEgiUXYv3muVCAOZJrjj8H1j1lKbOG6nSlVnOoGe6fp1LD4 TCacV1ixcT6qwXfynZByFjErAGeO1pxPXk28YKHUT9UOXHsRZu/08G1U2wuplTbsLYlH SMtLlM6jEzs3I6mh1nvieO2i1ItKNOyZihAb9nPWMHYH0AjUemAMdrB0bNcCOFjl30sa 6GMhSgkYnOOeYmIjBOUtnpMtjjwFfWRYFSTTQza7+p4hQYpRPgepweyRpWh/bY4W37aT QPmWEfu9T9sm3/dciLpQHVFQFzrDQ7Hz3oRDmKR9i/XmOydaqHHEb4V0jbnReSrMdNBr zCAQ== X-Gm-Message-State: APjAAAU8Y8ae2gFUKJ1k1X/zo7JCWe9Ss0VSPmHbhnbisNhvf0kgEq4i NJFDuhyVeRfgNp4O16/7wfEg6ssQIP1H9/q1SLc= X-Received: by 2002:a6b:dd18:: with SMTP id f24mr9312803ioc.97.1565193983972; Wed, 07 Aug 2019 09:06:23 -0700 (PDT) MIME-Version: 1.0 References: <20190807024917.27682-1-firo.yang@suse.com> <85aaefdf-d454-1823-5840-d9e2f71ffb19@oracle.com> <20190807083831.GA6811@linux-6qg8> <20190807160853.00001d71@gmail.com> In-Reply-To: <20190807160853.00001d71@gmail.com> From: Alexander Duyck Date: Wed, 7 Aug 2019 09:06:12 -0700 Message-ID: Subject: Re: [Intel-wired-lan] [PATCH v2 1/1] ixgbe: sync the first fragment unconditionally To: Maciej Fijalkowski Cc: Firo Yang , Netdev , LKML , intel-wired-lan , Jacob Wen , "davem@davemloft.net" , Alexander Duyck Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Aug 7, 2019 at 7:09 AM Maciej Fijalkowski wrote: > > On Wed, 7 Aug 2019 08:38:43 +0000 > Firo Yang wrote: > > > The 08/07/2019 15:56, Jacob Wen wrote: > > > I think the description is not correct. Consider using something like below. > > Thank you for comments. > > > > > > > > In Xen environment, due to memory fragmentation ixgbe may allocate a 'DMA' > > > buffer with pages that are not physically contiguous. > > Actually, I didn't look into the reason why ixgbe got a DMA buffer which > > was mapped to Xen-swiotlb area. > > I think that neither of these descriptions are telling us what was truly > broken. They lack what Alexander wrote on v1 thread of this patch. > > ixgbe_dma_sync_frag is called only on case when the current descriptor has EOP > bit set, skb was already allocated and you'll be adding a current buffer as a > frag. DMA unmapping for the first frag was intentionally skipped and we will be > unmapping it here, in ixgbe_dma_sync_frag. As Alex said, we're using the > DMA_ATTR_SKIP_CPU_SYNC attribute which obliges us to perform a sync manually > and it was missing. > > So IMHO the commit description should include descriptions from both xen and > ixgbe worlds, the v2 lacks info about ixgbe case. > > BTW Alex, what was the initial reason for holding off with unmapping the first > buffer? Asking because IIRC the i40e and ice behave a bit different here. We > don't look there for EOP at all when building/constructing skb and not delaying > the unmap of non-eop buffers. > > Thanks, > Maciej The reason why we have to hold off on unmapping the first buffer is because in the case of Receive Side Coalescing (RSC), also known as Large Receive Offload (LRO), the header of the packet is updated for each additional frame that is added. As such you can end up with the device writing data, header, data, header, data, header where each data write would update a new descriptor, but the header will only ever update the first descriptor in the chain. As such if we unmapped it earlier it would result in an IOMMU fault because the device would be rewriting the header after it had been unmapped. The devices supported by the ixgbe driver are the only ones that have RSC/LRO support. As such this behavior is present for ixgbe, but not for other Intel NIC drivers including igb, igbvf, ixgbevf, i40e, etc. - Alex