Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210709033502.3545820-1-stevensd@google.com> <20210709033502.3545820-2-stevensd@google.com>
 <20210802133037.GB28547@willie-the-truck>
In-Reply-To: <20210802133037.GB28547@willie-the-truck>
From:   David Stevens <stevensd@chromium.org>
Date:   Thu, 5 Aug 2021 12:40:40 +0900
Message-ID: <CAD=HUj6BoCZ31Dd-jeWKKbmt-jSWXiS2ZLwrFg40FFdPm0y7cA@mail.gmail.com>
Subject: Re: [PATCH v2 1/4] dma-iommu: fix sync_sg with swiotlb
To:     Will Deacon <will@kernel.org>
Cc:     Robin Murphy <robin.murphy@arm.com>,
        Joerg Roedel <joro@8bytes.org>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Tom Murphy <murphyt7@tcd.ie>, iommu@lists.linux-foundation.org,
        open list <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Aug 2, 2021 at 10:30 PM Will Deacon <will@kernel.org> wrote:
>
> On Fri, Jul 09, 2021 at 12:34:59PM +0900, David Stevens wrote:
> > From: David Stevens <stevensd@chromium.org>
> >
> > The is_swiotlb_buffer function takes the physical address of the swiotlb
> > buffer, not the physical address of the original buffer. The sglist
> > contains the physical addresses of the original buffer, so for the
> > sync_sg functions to work properly when a bounce buffer might have been
> > used, we need to use iommu_iova_to_phys to look up the physical address.
> > This is what sync_single does, so call that function on each sglist
> > segment.
> >
> > The previous code mostly worked because swiotlb does the transfer on map
> > and unmap. However, any callers which use DMA_ATTR_SKIP_CPU_SYNC with
> > sglists or which call sync_sg would not have had anything copied to the
> > bounce buffer.
> >
> > Fixes: 82612d66d51d ("iommu: Allow the dma-iommu api to use bounce buffers")
> > Signed-off-by: David Stevens <stevensd@chromium.org>
> > ---
> >  drivers/iommu/dma-iommu.c | 26 +++++++++++++-------------
> >  1 file changed, 13 insertions(+), 13 deletions(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 7bcdd1205535..eac65302439e 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -811,14 +811,14 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev,
> >       if (dev_is_dma_coherent(dev) && !dev_is_untrusted(dev))
> >               return;
> >
> > -     for_each_sg(sgl, sg, nelems, i) {
> > -             if (!dev_is_dma_coherent(dev))
> > -                     arch_sync_dma_for_cpu(sg_phys(sg), sg->length, dir);
> > -
> > -             if (is_swiotlb_buffer(sg_phys(sg)))
> > +     if (dev_is_untrusted(dev))
> > +             for_each_sg(sgl, sg, nelems, i)
> > +                     iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg),
> > +                                                   sg->length, dir);
> > +     else
> > +             for_each_sg(sgl, sg, nelems, i)
> >                       swiotlb_sync_single_for_cpu(dev, sg_phys(sg),
> >                                                   sg->length, dir);
>
> Doesn't this skip arch_sync_dma_for_cpu() for non-coherent trusted devices?

Whoops, this was supposed to be a call to arch_sync_dma_for_cpu, not
to swiotlb_sync_single_for_cpu. Similar to the sync_sg_for_device
case.

> Why not skip the extra dev_is_untrusted(dev) call here and just call
> iommu_dma_sync_single_for_cpu() for each entry regardless?

iommu_dma_sync_single_for_cpu calls iommu_iova_to_phys to translate
the dma_addr_t to a phys_addr_t. Since the physical address is readily
available, I think it's better to avoid that extra work.

> Will