From: Casey Leedom <leedom@chelsio.com>
To: "Raj, Ashok" <ashok.raj@intel.com>,
        Robin Murphy <robin.murphy@arm.com>
CC: "Harsh  Jain" <Harsh@chelsio.com>,
        Herbert Xuy <herbert@gondor.apana.org.au>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
        Dan Williams <dan.j.williams@intel.com>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>
Subject: Re: DMA error when sg->offset value is greater than PAGE_SIZE in
 Intel IOMMU
Thread-Topic: DMA error when sg->offset value is greater than PAGE_SIZE in
 Intel IOMMU
Thread-Index: AQHTLrKuCX/YGlya50aETYTEmnILyqK9cAoAgAAkZQCACFcnkv//5CkAgAAvAguAAJgAAIAAj7QAgAAh/4CAAANTgIAAAaSAgABj5z8=
Date: Tue, 26 Sep 2017 20:50:02 +0000
Message-ID: <MWHPR12MB16007D021D7067D0E134F938C87B0@MWHPR12MB1600.namprd12.prod.outlook.com>
References: <20170920080151.GA3348@gondor.apana.org.au>
 <26992a1e-edb3-ed78-ce8e-31e0739d75f4@arm.com>
 <MWHPR12MB160092A868C613FF64CA9EE9C87A0@MWHPR12MB1600.namprd12.prod.outlook.com>
 <20170925155430.GB131920@otc-nc-03>
 <MWHPR12MB16004E2A92CC3FFEC6313DDBC87A0@MWHPR12MB1600.namprd12.prod.outlook.com>
 <afa02763-4556-0e14-7d1b-1c044cdc1ff7@chelsio.com>
 <6d2af675-7b97-6eaf-4daa-d7bf80a05923@chelsio.com>
 <437a9bd8-d4d6-22ca-1a64-1a3e73f1101a@arm.com>
 <20170926143441.GA136940@otc-nc-03>,<20170926144032.GA90930@otc-nc-03>
In-Reply-To: <20170926144032.GA90930@otc-nc-03>
Accept-Language: en-US
Content-Language: en-US
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-1"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Sep 2017 20:50:02.7154
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 065db76d-a7ae-4c60-b78a-501e8fc17095
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1952
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by nfs id v8QKoqTK016396
Content-Length: 2017
Lines: 45

  So just to be 100% sure I understand the patch you're proposing, you got
the first use of VTD_PAGE_SHIFT wrong; it should have been VTD_PAGE_MASK?  I.e.

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6784a05..d43b566 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2254,10 +2254,13 @@ static int __domain_mapping(struct dmar_domain *domain, unsigned long iov_pfn,
                uint64_t tmp;

                if (!sg_res) {
+                       size_t off = sg->offset & ~VTD_PAGE_MASK;
                        sg_res = aligned_nrpages(sg->offset, sg->length);
-                       sg->dma_address = ((dma_addr_t)iov_pfn << VTD_PAGE_SHIFT) + sg->offset;
+                       sg->dma_address = ((dma_addr_t)
+                               (iov_pfn + sg->offset) << VTD_PAGE_SHIFT) + off;
                        sg->dma_length = sg->length;
-                       pteval = page_to_phys(sg_page(sg)) | prot;
+                       pteval = (page_to_phys(sg_page(sg)) +
+                               (sg->offset << VTD_PAGE_SHIFT)) | prot;
                        phys_pfn = pteval >> VTD_PAGE_SHIFT;
                }

???  And I'm still confused about this portion:

-                       sg->dma_address = ((dma_addr_t)iov_pfn << VTD_PAGE_SHIFT) + sg->offset;
+                       sg->dma_address = ((dma_addr_t)
+                               (iov_pfn + sg->offset) << VTD_PAGE_SHIFT) + off;
 
Isn't iov_pfn a Physical Page Number and we're adding a Byte Offset
to that?  I would have though that it would be more like:

    size_t page_off = sg->offset & ~VTD_PAGE_MASK;
    unsigned long pfn_off = sg->offset >> VTD_PAGE_MASK;
    ...
    sg->dma_address = ((dma_addr_t)
                       (iov_pfn + pfn_off) << VTD_PAGE_SHIFT) + page_off;

  I want to be sure that Harsh has a concrete patch to work with when he
wakes up.

  How about it Robin, Dan, David, Herbert, what do you guys think of Raj's
proposed patch?

Casey