Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752671AbdLERsf (ORCPT ); Tue, 5 Dec 2017 12:48:35 -0500 Received: from mail-lf0-f66.google.com ([209.85.215.66]:37928 "EHLO mail-lf0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752552AbdLERsc (ORCPT ); Tue, 5 Dec 2017 12:48:32 -0500 X-Google-Smtp-Source: AGs4zMZB46ZnsIM6MDrJabtalCn9XdMM45GNNW2yMKDg8LR7ILt5hQvnOYaMVa07VyJgjaAOv/z1mw== Date: Tue, 5 Dec 2017 20:48:37 +0300 From: Serge Semin To: Jon Mason Cc: Dave Jiang , "Hubbe, Allen" , "S-k, Shyam-sundar" , "Yu, Xiangliang" , Gary R Hook , Sergey.Semin@t-platforms.ru, linux-ntb , linux-kernel Subject: Re: [PATCH v2 02/15] NTB: Set dma mask and dma coherent mask to NTB devices Message-ID: <20171205174837.GC1701@mobilestation> References: <20171203191736.3399-1-fancer.lancer@gmail.com> <20171203191736.3399-3-fancer.lancer@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4903 Lines: 109 On Tue, Dec 05, 2017 at 11:51:08AM -0500, Jon Mason wrote: > On Sun, Dec 3, 2017 at 2:17 PM, Serge Semin wrote: > > The dma_mask and dma_coherent_mask fields of the NTB struct device > > weren't initialized in hardware drivers. In fact it should be done > > instead of PCIe interface usage, since NTB clients are supposed to > > use NTB API only and left unaware of real hardware implementation. > > In addition to that ntb_device_register() method shouldn't clear > > the passed ntb_dev structure, since it dma_mask is initialized > > by hardware drivers. > > This seems like a bug fix. Should this be broken out of this patch or > is it not noticed without your changes? > Yes, this is a bug fix. All the NTB API users shouldn't know what hardware type is hidden behind it, so the code would be portable for all NTB devices. In this case NTB device should have dma_masks being properly initialized. Since we pass DMA addresses to the PCI device then in hardware drivers, it's justified to copy the masks to NTB device, so the client drivers could allocate the proper memory addresses. It can't be broken out, since dma_alloc_coherent() is called in ntb_perf driver with NTB device passed now. We discovered the problem when the series was tested on the Intel platform. The same is with ntb_tool driver. Thanks, -Sergey > Thanks, > Jon > > > Signed-off-by: Serge Semin > > --- > > drivers/ntb/hw/amd/ntb_hw_amd.c | 4 ++++ > > drivers/ntb/hw/idt/ntb_hw_idt.c | 8 +++++++- > > drivers/ntb/hw/intel/ntb_hw_intel.c | 4 ++++ > > drivers/ntb/ntb.c | 1 - > > 4 files changed, 15 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/ntb/hw/amd/ntb_hw_amd.c b/drivers/ntb/hw/amd/ntb_hw_amd.c > > index f0788aae05c9..3cfa46876239 100644 > > --- a/drivers/ntb/hw/amd/ntb_hw_amd.c > > +++ b/drivers/ntb/hw/amd/ntb_hw_amd.c > > @@ -1020,6 +1020,10 @@ static int amd_ntb_init_pci(struct amd_ntb_dev *ndev, > > goto err_dma_mask; > > dev_warn(&pdev->dev, "Cannot DMA consistent highmem\n"); > > } > > + rc = dma_coerce_mask_and_coherent(&ndev->ntb.dev, > > + dma_get_mask(&pdev->dev)); > > + if (rc) > > + goto err_dma_mask; > > > > ndev->self_mmio = pci_iomap(pdev, 0, 0); > > if (!ndev->self_mmio) { > > diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.c b/drivers/ntb/hw/idt/ntb_hw_idt.c > > index 6fb87c0f0d97..d87dbd4bc82c 100644 > > --- a/drivers/ntb/hw/idt/ntb_hw_idt.c > > +++ b/drivers/ntb/hw/idt/ntb_hw_idt.c > > @@ -2426,7 +2426,7 @@ static int idt_init_pci(struct idt_ntb_dev *ndev) > > struct pci_dev *pdev = ndev->ntb.pdev; > > int ret; > > > > - /* Initialize the bit mask of DMA */ > > + /* Initialize the bit mask of PCI/NTB DMA */ > > ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(64)); > > if (ret != 0) { > > ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32)); > > @@ -2447,6 +2447,12 @@ static int idt_init_pci(struct idt_ntb_dev *ndev) > > dev_warn(&pdev->dev, > > "Cannot set consistent DMA highmem bit mask\n"); > > } > > + ret = dma_coerce_mask_and_coherent(&ndev->ntb.dev, > > + dma_get_mask(&pdev->dev)); > > + if (ret != 0) { > > + dev_err(&pdev->dev, "Failed to set NTB device DMA bit mask\n"); > > + return ret; > > + } > > > > /* > > * Enable the device advanced error reporting. It's not critical to > > diff --git a/drivers/ntb/hw/intel/ntb_hw_intel.c b/drivers/ntb/hw/intel/ntb_hw_intel.c > > index 4de074a86073..ea6c82f8655f 100644 > > --- a/drivers/ntb/hw/intel/ntb_hw_intel.c > > +++ b/drivers/ntb/hw/intel/ntb_hw_intel.c > > @@ -2586,6 +2586,10 @@ static int intel_ntb_init_pci(struct intel_ntb_dev *ndev, struct pci_dev *pdev) > > goto err_dma_mask; > > dev_warn(&pdev->dev, "Cannot DMA consistent highmem\n"); > > } > > + rc = dma_coerce_mask_and_coherent(&ndev->ntb.dev, > > + dma_get_mask(&pdev->dev)); > > + if (rc) > > + goto err_dma_mask; > > > > ndev->self_mmio = pci_iomap(pdev, 0, 0); > > if (!ndev->self_mmio) { > > diff --git a/drivers/ntb/ntb.c b/drivers/ntb/ntb.c > > index 03b80d89b980..79e50a5f8902 100644 > > --- a/drivers/ntb/ntb.c > > +++ b/drivers/ntb/ntb.c > > @@ -112,7 +112,6 @@ int ntb_register_device(struct ntb_dev *ntb) > > > > init_completion(&ntb->released); > > > > - memset(&ntb->dev, 0, sizeof(ntb->dev)); > > ntb->dev.bus = &ntb_bus; > > ntb->dev.parent = &ntb->pdev->dev; > > ntb->dev.release = ntb_dev_release; > > -- > > 2.12.0 > >