Received: by 10.213.65.68 with SMTP id h4csp251787imn; Mon, 12 Mar 2018 12:38:57 -0700 (PDT) X-Google-Smtp-Source: AG47ELvrXUCHU3pInr6OmocFfB6lGVy+tsReGEB9DDMKPnrtZrFE71H6AtRaRwSJ+KkC7AwrcZZj X-Received: by 2002:a17:902:6789:: with SMTP id g9-v6mr9348660plk.167.1520883537850; Mon, 12 Mar 2018 12:38:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520883537; cv=none; d=google.com; s=arc-20160816; b=kYanzb44629d16ig0hP+wgQ/1nQ32aL99Yt4vVz2tDBNF8GwnQP+spFhT4Unhhryr4 gJxHgrGrruqZ9M3H6FAmRAirNfTFb/OSA8sc1BbgoWyCE/H+7ZymxGQ/e/EZP8kqz4L5 KvrRqnao1U6kO3yuL6MFgQSforh9ixbw9dDCEqJhXnnO++0LqzCBkkqFTpTf3hjxO+4y KhUuRCWzX7TVN0cnGZ3mqLc9l7oJp5FZrJvzqsDYPVjyL3pCJfbYLnPsdSwyFhYgXK/E xObckYXsAMzsEEGvQgIfBTUKSRP6UxL+2HwHxKrCkWjf3DEJI6X8sqJq8KrbcGURn3qm M5zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:subject:references:in-reply-to:message-id :date:cc:to:from:arc-authentication-results; bh=TDvDHyQmGUP+UGy+Enuh0IpMR08+X789+yxnQjq0diA=; b=r6FBo4voz+jRuH13snDZ8FCQQUvAMW5u8MPzYrLVxdv3RzwP45lSH61aOr6i7pCva2 vCsWcG6HN6q4eYwi13U7FvBur9wdeRVNtswM+EP0Jeok8pWycoqoJOauZGesPf10yl9L kWUk4z43lYJz1RmfTJcCGrnKuKnewHHtZrJxVMpmGGf7LlAeyPA7MOpAfV4G2TScv0vC X2eTGR1fTx4LArxK60DzBx3St6Pt7FqLv2sjihEKxJG5ylEtovpkweY6NpxcLTB/u18w AgbNVLvCm8EDqYxStU41h1rUiTINgs3UPN97TfTBArMB4M+xBFVhHON4cVpP6SDfpuM2 rURA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z23-v6si6355341pll.729.2018.03.12.12.38.43; Mon, 12 Mar 2018 12:38:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932394AbeCLTfp (ORCPT + 99 others); Mon, 12 Mar 2018 15:35:45 -0400 Received: from ale.deltatee.com ([207.54.116.67]:54644 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932282AbeCLTfi (ORCPT ); Mon, 12 Mar 2018 15:35:38 -0400 Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1evTEJ-00066B-QG; Mon, 12 Mar 2018 13:35:33 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.89) (envelope-from ) id 1evTEH-0000lL-8R; Mon, 12 Mar 2018 13:35:29 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Stephen Bates , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Dan Williams , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Benjamin Herrenschmidt , Alex Williamson , Logan Gunthorpe Date: Mon, 12 Mar 2018 13:35:23 -0600 Message-Id: <20180312193525.2855-10-logang@deltatee.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180312193525.2855-1-logang@deltatee.com> References: <20180312193525.2855-1-logang@deltatee.com> X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-nvme@lists.infradead.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-rdma@vger.kernel.org, linux-block@vger.kernel.org, sbates@raithlin.com, hch@lst.de, axboe@kernel.dk, sagi@grimberg.me, bhelgaas@google.com, jgg@mellanox.com, maxg@mellanox.com, keith.busch@intel.com, dan.j.williams@intel.com, benh@kernel.crashing.org, jglisse@redhat.com, alex.williamson@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on ale.deltatee.com X-Spam-Level: X-Spam-Status: No, score=-6.5 required=5.0 tests=ALL_TRUSTED,BAYES_00, MYRULES_FREE,MYRULES_NO_TEXT,T_RP_MATCHES_RCVD autolearn=no autolearn_force=no version=3.4.1 Subject: [PATCH v3 09/11] nvme-pci: Add support for P2P memory in requests X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org For P2P requests, we must use the pci_p2pmem_[un]map_sg() functions instead of the dma_map_sg functions. With that, we can then indicate PCI_P2P support in the request queue. For this, we create an NVME_F_PCI_P2P flag which tells the core to set QUEUE_FLAG_PCI_P2P in the request queue. Signed-off-by: Logan Gunthorpe Reviewed-by: Sagi Grimberg --- drivers/nvme/host/core.c | 4 ++++ drivers/nvme/host/nvme.h | 1 + drivers/nvme/host/pci.c | 19 +++++++++++++++---- 3 files changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 7aeca5db7916..c7c5de116720 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2949,7 +2949,11 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) ns->queue = blk_mq_init_queue(ctrl->tagset); if (IS_ERR(ns->queue)) goto out_free_ns; + queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue); + if (ctrl->ops->flags & NVME_F_PCI_P2PDMA) + queue_flag_set_unlocked(QUEUE_FLAG_PCI_P2PDMA, ns->queue); + ns->queue->queuedata = ns; ns->ctrl = ctrl; diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index d733b14ede9d..1fb2b6603d49 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -290,6 +290,7 @@ struct nvme_ctrl_ops { unsigned int flags; #define NVME_F_FABRICS (1 << 0) #define NVME_F_METADATA_SUPPORTED (1 << 1) +#define NVME_F_PCI_P2PDMA (1 << 2) int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val); int (*reg_write32)(struct nvme_ctrl *ctrl, u32 off, u32 val); int (*reg_read64)(struct nvme_ctrl *ctrl, u32 off, u64 *val); diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 1fb57fa42dd0..0ebab7ab4d7e 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -796,8 +796,13 @@ static blk_status_t nvme_map_data(struct nvme_dev *dev, struct request *req, goto out; ret = BLK_STS_RESOURCE; - nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, dma_dir, - DMA_ATTR_NO_WARN); + + if (REQ_IS_PCI_P2PDMA(req)) + nr_mapped = pci_p2pdma_map_sg(dev->dev, iod->sg, iod->nents, + dma_dir); + else + nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents, + dma_dir, DMA_ATTR_NO_WARN); if (!nr_mapped) goto out; @@ -842,7 +847,12 @@ static void nvme_unmap_data(struct nvme_dev *dev, struct request *req) DMA_TO_DEVICE : DMA_FROM_DEVICE; if (iod->nents) { - dma_unmap_sg(dev->dev, iod->sg, iod->nents, dma_dir); + if (REQ_IS_PCI_P2PDMA(req)) + pci_p2pdma_unmap_sg(dev->dev, iod->sg, iod->nents, + dma_dir); + else + dma_unmap_sg(dev->dev, iod->sg, iod->nents, dma_dir); + if (blk_integrity_rq(req)) { if (req_op(req) == REQ_OP_READ) nvme_dif_remap(req, nvme_dif_complete); @@ -2426,7 +2436,8 @@ static int nvme_pci_reg_read64(struct nvme_ctrl *ctrl, u32 off, u64 *val) static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = { .name = "pcie", .module = THIS_MODULE, - .flags = NVME_F_METADATA_SUPPORTED, + .flags = NVME_F_METADATA_SUPPORTED | + NVME_F_PCI_P2PDMA, .reg_read32 = nvme_pci_reg_read32, .reg_write32 = nvme_pci_reg_write32, .reg_read64 = nvme_pci_reg_read64, -- 2.11.0