Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
To:     Chaitanya Kulkarni <chaitanyak@nvidia.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Cc:     Stephen Bates <sbates@raithlin.com>,
        Christoph Hellwig <hch@lst.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Jason Gunthorpe <jgg@ziepe.ca>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>,
        John Hubbard <jhubbard@nvidia.com>,
        Don Dutile <ddutile@redhat.com>,
        Matthew Wilcox <willy@infradead.org>,
        Daniel Vetter <daniel.vetter@ffwll.ch>,
        Jakowski Andrzej <andrzej.jakowski@intel.com>,
        Minturn Dave B <dave.b.minturn@intel.com>,
        Jason Ekstrand <jason@jlekstrand.net>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Xiong Jianxin <jianxin.xiong@intel.com>,
        Bjorn Helgaas <helgaas@kernel.org>,
        Ira Weiny <ira.weiny@intel.com>,
        Robin Murphy <robin.murphy@arm.com>,
        Martin Oliveira <martin.oliveira@eideticom.com>,
        Chaitanya Kulkarni <ckulkarnilinux@gmail.com>
References: <20210916234100.122368-1-logang@deltatee.com>
 <20210916234100.122368-10-logang@deltatee.com>
 <c04215bd-442a-6520-4fc7-5035c1310662@nvidia.com>
From:   Logan Gunthorpe <logang@deltatee.com>
Message-ID: <acaa91df-f932-7449-3623-4209c41634d9@deltatee.com>
Date:   Thu, 30 Sep 2021 10:51:22 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <c04215bd-442a-6520-4fc7-5035c1310662@nvidia.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-CA
Content-Transfer-Encoding: 7bit
Subject: Re: [PATCH v3 09/20] nvme-pci: check DMA ops when indicating support
 for PCI P2PDMA
Precedence: bulk


On 2021-09-29 11:06 p.m., Chaitanya Kulkarni wrote:
> Logan,
> 
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index 7efb31b87f37..916750a54f60 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -3771,7 +3771,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
>>                  blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES, ns->queue);
>>
>>          blk_queue_flag_set(QUEUE_FLAG_NONROT, ns->queue);
>> -       if (ctrl->ops->flags & NVME_F_PCI_P2PDMA)
>> +       if (ctrl->ops->supports_pci_p2pdma &&
>> +           ctrl->ops->supports_pci_p2pdma(ctrl))
>>                  blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, ns->queue);
>>
>>          ns->ctrl = ctrl;
>> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
>> index 9871c0c9374c..fb9bfc52a6d7 100644
>> --- a/drivers/nvme/host/nvme.h
>> +++ b/drivers/nvme/host/nvme.h
>> @@ -477,7 +477,6 @@ struct nvme_ctrl_ops {
>>          unsigned int flags;
>>   #define NVME_F_FABRICS                 (1 << 0)
>>   #define NVME_F_METADATA_SUPPORTED      (1 << 1)
>> -#define NVME_F_PCI_P2PDMA              (1 << 2)
>>          int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val);
>>          int (*reg_write32)(struct nvme_ctrl *ctrl, u32 off, u32 val);
>>          int (*reg_read64)(struct nvme_ctrl *ctrl, u32 off, u64 *val);
>> @@ -485,6 +484,7 @@ struct nvme_ctrl_ops {
>>          void (*submit_async_event)(struct nvme_ctrl *ctrl);
>>          void (*delete_ctrl)(struct nvme_ctrl *ctrl);
>>          int (*get_address)(struct nvme_ctrl *ctrl, char *buf, int size);
>> +       bool (*supports_pci_p2pdma)(struct nvme_ctrl *ctrl);
>>   };
>>
> 
> Is this new ops only needed for the PCIe transport ? or do you have 
> following patches to use this op for the other transports ?

No, I don't think this will make sense for transports that are not based
on PCI devices.

> If it is only needed for the PCIe then we need to find a way to
> not add this somehow...

I don't see how we can do that. The core code needs to know whether the
transport supports this and must have an operation to query it.

Logan