Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751883AbdDBVEE (ORCPT ); Sun, 2 Apr 2017 17:04:04 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:40006 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751822AbdDBVEB (ORCPT ); Sun, 2 Apr 2017 17:04:01 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 8CF1760DD5 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=okaya@codeaurora.org Subject: Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device To: Logan Gunthorpe References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <1490911959-5146-2-git-send-email-logang@deltatee.com> <7158f2e8-2016-f398-e77f-0fcbe6cb41dd@deltatee.com> <0280fbb4-ba9e-ac64-6bb3-b72590a54e57@deltatee.com> <0ae27bca-21be-b89c-aba4-6cc9766ebd7b@codeaurora.org> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> Cc: Christoph Hellwig , Sagi Grimberg , "James E.J. Bottomley" , "Martin K. Petersen" , Jens Axboe , Steve Wise , Stephen Bates , Max Gurtovoy , Dan Williams , Keith Busch , Jason Gunthorpe , linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, Alex Williamson , Bjorn Helgaas From: Sinan Kaya Message-ID: Date: Sun, 2 Apr 2017 17:03:53 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2340 Lines: 46 On 4/2/2017 1:21 PM, Logan Gunthorpe wrote: >> This is when the BIOS date helps so that you don't break existing systems. > I'm not that worried about this code breaking existing systems. There > are significant trade-offs with using p2pmem (ie. you are quite likely > sacrificing performance for memory QOS or upstream PCI bandwidth), and > therefore the user _has_ to specifically say to use it. This is why > we've put a flag in the nvme target code that defaults to off. Thus we > are not going to have a situation where people upgrade their kernels and > see broken or slow systems. People _have_ to make the decision to turn > it on and decide based on their use case whether it's appropriate. > OK. I didn't know the feature was not enabled by default. This is even easier now. Push the decision all the way to the user. Let them decide whether they want this feature to work on a root port connected port or under the switch. >> We can't guarentee all switches will work either. See above for instructions >> on when this feature should be enabled. > It's a lot easier to say that all switches will work than it is for root > ports. This is essentially what switches are designed for, so I'd be > surprised to find one that doesn't work. Root ports are the trouble here > seeing it's a lot more likely for them to be designed without > considering that traffic needs to move between ports efficiently. If we > do find extremely broken switches that don't support this then we'd > probably want to create a black list for that. Also, there's > significantly fewer PCI switch products on the market than there are > root port instances, so a black list would be much easier to manage there. > I thought the issue was feature didn't work at all with some root ports or there was some kind of memory corruption issue that you were trying to avoid with the existing systems. If you are just worried about performance, the switch recommendation belongs to your particular product tuning guide or a howto document not into the actual code itself. I think you should get rid of all pci searching business in your code. -- Sinan Kaya Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.