Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757261AbcKXRzq (ORCPT ); Thu, 24 Nov 2016 12:55:46 -0500 Received: from ale.deltatee.com ([207.54.116.67]:38740 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754307AbcKXRzo (ORCPT ); Thu, 24 Nov 2016 12:55:44 -0500 To: =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Dan Williams References: <75a1f44f-c495-7d1e-7e1c-17e89555edba@amd.com> <45c6e878-bece-7987-aee7-0e940044158c@deltatee.com> <20161123190515.GA12146@obsidianresearch.com> <7bc38037-b6ab-943f-59db-6280e16901ab@amd.com> <20161123193228.GC12146@obsidianresearch.com> <20161123203332.GA15062@obsidianresearch.com> <20161123215510.GA16311@obsidianresearch.com> <20161123232503.GA13965@obsidianresearch.com> Cc: Serguei Sagalovitch , "Deucher, Alexander" , "linux-nvdimm@lists.01.org" , "linux-rdma@vger.kernel.org" , "linux-pci@vger.kernel.org" , "Kuehling, Felix" , "Bridgman, John" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , "Sander, Ben" , "Suthikulpanit, Suravee" , "Blinzer, Paul" , "Linux-media@vger.kernel.org" , Haggai Eran From: Logan Gunthorpe Message-ID: <5e1de9ee-34f5-136d-a07e-f949d492864f@deltatee.com> Date: Thu, 24 Nov 2016 10:55:39 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 172.16.1.111 X-SA-Exim-Rcpt-To: haggaie@mellanox.com, Linux-media@vger.kernel.org, Paul.Blinzer@amd.com, Suravee.Suthikulpanit@amd.com, ben.sander@amd.com, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, John.Bridgman@amd.com, Felix.Kuehling@amd.com, linux-pci@vger.kernel.org, linux-rdma@vger.kernel.org, linux-nvdimm@ml01.01.org, Alexander.Deucher@amd.com, serguei.sagalovitch@amd.com, dan.j.williams@intel.com, jgunthorpe@obsidianresearch.com, christian.koenig@amd.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: Enabling peer to peer device transactions for PCIe devices X-SA-Exim-Version: 4.2.1 (built Mon, 26 Dec 2011 16:24:06 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 896 Lines: 20 Hey, On 24/11/16 02:45 AM, Christian K?nig wrote: > E.g. it can happen that PCI device A exports it's BAR using ZONE_DEVICE. > Not PCI device B (a SATA device) can directly read/write to it because > it is on the same bus segment, but PCI device C (a network card for > example) can't because it is on a different bus segment and the bridge > can't handle P2P transactions. Yeah, that could be an issue but in our experience we have yet to see it. We've tested with two separate PCI buses on different CPUs connected through QPI links and it works fine. (It is rather slow but I understand Intel has improved the bottleneck in newer CPUs than the ones we tested.) It may just be older hardware that has this issue. I expect that as long as a failed transfer can be handled gracefully by the initiator I don't see a need to predetermine whether a device can see another devices memory. Logan