Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932750AbcKYQqI (ORCPT ); Fri, 25 Nov 2016 11:46:08 -0500 Received: from ale.deltatee.com ([207.54.116.67]:42155 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754935AbcKYQpt (ORCPT ); Fri, 25 Nov 2016 11:45:49 -0500 To: =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Dan Williams References: <75a1f44f-c495-7d1e-7e1c-17e89555edba@amd.com> <45c6e878-bece-7987-aee7-0e940044158c@deltatee.com> <20161123190515.GA12146@obsidianresearch.com> <7bc38037-b6ab-943f-59db-6280e16901ab@amd.com> <20161123193228.GC12146@obsidianresearch.com> <20161123203332.GA15062@obsidianresearch.com> <20161123215510.GA16311@obsidianresearch.com> <20161123232503.GA13965@obsidianresearch.com> <5e1de9ee-34f5-136d-a07e-f949d492864f@deltatee.com> Cc: Serguei Sagalovitch , "Deucher, Alexander" , "linux-nvdimm@lists.01.org" , "linux-rdma@vger.kernel.org" , "linux-pci@vger.kernel.org" , "Kuehling, Felix" , "Bridgman, John" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , "Sander, Ben" , "Suthikulpanit, Suravee" , "Blinzer, Paul" , "Linux-media@vger.kernel.org" , Haggai Eran From: Logan Gunthorpe Message-ID: <209107c7-3098-ca70-7d62-b55021d01faa@deltatee.com> Date: Fri, 25 Nov 2016 09:45:27 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 172.16.1.111 X-SA-Exim-Rcpt-To: haggaie@mellanox.com, Linux-media@vger.kernel.org, Paul.Blinzer@amd.com, Suravee.Suthikulpanit@amd.com, ben.sander@amd.com, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, John.Bridgman@amd.com, Felix.Kuehling@amd.com, linux-pci@vger.kernel.org, linux-rdma@vger.kernel.org, linux-nvdimm@ml01.01.org, Alexander.Deucher@amd.com, serguei.sagalovitch@amd.com, dan.j.williams@intel.com, jgunthorpe@obsidianresearch.com, christian.koenig@amd.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: Enabling peer to peer device transactions for PCIe devices X-SA-Exim-Version: 4.2.1 (built Mon, 26 Dec 2011 16:24:06 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1075 Lines: 26 On 25/11/16 06:06 AM, Christian K?nig wrote: > Well Serguei send me a couple of documents about QPI when we started to > discuss this internally as well and that's exactly one of the cases I > had in mind when writing this. > > If I understood it correctly for such systems P2P is technical possible, > but not necessary a good idea. Usually it is faster to just use a > bouncing buffer when the peers are a bit "father" apart. > > That this problem is solved on newer hardware is good, but doesn't helps > us at all if we at want to support at least systems from the last five > years or so. Well we have been testing with Sandy Bridge, I think the problem was supposed to be fixed in Ivy but we never tested it so I can't say what the performance turned out to be. Ivy is nearly 5 years old. I expect this is something that will be improved more and more with subsequent generations. A white list may end up being rather complicated if it has to cover different CPU generations and system architectures. I feel this is a decision user space could easily make. Logan