Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752097AbdHDU3C (ORCPT ); Fri, 4 Aug 2017 16:29:02 -0400 Received: from mga02.intel.com ([134.134.136.20]:21696 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751260AbdHDU3A (ORCPT ); Fri, 4 Aug 2017 16:29:00 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,322,1498546800"; d="scan'208";a="886589951" Date: Fri, 4 Aug 2017 13:21:41 -0700 From: "Raj, Ashok" To: Casey Leedom Cc: Ding Tianhong , Alexander Duyck , Alex Williamson , Sinan Kaya , "bhelgaas@google.com" , "helgaas@kernel.org" , Michael Werner , Ganesh GR , "asit.k.mallick@intel.com" , "patrick.j.cramer@intel.com" , "Suravee.Suthikulpanit@amd.com" , "Bob.Shaw@amd.com" , "l.stach@pengutronix.de" , "amira@mellanox.com" , "gabriele.paoloni@huawei.com" , "David.Laight@aculab.com" , "jeffrey.t.kirsher@intel.com" , "catalin.marinas@arm.com" , "will.deacon@arm.com" , "mark.rutland@arm.com" , "robin.murphy@arm.com" , "davem@davemloft.net" , "linux-arm-kernel@lists.infradead.org" , "netdev@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linuxarm@huawei.com" , ashok.raj@intel.com Subject: Re: [PATCH v7 2/3] PCI: Enable PCIe Relaxed Ordering if supported Message-ID: <20170804202141.GA39054@otc-nc-03> References: <67be791f-e0cf-8284-9229-17174dc741ef@codeaurora.org> <5f9b8bfb-41a8-a17c-6fea-581aec1d5573@huawei.com> <20170724090516.2e0f0d2a@w520.home> <75213fca-4522-2297-3cb8-338e643d3552@huawei.com> <14218972-553d-eb60-0207-460ac7f4b064@huawei.com> <20170803083153.GB4883@otc-nc-03> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1739 Lines: 36 On Fri, Aug 04, 2017 at 08:20:37PM +0000, Casey Leedom wrote: > | From: Raj, Ashok > | Sent: Thursday, August 3, 2017 1:31 AM > | > | I don't understand this completely.. So your driver would know not to send > | RO TLP's to root complex. But you want to send RO to the NVMe device? This > | is the peer-2-peer case correct? > > Yes, this is the "heavy hammer" issue which you alluded to later. There are > applications where a device will want to send TLPs to a Root Complex without > Relaxed Ordering set, but will want to use it when sending TLPs to a Peer > device (say, an NVMe storage device). The current approach doesn't make > that easy ... and in fact, I still don't kow how to code a solution for this > with the proposed APIs. This means that we may be trading off one > performance problem for another and that Relaxed Ordering may be doomed for > use under Linux for the foreseeable future. > > As I've noted a number of times, it would be great if the Intel Hardware > Engineers who attempted to implement the Relaxed Ordering semantics in the > current generation of Root Complexes had left the ability to turn off the > logic which is obviously not working. If there was a way to disable the > logic via an undocumented register, then we could have the Linux PCI Quirk > do that. Since Relaxed Ordering is just a hint, it's completely legitimate > to completely ignore it. Suppose you are looking for the existence of a chicken bit to instruct the port to ignore RO traffic. So all we would do is turn the chicken bit on but would permit p2p traffic to be allowed since we won't turn off the capability as currently proposed. Let me look into that keep you posted. Cheers, Ashok > > Casey