Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753338AbdF2V2Q (ORCPT ); Thu, 29 Jun 2017 17:28:16 -0400 Received: from esa6.dell-outbound.iphmx.com ([68.232.149.229]:64174 "EHLO esa6.dell-outbound.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751626AbdF2V2O (ORCPT ); Thu, 29 Jun 2017 17:28:14 -0400 From: "Hubbe, Allen" X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com v5TLSALq025997 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com v5TLSALq025997 To: Logan Gunthorpe , "linux-ntb@googlegroups.com" , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" CC: "'Jon Mason'" , "'Dave Jiang'" , "'Bjorn Helgaas'" , "'Greg Kroah-Hartman'" , "'Kurt Schwemmer'" , "'Stephen Bates'" , "'Serge Semin'" Subject: RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align Thread-Topic: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align Thread-Index: AQHS8IeVz3JuZyOpkUW5S6r3xXIYA6I78leggACDfwD//8zpcIAASJYA///H4LA= Date: Thu, 29 Jun 2017 21:27:50 +0000 Message-ID: <40F65EF2B5E2254199711F58E3ACB84DB00F8F8A@MX104CL02.corp.emc.com> References: <20170629032648.3073-1-logang@deltatee.com> <20170629032648.3073-7-logang@deltatee.com> <000301d2f103$3310ca60$99325f20$@dell.com> <079047c1-6612-2814-e147-844fabd56b75@deltatee.com> <000401d2f114$3499f2b0$9dcdd810$@dell.com> <63c4b0fa-2f03-6313-2d60-c6dab53bc3a5@deltatee.com> In-Reply-To: <63c4b0fa-2f03-6313-2d60-c6dab53bc3a5@deltatee.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.230.149] Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: public Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id v5TLSOJI029607 Content-Length: 1177 Lines: 28 From: Logan Gunthorpe > On 6/29/2017 2:13 PM, Allen Hubbe wrote: > > Unfortunately, it is to work around hardware errata. That is not so trivial to fix. > > Can you describe more what the work around is doing? Can you share the > code? It seems odd that a workaround is based on the alignment > restrictions of the mws. Sure, while not making any claim that this is ready for upstream. It is not a workaround for alignment restrictions of the mws. It is a restriction to avoid the use of doorbells and scratchpads. Memory windows are used exclusively. Read msi-x local and send that to the peer: https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L603 Transform peer's addr to the memory window region: https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L622 Append a dma immediate value operation after other operations, to write the data at addr: https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L1208 Above describes the workaround. Used in the context of a rdma-over-ntb driver here: https://github.com/ntrdma/ntrdma/blob/master/drivers/infiniband/hw/ntrdma/ntrdma_qp.c#L1585 > > Logan