Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754573AbbERWID (ORCPT ); Mon, 18 May 2015 18:08:03 -0400 Received: from mail-bl2on0059.outbound.protection.outlook.com ([65.55.169.59]:30534 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753159AbbERWH7 (ORCPT ); Mon, 18 May 2015 18:07:59 -0400 X-Greylist: delayed 15381 seconds by postgrey-1.27 at vger.kernel.org; Mon, 18 May 2015 18:07:59 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=David.Daney@caviumnetworks.com; Message-ID: <555A5F1E.1010509@caviumnetworks.com> Date: Mon, 18 May 2015 14:52:30 -0700 From: David Daney User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7 MIME-Version: 1.0 To: David Miller CC: , , , , , , , , , , , , , Subject: Re: [PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller References: <1431747401-20847-1-git-send-email-aleksey.makarov@auriga.com> <1431747401-20847-3-git-send-email-aleksey.makarov@auriga.com> <20150518.160931.2235555994068443513.davem@davemloft.net> In-Reply-To: <20150518.160931.2235555994068443513.davem@davemloft.net> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [64.2.3.194] X-ClientProxiedBy: BY2PR07CA066.namprd07.prod.outlook.com (10.141.251.41) To BN3PR0701MB1719.namprd07.prod.outlook.com (25.163.39.18) X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1719;UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1736; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(5005006)(3002001);SRVR:BN3PR0701MB1719;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1719; X-Forefront-PRVS: 058043A388 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(377454003)(24454002)(76104003)(164054003)(479174004)(199003)(51704005)(189002)(80316001)(77156002)(64706001)(66066001)(65956001)(4001350100001)(4001540100001)(81156007)(50466002)(19580405001)(110136002)(5001860100001)(189998001)(5001960100002)(87266999)(65816999)(50986999)(47776003)(76176999)(65806001)(5001830100001)(54356999)(40100003)(97736004)(36756003)(2950100001)(69596002)(68736005)(122386002)(19580395003)(62966003)(92566002)(83506001)(46102003)(87976001)(33656002)(101416001)(106356001)(53416004)(42186005)(105586002)(59896002)(23756003)(64126003);DIR:OUT;SFP:1101;SCL:1;SRVR:BN3PR0701MB1719;H:dl.caveonetworks.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2015 21:52:34.3318 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1719 X-OriginatorOrg: caviumnetworks.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2284 Lines: 66 On 05/18/2015 01:09 PM, David Miller wrote: > From: Aleksey Makarov > Date: Fri, 15 May 2015 20:36:39 -0700 > >> +/* Register read/write APIs */ >> +static void nic_reg_write(struct nicpf *nic, u64 offset, u64 val) >> +{ >> + writeq_relaxed(val, nic->reg_base + offset); >> +} >> + >> +static u64 nic_reg_read(struct nicpf *nic, u64 offset) >> +{ >> + return readq_relaxed(nic->reg_base + offset); >> +} > > Are you really sure it's OK to used relaxed ordering for all register > accesses like this? Yes. > > Personally, I think it's asking for trouble. > > Maybe in _extremely_ specific situations in the packet processing > fast path where you can clearly define the ordering needs when > programming the mailbox registers, I'd say it's OK. > > But universally across the entire driver? No way, no way at all. > I think for an ordinary NIC you would be 100% correct. However, ... ... there is something that may not be evident from the patch: The Cavium ThunderX network controller can *only* be found in SoCs containing the ThunderX ARM64 CPU implementation. So, we know quite a bit (i.e. everything) about the memory ordering semantics of the system. A slightly more detailed explanation of why it is safe to use writeq_relaxed/readq_relaxed is that all accesses to the device registers are implicitly strongly ordered with respect to memory accesses, so even though the function names imply otherwise, there is no relaxed ordering. The readq/writeq functions add explicit ordering operation which in this case are redundant, and only add overhead. So this leaves us with the question: What should we do? As I see it there are two options: 1) Keep the writeq_relaxed()/readq_relaxed(), but add a comment about why they are safe. 2) Change the patch so that we are using writeq()/readq() and suffer a decrease in performance. I/we wouldn't object to either of these, so it is really up to you. Please let us know what you think the best way forward is. Thanks, David Daney -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/