Received: by 2002:ac0:aed5:0:0:0:0:0 with SMTP id t21csp4555278imb; Wed, 6 Mar 2019 16:48:42 -0800 (PST) X-Google-Smtp-Source: APXvYqxweekekiC7KNl0+F0IM4eBtPxfIO8GzQ1Z8VlADAvmuEQEuHCOgZACT+P+nTDH+/qIyTXf X-Received: by 2002:a62:1212:: with SMTP id a18mr9979518pfj.177.1551919722566; Wed, 06 Mar 2019 16:48:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551919722; cv=none; d=google.com; s=arc-20160816; b=rnIThIBAo95glACpcKZSRjeU8X5Gt+OPW5BhvzNlmSaqwGOczrNUr2FlDx7Ep2JfKZ wJhfgTRSKTaWyHE/09ttwm9LSxXckjqjTQedPrwyGe2dAkWxKuks8fxbkkXQrmsjWw92 aalKH1HVl6ASxI02UBDNBN5k6EY4jk47BY1wvXiOzIv6gVfKpJjDsY1PlAMhqL4EfHXY 3uiJ0LfMuFLhzxXGhTC4T1o5C8GiJlTqLXVfPYII7gdZ+GfbH12obi8EJgjKsfUwNDml nHcBfnnjkYwSroPByJYiHVxR3IqQIRm9Yn+m/QJ4UElQqVUF4qpg4081gIFTNaISA5yS a+rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from; bh=hyuE3vbXjULD8MLo/4gSHK6B5JQnVtsdUX7FQqc2tyY=; b=VQDqpe8aRnXRYZVv1M5n6CR6oJp7/idMNSWzxUyQ2MbE9uxF/nf4+LnxzeIGvvwuAp 8J5NMfmXgn3B48vpQuMgQD220E1ISjeJGx/O645hNst5ekefzREXmllGcCBk9aBTGrf2 9NcYanmRU1S79mBBO05KPzOwgQj2YwByx6sWKX26LkW3OEyYs+Cte4hERM0LB8HKBaQP vgzQU96J0XWiux+4j+Kl0RDNogrUBI+2B76ZWJsqDBW5bBDcxWrYmZAphdO3BZohxtI6 OAipZpKAGN4pSt25SSeA0vKSL3RYR1Tm+EBNUJ/nL70CkXma9Y3yPg43V3iKw/Bm1gHN sS8A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h78si2927500pfj.70.2019.03.06.16.48.26; Wed, 06 Mar 2019 16:48:42 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726299AbfCGAsB (ORCPT + 99 others); Wed, 6 Mar 2019 19:48:01 -0500 Received: from ozlabs.org ([203.11.71.1]:33331 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726128AbfCGAsB (ORCPT ); Wed, 6 Mar 2019 19:48:01 -0500 Received: from authenticated.ozlabs.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.ozlabs.org (Postfix) with ESMTPSA id 44FBn60GFVz9sBp; Thu, 7 Mar 2019 11:47:53 +1100 (AEDT) From: Michael Ellerman To: Linus Torvalds Cc: Nicholas Piggin , linux-arch , Will Deacon , Andrea Parri , Arnd Bergmann , Benjamin Herrenschmidt , Rich Felker , David Howells , Daniel Lustig , Linux List Kernel Mailing , "Maciej W. Rozycki" , Ingo Molnar , Palmer Dabbelt , Paul Burton , "Paul E. McKenney" , Peter Zijlstra , Alan Stern , Tony Luck , Yoshinori Sato Subject: Re: [PATCH 01/20] asm-generic/mmiowb: Add generic implementation of mmiowb() tracking In-Reply-To: References: <20190301140348.25175-1-will.deacon@arm.com> <20190301140348.25175-2-will.deacon@arm.com> <1551575210.6lwpiqtg5k.astroid@bobo.none> <87ef7n7x9v.fsf@concordia.ellerman.id.au> Date: Thu, 07 Mar 2019 11:47:53 +1100 Message-ID: <87k1hbv7ba.fsf@concordia.ellerman.id.au> MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Linus Torvalds writes: > On Mon, Mar 4, 2019 at 2:24 AM Michael Ellerman wrote: >> >> Without wading into the rest of the discussion, this does raise an >> interesting point, ie. what about eg. rwlock's? >> >> They're basically equivalent to spinlocks, and so could reasonably be >> expected to have the same behaviour. >> >> But we don't check the io_sync flag in arch_read/write_unlock() etc. and >> both of those use lwsync. > > I think technically rwlocks should do the same thing, at least when > they are used for exclusion. OK. > Because of the exclusion argument, we can presubably limit it to just > write_unlock(), although at least in theory I guess you could have > some "one reader does IO, then a writer comes in" situation.. It's a bit hard to grep for, but I did find one case: static void netxen_nic_io_write_128M(struct netxen_adapter *adapter, void __iomem *addr, u32 data) { read_lock(&adapter->ahw.crb_lock); writel(data, addr); read_unlock(&adapter->ahw.crb_lock); } It looks like that driver is using the rwlock to exclude cases that can just do a readl()/writel() (readers) vs another case that has to reconfigure a window or something, before doing readl()/writel() and then configuring the window back. So that seems like a valid use for a rwlock. Whether we want to penalise all read_unlock() usages with a mmiowb() check just to support that one driver is another question. > Perhaps more importantly, what about sleeping locks? When they > actually *block*, they get the barrier thanks to the scheduler, but > you can have a nice non-contended sequence that never does that. Yeah. The mutex unlock fast path is just: if (atomic_long_cmpxchg_release(&lock->owner, curr, 0UL) == curr) return true; And because it's the "release" variant we just use lwsync, which doesn't order MMIO. If it was just atomic_long_cmpxchg() that would work because we use sync for those. __up_write() uses atomic_long_sub_return_release(), so same story. > I guess the fact that these cases have never even shown up as an issue > means that we could just continue to ignore it. > > We could even give that approach some fancy name, and claim it as a > revolutionary new programming paradigm ("ostrich programming" to go > with "agile" and "pair programming"). Maybe. On power we have the double whammy of weaker ordering than other arches and infinitesimal market share, which makes me worry that there are bugs lurking that we just haven't found, it's happened before. cheers