Received: by 2002:ac0:aed5:0:0:0:0:0 with SMTP id t21csp484666imb; Fri, 1 Mar 2019 06:06:19 -0800 (PST) X-Google-Smtp-Source: APXvYqyMJUc24FIxJ76YWMFxayndoo9+XbQwjOEHLxAgYC4VUotjXuInvjklTE1tpKIIE5Oy8tvU X-Received: by 2002:a63:69c2:: with SMTP id e185mr5016849pgc.4.1551449179107; Fri, 01 Mar 2019 06:06:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551449179; cv=none; d=google.com; s=arc-20160816; b=eTXNj0HBHCPa1IJKGXTjJTXU+K6mWmJJIrnBe3Lmg/+cdp0UEfxzadtGPxcxiB3g+t 6EM0dSzBbYP7p0RhccFdgk2Feqt2bxV4GwEpIoxKxU+v5NcErsFcFqCJgrAfdipmRddF CaRD/Zlwwwq8rK9OoJ8y2z4UAhTQkTFeh2vIaQtIMXRystBBF4jEp+m6/Tyv7RIvt9my p4PWGr/F5Ljqxf1SFbi4l2AS6x2fNFVoQCR6F0A6lb/BShSHWwS28B7aLGl65O6r8hxv lGAW0rZXTGSF1R8D42RmoLRZ1aio3yOg/0TbGS2OADfJBfYsio16WvaZTcGdeyVTVujE yHPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=eqsCS+gKmyXcUgnKmIYP1Lu3hdCDXdkCXrsbM66Nj4A=; b=WxnfjZ8WHt/Rnn0qQE+Vqg/qPRHHqwvOY0GCHcujJWq0bWMHnNpu3k3XQFvbTBMHgv lj4WHucetFPVPnsmYuat0SXg7GGy/DDc/nH9DoVDA9dtJb/xj4E/ReXOi8FIxiq+BjZw dYJiwNqmyeWQq2KwKAGYF1f7w2Nte36aKP8Aj0mLZdzpmTAGB+2DWyIKbZw45Yd2Qlu3 7NbP7wKjqOLK4wPKxPeccq0EjNboxKAkln3ImkhCNKitxaPxQhgHdnUgk9XLpk4Qwmr9 mrVsQOq+WrqFEat/Emqj9WM/40tqsfZBHdxmI8BD51CNzl3JZ+7uwByAhQtSoqG/qUIQ QN3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f21si18218319pgv.28.2019.03.01.06.06.03; Fri, 01 Mar 2019 06:06:19 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388594AbfCAOEw (ORCPT + 99 others); Fri, 1 Mar 2019 09:04:52 -0500 Received: from foss.arm.com ([217.140.101.70]:36026 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388581AbfCAOEv (ORCPT ); Fri, 1 Mar 2019 09:04:51 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 02CAC1C00; Fri, 1 Mar 2019 06:04:51 -0800 (PST) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BBE963F720; Fri, 1 Mar 2019 06:04:47 -0800 (PST) From: Will Deacon To: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , "Paul E. McKenney" , Benjamin Herrenschmidt , Michael Ellerman , Arnd Bergmann , Peter Zijlstra , Andrea Parri , Palmer Dabbelt , Daniel Lustig , David Howells , Alan Stern , Linus Torvalds , "Maciej W. Rozycki" , Paul Burton , Ingo Molnar , Yoshinori Sato , Rich Felker , Tony Luck Subject: [PATCH 14/20] Documentation: Kill all references to mmiowb() Date: Fri, 1 Mar 2019 14:03:42 +0000 Message-Id: <20190301140348.25175-15-will.deacon@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190301140348.25175-1-will.deacon@arm.com> References: <20190301140348.25175-1-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The guarantees provided by mmiowb() are now provided implicitly by spin_unlock(), so we can remove all references to this most confusing of barriers from our Documentation. Good riddance. Signed-off-by: Will Deacon --- Documentation/driver-api/device-io.rst | 45 -------------- Documentation/driver-api/pci/p2pdma.rst | 4 -- Documentation/memory-barriers.txt | 103 ++------------------------------ 3 files changed, 4 insertions(+), 148 deletions(-) diff --git a/Documentation/driver-api/device-io.rst b/Documentation/driver-api/device-io.rst index b00b23903078..0e389378f71d 100644 --- a/Documentation/driver-api/device-io.rst +++ b/Documentation/driver-api/device-io.rst @@ -103,51 +103,6 @@ continuing execution:: ha->flags.ints_enabled = 0; } -In addition to write posting, on some large multiprocessing systems -(e.g. SGI Challenge, Origin and Altix machines) posted writes won't be -strongly ordered coming from different CPUs. Thus it's important to -properly protect parts of your driver that do memory-mapped writes with -locks and use the :c:func:`mmiowb()` to make sure they arrive in the -order intended. Issuing a regular readX() will also ensure write ordering, -but should only be used when the -driver has to be sure that the write has actually arrived at the device -(not that it's simply ordered with respect to other writes), since a -full readX() is a relatively expensive operation. - -Generally, one should use :c:func:`mmiowb()` prior to releasing a spinlock -that protects regions using :c:func:`writeb()` or similar functions that -aren't surrounded by readb() calls, which will ensure ordering -and flushing. The following pseudocode illustrates what might occur if -write ordering isn't guaranteed via :c:func:`mmiowb()` or one of the -readX() functions:: - - CPU A: spin_lock_irqsave(&dev_lock, flags) - CPU A: ... - CPU A: writel(newval, ring_ptr); - CPU A: spin_unlock_irqrestore(&dev_lock, flags) - ... - CPU B: spin_lock_irqsave(&dev_lock, flags) - CPU B: writel(newval2, ring_ptr); - CPU B: ... - CPU B: spin_unlock_irqrestore(&dev_lock, flags) - -In the case above, newval2 could be written to ring_ptr before newval. -Fixing it is easy though:: - - CPU A: spin_lock_irqsave(&dev_lock, flags) - CPU A: ... - CPU A: writel(newval, ring_ptr); - CPU A: mmiowb(); /* ensure no other writes beat us to the device */ - CPU A: spin_unlock_irqrestore(&dev_lock, flags) - ... - CPU B: spin_lock_irqsave(&dev_lock, flags) - CPU B: writel(newval2, ring_ptr); - CPU B: ... - CPU B: mmiowb(); - CPU B: spin_unlock_irqrestore(&dev_lock, flags) - -See tg3.c for a real world example of how to use :c:func:`mmiowb()` - PCI ordering rules also guarantee that PIO read responses arrive after any outstanding DMA writes from that bus, since for some devices the result of a readb() call may signal to the driver that a DMA transaction is diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver-api/pci/p2pdma.rst index 6d85b5a2598d..44deb52beeb4 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -132,10 +132,6 @@ precludes passing these pages to userspace. P2P memory is also technically IO memory but should never have any side effects behind it. Thus, the order of loads and stores should not be important and ioreadX(), iowriteX() and friends should not be necessary. -However, as the memory is not cache coherent, if access ever needs to -be protected by a spinlock then :c:func:`mmiowb()` must be used before -unlocking the lock. (See ACQUIRES VS I/O ACCESSES in -Documentation/memory-barriers.txt) P2P DMA Support Library diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 158947ae78c2..6d6eff413462 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1937,21 +1937,6 @@ There are some more advanced barrier functions: information on consistent memory. -MMIO WRITE BARRIER ------------------- - -The Linux kernel also has a special barrier for use with memory-mapped I/O -writes: - - mmiowb(); - -This is a variation on the mandatory write barrier that causes writes to weakly -ordered I/O regions to be partially ordered. Its effects may go beyond the -CPU->Hardware interface and actually affect the hardware at some level. - -See the subsection "Acquires vs I/O accesses" for more information. - - =============================== IMPLICIT KERNEL MEMORY BARRIERS =============================== @@ -2317,75 +2302,6 @@ But it won't see any of: *E, *F or *G following RELEASE Q - -ACQUIRES VS I/O ACCESSES ------------------------- - -Under certain circumstances (especially involving NUMA), I/O accesses within -two spinlocked sections on two different CPUs may be seen as interleaved by the -PCI bridge, because the PCI bridge does not necessarily participate in the -cache-coherence protocol, and is therefore incapable of issuing the required -read memory barriers. - -For example: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - writel(1, DATA); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - writel(5, DATA); - spin_unlock(Q); - -may be seen by the PCI bridge as follows: - - STORE *ADDR = 0, STORE *ADDR = 4, STORE *DATA = 1, STORE *DATA = 5 - -which would probably cause the hardware to malfunction. - - -What is necessary here is to intervene with an mmiowb() before dropping the -spinlock, for example: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - writel(1, DATA); - mmiowb(); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - writel(5, DATA); - mmiowb(); - spin_unlock(Q); - -this will ensure that the two stores issued on CPU 1 appear at the PCI bridge -before either of the stores issued on CPU 2. - - -Furthermore, following a store by a load from the same device obviates the need -for the mmiowb(), because the load forces the store to complete before the load -is performed: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - a = readl(DATA); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - b = readl(DATA); - spin_unlock(Q); - - -See Documentation/driver-api/device-io.rst for more information. - - ================================= WHERE ARE MEMORY BARRIERS NEEDED? ================================= @@ -2532,16 +2448,9 @@ the device to malfunction. Inside of the Linux kernel, I/O should be done through the appropriate accessor routines - such as inb() or writel() - which know how to make such accesses appropriately sequential. While this, for the most part, renders the explicit -use of memory barriers unnecessary, there are a couple of situations where they -might be needed: - - (1) On some systems, I/O stores are not strongly ordered across all CPUs, and - so for _all_ general drivers locks should be used and mmiowb() must be - issued prior to unlocking the critical section. - - (2) If the accessor functions are used to refer to an I/O memory window with - relaxed memory access properties, then _mandatory_ memory barriers are - required to enforce ordering. +use of memory barriers unnecessary, if the accessor functions are used to refer +to an I/O memory window with relaxed memory access properties, then _mandatory_ +memory barriers are required to enforce ordering. See Documentation/driver-api/device-io.rst for more information. @@ -2586,8 +2495,7 @@ explicit barriers are used. Normally this won't be a problem because the I/O accesses done inside such sections will include synchronous load operations on strictly ordered I/O -registers that form implicit I/O barriers. If this isn't sufficient then an -mmiowb() may need to be used explicitly. +registers that form implicit I/O barriers. A similar situation may occur between an interrupt routine and two routines @@ -2687,9 +2595,6 @@ guarantees: All of these accessors assume that the underlying peripheral is little-endian, and will therefore perform byte-swapping operations on big-endian architectures. -Composing I/O ordering barriers with SMP ordering barriers and LOCK/UNLOCK -operations is a dangerous sport which may require the use of mmiowb(). See the -subsection "Acquires vs I/O accesses" for more information. ======================================== ASSUMED MINIMUM EXECUTION ORDERING MODEL -- 2.11.0