Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752206AbdGDSRz (ORCPT ); Tue, 4 Jul 2017 14:17:55 -0400 Received: from pandora.armlinux.org.uk ([78.32.30.218]:42490 "EHLO pandora.armlinux.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751960AbdGDSRx (ORCPT ); Tue, 4 Jul 2017 14:17:53 -0400 Date: Tue, 4 Jul 2017 19:17:37 +0100 From: Russell King - ARM Linux To: Bjorn Helgaas Cc: Marc Gonzalez , Mark Rutland , Mason , Marc Zyngier , linux-pci , Thibaud Cornic , Ard Biesheuvel , LKML , Greg Kroah-Hartman , Thomas Gleixner , Linux ARM Subject: Re: [PATCH v9 2/3] PCI: Add tango PCIe host bridge support Message-ID: <20170704181737.GF4902@n2100.armlinux.org.uk> References: <987fac41-80dc-f1d0-ec0b-91ae57b91bfd@sigmadesigns.com> <20170702231811.GJ18324@bhelgaas-glaptop.roam.corp.google.com> <79382219-c730-da78-3e5f-5abf3173d7ac@sigmadesigns.com> <20170703134031.GG13824@bhelgaas-glaptop.roam.corp.google.com> <20170703181128.GD4902@n2100.armlinux.org.uk> <20170704151502.GH13824@bhelgaas-glaptop.roam.corp.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170704151502.GH13824@bhelgaas-glaptop.roam.corp.google.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3089 Lines: 61 On Tue, Jul 04, 2017 at 10:15:02AM -0500, Bjorn Helgaas wrote: > On Mon, Jul 03, 2017 at 07:11:28PM +0100, Russell King - ARM Linux wrote: > > On Mon, Jul 03, 2017 at 08:40:31AM -0500, Bjorn Helgaas wrote: > > > The problem is serializing vs. memory accesses, since they don't use > > > any wrappers. However, they are ioremapped(), so it's at least > > > conceivable that another solution would be to use VM to trap those > > > accesses. I'm not a VM person, so I don't know whether that's > > > feasible in Linux. > > > > Bjorn, > > > > You're forgetting that MMIO (iow, memory returned by ioremap()) must > > be accessed through the appropriate accessors, and must not be > > directly dereferenced in C. (We do have buggy drivers that do that > > but they are buggy, and in many cases are getting attention to fix > > that.) > > Oh, you're right, thank you! I guess you're referring to readb() > and friends. I haven't found an actual prohibition on directly > dereferencing addresses returned from ioremap(), but > Documentation/driver-api/device-io.rst is clear that they're > suitable for passing to readb(), etc. There was a strong suggestion years ago that what is returned from ioremap() is a cookie that must not be dereferenced by drivers, and that there was a suggestion that having ioremap() return the virtual address with an offset (which read*() and friends would undo) would be a good idea. However, even back then, we had some cases where drivers would directly dereference the pointer. We have sparse today which helps point these places out (provided drivers stay away from __force, but unfortunately, I think we've ended up with people who think that silencing sparse warnings with __force is more preferable than leaving them there to point out where things are actually wrong.) So, imho, unfortunately sparse has lost its usefulness in this regard. > I recently told someone else my mistaken idea that ioremap() must > return a valid virtual address. I wish I remembered who it was, so I > could correct that. Documentation/DMA-API-HOWTO.txt also suggests > that ioremap() returns a virtual address -- I think I wrote that, and > maybe that virtual address reference should be tweaked a bit. For most implementations, ioremap() does indeed return a virtual address, but that was never how the API was defined in the first place - it was always referred to as returning a cookie. > Another wrinkle is that the pci_mmap_resource() interface is exposed > via sysfs and allows direct userspace mmap of PCI MMIO resources. In > that case, there is no accessor available. I wonder if we need some > way to disable this mmap when readb() is non-trivial. Hmm, no comment, except that while the PCI MMIO space is available to userspace, and userspace is capable of running that thread on any CPU, PCI MMIO space can't be switched to config space. That's another nail in this coffin... -- RMK's Patch system: http://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.