Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966148AbbDVRHl (ORCPT ); Wed, 22 Apr 2015 13:07:41 -0400 Received: from cantor2.suse.de ([195.135.220.15]:56403 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934167AbbDVRHf (ORCPT ); Wed, 22 Apr 2015 13:07:35 -0400 Date: Wed, 22 Apr 2015 19:07:31 +0200 From: "Luis R. Rodriguez" To: Andy Lutomirski Cc: Jason Gunthorpe , Mike Marciniszyn , Mike Marciniszyn , linux-rdma@vger.kernel.org, Andy Walls , Toshi Kani , "H. Peter Anvin" , Ingo Molnar , "linux-kernel@vger.kernel.org" , Hal Rosenstock , Sean Hefty , Suresh Siddha , Rickard Strandqvist , Roland Dreier , Juergen Gross , Mauro Carvalho Chehab , Borislav Petkov , Mel Gorman , Vlastimil Babka , Davidlohr Bueso , Dave Hansen , Jean-Christophe Plagniol-Villard , Thomas Gleixner , Ville Syrj?l? , Linux Fbdev development list , linux-media@vger.kernel.org, X86 ML , "Luis R. Rodriguez" Subject: Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR Message-ID: <20150422170731.GH5622@wotan.suse.de> References: <20150421224601.GY5622@wotan.suse.de> <20150421225732.GA17356@obsidianresearch.com> <20150421233907.GA5622@wotan.suse.de> <20150422053939.GA29609@obsidianresearch.com> <20150422152328.GB5622@wotan.suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3910 Lines: 88 On Wed, Apr 22, 2015 at 09:53:03AM -0700, Andy Lutomirski wrote: > On Wed, Apr 22, 2015 at 8:23 AM, Luis R. Rodriguez wrote: > > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote: > >> On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote: > >> > > Mike, do you think the time is right to just remove the iPath driver? > >> > > >> > With PAT now being default the driver effectively won't work > >> > with write-combining on modern kernels. Even if systems are old > >> > they likely had PAT support, when upgrading kernels PAT will work > >> > but write-combing won't on ipath. > >> > >> Sorry, do you mean the driver already doesn't get WC? Or do you mean > >> after some more pending patches are applied? > > > > No, you have to consider the system used and the effects of calls used > > on the driver in light of this table: > > > > ---------------------------------------------------------------------- > > MTRR Non-PAT PAT Linux ioremap value Effective memory type > > ---------------------------------------------------------------------- > > Non-PAT | PAT > > PAT > > |PCD > > ||PWT > > ||| > > WC 000 WB _PAGE_CACHE_MODE_WB WC | WC > > WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC > > WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC > > WC 011 UC _PAGE_CACHE_MODE_UC UC | UC > > ---------------------------------------------------------------------- > > > > (*) denotes implementation defined and is discouraged > > > > ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today, > > in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC > > the default. When that flip occurs it will mean ipath cannot get > > write-combining on both non-PAT and PAT systems. Now that is for > > the future, lets review the current situation for ipath. > > > > For PAT capable systems if mtrr_add() is used today on a Linux system on a > > region mapped with ioremap_nocache() that will mean you effectively nullify the > > mtrr_add() effect as the combinatorial effect above yields an effective memory > > type of UC. > > Are you sure? Well lets double check. > I thought that ioremap_nocache currently is UC-, It is. > so mtrr_add + ioremap_nocache gets WC even on PAT systems. https://www-ssl.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf As per Intel SDM "11.5.2.2 Selecting Memory Types for Pentium III and More Recent Processor Families" the ffect of a WC MTRR for a region with a PAT entry value of UC will be UC. The effect of a WC MTRR on a region with a PAT entry UC- will be WC. The effect of a WC MTRR on a regoin with PAT entry WC is WC. And indeed as per table 11-7 mtrr WC on PAT UC- yields WC. So ineed the above table needs adjustment for this. So for PAT systems write-combing would be effective with mtrr_add(), but once strong UC (_PAGE_CACHE_MODE_UC) is used by default for ioremap_nocache() what I mentioned will be true. Furhtermore if we switch the drivers to use arch_phys_wc_add() then for sure write-combining will also not be effective. Jason, Andy, is the change still a reasonable compromise? We'd just be asking users to boot with noat for users for ipath, ivtv until the drivers gets proper PAT support with a split. There are two motivations for this: * help move to strong UC as default * bury MTRR > Going forward, when mtrr_add is gone, this will change, of course. Indeed. Luis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/