Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752559AbbGTTW0 (ORCPT ); Mon, 20 Jul 2015 15:22:26 -0400 Received: from g2t2352.austin.hp.com ([15.217.128.51]:34546 "EHLO g2t2352.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751014AbbGTTWZ (ORCPT ); Mon, 20 Jul 2015 15:22:25 -0400 Message-ID: <1437420078.3214.185.camel@hp.com> Subject: Re: Regression in v4.2-rc1: vmalloc_to_page with ioremap From: Toshi Kani To: Ashutosh Dixit Cc: "linux-kernel@vger.kernel.org" , "Dutt, Sudeep" , "Rao, Nikhil" , "Williams, Dan J" Date: Mon, 20 Jul 2015 13:21:18 -0600 In-Reply-To: References: <6DC2528F945B4149AB6566DFB5F22ED39BC5540B@ORSMSX115.amr.corp.intel.com> <1437407695.3214.156.camel@hp.com> <1437407942.3214.159.camel@hp.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.3 (3.16.3-2.fc22) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3382 Lines: 81 On Mon, 2015-07-20 at 11:33 -0700, Ashutosh Dixit wrote: > On Mon, Jul 20 2015 at 08:59:02 AM, Toshi Kani wrote: > > > Can you also try the 'nohugeiomap' kernel option to the 4.2-rc1 kernel > > to see if the problem goes away? > > Yes the problem goes away with 'nohugeiomap'. > > > Yes, vmalloc_to_page() assumes 4KB mappings. But ioremap with huge > > pages was enabled in v4.1, while v4.2-rc1 has update for the check with > > MTRRs. > > Yes we had bisected the failure down to the following commit, so the > problem goes away if this commit is reverted. So we think that without > this > commit ioremap actually maps device memory in 4K pages and that is why we > don't see the issue. > > commit b73522e0c1be58d3c69b124985b8ccf94e3677f7 > Author: Toshi Kani > Date: Tue May 26 10:28:10 2015 +0200 > > x86/mm/mtrr: Enhance MTRR checks in kernel mapping helpers Thanks for checking into this! That makes sense. Your system has the MTRR default type set to UC, which covers device memory ranges. In v4.1, large page mappings are not allowed for the ranges covered by MTRRs with non-WB types. In v4.2, large page mappings are allowed as long as the request type does not conflict with MTRRs. So, v4.2 enabled large page mappings to the driver. > > Can you send me outputs of the following files? If the driver > > fails to load in v4.2-rc1, you can obtain the info in v4.1. > > > > /proc/mtrr > > /proc/iomem > > /proc/vmallocinfo > > /sys/kernel/debug/kernel_page_tables (need CONFIG_X86_PTDUMP set) > > Since the outputs are large I have sent you the outputs in a separate > mail > outside the mailing list. Did you collect the info with your driver loaded? I did not see ioremap requests from your driver in the vmallocinfo output. > > Also, does the driver map a regular memory range with ioremap? If not, > > how does 'struct page' get allocated for the range (since > > vmalloc_to_page returns a page pointer)? > > No the driver does not map regular memory with ioremap, only device > memory. vmalloc_to_page was returning a valid 'struct page' in this case > too. It appears it can do this correctly using pte_page as long as all > four > page table levels (pgd, pud, pmd, pte) are present and the problem seems > to > be happening because in the case of huge pages they are not. For us the > bar > size is 8 G so we think that with the new ioremap maps the bars using 1 G > pages. Well, it is probably not a valid `struct page` pointer you got from vmalloc_to_page... pte_page(pte) simply adds a pfn to vmemmap. So, yes, you get a pointer, which can be put back to the pfn by subtracting vmemmap. It may look fine, but this pointer does not point to a struct page unless it is part of the regular memory (or you somehow allocated struct page for the range). Can you check to see if the struct page pointer from vmalloc_to_page actually points a struct page entry? /* memmap is virtually contiguous. */ #define __pfn_to_page(pfn) (vmemmap + (pfn)) #define __page_to_pfn(page) (unsigned long)((page) - vmemmap) Thanks, -Toshi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/