Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755313AbbGUPRG (ORCPT ); Tue, 21 Jul 2015 11:17:06 -0400 Received: from mga02.intel.com ([134.134.136.20]:48985 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753327AbbGUPRE (ORCPT ); Tue, 21 Jul 2015 11:17:04 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,516,1432623600"; d="scan'208";a="766731678" From: Ashutosh Dixit To: Toshi Kani Cc: "linux-kernel\@vger.kernel.org" , "Dutt\, Sudeep" , "Rao\, Nikhil" , "Williams\, Dan J" Subject: Re: Regression in v4.2-rc1: vmalloc_to_page with ioremap References: <6DC2528F945B4149AB6566DFB5F22ED39BC5540B@ORSMSX115.amr.corp.intel.com> <1437407695.3214.156.camel@hp.com> <1437407942.3214.159.camel@hp.com> <1437420078.3214.185.camel@hp.com> Date: Tue, 21 Jul 2015 08:17:02 -0700 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3224 Lines: 70 On Mon, Jul 20 2015 at 12:21:18 PM, Toshi Kani wrote: >> > Can you send me outputs of the following files? If the driver >> > fails to load in v4.2-rc1, you can obtain the info in v4.1. >> > >> > /proc/mtrr >> > /proc/iomem >> > /proc/vmallocinfo >> > /sys/kernel/debug/kernel_page_tables (need CONFIG_X86_PTDUMP set) >> >> Since the outputs are large I have sent you the outputs in a separate >> mail outside the mailing list. > > Did you collect the info with your driver loaded? I did not see ioremap > requests from your driver in the vmallocinfo output. Sorry, yes the driver was loaded. The ioremap itself is not done by scif.ko but by mic_host.ko and then the addresses are passed to scif.ko. So I think what you are looking for is this: 0xffffc90040000000-0xffffc90240001000 8589938688 mic_probe+0x281/0x5c0 \ [mic_host] phys=3c7e00000000 ioremap 0xffffc90280000000-0xffffc90480001000 8589938688 mic_probe+0x281/0x5c0 \ [mic_host] phys=3c7c00000000 ioremap >> > Also, does the driver map a regular memory range with ioremap? If not, >> > how does 'struct page' get allocated for the range (since >> > vmalloc_to_page returns a page pointer)? >> >> No the driver does not map regular memory with ioremap, only device >> memory. vmalloc_to_page was returning a valid 'struct page' in this case >> too. It appears it can do this correctly using pte_page as long as all >> four >> page table levels (pgd, pud, pmd, pte) are present and the problem seems >> to >> be happening because in the case of huge pages they are not. For us the >> bar >> size is 8 G so we think that with the new ioremap maps the bars using 1 G >> pages. > > Well, it is probably not a valid `struct page` pointer you got from > vmalloc_to_page... pte_page(pte) simply adds a pfn to vmemmap. So, yes, > you get a pointer, which can be put back to the pfn by subtracting vmemmap. > It may look fine, but this pointer does not point to a struct page unless > it is part of the regular memory (or you somehow allocated struct page for > the range). Can you check to see if the struct page pointer from > vmalloc_to_page actually points a struct page entry? > > /* memmap is virtually contiguous. */ > #define __pfn_to_page(pfn) (vmemmap + (pfn)) > #define __page_to_pfn(page) (unsigned long)((page) - vmemmap) Yes, you are correct, the 'struct page' pointer returned by vmalloc_to_page does not point to a "real" struct page entry. Neither have we allocated a struct page for the range. However, because we pass the returned pointer to streaming DMA mapping API's (dma_map_ops->dma_map_sg or dma_map_ops->dma_map_page) and all those functions do is call page_to_phys, they only care about the physical address, it used to work. Would it be possible to have a different API which can do this or can vmalloc_to_page be updated to handle huge ioremaps without crashing? Or would you have a suggestion for doing this differently? Thanks, Ashutosh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/