From: Bill Waddington <william.waddington@beezmo.com>
To: linux-kernel@vger.kernel.org
Cc: Robert Hancock <hancockr@shaw.ca>
Subject: Re: DMA mapping API on 32-bit X86 with CONFIG_HIGHMEM64G
Date: Sun, 17 Feb 2008 07:31:53 -0800
Message-ID: <qakgr39onfg8q408qhpstffuf5lhpt7c20@4ax.com>
References: <fa.UILz1HxdpfM019FfTnX6mSuY9KE@ifi.uio.no>
In-Reply-To: <fa.UILz1HxdpfM019FfTnX6mSuY9KE@ifi.uio.no>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3148
Lines: 69

On Tue, 12 Feb 2008 04:16:58 UTC, in fa.linux.kernel you wrote:

>I was looking at the out-of-tree driver for a PCI high-security module 
>(from a vendor who shall remain nameless) today, as we had a problem 
>reported where the device didn't work properly if the computer had more 
>than 4GB of RAM (this is x86 32-bit, with CONFIG_HIGHMEM64G enabled).
>
>Essentially what it was doing was taking some memory that the userspace 
>app was transferring to/from the device, doing get_user_pages on it, and 
>then using the old-style page_to_phys, etc. functions to DMA on that 
>memory instead of the modern DMA API.
>
>However, I'm not sure this strategy would have worked on this platform 
>even if it had been using the proper DMA API. This device has 32-bit DMA 
>limits and is transferring userspace buffers which with HIGHMEM64G 
>enabled could easily have physical addresses over 4GB. The strategy that 
>Linux Device Drivers, 3rd Edition (chapter 15) suggests is doing 
>get_user_pages, creating an SG list from the returned pages and then 
>using dma_map_sg on that list. However, essentially all dma_map_sg in 
>include/asm-x86/dma-mapping_32.h is:
>
>	for_each_sg(sglist, sg, nents, i) {
>		BUG_ON(!sg_page(sg));
>
>		sg->dma_address = sg_phys(sg);
>	}
>
>which does nothing to ensure that the returned physical address is 
>within the device's DMA mask. On 64-bit this triggers IOMMU mapping but 
>on 32-bit it doesn't seem like this case is handled at all. I believe 
>the block and networking layers have their own ways of ensuring that 
>they don't feed such buffers to their drivers if they can't handle it, 
>but a basic character device driver is kind of left out in the cold here 
>and the DMA API doesn't appear to work as documented in this case. Given 
>that x86-32 kernels don't implement any IOMMU support I'm not sure what 
>it actually could do, other than implementing some kind of software 
>bounce buffering of its own..
>
>Are there any in-tree drivers that use this DMA mapping on 
>get_user_pages strategy that could be affected by this?

No takers?  This got me worried about _my_ out-of-tree driver...

>I think the get_free_pages trick is actually pretty silly in this case, 
>the size of the data being transferred is likely such that it would be 
>just as fast or faster to copy to a kernel buffer and DMA to/from there..

That's what I do currently.  If HIGHMEM64G is defined I switch from
user space DMA to an in-driver copy/DMA buffer.

Is there a more elegant/simpler way to do this?  At one time I thought
there was a kernel bounce-buffer hidden behind the DMA API - at least
on some architectures and some memory configurations.

Just my imagination, or is this problem already taken care of in the
kernel?

Thanks,
Bill
-- 
William D Waddington
william.waddington@beezmo.com
"Even bugs...are unexpected signposts on
the long road of creativity..." - Ken Burtch
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/