2006-09-30 17:00:20

by Miklos Szeredi

[permalink] [raw]
Subject: get_user_pages() cache issues on ARM

Hi Russell,

The get_user_pages() vs dcache coherency issue still seems to be
unresolved on ARM.

See flush_anon_page() and flush_kernel_dcache_page() in
Documentation/cachetlb.txt and their implementation on PARISC.

Can you please take a look at this?

Thanks,
Miklos

------- Start of forwarded message -------
Date: Sun, 24 Sep 2006 20:20:15 -0700
From: "Dave Hylands" <[email protected]>
To: "Miklos Szeredi" <[email protected]>
Subject: Re: [fuse-devel] ARM cross build issues
Cc: [email protected], [email protected]
In-Reply-To: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi Miklos,

Sending this to the list...

> > 2.6.18 still calls get_user_pages will the NULL vma. I verified that
> > the flush_cache_all patch made things work for me under 2.6.11
> > (big-endian ARM) and 2.6.17 (little-endian ARM).
>
> get_user_pages() should do the right thing on it's own. So passing
> NULL vma is the normal thing to do, and the workaround should not be
> needed.
>
> So if 2.6.18 works without the patch, then it's fixed, otherwise not.

So 2.6.18 does NOT work on the ARM using the fuse.ko that gets built from the
kernel sources.

If I add the flush_cache_all() patch or the DCAHCE_BUG patch, then
everything works fine.

This was tested on my gumstix, which is an ARM XScale PXA255 running
in little-endian mode, using 2.6.18. I was using the hello filesystem
for the tests.

- --
Dave Hylands
Vancouver, BC, Canada
http://www.DaveHylands.com/
------- End of forwarded message -------


2006-09-30 17:05:58

by Russell King

[permalink] [raw]
Subject: Re: get_user_pages() cache issues on ARM

On Sat, Sep 30, 2006 at 06:59:50PM +0200, Miklos Szeredi wrote:
> Hi Russell,
>
> The get_user_pages() vs dcache coherency issue still seems to be
> unresolved on ARM.
>
> See flush_anon_page() and flush_kernel_dcache_page() in
> Documentation/cachetlb.txt and their implementation on PARISC.
>
> Can you please take a look at this?

I'm sorry, I don't think I have sufficient understanding of the Linux VM
to look at these issues anymore.

The questions I have are:

- where do these pages that get_user_pages() finds and calls flush_anon_page()
on come from?
- why is the current ARM flush_dcache_page() (which is also called after
flush_anon_page()) not sufficient?
- if we implement flush_anon_page() does that mean that we end up flushing
multiple times in some circumstances? If so, how do we avoid this?

I'm really serious - I no longer understand the Linux VM sufficiently to
get this stuff right.

--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 Serial core

2006-09-30 17:19:31

by Miklos Szeredi

[permalink] [raw]
Subject: Re: get_user_pages() cache issues on ARM

Added James Bottomley to the CC. He designed this interface, and he
explained to me how it's supposed to work, but I since forgot.

James, do you have some memory of these issues?

Thanks,
Miklos

> > Hi Russell,
> >
> > The get_user_pages() vs dcache coherency issue still seems to be
> > unresolved on ARM.
> >
> > See flush_anon_page() and flush_kernel_dcache_page() in
> > Documentation/cachetlb.txt and their implementation on PARISC.
> >
> > Can you please take a look at this?
>
> I'm sorry, I don't think I have sufficient understanding of the Linux VM
> to look at these issues anymore.
>
> The questions I have are:
>
> - where do these pages that get_user_pages() finds and calls flush_anon_page()
> on come from?
> - why is the current ARM flush_dcache_page() (which is also called after
> flush_anon_page()) not sufficient?
> - if we implement flush_anon_page() does that mean that we end up flushing
> multiple times in some circumstances? If so, how do we avoid this?
>
> I'm really serious - I no longer understand the Linux VM sufficiently to
> get this stuff right.

2006-10-07 13:14:34

by James Bottomley

[permalink] [raw]
Subject: Re: get_user_pages() cache issues on ARM

On Sat, 2006-09-30 at 19:18 +0200, Miklos Szeredi wrote:
> Added James Bottomley to the CC. He designed this interface, and he
> explained to me how it's supposed to work, but I since forgot.
>
> James, do you have some memory of these issues?

I can try, but the problems were very parisc specific.

> > The questions I have are:
> >
> > - where do these pages that get_user_pages() finds and calls flush_anon_page()
> > on come from?

Anonymous pages are non-file backed pages. Fuse has them because it
uses non file backed buffers to operate the user level filesystem
(basically any memory you malloc at user level can end up as an
anonymous page).

> > - why is the current ARM flush_dcache_page() (which is also called after
> > flush_anon_page()) not sufficient?

I think you do the same as we do on parisc: traverse the
page_mapping(page) list and flush. Unfortunately, anonymous pages don't
have entries in the page_mapping(page) list (because they're not file
backed), so our flush_dcache_page() wasn't flushing them, hence the need
for an additional API for fuse.

if you have an archive of linux-arch, the original discussion is under
the subject

Potential problem with the page_mapping macro and flush_dcache_page()

and begins on 19 March 2006

> > - if we implement flush_anon_page() does that mean that we end up flushing
> > multiple times in some circumstances? If so, how do we avoid this?

In theory no, because the way I coded the parisc version, it ignores
non-anon pages, which are flushed by flush_dcache_page(); but it would
depend on how you implement this.

James