2007-11-21 20:56:34

by Timo Sirainen

[permalink] [raw]
Subject: Re: [NFS] Cache flushing

On Wed, 2007-11-21 at 15:39 -0500, Trond Myklebust wrote:
> On Wed, 2007-11-21 at 22:36 +0200, Timo Sirainen wrote:
> > On 21.11.2007, at 15.56, Trond Myklebust wrote:
> >
> > > On Wed, 2007-11-21 at 15:14 +0200, Timo Sirainen wrote:
> > >
> > >> Now that I think of it, I guess the reason is that my 2.6.22 setup
> > >> has
> > >> Linux+ext3 as NFS server and 2.6.17-rc2 setup has NetApp as
> > >> server. So
> > >> even though open()+close() flushes the directory's attribute
> > >> cache, it
> > >> doesn't flush the file name -> NFS handle cache unless mtime changes?
> > >
> > > Right. You're hitting the principal limitation of ext3 as an NFS
> > > backend
> > > filesystem.
> > > As I said in an earlier mail, the resolution on the time is 1
> > > second, so
> > > client 2 can basically not see any changes that happen within < 1
> > > second
> > > on client 1. The reason is that the mtime stays the same.
> >
> > With files it's possible to work around this limitation with fcntl
> > locking, but directories can't be locked. So back to my original
> > question: Would it be possible to have chmod(dir, (uid_t)-1,
> > (gid_t)-1) flush its cache? I don't see what harm there could be in
> > this, since normally no-one would do it anyway and the change is simple


2007-11-21 21:15:42

by Trond Myklebust

[permalink] [raw]
Subject: Re: [NFS] Cache flushing


On Wed, 2007-11-21 at 22:56 +0200, Timo Sirainen wrote:
> On Wed, 2007-11-21 at 15:39 -0500, Trond Myklebust wrote:
> > I thought you said this was for flushing the attribute cache?
>
> That was it originally, but this one's more important. :) Also I
> originally thought that directory's attribute cache flushing also
> flushed its file handle cache. I'm not actually sure what file handle
> cache should even really be called. It looks like Linux handles it with
> page cache the same way as for files?
>
> So I guess I'm asking for chown(-1,-1) call to invalidate the file's
> attribute and page cache. Although it's interesting why chown(0,-1)
> flushes a directory's file handle cache (==page cache?), but the same
> call for a file doesn't flush its page cache. Maybe I'm still
> misunderstanding something how all of this works.

No. I'm not at all comfortable with the idea of overloading chown() to
invalidate data caches.

It is one thing to have it revalidate the attribute cache (you might be
able to argue that this is consistent with the mission of chown(),
although I'm not yet convinced of that) but the only effect that chown()
has on data caching is that we force a sync to disk in order to avoid
trouble with cached writes that are no longer accepted by the server. It
has never forced a cache invalidation.

IMO, the right interface for manipulating the page cache is rather
posix_fadvise(). I can certainly see an argument for adding a
"POSIX_FADV_UNCACHE" option in order to force a cache invalidation for
those applications that need stronger cache consistency.

Note, however, that even if such a thing is accepted into the mainstream
kernel, you still have to convince all the distros to backport this
interface to their older kernels. If not, you will still have a problem
with legacy clients...

Cheers
Trond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued.
Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs