Greetings kernel-hackers -
I'm setting up a web farm with a number of Linux machines (ia32) attached
via FC-AL to a shared SAN. There are 3 web servers, one of them mounts
the filesystem read/write, the other 2 read/only. (Filesystem is
currently ext3.)
Everything works "fine", until of course I try to modify files on the r/w
server. Then we start seeing cache incoherency - the r/o systems now have
"stale" information cached. Old versions of files and directories are not
replaced.
I'm looking for a way to flush or invalidate the cache on the block
device/filesystem, so that the system is forced to go all the way to the
disk. Unmounting and remounting would accomplish this, of course, but
that's tough to do in production.
This is a "read-mostly" application - I really only need to update the
filesystem once a day or so - but I'd like to find a nice way to do it,
without having to use NFS.
More info, if needed: Running Linux 2.4.18 (under Debian woody.) Using
the qla2200 driver from Qlogic, although I don't think that matters - it
looks like the caching happens in the VFS somewhere...
I've tried blockdev --flushbufs, which appears to do a BLKFLSBUF, but that
seems to be equivalent to "sync" - just pushes the dirty buffers to disk,
which doesn't help me.
Looking in fs/dcache.c, I see shrink_dcache_sb(struct super_block *),
which *might* do what I want, but there doesn't seem to be any way to call
that from user-land. And that probably just updates the dentries - I
don't know if that has any effect on the file data.
Hoping someone with more kernel experience can enlighten me -
Jim Lawson
On Nov 05, 2002 10:18 -0500, Jim Lawson wrote:
> I'm looking for a way to flush or invalidate the cache on the block
> device/filesystem, so that the system is forced to go all the way to the
> disk. Unmounting and remounting would accomplish this, of course, but
> that's tough to do in production.
>
> I've tried blockdev --flushbufs, which appears to do a BLKFLSBUF, but that
> seems to be equivalent to "sync" - just pushes the dirty buffers to disk,
> which doesn't help me.
>
> Looking in fs/dcache.c, I see shrink_dcache_sb(struct super_block *),
> which *might* do what I want, but there doesn't seem to be any way to call
> that from user-land. And that probably just updates the dentries - I
> don't know if that has any effect on the file data.
You may be in luck - we likely need to have an ioctl to do this from
e2fsck because the ext3 htree repacking of bad directories is causing
a bunch of problems. In theory, the startup scripts should reboot the
system in this case, but there have also been cases reported where
people ran "e2fsck -D" on an ro-mounted root and then read-write mounted
it again.
Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
> You may be in luck - we likely need to have an ioctl to do this from
> e2fsck because the ext3 htree repacking of bad directories is causing
> a bunch of problems. In theory, the startup scripts should reboot the
> system in this case, but there have also been cases reported where
> people ran "e2fsck -D" on an ro-mounted root and then read-write mounted
> it again.
It would be great if we could avoid the reboot, the biggest machines
I have access to can take 45 minutes to to pass firmware checks. I
hate it when fsck finds and error and causes another reboot :)
Anton