After two days uptime on my NFS4 server with 2.6.31-rc4 and a few
2.6.28 clients, I hit the file write-count
WARN_ON(f->f_mnt_write_state != 0) in file_take_write() in the
nfsd4_open path [1].
I can't find this reported elsewhere; let me know if it's preferred in
bugzilla.kernel.org etc.
Thanks,
Daniel
--- [1]
------------[ cut here ]------------
WARNING: at include/linux/fs.h:953 nfsd4_process_open2+0x9c3/0xc90()
Hardware name: OEM
Modules linked in: coretemp w83627ehf hwmon_vid ath9k
snd_hda_codec_realtek mac80211 led_class ath snd_hda_intel
snd_hda_codec snd_pcm snd_timer snd pl2303 soundcore snd_page_alloc
Pid: 2970, comm: nfsd Tainted: G W 2.6.31-rc4-274sd #1
Call Trace:
[<ffffffff81190413>] ? nfsd4_process_open2+0x9c3/0xc90
[<ffffffff810490f8>] warn_slowpath_common+0x78/0xd0
[<ffffffff8104915f>] warn_slowpath_null+0xf/0x20
[<ffffffff81190413>] nfsd4_process_open2+0x9c3/0xc90
[<ffffffff8118420f>] ? do_open_lookup+0x28f/0x300
[<ffffffff811844b9>] nfsd4_open+0x239/0x420
[<ffffffff81183c67>] nfsd4_proc_compound+0x2d7/0x530
[<ffffffff81172115>] nfsd_dispatch+0x115/0x260
[<ffffffff813f3d12>] svc_process+0x492/0x800
[<ffffffff8142e507>] ? down_read+0x77/0x80
[<ffffffff81172740>] ? nfsd+0x0/0x160
[<ffffffff8117281d>] nfsd+0xdd/0x160
[<ffffffff81062c9e>] kthread+0x9e/0xb0
[<ffffffff8100ce9a>] child_rip+0xa/0x20
[<ffffffff8100c83c>] ? restore_args+0x0/0x30
[<ffffffff81062c00>] ? kthread+0x0/0xb0
[<ffffffff8100ce90>] ? child_rip+0x0/0x20
---[ end trace 8ecb5c2eb0ae54d8 ]---
--
Daniel J Blueman
On Mon, Jul 27, 2009 at 10:49:03PM +0100, Daniel J Blueman wrote:
> After two days uptime on my NFS4 server with 2.6.31-rc4 and a few
> 2.6.28 clients, I hit the file write-count
> WARN_ON(f->f_mnt_write_state != 0) in file_take_write() in the
> nfsd4_open path [1].
Hm, so probably introduced by:
e518f0560a191269bd345178c899c790eb1ad4c8 "nfsd: take file and mnt write
in nfs4_upgrade_open".
The other possible file_take_write() caller here is dentry_open (which
calls it in the (f->f_mode & FMODE_WRITE) case).
Looks like nfs4_upgrade_open() isn't handling error case cleanup
correctly. Perhaps that could explain this.
--b.
>
> I can't find this reported elsewhere; let me know if it's preferred in
> bugzilla.kernel.org etc.
>
> Thanks,
> Daniel
>
> --- [1]
>
> ------------[ cut here ]------------
>
> WARNING: at include/linux/fs.h:953 nfsd4_process_open2+0x9c3/0xc90()
>
> Hardware name: OEM
>
> Modules linked in: coretemp w83627ehf hwmon_vid ath9k
> snd_hda_codec_realtek mac80211 led_class ath snd_hda_intel
> snd_hda_codec snd_pcm snd_timer snd pl2303 soundcore snd_page_alloc
>
> Pid: 2970, comm: nfsd Tainted: G W 2.6.31-rc4-274sd #1
>
> Call Trace:
>
> [<ffffffff81190413>] ? nfsd4_process_open2+0x9c3/0xc90
>
> [<ffffffff810490f8>] warn_slowpath_common+0x78/0xd0
>
> [<ffffffff8104915f>] warn_slowpath_null+0xf/0x20
>
> [<ffffffff81190413>] nfsd4_process_open2+0x9c3/0xc90
>
> [<ffffffff8118420f>] ? do_open_lookup+0x28f/0x300
>
> [<ffffffff811844b9>] nfsd4_open+0x239/0x420
>
> [<ffffffff81183c67>] nfsd4_proc_compound+0x2d7/0x530
>
> [<ffffffff81172115>] nfsd_dispatch+0x115/0x260
>
> [<ffffffff813f3d12>] svc_process+0x492/0x800
>
> [<ffffffff8142e507>] ? down_read+0x77/0x80
>
> [<ffffffff81172740>] ? nfsd+0x0/0x160
>
> [<ffffffff8117281d>] nfsd+0xdd/0x160
>
> [<ffffffff81062c9e>] kthread+0x9e/0xb0
>
> [<ffffffff8100ce9a>] child_rip+0xa/0x20
>
> [<ffffffff8100c83c>] ? restore_args+0x0/0x30
>
> [<ffffffff81062c00>] ? kthread+0x0/0xb0
>
> [<ffffffff8100ce90>] ? child_rip+0x0/0x20
>
> ---[ end trace 8ecb5c2eb0ae54d8 ]---
>
> --
> Daniel J Blueman
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Jul 28, 2009 at 12:18:14PM -0400, bfields wrote:
> On Mon, Jul 27, 2009 at 10:49:03PM +0100, Daniel J Blueman wrote:
> > After two days uptime on my NFS4 server with 2.6.31-rc4 and a few
> > 2.6.28 clients, I hit the file write-count
> > WARN_ON(f->f_mnt_write_state != 0) in file_take_write() in the
> > nfsd4_open path [1].
>
> Hm, so probably introduced by:
>
> e518f0560a191269bd345178c899c790eb1ad4c8 "nfsd: take file and mnt write
> in nfs4_upgrade_open".
>
> The other possible file_take_write() caller here is dentry_open (which
> calls it in the (f->f_mode & FMODE_WRITE) case).
>
> Looks like nfs4_upgrade_open() isn't handling error case cleanup
> correctly. Perhaps that could explain this.
The below (untested) should fix the error handling, at the expense of
making nfs4_upgrade_open() just a little more byzantine.
There's another problem, though: if our server gets a sequence like:
- OPEN for read
- OPEN for write
- OPEN_DOWNGRADE to read
- OPEN for write
all with the same open owner and file, then at the vfs level we do all
this with a single filp, like:
- OPEN for read
dentry_open();
- OPEN for write
get_write_access(inode);
mnt_want_write(mnt);
file_take_write(file);
- OPEN_DOWNGRADE to read
put_write_access(inode);
mnt_drop_write(mnt);
file_release_write(file);
- OPEN for write
get_write_access(inode);
mnt_want_write(mnt);
file_take_write(file);
But examination of the code shows that file_take_write() doesn't allow
this kind of use: you'll hit the warning you found, because
f_mnt_write_state == FILE_MNT_WRITE_TAKEN | FILE_MNT_WRITE_RELEASE;
We could fix that by replacing our file_release_write() with a
file_reset_write(). But I suspect we're abusing the VFS interface, and
we should really just keep separate filp's for read and write.
--b.
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 7729d09..3018839 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -2378,14 +2378,19 @@ nfs4_upgrade_open(struct svc_rqst *rqstp, struct svc_fh *cur_fh, struct nfs4_sta
if (err)
return nfserrno(err);
err = mnt_want_write(cur_fh->fh_export->ex_path.mnt);
- if (err)
+ if (err) {
+ put_write_access(inode);
return nfserrno(err);
+ }
file_take_write(filp);
}
status = nfsd4_truncate(rqstp, cur_fh, open);
if (status) {
- if (new_writer)
+ if (new_writer) {
+ file_reset_write(filp);
+ mnt_drop_write(cur_fh->fh_export->ex_path.mnt);
put_write_access(inode);
+ }
return status;
}
/* remember the open */