2008-06-13 23:22:30

by Maurice Volaski

[permalink] [raw]
Subject: [NFS] the infamous" device is busy" bug

I'm running 2.6.25-gentoo-r1 on an AMD 64 box along with
nfs-utils-1.1.2-r1 and I have /var/lib/nfs symlinked to a separate
filesystem, /database/nfs

Now the system is running drbd and heartbeat and /database is on
drbd, so that if the server fails, the secondary server will be able
to immediately take over nfs using the identical data on its
/database/nfs.

If, however, I want to manually stop heartbeat on the primary server,
I cannot because /database will not umount, complaining "the device
is busy". I've tried to manually by shutting down all things nfs, but
something is still holding it, and nfsd in the kernel is the culprit.
I tested that by loading the module and from that point forward,
trying to unmount /database just results in the "device is busy"
complaint. That means the only way I can shutdown heartbeat is to
physically shut down the server.
--

Maurice Volaski, mvolaski-Q33DkHiyi7z2fBVCVOL8/[email protected]
Computing Support, Rose F. Kennedy Center
Albert Einstein College of Medicine of Yeshiva University

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued.
Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs



2008-06-16 18:42:54

by J. Bruce Fields

[permalink] [raw]
Subject: Re: [NFS] the infamous" device is busy" bug

On Sat, Jun 14, 2008 at 01:20:26AM -0400, Maurice Volaski wrote:
> >should release it, though if there are active clients, one could grab it
> >again very quickly.
> >Alternately
> > rpc.nfsd 0
> > exportfs -f
> > umount
> > rpc.nfsd 16
>
> The rpc.nfsd 0 alone stopped this complaint and allowed the umount to
> succeed. Thanks. (Do I really want the last line "rpc.nfsd 16"? After
> all, I'm trying to turn nfs off.)
>
> It sounds as if there is a bug here,

It may be the nfsv4 reboot recovey code holding a reference to
/var/lib/nfs/v4recovery/.

> but it's Gentoo nfs init script
> that's at fault because presumably it should be executing "rpc.nfsd
> "0 and "exportfs -f" in response to "/etc/init.d/nfs stop".

I'd think it should be, yes.

--b.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued.
Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs


2008-06-14 00:59:34

by NeilBrown

[permalink] [raw]
Subject: Re: [NFS] the infamous" device is busy" bug

On Sat, June 14, 2008 9:22 am, Maurice Volaski wrote:
> I'm running 2.6.25-gentoo-r1 on an AMD 64 box along with
> nfs-utils-1.1.2-r1 and I have /var/lib/nfs symlinked to a separate
> filesystem, /database/nfs
>
> Now the system is running drbd and heartbeat and /database is on
> drbd, so that if the server fails, the secondary server will be able
> to immediately take over nfs using the identical data on its
> /database/nfs.
>
> If, however, I want to manually stop heartbeat on the primary server,
> I cannot because /database will not umount, complaining "the device
> is busy". I've tried to manually by shutting down all things nfs, but
> something is still holding it, and nfsd in the kernel is the culprit.
> I tested that by loading the module and from that point forward,
> trying to unmount /database just results in the "device is busy"
> complaint. That means the only way I can shutdown heartbeat is to
> physically shut down the server.

If nfsd is keeping the filesystem active, then
exportfs -f
should release it, though if there are active clients, one could grab it
again very quickly.
Alternately
rpc.nfsd 0
exportfs -f
umount
rpc.nfsd 16

NeilBrown



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued.
Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs


2008-06-14 05:20:57

by Maurice Volaski

[permalink] [raw]
Subject: Re: [NFS] the infamous" device is busy" bug

>should release it, though if there are active clients, one could grab it
>again very quickly.
>Alternately
> rpc.nfsd 0
> exportfs -f
> umount
> rpc.nfsd 16

The rpc.nfsd 0 alone stopped this complaint and allowed the umount to
succeed. Thanks. (Do I really want the last line "rpc.nfsd 16"? After
all, I'm trying to turn nfs off.)

It sounds as if there is a bug here, but it's Gentoo nfs init script
that's at fault because presumably it should be executing "rpc.nfsd
"0 and "exportfs -f" in response to "/etc/init.d/nfs stop".
--

Maurice Volaski, mvolaski-Q33DkHiyi7z2fBVCVOL8/[email protected]
Computing Support, Rose F. Kennedy Center
Albert Einstein College of Medicine of Yeshiva University

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued.
Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs