2008-05-23 00:53:15

by Murata, Dennis

[permalink] [raw]
Subject: extremely long time for exportfs to display

Very recently we have seen a very serious slowdown with the exportfs
command on a file server. Usually the exportfs -r command or even just
the exportfs command to display the exported directories is very quick.
Now it can take 20 minutes to display the exported directories.
An strace shows a lookup at /etc/hosts, from yp, and from dns. The
hosts lookup in /etc/nsswitch is files nis dns, so this is as expected,
the time it is taking is not.
The yp slave that the server is bound to seems to be having some
problems, could this be the issue?

Wayne


2008-05-23 15:20:47

by J. Bruce Fields

[permalink] [raw]
Subject: Re: extremely long time for exportfs to display

On Thu, May 22, 2008 at 05:41:05PM -0700, Murata, Dennis wrote:
> Very recently we have seen a very serious slowdown with the exportfs
> command on a file server. Usually the exportfs -r command or even just
> the exportfs command to display the exported directories is very quick.
> Now it can take 20 minutes to display the exported directories.
> An strace shows a lookup at /etc/hosts, from yp, and from dns. The
> hosts lookup in /etc/nsswitch is files nis dns, so this is as expected,
> the time it is taking is not.

Have you run strace with something like -t to work out exactly where the
time is going?

--b.

> The yp slave that the server is bound to seems to be having some
> problems, could this be the issue?
>
> Wayne
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2008-05-23 20:35:39

by Murata, Dennis

[permalink] [raw]
Subject: RE: extremely long time for exportfs to display

A little history, I have tested the exports file on two servers, one in
production one a test server. The two servers are modified RHEL4.2.
production - kernel-smp-2.6.9-67.EL.x86_64
nfs-utils-1.0.6-84.EL4.x86_64
nfs-utils-lib-1.0.6-8.z1.x86_64
util-linux-2.12a-16.EL4.29.x86_64

test - kernel-smp-2.6.9-67.0.4.EL.x86_64
nfs-utils-1.0.6-84.EL4.x86_64
nfs-utils-lib-1.0.6-8.z1.x86_64
util-linux-2.12a-16.EL4.29.x86_64

The test server has a slightly newer kernel. The exports file is large,
186 lines, some lines with 20+ workstation entries.
using strace -t, the two servers look at /etc/resolv.conf, do an lstat
on the exported directories, I am double checking not to see if it is
all directories or not.
On the production server, after the lstat on the directories, there is a
call to open /etc/nsswitch.conf, /etc/host.conf, /etc/hosts, and
/var/yp/binding/domainname. All of the extra time in within this piece.
On the test server, there never is a call to open nsswitch.conf,
/etc/host.conf, /etc/hosts, or/var/yp/binding/domainname.
I have the strace of the execution of exportfs if needed.

Wayne
> -----Original Message-----
> From: J. Bruce Fields [mailto:[email protected]]
> Sent: Friday, May 23, 2008 10:21 AM
> To: Murata, Dennis
> Cc: [email protected]
> Subject: Re: extremely long time for exportfs to display
>
> On Thu, May 22, 2008 at 05:41:05PM -0700, Murata, Dennis wrote:
> > Very recently we have seen a very serious slowdown with the
> exportfs
> > command on a file server. Usually the exportfs -r command or even
> > just the exportfs command to display the exported
> directories is very quick.
> > Now it can take 20 minutes to display the exported directories.
> > An strace shows a lookup at /etc/hosts, from yp, and from dns. The
> > hosts lookup in /etc/nsswitch is files nis dns, so this is as
> > expected, the time it is taking is not.
>
> Have you run strace with something like -t to work out
> exactly where the time is going?
>
> --b.
>
> > The yp slave that the server is bound to seems to be having some
> > problems, could this be the issue?
> >
> > Wayne
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> linux-nfs"
> > in the body of a message to [email protected] More
> majordomo
> > info at http://vger.kernel.org/majordomo-info.html
>