2009-04-15 10:50:55

by Hans-Peter Jansen

[permalink] [raw]
Subject: klibc's nfsmount failure with 2.6.27.21, while 2.6.25.20 was fine

Hi,

I suffer from a problem, where my diskless environment doesn't work anymore
after upgrading the server from openSUSE 11.0 (2.6.25.20) to openSUSE 11.1
(2.6.27.21).

The diskless environment gets set up with an customized initrd via pxelinux.
The initrd uses klibc's nfsmount, and I've tried both protocols: udp and
tcp. It's not an iptables issue, since I can mount these directories from
other (fully booted) systems just fine.

Using the debug version of klibc's nfsmount prints these messages:
short read: 0 < 28
Port for 100003/3[udp]: 0
short read: 0 < 28
Port for 100005/3[udp]: 0
NFS params:
server = xxx.xx.xx.xx, path = "/roroot", version = 3, proto = udp,
mount_port = 627, nfs_port = 2049, flags = 00010282,
rsize = 0, wsize = 0, timeo = 7, retrans = 3,
acreg (min,max) = (3,60), acdir (min,max) = (30,60),
soft = 0, intr = 1, posix = 0, nocto = 0, noac = 0

tcpdumping the communication shows, that for whatever reason, the client
sends the mount request from source port 798 to destination port 627, but
the server doesn't listen on 627:
# rpcinfo -p
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100005 1 udp 41656 mountd
100005 1 tcp 49063 mountd
100005 2 udp 41656 mountd
100005 2 tcp 49063 mountd
100005 3 udp 41656 mountd
100005 3 tcp 49063 mountd
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 4 udp 2049 nfs
100021 1 udp 33144 nlockmgr
100021 3 udp 33144 nlockmgr
100021 4 udp 33144 nlockmgr
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100021 1 tcp 57268 nlockmgr
100021 3 tcp 57268 nlockmgr
100021 4 tcp 57268 nlockmgr
100024 1 udp 39508 status
100024 1 tcp 45430 status


I've attached the relevant tcpdump section.

I would be very grateful, if some kind soul could enlight me here, as I'm
fully puzzled..

Pete


Attachments:
(No filename) (2.20 kB)
shark-nfs-mount.dump (4.71 kB)
Download all attachments

2009-04-15 12:38:39

by Trond Myklebust

[permalink] [raw]
Subject: Re: klibc's nfsmount failure with 2.6.27.21, while 2.6.25.20 was fine

On Wed, 2009-04-15 at 12:50 +0200, Hans-Peter Jansen wrote:
> Hi,
>
> I suffer from a problem, where my diskless environment doesn't work anymore
> after upgrading the server from openSUSE 11.0 (2.6.25.20) to openSUSE 11.1
> (2.6.27.21).
>
> The diskless environment gets set up with an customized initrd via pxelinux.
> The initrd uses klibc's nfsmount, and I've tried both protocols: udp and
> tcp. It's not an iptables issue, since I can mount these directories from
> other (fully booted) systems just fine.
>
> Using the debug version of klibc's nfsmount prints these messages:
> short read: 0 < 28
> Port for 100003/3[udp]: 0
> short read: 0 < 28
> Port for 100005/3[udp]: 0
> NFS params:
> server = xxx.xx.xx.xx, path = "/roroot", version = 3, proto = udp,
> mount_port = 627, nfs_port = 2049, flags = 00010282,
^^^^^^^^^^^^^^^^

> rsize = 0, wsize = 0, timeo = 7, retrans = 3,
> acreg (min,max) = (3,60), acdir (min,max) = (30,60),
> soft = 0, intr = 1, posix = 0, nocto = 0, noac = 0
>
> tcpdumping the communication shows, that for whatever reason, the client
> sends the mount request from source port 798 to destination port 627, but
> the server doesn't listen on 627:

The question is why is klibc specifying mount port = 627?

Trond


2009-04-15 13:48:32

by Hans-Peter Jansen

[permalink] [raw]
Subject: Re: klibc's nfsmount failure with 2.6.27.21, while 2.6.25.20 was fine

Am Mittwoch, 15. April 2009 schrieb Trond Myklebust:
> On Wed, 2009-04-15 at 12:50 +0200, Hans-Peter Jansen wrote:
> > Hi,
> >
> > I suffer from a problem, where my diskless environment doesn't work
> > anymore after upgrading the server from openSUSE 11.0 (2.6.25.20) to
> > openSUSE 11.1 (2.6.27.21).
> >
> > The diskless environment gets set up with an customized initrd via
> > pxelinux. The initrd uses klibc's nfsmount, and I've tried both
> > protocols: udp and tcp. It's not an iptables issue, since I can mount
> > these directories from other (fully booted) systems just fine.
> >
> > Using the debug version of klibc's nfsmount prints these messages:
> > short read: 0 < 28
> > Port for 100003/3[udp]: 0
> > short read: 0 < 28
> > Port for 100005/3[udp]: 0
> > NFS params:
> > server = xxx.xx.xx.xx, path = "/roroot", version = 3, proto = udp,
> > mount_port = 627, nfs_port = 2049, flags = 00010282,
>
> ^^^^^^^^^^^^^^^^
>
> > rsize = 0, wsize = 0, timeo = 7, retrans = 3,
> > acreg (min,max) = (3,60), acdir (min,max) = (30,60),
> > soft = 0, intr = 1, posix = 0, nocto = 0, noac = 0
> >
> > tcpdumping the communication shows, that for whatever reason, the
> > client sends the mount request from source port 798 to destination port
> > 627, but the server doesn't listen on 627:
>
> The question is why is klibc specifying mount port = 627?

Well, this is exactly my problem. That's the mount port, but this one cannot
be set to a manual value, thus the rpc stuff infront of it must have gone
wrong.

[Update]
Issue solved. It was that silly rpcbind package (0.1.6+git20080930-5.2),
that has gone wrong. Replacing it with portmap (6.0+git20070716-31.37) did
it. Looks like this deserves a new bugzilla record.

Sorry for the churn and thanks for your care, Trond.

Pete