I've set up nfsroot based systems before but I am now attempting to do
the same thing using nfsv4 using the _in-kernel_ nfsroot support (ie: no
initrd with script that does the mounting). I have been unable to find
"authoritative" references to how to accomplish this and wanted to ask
for solid references before I start polluting the mailing list with my
probably completely erroneous approach. Things that might be worth
mentioning:
Some basic information, here is a sequence where I mount the nfsv4
export onto the same machine exporting the tree:
/etc/exports, using FSID=0/root:
livecd ~ # exportfs -v
/tftproot/nfsroot/x86_64
10.0.0.0/255.255.255.0(ro,async,wdelay,no_root_squash,no_subtree_check,fsid=0)
Simple call to mount won't work, I'm assuming mount defaults to nfsv3:
livecd ~ # mount 10.0.0.1:/ /mnt/gentoo/
mount.nfs: access denied by server while mounting 10.0.0.1:/
Specifying nfs4 works:
livecd ~ # mount -t nfs4 10.0.0.1:/ /mnt/gentoo/
Attempts to use the nfsv3 syntax fails (as expected since I put
`--no-nfs-version 3 --no-nfs-version 2` as a startup arg for nfsd):
livecd ~ # mount -t nfs4 10.0.0.1:/tftproot/nfsroot/x86_64 /mnt/cdrom/
mount.nfs4: mounting 10.0.0.1:/tftproot/nfsroot/x86_64 failed,
reason given by server:
No such file or directory
Booting the diskless node, I get the following:
1- If the args are (v3 syntax, in pxelinux.cfg/default):
append ip=dhcp nfsroot=10.0.0.1:/tftproot/nfsroot/x86_64,hard,intr
init=/boot/stateless.sh
The node boots up to the point of calling init and panics because it
can't find "/boot/stateless.sh" (screen shot:
http://wiki.neuralbs.com/~kyron/soc2008/screenshoots/NFSv4WithV3CommandLine_no_init.png
) But stateless is there and accessible (from the earlyer local mounting
of the root):
livecd ~ # ls -l /mnt/gentoo/boot/stateless.sh
-rwxr-xr-x 1 4294967294 4294967294 3964 Jul 11 04:22
/mnt/gentoo/boot/stateless.sh
2- If the args are (v4 syntax, in pxelinux.cfg/default):
append ip=dhcp nfsroot=10.0.0.1:/,hard,intr init=/boot/stateless.sh
The node boots up but panics on mounting root (screen shot :
http://wiki.neuralbs.com/~kyron/soc2008/screenshoots/NFSv4PanicOnMount.png )
What can be seen in the logs:
livecd ~ # egrep -i '(nfs|mount)' /var/log/messages
Jul 16 01:37:23 livecd Mount-cache hash table entries: 256
Jul 16 01:37:36 livecd rc-scripts: ERROR: cannot start nfsmount as
rpc.statd could not start
Jul 16 01:52:00 livecd aufs test_add:377:mount[27642]: uid/gid/perm
/mnt/livecd/tftproot 0/0/0755, 0/0/01777
Jul 16 01:52:11 livecd Installing knfsd (copyright (C) 1996
[email protected]).
Jul 16 01:52:12 livecd NFSD: Using /var/lib/nfs/v4recovery as the
NFSv4 state recovery directory
Jul 16 01:52:12 livecd NFSD: starting 90-second grace period
Jul 16 02:05:17 livecd mountd[28146]: Caught signal 15,
un-registering and exiting.
Jul 16 02:05:17 livecd nfsd: last server has exited
Jul 16 02:05:17 livecd nfsd: unexporting all filesystems
Jul 16 02:05:17 livecd NFSD: Using /var/lib/nfs/v4recovery as the
NFSv4 state recovery directory
Jul 16 02:05:17 livecd NFSD: starting 90-second grace period
Jul 16 02:05:40 livecd mountd[28441]: refused mount request from
10.0.0.1 for / (/): not exported
Jul 16 02:05:49 livecd nfsd: nfsv4 idmapping failing: has idmapd not
been started?
Jul 16 02:12:52 livecd dnsmasq[27954]: TFTP sent
/tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
Jul 16 02:13:10 livecd mountd[28441]: authenticated mount request
from 10.0.0.55:955 for /tftproot/nfsroot/x86_64
(/tftproot/nfsroot/x86_64)
Jul 16 02:13:10 livecd aufs aufs_encode_fh:529:nfsd[28447]: NFSv2
client (max_len 6)?
Jul 16 02:14:44 livecd dnsmasq[27954]: TFTP sent
/tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
Jul 16 02:15:01 livecd mountd[28441]: authenticated mount request
from 10.0.0.55:682 for /tftproot/nfsroot/x86_64
(/tftproot/nfsroot/x86_64)
Jul 16 02:24:55 livecd dnsmasq[28619]: TFTP sent
/tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
Jul 16 02:25:12 livecd mountd[28441]: refused mount request from
10.0.0.55 for / (/): not exported
NFS-booting-node's kernel config (nfs stuff), notice v3 client is disabled:
kyron clustering-livecd # grep -i nfs
/tmp/nfsboot_kernel/gentoo/usr/src/linux-2.6.26-rc5-mm3/.config
CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not set
CONFIG_NFS_V4=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFS_COMMON=y
The server's kernel config (nfs):
kyron clustering-livecd # grep -i nfs
/var/tmp/catalyst/kerncache/default/livecd-stage2-amd64-2008.0/gentoo/usr/src/linux-2.6.24-gentoo-r8/.config
CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not set
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_ROOT_NFS=y
CONFIG_NFS_COMMON=y
...so, it seems kernel boot time nfs is not NFSv4 compatible (only v2/3
is supported _in kernel_).
Eric
On Tue, 2008-07-15 at 23:59 -0400, Eric Thibodeau wrote:
> I've set up nfsroot based systems before but I am now attempting to do
> the same thing using nfsv4 using the _in-kernel_ nfsroot support (ie: no
> initrd with script that does the mounting). I have been unable to find
> "authoritative" references to how to accomplish this and wanted to ask
> for solid references before I start polluting the mailing list with my
> probably completely erroneous approach. Things that might be worth
> mentioning:
nfsroot for NFSv4 is not supported. You can still do it by using an
initrd file in order to bootstrap the necessary user-space utilities
like the idmapper (and possibly rpc.gssd) before mounting the partition
and pivoting.
Cheers
Trond
