■Reference
http://wiki.linux-nfs.org/wiki/index.php/Configuring_pNFS/spnfsd
http://osdir.com/ml/general/2010-05/msg10129.html
■My environment
kernel-2.6.32.11-107.2.3.pnfs.fc12.x86_64.rpm
kernel-devel-2.6.32.11-107.2.3.pnfs.fc12.x86_64.rpm
kernel-firmware-2.6.32.11-107.2.3.pnfs.fc12.noarch.rpm
kernel-headers-2.6.32.11-107.2.3.pnfs.fc12.x86_64.rpm
nfs-utils-1.2.2-4.1.pnfs.x86_64.rpm
MDS: 192.168.1.60
DS: 192.168.1.45
Client: 192.168.1.62 ( created on the KVM's virtual machine on the MDS )
■/etc/exports
- MDS's
/ *(rw,sync,pnfs,fsid=0,insecure,no_subtree_check,no_root_squash)
- DS's
/ *(rw,sync,fsid=0,insecure,no_subtree_check,pnfs,no_root_squash)
■/etc/fstab
- MDS's
UUID=52ab88e8-5d93-440c-af45-0a85923898ec / ext4
defaults 1 1
UUID=90d25a9c-54b4-4db4-979a-26ea0d67fe60 swap swap
defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
192.168.1.45:/pnfs /spnfs/192.168.1.45 nfs4 minorversion=1 0 0
■/etc/spnfsd.conf of the MDS
[General]
Verbosity = 1
Stripe-size = 8192
Dense-striping = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
DS-Mount-Directory = /spnfs
[DataServers]
NumDS = 1
DS1_IP = 192.168.1.45
DS1_PORT = 2049
DS1_ROOT = /pnfs
DS1_ID = 1
■Preparation nad Assumption
-The NFS related ports has opened and has confirmed
- on the DS
$mkdir /pnfs
- on the MDS
$mkdir /spnfs
$mkdir /spnfs/192.168.1.45
$reboot
$touch /spnfs/192.168.1.45/test <- write succeeded to the DS's /spnfs
The MDS has mounted the exported directory of DS and able to read and write.
The MDS's /proc/self/mountstats includes lines that means the PNFS has turned on
like below
PNFS_LAYOUTGET: 0 0 0 0 0 0 0 0
PNFS_LAYOUTCOMMIT: 0 0 0 0 0 0 0 0
PNFS_LAYOUTRETURN: 0 0 0 0 0 0 0 0
PNFS_GETDEVICELIST: 0 0 0 0 0 0 0 0
PNFS_GETDEVICEINFO: 0 0 0 0 0 0 0 0
PNFS_WRITE: 0 0 0 0 0 0 0 0
PNFS_COMMIT: 0 0 0 0 0 0 0 0
■issue
- on the client
$mkdir /mnt/pnfs
$modprobe nfslayoutdriver;modprobe blocklayoutdriver
$mount -t nfs4 -o minorversion=1 192.168.1.60:/ /mnt/pnfs <- SUCCEEDED
$echo "heeeeeeeeeeeeeeeey" >> /mnt/pnfs/hello.txt <- has not been wrote to the
DS but has been worte under / of the MDS.