MIME-Version: 1.0
References: <CAP8UQm+od+oU63049_hgitTcBepCeG0vFBPHG90=WruVXGU_mw@mail.gmail.com>
In-Reply-To: <CAP8UQm+od+oU63049_hgitTcBepCeG0vFBPHG90=WruVXGU_mw@mail.gmail.com>
From:   Kanika <jr.kanika@gmail.com>
Date:   Wed, 6 Mar 2019 16:09:43 +0530
Message-ID: <CAP8UQm+gQZW5_FD5YMuh_fev0H7SAanwz92CpoA1Yyr7ooj7Hg@mail.gmail.com>
Subject: pnfs setup on Linux 4.15
To:     linux-nfs@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hi,
I am trying to deploy pnfs using the mainstream-ed NFSv4 server/client
available with 4.15 kernels. I have been able to setup the MDS and
client using the flexfile layout. I can't find any documentation on
how to inform MDS about the data servers.

The setup I have so far:
Metadata server and data server are running Ubuntu 18.04
Client is Centos 7.5

MDS (intended), ip: 192.168.122.92
root@ubuntu1804:~/#cat /etc/exports
/sudosrv *(rw,sync,fsid=0,no_subtree_check,no_root_squash,pnfs) <-- PNFS set
/sudosrv/share1 *(rw,sync,fsid=1,no_subtree_check,no_root_squash)

Intended DS, ip: 192.168.122.83
root@ubuntu18_04_2:~#cat /etc/exports
/srv/homes *(rw,sync,fsid=4,no_root_squash,no_subtree_check)

Client, ip:192.168.122.5 (Centos 7.5)
Mount the pseudo root filesystem exported by MDS

#mount -t nfs -o v4.2 -o rw 192.168.122.92:/ /mnt/
#cat /proc/self/mountstats

device nfsd mounted on /proc/fs/nfsd with fstype nfsd
device 192.168.122.92:/ mounted on /mnt with fstype nfs4 statvers=1.1
opts: rw,vers=4.2,rsize=524288,wsize=524288,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.122.5,local_lock=none
age: 127
impl_id: name='',domain='',date='0,0'
caps: caps=0x1fbffdf,wtmult=512,dtsize=32768,bsize=0,namlen=255
nfsv4: bm0=0xfdffbfff,bm1=0x40f9be3e,bm2=0x20803,acl=0x3,sessions,pnfs=LAYOUT_FLEX_FILES
 <--- flex file layout
sec: flavor=1,pseudoflavor=1

I tried 2 ways to link MDS and DS
1. Use the "refer" while exporting the DS share from MDS. This works
as referrals are expected to work but not like "pnfs".
2. Mount the DS on MDS in a subdir of /sudosrv/share1. All IO
operations go only to the MDS as if it were a local share. No exchange
between the client and the DS.

How can the DS be known to an MDS for a local filesystem like ext4?
Any help will be appreciated.

Thanks,
jrk