Hello,
I'm seeing an issue where stat (and ls) reports wrong mode bits on
referral directories. Actual permissions are 755; but Linux client
displays 555. This causes some operations like setattr (chmod) to
fail. Traversing to the directory fixes the issue.
Kernel version : 5.8.6-2.el7.elrepo.x86_64
[nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
[nfstest@centos77 ~]$ ls -l /mnt/nfsh1
total 3
dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
[nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
File: ‘/mnt/nfsh1/dir.1’
Size: 2 Blocks: 1 IO Block: 1048576 directory
Device: 30h/48d Inode: 3940649673949864 Links: 2
Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
Context: system_u:object_r:nfs_t:s0
Access: 2020-09-03 17:55:59.082327209 -0400
Modify: 2020-09-03 17:55:59.082327209 -0400
Change: 2020-09-03 17:55:59.082327209 -0400
Birth: -
[nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into the
dir, see the mode bits in stat after traversal.
[nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
File: ‘/mnt/nfsh1/dir.1’
Size: 2 Blocks: 1 IO Block: 32768 directory
Device: 32h/50d Inode: 3940649673949864 Links: 2
Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
Context: system_u:object_r:nfs_t:s0
Access: 2020-09-03 17:55:59.082327209 -0400
Modify: 2020-09-03 17:55:59.082327209 -0400
Change: 2020-09-03 17:55:59.082327209 -0400
Birth: -
Attached is the tcpdump for requests. It looks like the server sends
back correct attributes; but the client somehow is ignoring it. Any
ideas why?
Thanks,
Pradeep
Just to add, if you look at packet 100 (READDIR response) in tcpdump,
the mode bits are set to 0755. But what is displayed by "ls" is 0555.
I'm trying to figure out where that one bit gets lost.
On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]> wrote:
>
> Hello,
>
> I'm seeing an issue where stat (and ls) reports wrong mode bits on
> referral directories. Actual permissions are 755; but Linux client
> displays 555. This causes some operations like setattr (chmod) to
> fail. Traversing to the directory fixes the issue.
>
> Kernel version : 5.8.6-2.el7.elrepo.x86_64
>
> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
> total 3
> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> File: ‘/mnt/nfsh1/dir.1’
> Size: 2 Blocks: 1 IO Block: 1048576 directory
> Device: 30h/48d Inode: 3940649673949864 Links: 2
> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
> Context: system_u:object_r:nfs_t:s0
> Access: 2020-09-03 17:55:59.082327209 -0400
> Modify: 2020-09-03 17:55:59.082327209 -0400
> Change: 2020-09-03 17:55:59.082327209 -0400
> Birth: -
> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into the
> dir, see the mode bits in stat after traversal.
> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> File: ‘/mnt/nfsh1/dir.1’
> Size: 2 Blocks: 1 IO Block: 32768 directory
> Device: 32h/50d Inode: 3940649673949864 Links: 2
> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
> Context: system_u:object_r:nfs_t:s0
> Access: 2020-09-03 17:55:59.082327209 -0400
> Modify: 2020-09-03 17:55:59.082327209 -0400
> Change: 2020-09-03 17:55:59.082327209 -0400
> Birth: -
>
> Attached is the tcpdump for requests. It looks like the server sends
> back correct attributes; but the client somehow is ignoring it. Any
> ideas why?
>
> Thanks,
> Pradeep
Probably in nfs_fixup_secinfo_attributes(), and looks deliberate. I'm not
sure about the reasoning behind it, though. Maybe it's to clarify that you
can't traverse this directory.
Ben
On 4 Sep 2020, at 12:57, Pradeep wrote:
> Just to add, if you look at packet 100 (READDIR response) in tcpdump,
> the mode bits are set to 0755. But what is displayed by "ls" is 0555.
> I'm trying to figure out where that one bit gets lost.
>
> On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]> wrote:
>>
>> Hello,
>>
>> I'm seeing an issue where stat (and ls) reports wrong mode bits on
>> referral directories. Actual permissions are 755; but Linux client
>> displays 555. This causes some operations like setattr (chmod) to
>> fail. Traversing to the directory fixes the issue.
>>
>> Kernel version : 5.8.6-2.el7.elrepo.x86_64
>>
>> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
>> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
>> total 3
>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
>> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>> File: ‘/mnt/nfsh1/dir.1’
>> Size: 2 Blocks: 1 IO Block: 1048576 directory
>> Device: 30h/48d Inode: 3940649673949864 Links: 2
>> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
>> Context: system_u:object_r:nfs_t:s0
>> Access: 2020-09-03 17:55:59.082327209 -0400
>> Modify: 2020-09-03 17:55:59.082327209 -0400
>> Change: 2020-09-03 17:55:59.082327209 -0400
>> Birth: -
>> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into the
>> dir, see the mode bits in stat after traversal.
>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>> File: ‘/mnt/nfsh1/dir.1’
>> Size: 2 Blocks: 1 IO Block: 32768 directory
>> Device: 32h/50d Inode: 3940649673949864 Links: 2
>> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
>> Context: system_u:object_r:nfs_t:s0
>> Access: 2020-09-03 17:55:59.082327209 -0400
>> Modify: 2020-09-03 17:55:59.082327209 -0400
>> Change: 2020-09-03 17:55:59.082327209 -0400
>> Birth: -
>>
>> Attached is the tcpdump for requests. It looks like the server sends
>> back correct attributes; but the client somehow is ignoring it. Any
>> ideas why?
>>
>> Thanks,
>> Pradeep
Thanks Benjamin. Looks like it is added as part of commit
72de53ec4bca39c26709122a8f78bfefe7b6bca4 by Bryan Schumaker.
Given that neither chmod nor readdir traverses the referral mount
points, wouldn't this be an issue to return mode bits as 0555?
The issue I see is with rsync. rsync first creates the directory and
calls chmod. If this directory happens to be a referral mount, then
chmod fails because the user doesn't have write access (0555).
Thanks
On Fri, Sep 4, 2020 at 10:15 AM Benjamin Coddington <[email protected]> wrote:
>
> Probably in nfs_fixup_secinfo_attributes(), and looks deliberate. I'm not
> sure about the reasoning behind it, though. Maybe it's to clarify that you
> can't traverse this directory.
>
> Ben
>
> On 4 Sep 2020, at 12:57, Pradeep wrote:
>
> > Just to add, if you look at packet 100 (READDIR response) in tcpdump,
> > the mode bits are set to 0755. But what is displayed by "ls" is 0555.
> > I'm trying to figure out where that one bit gets lost.
> >
> > On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]> wrote:
> >>
> >> Hello,
> >>
> >> I'm seeing an issue where stat (and ls) reports wrong mode bits on
> >> referral directories. Actual permissions are 755; but Linux client
> >> displays 555. This causes some operations like setattr (chmod) to
> >> fail. Traversing to the directory fixes the issue.
> >>
> >> Kernel version : 5.8.6-2.el7.elrepo.x86_64
> >>
> >> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
> >> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
> >> total 3
> >> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
> >> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
> >> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
> >> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
> >> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
> >> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> >> File: ‘/mnt/nfsh1/dir.1’
> >> Size: 2 Blocks: 1 IO Block: 1048576 directory
> >> Device: 30h/48d Inode: 3940649673949864 Links: 2
> >> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
> >> Context: system_u:object_r:nfs_t:s0
> >> Access: 2020-09-03 17:55:59.082327209 -0400
> >> Modify: 2020-09-03 17:55:59.082327209 -0400
> >> Change: 2020-09-03 17:55:59.082327209 -0400
> >> Birth: -
> >> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into the
> >> dir, see the mode bits in stat after traversal.
> >> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> >> File: ‘/mnt/nfsh1/dir.1’
> >> Size: 2 Blocks: 1 IO Block: 32768 directory
> >> Device: 32h/50d Inode: 3940649673949864 Links: 2
> >> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/ wheel)
> >> Context: system_u:object_r:nfs_t:s0
> >> Access: 2020-09-03 17:55:59.082327209 -0400
> >> Modify: 2020-09-03 17:55:59.082327209 -0400
> >> Change: 2020-09-03 17:55:59.082327209 -0400
> >> Birth: -
> >>
> >> Attached is the tcpdump for requests. It looks like the server sends
> >> back correct attributes; but the client somehow is ignoring it. Any
> >> ideas why?
> >>
> >> Thanks,
> >> Pradeep
>
I don't understand how the client can create a directory that can be a
referral. Doesn't the directory have to be a mountpoint on the server?
I don't think that rsync can copy a namespace that contains refferal
mounts
from the client side, but maybe I am really misunderstanding things..
Ben
On 4 Sep 2020, at 14:09, Pradeep wrote:
> Thanks Benjamin. Looks like it is added as part of commit
> 72de53ec4bca39c26709122a8f78bfefe7b6bca4 by Bryan Schumaker.
>
> Given that neither chmod nor readdir traverses the referral mount
> points, wouldn't this be an issue to return mode bits as 0555?
> The issue I see is with rsync. rsync first creates the directory and
> calls chmod. If this directory happens to be a referral mount, then
> chmod fails because the user doesn't have write access (0555).
>
> Thanks
>
> On Fri, Sep 4, 2020 at 10:15 AM Benjamin Coddington
> <[email protected]> wrote:
>>
>> Probably in nfs_fixup_secinfo_attributes(), and looks deliberate.
>> I'm not
>> sure about the reasoning behind it, though. Maybe it's to clarify
>> that you
>> can't traverse this directory.
>>
>> Ben
>>
>> On 4 Sep 2020, at 12:57, Pradeep wrote:
>>
>>> Just to add, if you look at packet 100 (READDIR response) in
>>> tcpdump,
>>> the mode bits are set to 0755. But what is displayed by "ls" is
>>> 0555.
>>> I'm trying to figure out where that one bit gets lost.
>>>
>>> On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]>
>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm seeing an issue where stat (and ls) reports wrong mode bits on
>>>> referral directories. Actual permissions are 755; but Linux client
>>>> displays 555. This causes some operations like setattr (chmod) to
>>>> fail. Traversing to the directory fixes the issue.
>>>>
>>>> Kernel version : 5.8.6-2.el7.elrepo.x86_64
>>>>
>>>> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
>>>> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
>>>> total 3
>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
>>>> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>>>> File: ‘/mnt/nfsh1/dir.1’
>>>> Size: 2 Blocks: 1 IO Block: 1048576
>>>> directory
>>>> Device: 30h/48d Inode: 3940649673949864 Links: 2
>>>> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
>>>> wheel)
>>>> Context: system_u:object_r:nfs_t:s0
>>>> Access: 2020-09-03 17:55:59.082327209 -0400
>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
>>>> Change: 2020-09-03 17:55:59.082327209 -0400
>>>> Birth: -
>>>> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into
>>>> the
>>>> dir, see the mode bits in stat after traversal.
>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>>>> File: ‘/mnt/nfsh1/dir.1’
>>>> Size: 2 Blocks: 1 IO Block: 32768
>>>> directory
>>>> Device: 32h/50d Inode: 3940649673949864 Links: 2
>>>> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
>>>> wheel)
>>>> Context: system_u:object_r:nfs_t:s0
>>>> Access: 2020-09-03 17:55:59.082327209 -0400
>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
>>>> Change: 2020-09-03 17:55:59.082327209 -0400
>>>> Birth: -
>>>>
>>>> Attached is the tcpdump for requests. It looks like the server
>>>> sends
>>>> back correct attributes; but the client somehow is ignoring it. Any
>>>> ideas why?
>>>>
>>>> Thanks,
>>>> Pradeep
>>
Ah, maybe you're just trying to copy the NFS namespace and cross the
referral, not actually create a new referral. In that case - yes,
perhaps
this needs a fixup. We might look at how rsync crosses other
mountpoints,
because I think the same problem would exist. You'd want rsync to
create
the directory with the attributes of the root of the mounted filesystem,
not
the mountpoint.
Ben
On 4 Sep 2020, at 14:20, Benjamin Coddington wrote:
> I don't understand how the client can create a directory that can be a
> referral. Doesn't the directory have to be a mountpoint on the
> server?
>
> I don't think that rsync can copy a namespace that contains refferal
> mounts
> from the client side, but maybe I am really misunderstanding things..
>
> Ben
>
> On 4 Sep 2020, at 14:09, Pradeep wrote:
>
>> Thanks Benjamin. Looks like it is added as part of commit
>> 72de53ec4bca39c26709122a8f78bfefe7b6bca4 by Bryan Schumaker.
>>
>> Given that neither chmod nor readdir traverses the referral mount
>> points, wouldn't this be an issue to return mode bits as 0555?
>> The issue I see is with rsync. rsync first creates the directory and
>> calls chmod. If this directory happens to be a referral mount, then
>> chmod fails because the user doesn't have write access (0555).
>>
>> Thanks
>>
>> On Fri, Sep 4, 2020 at 10:15 AM Benjamin Coddington
>> <[email protected]> wrote:
>>>
>>> Probably in nfs_fixup_secinfo_attributes(), and looks deliberate.
>>> I'm not
>>> sure about the reasoning behind it, though. Maybe it's to clarify
>>> that you
>>> can't traverse this directory.
>>>
>>> Ben
>>>
>>> On 4 Sep 2020, at 12:57, Pradeep wrote:
>>>
>>>> Just to add, if you look at packet 100 (READDIR response) in
>>>> tcpdump,
>>>> the mode bits are set to 0755. But what is displayed by "ls" is
>>>> 0555.
>>>> I'm trying to figure out where that one bit gets lost.
>>>>
>>>> On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I'm seeing an issue where stat (and ls) reports wrong mode bits on
>>>>> referral directories. Actual permissions are 755; but Linux client
>>>>> displays 555. This causes some operations like setattr (chmod) to
>>>>> fail. Traversing to the directory fixes the issue.
>>>>>
>>>>> Kernel version : 5.8.6-2.el7.elrepo.x86_64
>>>>>
>>>>> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
>>>>> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
>>>>> total 3
>>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
>>>>> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
>>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
>>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
>>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
>>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>>>>> File: ‘/mnt/nfsh1/dir.1’
>>>>> Size: 2 Blocks: 1 IO Block: 1048576
>>>>> directory
>>>>> Device: 30h/48d Inode: 3940649673949864 Links: 2
>>>>> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
>>>>> wheel)
>>>>> Context: system_u:object_r:nfs_t:s0
>>>>> Access: 2020-09-03 17:55:59.082327209 -0400
>>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
>>>>> Change: 2020-09-03 17:55:59.082327209 -0400
>>>>> Birth: -
>>>>> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into
>>>>> the
>>>>> dir, see the mode bits in stat after traversal.
>>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
>>>>> File: ‘/mnt/nfsh1/dir.1’
>>>>> Size: 2 Blocks: 1 IO Block: 32768
>>>>> directory
>>>>> Device: 32h/50d Inode: 3940649673949864 Links: 2
>>>>> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
>>>>> wheel)
>>>>> Context: system_u:object_r:nfs_t:s0
>>>>> Access: 2020-09-03 17:55:59.082327209 -0400
>>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
>>>>> Change: 2020-09-03 17:55:59.082327209 -0400
>>>>> Birth: -
>>>>>
>>>>> Attached is the tcpdump for requests. It looks like the server
>>>>> sends
>>>>> back correct attributes; but the client somehow is ignoring it.
>>>>> Any
>>>>> ideas why?
>>>>>
>>>>> Thanks,
>>>>> Pradeep
>>>
Yes Benjamin. If the referral mounts already exist, then rsync will
try to chmod to the source. That will not work because of mode bits.
rsync will continue to traverse into the referral mount and create
inner directories and copy other stuff. At the end, it will copy
everything except the attributes for referral mounts and also displays
an error. This is what I see:
[nfstest@centos77 ~]$ mkdir rsync-dataset
[nfstest@centos77 ~]$ mkdir -p rsync-dataset/dir.{1..5}
[nfstest@centos77 ~]$ touch rsync-dataset/dir.{1..5}/file
[nfstest@centos77 ~]$ rsync -avz rsync-dataset/* /mnt/nfsh1/
sending incremental file list
rsync: failed to set times on "/mnt/nfsh1/dir.1": Permission denied (13)
rsync: failed to set times on "/mnt/nfsh1/dir.3": Permission denied (13)
rsync: failed to set times on "/mnt/nfsh1/dir.4": Permission denied (13)
rsync: failed to set times on "/mnt/nfsh1/dir.5": Permission denied (13)
dir.1/
dir.1/file
dir.2/
dir.2/file
dir.3/
dir.3/file
dir.4/
dir.4/file
dir.5/
dir.5/file
sent 432 bytes received 439 bytes 1,742.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1179) [sender=3.1.2]
On Fri, Sep 4, 2020 at 11:30 AM Benjamin Coddington <[email protected]> wrote:
>
> Ah, maybe you're just trying to copy the NFS namespace and cross the
> referral, not actually create a new referral. In that case - yes,
> perhaps
> this needs a fixup. We might look at how rsync crosses other
> mountpoints,
> because I think the same problem would exist. You'd want rsync to
> create
> the directory with the attributes of the root of the mounted filesystem,
> not
> the mountpoint.
>
> Ben
>
> On 4 Sep 2020, at 14:20, Benjamin Coddington wrote:
>
> > I don't understand how the client can create a directory that can be a
> > referral. Doesn't the directory have to be a mountpoint on the
> > server?
> >
> > I don't think that rsync can copy a namespace that contains refferal
> > mounts
> > from the client side, but maybe I am really misunderstanding things..
> >
> > Ben
> >
> > On 4 Sep 2020, at 14:09, Pradeep wrote:
> >
> >> Thanks Benjamin. Looks like it is added as part of commit
> >> 72de53ec4bca39c26709122a8f78bfefe7b6bca4 by Bryan Schumaker.
> >>
> >> Given that neither chmod nor readdir traverses the referral mount
> >> points, wouldn't this be an issue to return mode bits as 0555?
> >> The issue I see is with rsync. rsync first creates the directory and
> >> calls chmod. If this directory happens to be a referral mount, then
> >> chmod fails because the user doesn't have write access (0555).
> >>
> >> Thanks
> >>
> >> On Fri, Sep 4, 2020 at 10:15 AM Benjamin Coddington
> >> <[email protected]> wrote:
> >>>
> >>> Probably in nfs_fixup_secinfo_attributes(), and looks deliberate.
> >>> I'm not
> >>> sure about the reasoning behind it, though. Maybe it's to clarify
> >>> that you
> >>> can't traverse this directory.
> >>>
> >>> Ben
> >>>
> >>> On 4 Sep 2020, at 12:57, Pradeep wrote:
> >>>
> >>>> Just to add, if you look at packet 100 (READDIR response) in
> >>>> tcpdump,
> >>>> the mode bits are set to 0755. But what is displayed by "ls" is
> >>>> 0555.
> >>>> I'm trying to figure out where that one bit gets lost.
> >>>>
> >>>> On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]>
> >>>> wrote:
> >>>>>
> >>>>> Hello,
> >>>>>
> >>>>> I'm seeing an issue where stat (and ls) reports wrong mode bits on
> >>>>> referral directories. Actual permissions are 755; but Linux client
> >>>>> displays 555. This causes some operations like setattr (chmod) to
> >>>>> fail. Traversing to the directory fixes the issue.
> >>>>>
> >>>>> Kernel version : 5.8.6-2.el7.elrepo.x86_64
> >>>>>
> >>>>> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
> >>>>> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
> >>>>> total 3
> >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
> >>>>> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
> >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
> >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
> >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
> >>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> >>>>> File: ‘/mnt/nfsh1/dir.1’
> >>>>> Size: 2 Blocks: 1 IO Block: 1048576
> >>>>> directory
> >>>>> Device: 30h/48d Inode: 3940649673949864 Links: 2
> >>>>> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
> >>>>> wheel)
> >>>>> Context: system_u:object_r:nfs_t:s0
> >>>>> Access: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Change: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Birth: -
> >>>>> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into
> >>>>> the
> >>>>> dir, see the mode bits in stat after traversal.
> >>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> >>>>> File: ‘/mnt/nfsh1/dir.1’
> >>>>> Size: 2 Blocks: 1 IO Block: 32768
> >>>>> directory
> >>>>> Device: 32h/50d Inode: 3940649673949864 Links: 2
> >>>>> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
> >>>>> wheel)
> >>>>> Context: system_u:object_r:nfs_t:s0
> >>>>> Access: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Change: 2020-09-03 17:55:59.082327209 -0400
> >>>>> Birth: -
> >>>>>
> >>>>> Attached is the tcpdump for requests. It looks like the server
> >>>>> sends
> >>>>> back correct attributes; but the client somehow is ignoring it.
> >>>>> Any
> >>>>> ideas why?
> >>>>>
> >>>>> Thanks,
> >>>>> Pradeep
> >>>
>
Hi Benjamin,
I'm looking at this path nfs4_proc_lookup() ->
nfs4_proc_lookup_common(). I don't see how client pointer can get
modified for "-NFS4ERR_MOVED" case. So, nfs_fixup_secinfo_attributes()
may not be called, right?
Looking at nfs4_get_referral() -> nfs_fixup_referral_attributes().
This also seems to set mode bits to 0555. But then if inode already
exists, then it shouldn't overwrite mode bits in nfs_fhget().
Thanks,
Pradeep
On Fri, Sep 4, 2020 at 11:45 AM Pradeep <[email protected]> wrote:
>
> Yes Benjamin. If the referral mounts already exist, then rsync will
> try to chmod to the source. That will not work because of mode bits.
> rsync will continue to traverse into the referral mount and create
> inner directories and copy other stuff. At the end, it will copy
> everything except the attributes for referral mounts and also displays
> an error. This is what I see:
>
> [nfstest@centos77 ~]$ mkdir rsync-dataset
> [nfstest@centos77 ~]$ mkdir -p rsync-dataset/dir.{1..5}
> [nfstest@centos77 ~]$ touch rsync-dataset/dir.{1..5}/file
> [nfstest@centos77 ~]$ rsync -avz rsync-dataset/* /mnt/nfsh1/
> sending incremental file list
> rsync: failed to set times on "/mnt/nfsh1/dir.1": Permission denied (13)
> rsync: failed to set times on "/mnt/nfsh1/dir.3": Permission denied (13)
> rsync: failed to set times on "/mnt/nfsh1/dir.4": Permission denied (13)
> rsync: failed to set times on "/mnt/nfsh1/dir.5": Permission denied (13)
> dir.1/
> dir.1/file
> dir.2/
> dir.2/file
> dir.3/
> dir.3/file
> dir.4/
> dir.4/file
> dir.5/
> dir.5/file
>
> sent 432 bytes received 439 bytes 1,742.00 bytes/sec
> total size is 0 speedup is 0.00
> rsync error: some files/attrs were not transferred (see previous
> errors) (code 23) at main.c(1179) [sender=3.1.2]
>
> On Fri, Sep 4, 2020 at 11:30 AM Benjamin Coddington <[email protected]> wrote:
> >
> > Ah, maybe you're just trying to copy the NFS namespace and cross the
> > referral, not actually create a new referral. In that case - yes,
> > perhaps
> > this needs a fixup. We might look at how rsync crosses other
> > mountpoints,
> > because I think the same problem would exist. You'd want rsync to
> > create
> > the directory with the attributes of the root of the mounted filesystem,
> > not
> > the mountpoint.
> >
> > Ben
> >
> > On 4 Sep 2020, at 14:20, Benjamin Coddington wrote:
> >
> > > I don't understand how the client can create a directory that can be a
> > > referral. Doesn't the directory have to be a mountpoint on the
> > > server?
> > >
> > > I don't think that rsync can copy a namespace that contains refferal
> > > mounts
> > > from the client side, but maybe I am really misunderstanding things..
> > >
> > > Ben
> > >
> > > On 4 Sep 2020, at 14:09, Pradeep wrote:
> > >
> > >> Thanks Benjamin. Looks like it is added as part of commit
> > >> 72de53ec4bca39c26709122a8f78bfefe7b6bca4 by Bryan Schumaker.
> > >>
> > >> Given that neither chmod nor readdir traverses the referral mount
> > >> points, wouldn't this be an issue to return mode bits as 0555?
> > >> The issue I see is with rsync. rsync first creates the directory and
> > >> calls chmod. If this directory happens to be a referral mount, then
> > >> chmod fails because the user doesn't have write access (0555).
> > >>
> > >> Thanks
> > >>
> > >> On Fri, Sep 4, 2020 at 10:15 AM Benjamin Coddington
> > >> <[email protected]> wrote:
> > >>>
> > >>> Probably in nfs_fixup_secinfo_attributes(), and looks deliberate.
> > >>> I'm not
> > >>> sure about the reasoning behind it, though. Maybe it's to clarify
> > >>> that you
> > >>> can't traverse this directory.
> > >>>
> > >>> Ben
> > >>>
> > >>> On 4 Sep 2020, at 12:57, Pradeep wrote:
> > >>>
> > >>>> Just to add, if you look at packet 100 (READDIR response) in
> > >>>> tcpdump,
> > >>>> the mode bits are set to 0755. But what is displayed by "ls" is
> > >>>> 0555.
> > >>>> I'm trying to figure out where that one bit gets lost.
> > >>>>
> > >>>> On Fri, Sep 4, 2020 at 8:55 AM Pradeep <[email protected]>
> > >>>> wrote:
> > >>>>>
> > >>>>> Hello,
> > >>>>>
> > >>>>> I'm seeing an issue where stat (and ls) reports wrong mode bits on
> > >>>>> referral directories. Actual permissions are 755; but Linux client
> > >>>>> displays 555. This causes some operations like setattr (chmod) to
> > >>>>> fail. Traversing to the directory fixes the issue.
> > >>>>>
> > >>>>> Kernel version : 5.8.6-2.el7.elrepo.x86_64
> > >>>>>
> > >>>>> [nfstest@centos77 ~]$ mkdir /mnt/nfsh1/dir.{1..5}
> > >>>>> [nfstest@centos77 ~]$ ls -l /mnt/nfsh1
> > >>>>> total 3
> > >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.1
> > >>>>> drwxr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.2
> > >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.3
> > >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.4
> > >>>>> dr-xr-xr-x. 2 nfstest wheel 2 Sep 3 17:55 dir.5
> > >>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> > >>>>> File: ‘/mnt/nfsh1/dir.1’
> > >>>>> Size: 2 Blocks: 1 IO Block: 1048576
> > >>>>> directory
> > >>>>> Device: 30h/48d Inode: 3940649673949864 Links: 2
> > >>>>> Access: (0555/dr-xr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
> > >>>>> wheel)
> > >>>>> Context: system_u:object_r:nfs_t:s0
> > >>>>> Access: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Change: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Birth: -
> > >>>>> [nfstest@centos77 ~]$ ls /mnt/nfsh1/dir.1 <-- Try traversing into
> > >>>>> the
> > >>>>> dir, see the mode bits in stat after traversal.
> > >>>>> [nfstest@centos77 ~]$ stat /mnt/nfsh1/dir.1
> > >>>>> File: ‘/mnt/nfsh1/dir.1’
> > >>>>> Size: 2 Blocks: 1 IO Block: 32768
> > >>>>> directory
> > >>>>> Device: 32h/50d Inode: 3940649673949864 Links: 2
> > >>>>> Access: (0755/drwxr-xr-x) Uid: ( 2000/ nfstest) Gid: ( 10/
> > >>>>> wheel)
> > >>>>> Context: system_u:object_r:nfs_t:s0
> > >>>>> Access: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Modify: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Change: 2020-09-03 17:55:59.082327209 -0400
> > >>>>> Birth: -
> > >>>>>
> > >>>>> Attached is the tcpdump for requests. It looks like the server
> > >>>>> sends
> > >>>>> back correct attributes; but the client somehow is ignoring it.
> > >>>>> Any
> > >>>>> ideas why?
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Pradeep
> > >>>
> >