I have met application that is badly broken when installed on NFS. The
reason is - it expects files to belong to specific group. It switches
to this group on startup (explicit setgid) and creates files. But
files come out as belonging to GID 0.
I finally reduced it to this trivial script:
=== cut here ===
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
main()
{
int fd;
setgid(107);
fd = open("bar", O_CREAT, 0666);
close(fd);
}
=== cut here ===
On local storage file comes with GID 107; on NFS file comes with GID 0.
Linux is SLES10 SP3 with relatively old kernel: 2.6.16.60-0.89.1-smp,
server(s) are NetApp with different Data ONTAP versions (7.x and 8.1.1
as the last).
Client passes correct credentials (UID:0, GID:107), but does not
explicitly request file ownership in CREATE call (uid set_it - 0, gid
set_it - 0).
I am not sure whether this is a bug and if yes, on which side. I
appreciate any hint. Thank you!
-andrey
On Thu, Nov 8, 2012 at 7:12 AM, Andrey Borzenkov <[email protected]> wrote:
> В Wed, 7 Nov 2012 20:35:55 +0000
> "Myklebust, Trond" <[email protected]> пишет:
>
>> > -----Original Message-----
>> > From: [email protected] [mailto:linux-nfs-
>> > [email protected]] On Behalf Of Andrey Borzenkov
>> > Sent: Wednesday, November 07, 2012 2:28 PM
>> > To: J. Bruce Fields
>> > Cc: [email protected]
>> > Subject: Re: Effective process GID is ignored when client creates
>> > file on NFS
>> >
>> > В Wed, 7 Nov 2012 14:13:36 -0500
>> > "J. Bruce Fields" <[email protected]> пишет:
>> >
>> > > On Mon, Oct 29, 2012 at 06:09:29PM +0400, Andrey Borzenkov wrote:
>> > > > I have met application that is badly broken when installed on
>> > > > NFS. The reason is - it expects files to belong to specific
>> > > > group. It switches to this group on startup (explicit setgid)
>> > > > and creates files. But files come out as belonging to GID 0.
>> > > >
>> > > > I finally reduced it to this trivial script:
>> > > >
>> > > > === cut here ===
>> > > > #include <sys/types.h>
>> > > > #include <sys/stat.h>
>> > > > #include <fcntl.h>
>> > > > #include <unistd.h>
>> > > >
>> > > > main()
>> > > > {
>> > > > int fd;
>> > > >
>> > > > setgid(107);
>> > > > fd = open("bar", O_CREAT, 0666);
>> > > > close(fd);
>> > > > }
>> > > > === cut here ===
>> > > >
>> > > > On local storage file comes with GID 107; on NFS file comes
>> > > > with GID 0.
>> > > >
>> > > > Linux is SLES10 SP3 with relatively old kernel:
>> > > > 2.6.16.60-0.89.1-smp, server(s) are NetApp with different Data
>> > > > ONTAP versions (7.x and 8.1.1 as the last).
>> > > >
>> > > > Client passes correct credentials (UID:0, GID:107), but does not
>> > >
>> > > Those are the credentials in the rpc header on the CREATE call?
>> > >
>> >
>> > Yes.
>> >
>> > > > explicitly request file ownership in CREATE call (uid set_it -
>> > > > 0, gid set_it - 0).
>> > >
>> > > The client shouldn't have to set the owner or group itself.
>> > >
>> > > So this is server behavior.
>> > >
>> > > Have you checked that the directory you're creating in doesn't
>> > > have the sgid bit?
>> >
>> > Yes. It does not.
>> >
>> > > Or perhaps there's some other server configuration
>> > > that causes this.
>> >
>> > It appears that server ignores passed group if UID == 0. It
>> > correctly creates files if UID != 0. I got bug number from support
>> > but it is non-public and no information is visible so far. I asked
>> > about possible workaround (or undocumented options) but have not
>> > got any reply as yet.
>>
>> So is it possible that you forgot to turn off root squashing on the
>> server before testing?
>>
>
> All filesystems are exported with anon=0 in this configuration. But if
> root suqashing is active, files should not appear as owned by UID 0,
> should not they?
OK this gave me idea. So the problem is indeed anon=0. If I reexport
the same filesystems with root=... option, files are created with
correct group ownership. I have to think about it, but on the first
glance NetApp behavior is correct. As root= option is missing it maps
root to anonymous user, and as anon=0 is present, this anonymous user
is root. What do standards say about group ownership in this case?
On Mon, Oct 29, 2012 at 06:09:29PM +0400, Andrey Borzenkov wrote:
> I have met application that is badly broken when installed on NFS. The
> reason is - it expects files to belong to specific group. It switches
> to this group on startup (explicit setgid) and creates files. But
> files come out as belonging to GID 0.
>
> I finally reduced it to this trivial script:
>
> === cut here ===
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
>
> main()
> {
> int fd;
>
> setgid(107);
> fd = open("bar", O_CREAT, 0666);
> close(fd);
> }
> === cut here ===
>
> On local storage file comes with GID 107; on NFS file comes with GID 0.
>
> Linux is SLES10 SP3 with relatively old kernel: 2.6.16.60-0.89.1-smp,
> server(s) are NetApp with different Data ONTAP versions (7.x and 8.1.1
> as the last).
>
> Client passes correct credentials (UID:0, GID:107), but does not
Those are the credentials in the rpc header on the CREATE call?
> explicitly request file ownership in CREATE call (uid set_it - 0, gid
> set_it - 0).
The client shouldn't have to set the owner or group itself.
So this is server behavior.
Have you checked that the directory you're creating in doesn't have the
sgid bit? Or perhaps there's some other server configuration that
causes this.
--b.
>
> I am not sure whether this is a bug and if yes, on which side. I
> appreciate any hint. Thank you!
>
> -andrey
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
? Wed, 7 Nov 2012 14:13:36 -0500
"J. Bruce Fields" <[email protected]> ?????:
> On Mon, Oct 29, 2012 at 06:09:29PM +0400, Andrey Borzenkov wrote:
> > I have met application that is badly broken when installed on NFS.
> > The reason is - it expects files to belong to specific group. It
> > switches to this group on startup (explicit setgid) and creates
> > files. But files come out as belonging to GID 0.
> >
> > I finally reduced it to this trivial script:
> >
> > === cut here ===
> > #include <sys/types.h>
> > #include <sys/stat.h>
> > #include <fcntl.h>
> > #include <unistd.h>
> >
> > main()
> > {
> > int fd;
> >
> > setgid(107);
> > fd = open("bar", O_CREAT, 0666);
> > close(fd);
> > }
> > === cut here ===
> >
> > On local storage file comes with GID 107; on NFS file comes with
> > GID 0.
> >
> > Linux is SLES10 SP3 with relatively old kernel:
> > 2.6.16.60-0.89.1-smp, server(s) are NetApp with different Data
> > ONTAP versions (7.x and 8.1.1 as the last).
> >
> > Client passes correct credentials (UID:0, GID:107), but does not
>
> Those are the credentials in the rpc header on the CREATE call?
>
Yes.
> > explicitly request file ownership in CREATE call (uid set_it - 0,
> > gid set_it - 0).
>
> The client shouldn't have to set the owner or group itself.
>
> So this is server behavior.
>
> Have you checked that the directory you're creating in doesn't have
> the sgid bit?
Yes. It does not.
> Or perhaps there's some other server configuration that
> causes this.
It appears that server ignores passed group if UID == 0. It correctly
creates files if UID != 0. I got bug number from support but it is
non-public and no information is visible so far. I asked about
possible workaround (or undocumented options) but have not got any reply
as yet.
? Wed, 7 Nov 2012 20:35:55 +0000
"Myklebust, Trond" <[email protected]> ?????:
> > -----Original Message-----
> > From: [email protected] [mailto:linux-nfs-
> > [email protected]] On Behalf Of Andrey Borzenkov
> > Sent: Wednesday, November 07, 2012 2:28 PM
> > To: J. Bruce Fields
> > Cc: [email protected]
> > Subject: Re: Effective process GID is ignored when client creates
> > file on NFS
> >
> > ? Wed, 7 Nov 2012 14:13:36 -0500
> > "J. Bruce Fields" <[email protected]> ?????:
> >
> > > On Mon, Oct 29, 2012 at 06:09:29PM +0400, Andrey Borzenkov wrote:
> > > > I have met application that is badly broken when installed on
> > > > NFS. The reason is - it expects files to belong to specific
> > > > group. It switches to this group on startup (explicit setgid)
> > > > and creates files. But files come out as belonging to GID 0.
> > > >
> > > > I finally reduced it to this trivial script:
> > > >
> > > > === cut here ===
> > > > #include <sys/types.h>
> > > > #include <sys/stat.h>
> > > > #include <fcntl.h>
> > > > #include <unistd.h>
> > > >
> > > > main()
> > > > {
> > > > int fd;
> > > >
> > > > setgid(107);
> > > > fd = open("bar", O_CREAT, 0666);
> > > > close(fd);
> > > > }
> > > > === cut here ===
> > > >
> > > > On local storage file comes with GID 107; on NFS file comes
> > > > with GID 0.
> > > >
> > > > Linux is SLES10 SP3 with relatively old kernel:
> > > > 2.6.16.60-0.89.1-smp, server(s) are NetApp with different Data
> > > > ONTAP versions (7.x and 8.1.1 as the last).
> > > >
> > > > Client passes correct credentials (UID:0, GID:107), but does not
> > >
> > > Those are the credentials in the rpc header on the CREATE call?
> > >
> >
> > Yes.
> >
> > > > explicitly request file ownership in CREATE call (uid set_it -
> > > > 0, gid set_it - 0).
> > >
> > > The client shouldn't have to set the owner or group itself.
> > >
> > > So this is server behavior.
> > >
> > > Have you checked that the directory you're creating in doesn't
> > > have the sgid bit?
> >
> > Yes. It does not.
> >
> > > Or perhaps there's some other server configuration
> > > that causes this.
> >
> > It appears that server ignores passed group if UID == 0. It
> > correctly creates files if UID != 0. I got bug number from support
> > but it is non-public and no information is visible so far. I asked
> > about possible workaround (or undocumented options) but have not
> > got any reply as yet.
>
> So is it possible that you forgot to turn off root squashing on the
> server before testing?
>
All filesystems are exported with anon=0 in this configuration. But if
root suqashing is active, files should not appear as owned by UID 0,
should not they?
> -----Original Message-----
> From: [email protected] [mailto:linux-nfs-
> [email protected]] On Behalf Of Andrey Borzenkov
> Sent: Wednesday, November 07, 2012 2:28 PM
> To: J. Bruce Fields
> Cc: [email protected]
> Subject: Re: Effective process GID is ignored when client creates file on NFS
>
> ? Wed, 7 Nov 2012 14:13:36 -0500
> "J. Bruce Fields" <[email protected]> ?????:
>
> > On Mon, Oct 29, 2012 at 06:09:29PM +0400, Andrey Borzenkov wrote:
> > > I have met application that is badly broken when installed on NFS.
> > > The reason is - it expects files to belong to specific group. It
> > > switches to this group on startup (explicit setgid) and creates
> > > files. But files come out as belonging to GID 0.
> > >
> > > I finally reduced it to this trivial script:
> > >
> > > === cut here ===
> > > #include <sys/types.h>
> > > #include <sys/stat.h>
> > > #include <fcntl.h>
> > > #include <unistd.h>
> > >
> > > main()
> > > {
> > > int fd;
> > >
> > > setgid(107);
> > > fd = open("bar", O_CREAT, 0666);
> > > close(fd);
> > > }
> > > === cut here ===
> > >
> > > On local storage file comes with GID 107; on NFS file comes with GID
> > > 0.
> > >
> > > Linux is SLES10 SP3 with relatively old kernel:
> > > 2.6.16.60-0.89.1-smp, server(s) are NetApp with different Data ONTAP
> > > versions (7.x and 8.1.1 as the last).
> > >
> > > Client passes correct credentials (UID:0, GID:107), but does not
> >
> > Those are the credentials in the rpc header on the CREATE call?
> >
>
> Yes.
>
> > > explicitly request file ownership in CREATE call (uid set_it - 0,
> > > gid set_it - 0).
> >
> > The client shouldn't have to set the owner or group itself.
> >
> > So this is server behavior.
> >
> > Have you checked that the directory you're creating in doesn't have
> > the sgid bit?
>
> Yes. It does not.
>
> > Or perhaps there's some other server configuration that
> > causes this.
>
> It appears that server ignores passed group if UID == 0. It correctly creates
> files if UID != 0. I got bug number from support but it is non-public and no
> information is visible so far. I asked about possible workaround (or
> undocumented options) but have not got any reply as yet.
So is it possible that you forgot to turn off root squashing on the server before testing?
Cheers
Trond
On Thu, Nov 08, 2012 at 10:43:04AM +0400, Andrey Borzenkov wrote:
> OK this gave me idea. So the problem is indeed anon=0. If I reexport
> the same filesystems with root=... option, files are created with
> correct group ownership. I have to think about it, but on the first
> glance NetApp behavior is correct. As root= option is missing it maps
> root to anonymous user, and as anon=0 is present, this anonymous user
> is root. What do standards say about group ownership in this case?
As far as the NFS specs are concerned this sort of thing is up to the
implementation.
The behavior you describe sounds reasonable to me, and is similar to
what the Linux server would do, for example.
--b.