2015-05-15 14:24:24

by Russell King - ARM Linux

[permalink] [raw]
Subject: NFS client broken in 4.1.0-rc2

While trying to update a kernel and modules on one of my test systems,
I was greeted by these errors:

tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/dvb-frontends/drx39xyj/drx39xyj.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/usb/em28xx/em28xx.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/option.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/ftdi_sio.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/net/wireless/brcm80211/brcmfmac/brcmfmac.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/drivers/input/mouse/psmouse.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/fs/udf/udf.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/fs/fuse/fuse.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/fs/nfsd/nfsd.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/sound/soc/codecs/snd-soc-wm8962.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/kernel/net/bluetooth/bluetooth.ko: Cannot utime
tar: lib/modules/4.1.0-rc2+/modules.alias.bin: Cannot utime
tar: lib/modules/4.1.0-rc2+/modules.alias: Cannot utime
tar: Exiting with failure status due to previous errors

Searching google wasn't helpful, as all the "Cannot utime" errors that
google could find are followed by an errno string.

stracing at first sight didn't seem to be helpful, as no syscalls (apart
from openat() with a pre-existing file) were failing.

Having recently updated to fc21 tar generating the archive, I thought
maybe it was a tar format bug between fc21 tar and the target's tar.
That was until I tried to "apt-get source tar" on the target, and was
greeted by the same error.

So I then tried untaring the tar source archive onto a ramfs, which
worked without complaint. The difference being that it's a root NFS
box, and so I was untaring onto NFS.

Here's the entry from /proc/mounts:

x.y.z.221:/var/boot/ci on / type nfs (rw,nolock,vers=4,addr=x.y.z.221,clientaddr=a.b.c.55)

Looking closer at the strace reveals this:

openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
unlinkat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", 0) = 0
openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
...
write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
dup2(4, 4) = 4
fstat64(4, {st_mode=0757221, st_size=13181880119170311768, ...}) = 21
write(2, "tar: ", 5) = 5
write(2, "lib/modules/4.1.0-rc2+/kernel/dr"..., 79) = 79
write(2, "\n", 1) = 1
fchown32(4, 0, 0) = 0
fchmod(4, 0664) = 0
close(4) = 0

Look closely at that fstat64, and you'll notice that it's returning crap.
The file is not 11 exabytes, and it definitely would not have an octal
mode of 0757221 at this point, having only just been created by the
kernel.

For comparison, untaring onto a ramfs filesystem gives this:

openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
...
write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
dup2(4, 4) = 4
fstat64(4, {st_mode=S_IFREG|0600, st_size=83088, ...}) = 0
utimensat(4, NULL, {{1431698625, 21832730}, {1431694673, 0}}, 0) = 0
fchown32(4, 0, 0) = 0
fchmod(4, 0664) = 0
close(4) = 0

The reason for the strange dup2() above is this code in tar:

/* Require that at least one of FD or FILE are valid. Works around
a Linux bug where futimens (AT_FDCWD, NULL) changes "." rather
than failing. */
if (!file)
{
if (fd < 0)
{
errno = EBADF;
return -1;
}
if (dup2 (fd, fd) != fd)
return -1;
}

The call path in tar is:

fdutimensat (fd, dir, file, ts, atflag)
`-futimens (fd, ts)
`-fdutimens (fd, NULL, ts);

I'm assuming that the reason for this fstat() call is:

# if __linux__
/* As recently as Linux kernel 2.6.32 (Dec 2009), several file
systems (xfs, ntfs-3g) have bugs with a single UTIME_OMIT,
but work if both times are either explicitly specified or
UTIME_NOW. Work around it with a preparatory [f]stat prior
to calling futimens/utimensat; fortunately, there is not much
timing impact due to the extra syscall even on file systems
where UTIME_OMIT would have worked. FIXME: Simplify this in
2012, when file system bugs are no longer common. */
if (adjustment_needed == 2)
{
if (fd < 0 ? stat (file, &st) : fstat (fd, &st))
return -1;
if (ts[0].tv_nsec == UTIME_OMIT)
ts[0] = get_stat_atime (&st);
else if (ts[1].tv_nsec == UTIME_OMIT)
ts[1] = get_stat_mtime (&st);
/* Note that st is good, in case utimensat gives ENOSYS. */
adjustment_needed++;
}
# endif /* __linux__ */
# if HAVE_UTIMENSAT
if (fd < 0)
{
result = utimensat (AT_FDCWD, file, ts, 0);
# ifdef __linux__
/* Work around a kernel bug:
http://bugzilla.redhat.com/442352
http://bugzilla.redhat.com/449910
It appears that utimensat can mistakenly return 280 rather
than -1 upon ENOSYS failure.
FIXME: remove in 2010 or whenever the offending kernels
are no longer in common use. */
if (0 < result)
errno = ENOSYS;
# endif /* __linux__ */
if (result == 0 || errno != ENOSYS)
{
utimensat_works_really = 1;
return result;
}
}
# endif /* HAVE_UTIMENSAT */

--
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.


2015-05-15 14:27:47

by Chuck Lever III

[permalink] [raw]
Subject: Re: NFS client broken in 4.1.0-rc2


On May 15, 2015, at 10:24 AM, Russell King - ARM Linux <[email protected]> wrote:

> While trying to update a kernel and modules on one of my test systems,
> I was greeted by these errors:
>
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/dvb-frontends/drx39xyj/drx39xyj.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/usb/em28xx/em28xx.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/option.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/ftdi_sio.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/net/wireless/brcm80211/brcmfmac/brcmfmac.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/drivers/input/mouse/psmouse.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/fs/udf/udf.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/fs/fuse/fuse.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/fs/nfsd/nfsd.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/sound/soc/codecs/snd-soc-wm8962.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/kernel/net/bluetooth/bluetooth.ko: Cannot utime
> tar: lib/modules/4.1.0-rc2+/modules.alias.bin: Cannot utime
> tar: lib/modules/4.1.0-rc2+/modules.alias: Cannot utime
> tar: Exiting with failure status due to previous errors
>
> Searching google wasn't helpful, as all the "Cannot utime" errors that
> google could find are followed by an errno string.
>
> stracing at first sight didn't seem to be helpful, as no syscalls (apart
> from openat() with a pre-existing file) were failing.
>
> Having recently updated to fc21 tar generating the archive, I thought
> maybe it was a tar format bug between fc21 tar and the target's tar.
> That was until I tried to "apt-get source tar" on the target, and was
> greeted by the same error.
>
> So I then tried untaring the tar source archive onto a ramfs, which
> worked without complaint. The difference being that it's a root NFS
> box, and so I was untaring onto NFS.
>
> Here's the entry from /proc/mounts:
>
> x.y.z.221:/var/boot/ci on / type nfs (rw,nolock,vers=4,addr=x.y.z.221,clientaddr=a.b.c.55)
>
> Looking closer at the strace reveals this:
>
> openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
> unlinkat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", 0) = 0
> openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
> write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> ...
> write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
> dup2(4, 4) = 4
> fstat64(4, {st_mode=0757221, st_size=13181880119170311768, ...}) = 21
> write(2, "tar: ", 5) = 5
> write(2, "lib/modules/4.1.0-rc2+/kernel/dr"..., 79) = 79
> write(2, "\n", 1) = 1
> fchown32(4, 0, 0) = 0
> fchmod(4, 0664) = 0
> close(4) = 0
>
> Look closely at that fstat64, and you'll notice that it's returning crap.

This is likely fixed by:

http://marc.info/?l=linux-nfs&m=143095122604344&w=2


> The file is not 11 exabytes, and it definitely would not have an octal
> mode of 0757221 at this point, having only just been created by the
> kernel.
>
> For comparison, untaring onto a ramfs filesystem gives this:
>
> openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
> write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> ...
> write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
> dup2(4, 4) = 4
> fstat64(4, {st_mode=S_IFREG|0600, st_size=83088, ...}) = 0
> utimensat(4, NULL, {{1431698625, 21832730}, {1431694673, 0}}, 0) = 0
> fchown32(4, 0, 0) = 0
> fchmod(4, 0664) = 0
> close(4) = 0
>
> The reason for the strange dup2() above is this code in tar:
>
> /* Require that at least one of FD or FILE are valid. Works around
> a Linux bug where futimens (AT_FDCWD, NULL) changes "." rather
> than failing. */
> if (!file)
> {
> if (fd < 0)
> {
> errno = EBADF;
> return -1;
> }
> if (dup2 (fd, fd) != fd)
> return -1;
> }
>
> The call path in tar is:
>
> fdutimensat (fd, dir, file, ts, atflag)
> `-futimens (fd, ts)
> `-fdutimens (fd, NULL, ts);
>
> I'm assuming that the reason for this fstat() call is:
>
> # if __linux__
> /* As recently as Linux kernel 2.6.32 (Dec 2009), several file
> systems (xfs, ntfs-3g) have bugs with a single UTIME_OMIT,
> but work if both times are either explicitly specified or
> UTIME_NOW. Work around it with a preparatory [f]stat prior
> to calling futimens/utimensat; fortunately, there is not much
> timing impact due to the extra syscall even on file systems
> where UTIME_OMIT would have worked. FIXME: Simplify this in
> 2012, when file system bugs are no longer common. */
> if (adjustment_needed == 2)
> {
> if (fd < 0 ? stat (file, &st) : fstat (fd, &st))
> return -1;
> if (ts[0].tv_nsec == UTIME_OMIT)
> ts[0] = get_stat_atime (&st);
> else if (ts[1].tv_nsec == UTIME_OMIT)
> ts[1] = get_stat_mtime (&st);
> /* Note that st is good, in case utimensat gives ENOSYS. */
> adjustment_needed++;
> }
> # endif /* __linux__ */
> # if HAVE_UTIMENSAT
> if (fd < 0)
> {
> result = utimensat (AT_FDCWD, file, ts, 0);
> # ifdef __linux__
> /* Work around a kernel bug:
> http://bugzilla.redhat.com/442352
> http://bugzilla.redhat.com/449910
> It appears that utimensat can mistakenly return 280 rather
> than -1 upon ENOSYS failure.
> FIXME: remove in 2010 or whenever the offending kernels
> are no longer in common use. */
> if (0 < result)
> errno = ENOSYS;
> # endif /* __linux__ */
> if (result == 0 || errno != ENOSYS)
> {
> utimensat_works_really = 1;
> return result;
> }
> }
> # endif /* HAVE_UTIMENSAT */

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com


2015-05-15 14:50:31

by Russell King - ARM Linux

[permalink] [raw]
Subject: Re: NFS client broken in 4.1.0-rc2

On Fri, May 15, 2015 at 10:26:48AM -0400, Chuck Lever wrote:
>
> On May 15, 2015, at 10:24 AM, Russell King - ARM Linux <[email protected]> wrote:
>
> > While trying to update a kernel and modules on one of my test systems,
> > I was greeted by these errors:
> >
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/dvb-frontends/drx39xyj/drx39xyj.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/usb/em28xx/em28xx.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/option.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/ftdi_sio.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/net/wireless/brcm80211/brcmfmac/brcmfmac.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/drivers/input/mouse/psmouse.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/fs/udf/udf.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/fs/fuse/fuse.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/fs/nfsd/nfsd.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/sound/soc/codecs/snd-soc-wm8962.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/kernel/net/bluetooth/bluetooth.ko: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/modules.alias.bin: Cannot utime
> > tar: lib/modules/4.1.0-rc2+/modules.alias: Cannot utime
> > tar: Exiting with failure status due to previous errors
> >
> > Searching google wasn't helpful, as all the "Cannot utime" errors that
> > google could find are followed by an errno string.
> >
> > stracing at first sight didn't seem to be helpful, as no syscalls (apart
> > from openat() with a pre-existing file) were failing.
> >
> > Having recently updated to fc21 tar generating the archive, I thought
> > maybe it was a tar format bug between fc21 tar and the target's tar.
> > That was until I tried to "apt-get source tar" on the target, and was
> > greeted by the same error.
> >
> > So I then tried untaring the tar source archive onto a ramfs, which
> > worked without complaint. The difference being that it's a root NFS
> > box, and so I was untaring onto NFS.
> >
> > Here's the entry from /proc/mounts:
> >
> > x.y.z.221:/var/boot/ci on / type nfs (rw,nolock,vers=4,addr=x.y.z.221,clientaddr=a.b.c.55)
> >
> > Looking closer at the strace reveals this:
> >
> > openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
> > unlinkat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", 0) = 0
> > openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
> > write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> > ...
> > write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
> > dup2(4, 4) = 4
> > fstat64(4, {st_mode=0757221, st_size=13181880119170311768, ...}) = 21
> > write(2, "tar: ", 5) = 5
> > write(2, "lib/modules/4.1.0-rc2+/kernel/dr"..., 79) = 79
> > write(2, "\n", 1) = 1
> > fchown32(4, 0, 0) = 0
> > fchmod(4, 0664) = 0
> > close(4) = 0
> >
> > Look closely at that fstat64, and you'll notice that it's returning crap.
>
> This is likely fixed by:
>
> http://marc.info/?l=linux-nfs&m=143095122604344&w=2

Yes, it appears to be fixed by that change. I'll add it to my tree so
my test systems work.

Many thanks.

--
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

2015-05-20 23:22:45

by Russell King - ARM Linux

[permalink] [raw]
Subject: Re: NFS client broken in 4.1.0-rc2

On Fri, May 15, 2015 at 03:50:14PM +0100, Russell King - ARM Linux wrote:
> On Fri, May 15, 2015 at 10:26:48AM -0400, Chuck Lever wrote:
> > On May 15, 2015, at 10:24 AM, Russell King - ARM Linux <[email protected]> wrote:
> > > While trying to update a kernel and modules on one of my test systems,
> > > I was greeted by these errors:
> > >
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/dvb-frontends/drx39xyj/drx39xyj.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/usb/em28xx/em28xx.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/option.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/ftdi_sio.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/net/wireless/brcm80211/brcmfmac/brcmfmac.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/drivers/input/mouse/psmouse.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/fs/udf/udf.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/fs/fuse/fuse.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/fs/nfsd/nfsd.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/sound/soc/codecs/snd-soc-wm8962.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/kernel/net/bluetooth/bluetooth.ko: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/modules.alias.bin: Cannot utime
> > > tar: lib/modules/4.1.0-rc2+/modules.alias: Cannot utime
> > > tar: Exiting with failure status due to previous errors
> > >
> > > Searching google wasn't helpful, as all the "Cannot utime" errors that
> > > google could find are followed by an errno string.
> > >
> > > stracing at first sight didn't seem to be helpful, as no syscalls (apart
> > > from openat() with a pre-existing file) were failing.
> > >
> > > Having recently updated to fc21 tar generating the archive, I thought
> > > maybe it was a tar format bug between fc21 tar and the target's tar.
> > > That was until I tried to "apt-get source tar" on the target, and was
> > > greeted by the same error.
> > >
> > > So I then tried untaring the tar source archive onto a ramfs, which
> > > worked without complaint. The difference being that it's a root NFS
> > > box, and so I was untaring onto NFS.
> > >
> > > Here's the entry from /proc/mounts:
> > >
> > > x.y.z.221:/var/boot/ci on / type nfs (rw,nolock,vers=4,addr=x.y.z.221,clientaddr=a.b.c.55)
> > >
> > > Looking closer at the strace reveals this:
> > >
> > > openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
> > > unlinkat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", 0) = 0
> > > openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
> > > write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> > > ...
> > > write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
> > > dup2(4, 4) = 4
> > > fstat64(4, {st_mode=0757221, st_size=13181880119170311768, ...}) = 21
> > > write(2, "tar: ", 5) = 5
> > > write(2, "lib/modules/4.1.0-rc2+/kernel/dr"..., 79) = 79
> > > write(2, "\n", 1) = 1
> > > fchown32(4, 0, 0) = 0
> > > fchmod(4, 0664) = 0
> > > close(4) = 0
> > >
> > > Look closely at that fstat64, and you'll notice that it's returning crap.
> >
> > This is likely fixed by:
> >
> > http://marc.info/?l=linux-nfs&m=143095122604344&w=2
>
> Yes, it appears to be fixed by that change. I'll add it to my tree so
> my test systems work.

I notice NFS patches were merged by Linus recently, but this patch was not
in the set. Is this patch scheduled to go in for 4.1?

--
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

2015-05-21 01:16:54

by Chuck Lever III

[permalink] [raw]
Subject: Re: NFS client broken in 4.1.0-rc2


On May 20, 2015, at 7:22 PM, Russell King - ARM Linux <[email protected]> wrote:

> On Fri, May 15, 2015 at 03:50:14PM +0100, Russell King - ARM Linux wrote:
>> On Fri, May 15, 2015 at 10:26:48AM -0400, Chuck Lever wrote:
>>> On May 15, 2015, at 10:24 AM, Russell King - ARM Linux <[email protected]> wrote:
>>>> While trying to update a kernel and modules on one of my test systems,
>>>> I was greeted by these errors:
>>>>
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/dvb-frontends/drx39xyj/drx39xyj.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/media/usb/em28xx/em28xx.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/option.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/usb/serial/ftdi_sio.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/net/wireless/brcm80211/brcmfmac/brcmfmac.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/drivers/input/mouse/psmouse.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/fs/udf/udf.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/fs/fuse/fuse.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/fs/nfsd/nfsd.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/sound/soc/codecs/snd-soc-wm8962.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/kernel/net/bluetooth/bluetooth.ko: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/modules.alias.bin: Cannot utime
>>>> tar: lib/modules/4.1.0-rc2+/modules.alias: Cannot utime
>>>> tar: Exiting with failure status due to previous errors
>>>>
>>>> Searching google wasn't helpful, as all the "Cannot utime" errors that
>>>> google could find are followed by an errno string.
>>>>
>>>> stracing at first sight didn't seem to be helpful, as no syscalls (apart
>>>> from openat() with a pre-existing file) were failing.
>>>>
>>>> Having recently updated to fc21 tar generating the archive, I thought
>>>> maybe it was a tar format bug between fc21 tar and the target's tar.
>>>> That was until I tried to "apt-get source tar" on the target, and was
>>>> greeted by the same error.
>>>>
>>>> So I then tried untaring the tar source archive onto a ramfs, which
>>>> worked without complaint. The difference being that it's a root NFS
>>>> box, and so I was untaring onto NFS.
>>>>
>>>> Here's the entry from /proc/mounts:
>>>>
>>>> x.y.z.221:/var/boot/ci on / type nfs (rw,nolock,vers=4,addr=x.y.z.221,clientaddr=a.b.c.55)
>>>>
>>>> Looking closer at the strace reveals this:
>>>>
>>>> openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
>>>> unlinkat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", 0) = 0
>>>> openat(AT_FDCWD, "lib/modules/4.1.0-rc2+/kernel/drivers/media/platform/coda/coda.ko", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC, 0600) = 4
>>>> write(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\1\0(\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
>>>> ...
>>>> write(4, "\300H\0\0\34\345\1\0\314H\0\0\34\345\1\0\330H\0\0\34\345\1\0<I\0\0\34\370\1\0"..., 7312) = 7312
>>>> dup2(4, 4) = 4
>>>> fstat64(4, {st_mode=0757221, st_size=13181880119170311768, ...}) = 21
>>>> write(2, "tar: ", 5) = 5
>>>> write(2, "lib/modules/4.1.0-rc2+/kernel/dr"..., 79) = 79
>>>> write(2, "\n", 1) = 1
>>>> fchown32(4, 0, 0) = 0
>>>> fchmod(4, 0664) = 0
>>>> close(4) = 0
>>>>
>>>> Look closely at that fstat64, and you'll notice that it's returning crap.
>>>
>>> This is likely fixed by:
>>>
>>> http://marc.info/?l=linux-nfs&m=143095122604344&w=2
>>
>> Yes, it appears to be fixed by that change. I'll add it to my tree so
>> my test systems work.
>
> I notice NFS patches were merged by Linus recently, but this patch was not
> in the set. Is this patch scheduled to go in for 4.1?

Trond sent Linus a pull request yesterday for 2 patches, including
the one you are asking about. See:

http://marc.info/?l=linux-nfs&m=143205093309624&w=2

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com