2018-10-31 12:50:40

by Leonard Crestez

[permalink] [raw]
Subject: [RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next

Since commit aa563d7bca6e ("iov_iter: Separate type from direction and
use accessor functions") the iov_iter_kvec and iov_iter_bvec functions
warn if the direction parameter contains anything other than READ/WRITE.

That commit also updated users of iov_iter_kvec/bvec but a new call was
added on a different branch by commit 277e4ab7d530 ("SUNRPC: Simplify
TCP receive code by switching to using iterators")

This causes a flood of warnings in linux-next like this:

------------[ cut here ]------------
WARNING: CPU: 0 PID: 110 at ../lib/iov_iter.c:1082 iov_iter_kvec+0x4c/0x5c
Modules linked in:
CPU: 0 PID: 110 Comm: kworker/u3:2 Tainted: G W 4.19.0-next-20181031 #157
Hardware name: Freescale i.MX6 SoloLite (Device Tree)
Workqueue: xprtiod xs_stream_data_receive_workfn
[<c0112908>] (unwind_backtrace) from [<c010d238>] (show_stack+0x10/0x14)
[<c010d238>] (show_stack) from [<c0b3894c>] (dump_stack+0xb0/0xe8)
[<c0b3894c>] (dump_stack) from [<c0126e14>] (__warn+0xe0/0x10c)
[<c0126e14>] (__warn) from [<c0126f54>] (warn_slowpath_null+0x3c/0x48)
[<c0126f54>] (warn_slowpath_null) from [<c04b225c>] (iov_iter_kvec+0x4c/0x5c)
[<c04b225c>] (iov_iter_kvec) from [<c0a2c650>] (xs_stream_data_receive_workfn+0x538/0x8e0)
[<c0a2c650>] (xs_stream_data_receive_workfn) from [<c014609c>] (process_one_work+0x2ac/0x6fc)
[<c014609c>] (process_one_work) from [<c014678c>] (worker_thread+0x2a0/0x574)
[<c014678c>] (worker_thread) from [<c014c43c>] (kthread+0x134/0x14c)
[<c014c43c>] (kthread) from [<c01010b4>] (ret_from_fork+0x14/0x20)
Exception stack(0xec9fbfb0 to 0xec9fbff8)
bfa0: 00000000 00000000 00000000 00000000
bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
bfe0: 00000000 00000000 00000000 00000000 00000013 00000000
irq event stamp: 91225
hardirqs last enabled at (91233): [<c0186630>] console_unlock+0x3e4/0x5d0
hardirqs last disabled at (91250): [<c01862c8>] console_unlock+0x7c/0x5d0
softirqs last enabled at (91266): [<c0102500>] __do_softirq+0x360/0x524
softirqs last disabled at (91277): [<c012e66c>] irq_exit+0xf8/0x1a4
---[ end trace bc12311e869d672a ]---

This fix updates sunrpc code and make nfs boot cleanly.

Signed-off-by: Leonard Crestez <[email protected]>
---
net/sunrpc/xprtsock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

You can treat this as a bug report; I did a brief search and didn't see
anyone else complain about this specific issue. Big ugly stacktrace was
included so that it hopefully shows up in searches.

Here is a mention of a different conflict between iov_iter and sunrpc:
https://lkml.org/lkml/2018/10/29/64

diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 1b51e04d3566..ae77c71c1f64 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -359,20 +359,20 @@ xs_sock_recvmsg(struct socket *sock, struct msghdr *msg, int flags, size_t seek)

static ssize_t
xs_read_kvec(struct socket *sock, struct msghdr *msg, int flags,
struct kvec *kvec, size_t count, size_t seek)
{
- iov_iter_kvec(&msg->msg_iter, READ | ITER_KVEC, kvec, 1, count);
+ iov_iter_kvec(&msg->msg_iter, READ, kvec, 1, count);
return xs_sock_recvmsg(sock, msg, flags, seek);
}

static ssize_t
xs_read_bvec(struct socket *sock, struct msghdr *msg, int flags,
struct bio_vec *bvec, unsigned long nr, size_t count,
size_t seek)
{
- iov_iter_bvec(&msg->msg_iter, READ | ITER_BVEC, bvec, nr, count);
+ iov_iter_bvec(&msg->msg_iter, READ, bvec, nr, count);
return xs_sock_recvmsg(sock, msg, flags, seek);
}

static ssize_t
xs_read_discard(struct socket *sock, struct msghdr *msg, int flags,
--
2.17.1



2018-10-31 14:32:32

by David Howells

[permalink] [raw]
Subject: Re: [RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next

Leonard Crestez <[email protected]> wrote:

> Since commit aa563d7bca6e ("iov_iter: Separate type from direction and
> use accessor functions") the iov_iter_kvec and iov_iter_bvec functions
> warn if the direction parameter contains anything other than READ/WRITE.
>
> That commit also updated users of iov_iter_kvec/bvec but a new call was
> added on a different branch by commit 277e4ab7d530 ("SUNRPC: Simplify
> TCP receive code by switching to using iterators")
>
> This causes a flood of warnings in linux-next like this:
>
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 110 at ../lib/iov_iter.c:1082 iov_iter_kvec+0x4c/0x5c
> Modules linked in:
> CPU: 0 PID: 110 Comm: kworker/u3:2 Tainted: G W 4.19.0-next-20181031 #157
> Hardware name: Freescale i.MX6 SoloLite (Device Tree)
> Workqueue: xprtiod xs_stream_data_receive_workfn
> [<c0112908>] (unwind_backtrace) from [<c010d238>] (show_stack+0x10/0x14)
> [<c010d238>] (show_stack) from [<c0b3894c>] (dump_stack+0xb0/0xe8)
> [<c0b3894c>] (dump_stack) from [<c0126e14>] (__warn+0xe0/0x10c)
> [<c0126e14>] (__warn) from [<c0126f54>] (warn_slowpath_null+0x3c/0x48)
> [<c0126f54>] (warn_slowpath_null) from [<c04b225c>] (iov_iter_kvec+0x4c/0x5c)
> [<c04b225c>] (iov_iter_kvec) from [<c0a2c650>] (xs_stream_data_receive_workfn+0x538/0x8e0)
> [<c0a2c650>] (xs_stream_data_receive_workfn) from [<c014609c>] (process_one_work+0x2ac/0x6fc)
> [<c014609c>] (process_one_work) from [<c014678c>] (worker_thread+0x2a0/0x574)
> [<c014678c>] (worker_thread) from [<c014c43c>] (kthread+0x134/0x14c)
> [<c014c43c>] (kthread) from [<c01010b4>] (ret_from_fork+0x14/0x20)
> Exception stack(0xec9fbfb0 to 0xec9fbff8)
> bfa0: 00000000 00000000 00000000 00000000
> bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> bfe0: 00000000 00000000 00000000 00000000 00000013 00000000
> irq event stamp: 91225
> hardirqs last enabled at (91233): [<c0186630>] console_unlock+0x3e4/0x5d0
> hardirqs last disabled at (91250): [<c01862c8>] console_unlock+0x7c/0x5d0
> softirqs last enabled at (91266): [<c0102500>] __do_softirq+0x360/0x524
> softirqs last disabled at (91277): [<c012e66c>] irq_exit+0xf8/0x1a4
> ---[ end trace bc12311e869d672a ]---
>
> This fix updates sunrpc code and make nfs boot cleanly.
>
> Signed-off-by: Leonard Crestez <[email protected]>

Reviewed-by: David Howells <[email protected]>

2018-10-31 23:34:00

by Stephen Rothwell

[permalink] [raw]
Subject: Re: [RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next

Hi all,

On Wed, 31 Oct 2018 12:49:52 +0000 Leonard Crestez <[email protected]> wrote:
>
> Since commit aa563d7bca6e ("iov_iter: Separate type from direction and
> use accessor functions") the iov_iter_kvec and iov_iter_bvec functions
> warn if the direction parameter contains anything other than READ/WRITE.
>
> That commit also updated users of iov_iter_kvec/bvec but a new call was
> added on a different branch by commit 277e4ab7d530 ("SUNRPC: Simplify
> TCP receive code by switching to using iterators")
>
> This causes a flood of warnings in linux-next like this:
>
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 110 at ../lib/iov_iter.c:1082 iov_iter_kvec+0x4c/0x5c
> Modules linked in:
> CPU: 0 PID: 110 Comm: kworker/u3:2 Tainted: G W 4.19.0-next-20181031 #157
> Hardware name: Freescale i.MX6 SoloLite (Device Tree)
> Workqueue: xprtiod xs_stream_data_receive_workfn
> [<c0112908>] (unwind_backtrace) from [<c010d238>] (show_stack+0x10/0x14)
> [<c010d238>] (show_stack) from [<c0b3894c>] (dump_stack+0xb0/0xe8)
> [<c0b3894c>] (dump_stack) from [<c0126e14>] (__warn+0xe0/0x10c)
> [<c0126e14>] (__warn) from [<c0126f54>] (warn_slowpath_null+0x3c/0x48)
> [<c0126f54>] (warn_slowpath_null) from [<c04b225c>] (iov_iter_kvec+0x4c/0x5c)
> [<c04b225c>] (iov_iter_kvec) from [<c0a2c650>] (xs_stream_data_receive_workfn+0x538/0x8e0)
> [<c0a2c650>] (xs_stream_data_receive_workfn) from [<c014609c>] (process_one_work+0x2ac/0x6fc)
> [<c014609c>] (process_one_work) from [<c014678c>] (worker_thread+0x2a0/0x574)
> [<c014678c>] (worker_thread) from [<c014c43c>] (kthread+0x134/0x14c)
> [<c014c43c>] (kthread) from [<c01010b4>] (ret_from_fork+0x14/0x20)
> Exception stack(0xec9fbfb0 to 0xec9fbff8)
> bfa0: 00000000 00000000 00000000 00000000
> bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> bfe0: 00000000 00000000 00000000 00000000 00000013 00000000
> irq event stamp: 91225
> hardirqs last enabled at (91233): [<c0186630>] console_unlock+0x3e4/0x5d0
> hardirqs last disabled at (91250): [<c01862c8>] console_unlock+0x7c/0x5d0
> softirqs last enabled at (91266): [<c0102500>] __do_softirq+0x360/0x524
> softirqs last disabled at (91277): [<c012e66c>] irq_exit+0xf8/0x1a4
> ---[ end trace bc12311e869d672a ]---
>
> This fix updates sunrpc code and make nfs boot cleanly.
>
> Signed-off-by: Leonard Crestez <[email protected]>
> ---
> net/sunrpc/xprtsock.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> You can treat this as a bug report; I did a brief search and didn't see
> anyone else complain about this specific issue. Big ugly stacktrace was
> included so that it hopefully shows up in searches.
>
> Here is a mention of a different conflict between iov_iter and sunrpc:
> https://lkml.org/lkml/2018/10/29/64
>
> diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
> index 1b51e04d3566..ae77c71c1f64 100644
> --- a/net/sunrpc/xprtsock.c
> +++ b/net/sunrpc/xprtsock.c
> @@ -359,20 +359,20 @@ xs_sock_recvmsg(struct socket *sock, struct msghdr *msg, int flags, size_t seek)
>
> static ssize_t
> xs_read_kvec(struct socket *sock, struct msghdr *msg, int flags,
> struct kvec *kvec, size_t count, size_t seek)
> {
> - iov_iter_kvec(&msg->msg_iter, READ | ITER_KVEC, kvec, 1, count);
> + iov_iter_kvec(&msg->msg_iter, READ, kvec, 1, count);
> return xs_sock_recvmsg(sock, msg, flags, seek);
> }
>
> static ssize_t
> xs_read_bvec(struct socket *sock, struct msghdr *msg, int flags,
> struct bio_vec *bvec, unsigned long nr, size_t count,
> size_t seek)
> {
> - iov_iter_bvec(&msg->msg_iter, READ | ITER_BVEC, bvec, nr, count);
> + iov_iter_bvec(&msg->msg_iter, READ, bvec, nr, count);
> return xs_sock_recvmsg(sock, msg, flags, seek);
> }
>
> static ssize_t
> xs_read_discard(struct socket *sock, struct msghdr *msg, int flags,
> --
> 2.17.1

I have added that as a merge resolution for the merge of the vfs tree
in linux-next today. Please remember to tell Linus about it.

--
Cheers,
Stephen Rothwell


Attachments:
(No filename) (499.00 B)
OpenPGP digital signature

2018-11-02 00:18:25

by Chen, Rong A

[permalink] [raw]
Subject: [LKP] [sunrpc] 6a7da2a288: kernel_BUG_at_lib/iov_iter.c

FYI, we noticed the following commit (built with gcc-7):

commit: 6a7da2a288ce412d7ac117a2912a7b0d9104ee6d ("[RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next")
url: https://github.com/0day-ci/linux/commits/Leonard-Crestez/sunrpc-Fix-flood-of-warnings-from-iov_iter_kvec-in-linux-next/20181101-070713
base: git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next

in testcase: boot

on test machine: qemu-system-x86_64 -enable-kvm -cpu kvm64,+ssse3 -smp 2 -m 8G

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):


+------------------------------------------------------------------------------------------+------------+------------+
| | 86bbd7422a | 6a7da2a288 |
+------------------------------------------------------------------------------------------+------------+------------+
| boot_successes | 8 | 0 |
| boot_failures | 6 | 35 |
| Kernel_panic-not_syncing:initrd_too_large_to_handle,disabling_initrd(#needed,#available) | 6 | 4 |
| kernel_BUG_at_lib/iov_iter.c | 0 | 31 |
| invalid_opcode:#[##] | 0 | 31 |
| RIP:iov_iter_kvec | 0 | 31 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 31 |
+------------------------------------------------------------------------------------------+------------+------------+



[ 103.400523] kernel BUG at lib/iov_iter.c:1067!
[ 103.415315] invalid opcode: 0000 [#1] SMP PTI
[ 103.425743] CPU: 0 PID: 358 Comm: kworker/u5:1 Not tainted 4.19.0-rc6-00095-g6a7da2a #1
[ 103.444219] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 103.464595] Workqueue: xprtiod xs_stream_data_receive_workfn
[ 103.477431] RIP: 0010:iov_iter_kvec+0x1d/0x20
[ 103.487834] Code: 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 40 f6 c6 02 74 17 89 37 48 89 57 18 48 89 4f 20 48 c7 47 08 00 00 00 00 4c 89 47 10 c3 <0f> 0b 90 40 f6 c6 04 74 17 89 37 48 89 57 18 48 89 4f 20 48 c7 47
[ 103.529675] RSP: 0000:ffffb760411a7dd0 EFLAGS: 00010246
[ 103.541990] RAX: ffff8902d4405d20 RBX: ffffb760411a7e00 RCX: 0000000000000001
[ 103.559385] RDX: ffffb760411a7df0 RSI: 0000000000000000 RDI: ffffb760411a7e10
[ 103.576569] RBP: 0000000000000000 R08: 000000000000000c R09: 8080808080808080
[ 103.593182] R10: 0000000000000000 R11: fefefefefefefeff R12: ffff8902d4405800
[ 103.609806] R13: ffff8902d8538c00 R14: 0000000000000000 R15: 000000000000000c
[ 103.626499] FS: 0000000000000000(0000) GS:ffff89033fc00000(0000) knlGS:0000000000000000
[ 103.645847] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 103.658781] CR2: 00005574746cf808 CR3: 00000001d8952000 CR4: 00000000000006f0
[ 103.674923] Call Trace:
[ 103.681009] xs_stream_data_receive+0xbb/0x4d0
[ 103.691127] ? __switch_to_asm+0x34/0x70
[ 103.700388] process_one_work+0x196/0x3a0
[ 103.710075] worker_thread+0x3c/0x3a0
[ 103.718542] ? process_one_work+0x3a0/0x3a0
[ 103.727843] kthread+0x11e/0x140
[ 103.735419] ? kthread_create_worker_on_cpu+0x70/0x70
[ 103.747120] ret_from_fork+0x35/0x40
[ 103.755907] Modules linked in: ppdev snd_pcm bochs_drm snd_timer ttm drm_kms_helper snd soundcore virtio_pci joydev drm virtio_ring pcspkr serio_raw virtio i6300esb ata_generic i2c_piix4 pata_acpi parport_pc parport floppy qemu_fw_cfg
[ 103.803280] ---[ end trace b47e3b48d3a08066 ]---


To reproduce:

git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email



Thanks,
lkp


Attachments:
(No filename) (4.05 kB)
config-4.19.0-rc6-00095-g6a7da2a (201.17 kB)
job-script (4.31 kB)
dmesg.xz (13.47 kB)
Download all attachments

2018-11-02 00:50:59

by David Howells

[permalink] [raw]
Subject: Re: [LKP] [sunrpc] 6a7da2a288: kernel_BUG_at_lib/iov_iter.c

kernel test robot <[email protected]> wrote:

> FYI, we noticed the following commit (built with gcc-7):
>
> commit: 6a7da2a288ce412d7ac117a2912a7b0d9104ee6d ("[RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next")
> url: https://github.com/0day-ci/linux/commits/Leonard-Crestez/sunrpc-Fix-flood-of-warnings-from-iov_iter_kvec-in-linux-next/20181101-070713
> base: git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next
>
> in testcase: boot
>
> on test machine: qemu-system-x86_64 -enable-kvm -cpu kvm64,+ssse3 -smp 2 -m 8G
>
> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):

Ummm... You can't just apply that commit to Trond's linux-next branch unless
that branch also includes the iov_iter changes from my afs-next branch.

Before those changes, ITER_KVEC is required:

BUG_ON(!(direction & ITER_KVEC));

and after, it will be prohibited:

WARN_ON(direction & ~(READ | WRITE));

The reason for this is that have yet more patches that split the direction
from the iov_iter::type member into their own member and turn the types into a
simple integer sequence instead of a bit mask.

David

2018-11-03 15:50:33

by Leonard Crestez

[permalink] [raw]
Subject: Re: [LKP] [sunrpc] 6a7da2a288: kernel_BUG_at_lib/iov_iter.c

On 11/2/2018 2:50 AM, David Howells wrote:
> kernel test robot <[email protected]> wrote:
>
>> FYI, we noticed the following commit (built with gcc-7):
>>
>> commit: 6a7da2a288ce412d7ac117a2912a7b0d9104ee6d ("[RFC] sunrpc: Fix flood of warnings from iov_iter_kvec in linux-next")
>> url: https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-ci%2Flinux%2Fcommits%2FLeonard-Crestez%2Fsunrpc-Fix-flood-of-warnings-from-iov_iter_kvec-in-linux-next%2F20181101-070713&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7C0b22a6d58a5242277ca008d6405d28b3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636767166190134069&amp;sdata=335WGS8H6GRfqp%2BDmocDQ8mowhe7a0RxKbfM6rSgJ3g%3D&amp;reserved=0
>> base: git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next
>>
>> in testcase: boot
>>
>> on test machine: qemu-system-x86_64 -enable-kvm -cpu kvm64,+ssse3 -smp 2 -m 8G
>>
>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>
> Ummm... You can't just apply that commit to Trond's linux-next branch unless
> that branch also includes the iov_iter changes from my afs-next branch.
>
> Before those changes, ITER_KVEC is required:
>
> BUG_ON(!(direction & ITER_KVEC));
>
> and after, it will be prohibited:
>
> WARN_ON(direction & ~(READ | WRITE));
>
> The reason for this is that have yet more patches that split the direction
> from the iov_iter::type member into their own member and turn the types into a
> simple integer sequence instead of a bit mask.

It's not clear how such conflicts should be handled. Maybe patches from
one tree should be pulled into the other earlier so that merging "just
works"?

--
Regards,
Leonard