vhost_worker will call tun call backs to receive packets. If too many
illegal packets arrives, tun_do_read will keep dumping packet contents.
When console is enabled, it will costs much more cpu time to dump
packet and soft lockup will be detected.
net_ratelimit mechanism can be used to limit the dumping rate.
PID: 33036 TASK: ffff949da6f20000 CPU: 23 COMMAND: "vhost-32980"
#0 [fffffe00003fce50] crash_nmi_callback at ffffffff89249253
#1 [fffffe00003fce58] nmi_handle at ffffffff89225fa3
#2 [fffffe00003fceb0] default_do_nmi at ffffffff8922642e
#3 [fffffe00003fced0] do_nmi at ffffffff8922660d
#4 [fffffe00003fcef0] end_repeat_nmi at ffffffff89c01663
[exception RIP: io_serial_in+20]
RIP: ffffffff89792594 RSP: ffffa655314979e8 RFLAGS: 00000002
RAX: ffffffff89792500 RBX: ffffffff8af428a0 RCX: 0000000000000000
RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff8af428a0
RBP: 0000000000002710 R8: 0000000000000004 R9: 000000000000000f
R10: 0000000000000000 R11: ffffffff8acbf64f R12: 0000000000000020
R13: ffffffff8acbf698 R14: 0000000000000058 R15: 0000000000000000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#5 [ffffa655314979e8] io_serial_in at ffffffff89792594
#6 [ffffa655314979e8] wait_for_xmitr at ffffffff89793470
#7 [ffffa65531497a08] serial8250_console_putchar at ffffffff897934f6
#8 [ffffa65531497a20] uart_console_write at ffffffff8978b605
#9 [ffffa65531497a48] serial8250_console_write at ffffffff89796558
#10 [ffffa65531497ac8] console_unlock at ffffffff89316124
#11 [ffffa65531497b10] vprintk_emit at ffffffff89317c07
#12 [ffffa65531497b68] printk at ffffffff89318306
#13 [ffffa65531497bc8] print_hex_dump at ffffffff89650765
#14 [ffffa65531497ca8] tun_do_read at ffffffffc0b06c27 [tun]
#15 [ffffa65531497d38] tun_recvmsg at ffffffffc0b06e34 [tun]
#16 [ffffa65531497d68] handle_rx at ffffffffc0c5d682 [vhost_net]
#17 [ffffa65531497ed0] vhost_worker at ffffffffc0c644dc [vhost]
#18 [ffffa65531497f10] kthread at ffffffff892d2e72
#19 [ffffa65531497f50] ret_from_fork at ffffffff89c0022f
Fixes: ef3db4a59542 (\"tun: avoid BUG, dump packet on GSO errors\")
Signed-off-by: Lei Chen <[email protected]>
Reviewed-by: Willem de Bruijn <[email protected]>
Suggested-by: Paolo Abeni <[email protected]>
Acked-by: Jason Wang <[email protected]>
Changes from v2:
https://lore.kernel.org/netdev/[email protected]/
1. Add net-dev to patch subject-prefix.
2. Add fix tag.
Changes from v1:
https://lore.kernel.org/all/[email protected]/
1. Use net_ratelimit instead of raw __ratelimit.
2. Use netdev_err instead of pr_err to print more info abort net dev.
3. Adjust git commit message to make git am happy.
---
drivers/net/tun.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 0b3f21cba552..ca9b4bc89de7 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -2125,14 +2125,16 @@ static ssize_t tun_put_user(struct tun_struct *tun,
tun_is_little_endian(tun), true,
vlan_hlen)) {
struct skb_shared_info *sinfo = skb_shinfo(skb);
- pr_err("unexpected GSO type: "
- "0x%x, gso_size %d, hdr_len %d\n",
- sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size),
- tun16_to_cpu(tun, gso.hdr_len));
- print_hex_dump(KERN_ERR, "tun: ",
- DUMP_PREFIX_NONE,
- 16, 1, skb->head,
- min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true);
+
+ if (net_ratelimit()) {
+ netdev_err(tun->dev, "unexpected GSO type: 0x%x, gso_size %d, hdr_len %d\n",
+ sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size),
+ tun16_to_cpu(tun, gso.hdr_len));
+ print_hex_dump(KERN_ERR, "tun: ",
+ DUMP_PREFIX_NONE,
+ 16, 1, skb->head,
+ min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true);
+ }
WARN_ON_ONCE(1);
return -EINVAL;
}
--
2.44.0
Lei Chen wrote:
> vhost_worker will call tun call backs to receive packets. If too many
> illegal packets arrives, tun_do_read will keep dumping packet contents.
> When console is enabled, it will costs much more cpu time to dump
> packet and soft lockup will be detected.
>
> net_ratelimit mechanism can be used to limit the dumping rate.
>
> PID: 33036 TASK: ffff949da6f20000 CPU: 23 COMMAND: "vhost-32980"
> #0 [fffffe00003fce50] crash_nmi_callback at ffffffff89249253
> #1 [fffffe00003fce58] nmi_handle at ffffffff89225fa3
> #2 [fffffe00003fceb0] default_do_nmi at ffffffff8922642e
> #3 [fffffe00003fced0] do_nmi at ffffffff8922660d
> #4 [fffffe00003fcef0] end_repeat_nmi at ffffffff89c01663
> [exception RIP: io_serial_in+20]
> RIP: ffffffff89792594 RSP: ffffa655314979e8 RFLAGS: 00000002
> RAX: ffffffff89792500 RBX: ffffffff8af428a0 RCX: 0000000000000000
> RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff8af428a0
> RBP: 0000000000002710 R8: 0000000000000004 R9: 000000000000000f
> R10: 0000000000000000 R11: ffffffff8acbf64f R12: 0000000000000020
> R13: ffffffff8acbf698 R14: 0000000000000058 R15: 0000000000000000
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #5 [ffffa655314979e8] io_serial_in at ffffffff89792594
> #6 [ffffa655314979e8] wait_for_xmitr at ffffffff89793470
> #7 [ffffa65531497a08] serial8250_console_putchar at ffffffff897934f6
> #8 [ffffa65531497a20] uart_console_write at ffffffff8978b605
> #9 [ffffa65531497a48] serial8250_console_write at ffffffff89796558
> #10 [ffffa65531497ac8] console_unlock at ffffffff89316124
> #11 [ffffa65531497b10] vprintk_emit at ffffffff89317c07
> #12 [ffffa65531497b68] printk at ffffffff89318306
> #13 [ffffa65531497bc8] print_hex_dump at ffffffff89650765
> #14 [ffffa65531497ca8] tun_do_read at ffffffffc0b06c27 [tun]
> #15 [ffffa65531497d38] tun_recvmsg at ffffffffc0b06e34 [tun]
> #16 [ffffa65531497d68] handle_rx at ffffffffc0c5d682 [vhost_net]
> #17 [ffffa65531497ed0] vhost_worker at ffffffffc0c644dc [vhost]
> #18 [ffffa65531497f10] kthread at ffffffff892d2e72
> #19 [ffffa65531497f50] ret_from_fork at ffffffff89c0022f
>
> Fixes: ef3db4a59542 (\"tun: avoid BUG, dump packet on GSO errors\")
Nit: no escaped parentheses in the Fixes string.
Since there is this small issue, another: target is tun, not net:tun
> Signed-off-by: Lei Chen <[email protected]>
> Reviewed-by: Willem de Bruijn <[email protected]>
> Suggested-by: Paolo Abeni <[email protected]>
> Acked-by: Jason Wang <[email protected]>
>
> Changes from v2:
> https://lore.kernel.org/netdev/[email protected]/
> 1. Add net-dev to patch subject-prefix.
> 2. Add fix tag.
>
> Changes from v1:
> https://lore.kernel.org/all/[email protected]/
> 1. Use net_ratelimit instead of raw __ratelimit.
> 2. Use netdev_err instead of pr_err to print more info abort net dev.
> 3. Adjust git commit message to make git am happy.
> ---
> drivers/net/tun.c | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> index 0b3f21cba552..ca9b4bc89de7 100644
> --- a/drivers/net/tun.c
> +++ b/drivers/net/tun.c
> @@ -2125,14 +2125,16 @@ static ssize_t tun_put_user(struct tun_struct *tun,
> tun_is_little_endian(tun), true,
> vlan_hlen)) {
> struct skb_shared_info *sinfo = skb_shinfo(skb);
> - pr_err("unexpected GSO type: "
> - "0x%x, gso_size %d, hdr_len %d\n",
> - sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size),
> - tun16_to_cpu(tun, gso.hdr_len));
> - print_hex_dump(KERN_ERR, "tun: ",
> - DUMP_PREFIX_NONE,
> - 16, 1, skb->head,
> - min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true);
> +
> + if (net_ratelimit()) {
> + netdev_err(tun->dev, "unexpected GSO type: 0x%x, gso_size %d, hdr_len %d\n",
> + sinfo->gso_type, tun16_to_cpu(tun, gso.gso_size),
> + tun16_to_cpu(tun, gso.hdr_len));
> + print_hex_dump(KERN_ERR, "tun: ",
> + DUMP_PREFIX_NONE,
> + 16, 1, skb->head,
> + min((int)tun16_to_cpu(tun, gso.hdr_len), 64), true);
> + }
> WARN_ON_ONCE(1);
> return -EINVAL;
> }
> --
> 2.44.0
>
On Fri, 2024-04-12 at 02:58 -0400, Lei Chen wrote:
> vhost_worker will call tun call backs to receive packets. If too many
> illegal packets arrives, tun_do_read will keep dumping packet contents.
> When console is enabled, it will costs much more cpu time to dump
> packet and soft lockup will be detected.
>
> net_ratelimit mechanism can be used to limit the dumping rate.
>
> PID: 33036 TASK: ffff949da6f20000 CPU: 23 COMMAND: "vhost-32980"
> #0 [fffffe00003fce50] crash_nmi_callback at ffffffff89249253
> #1 [fffffe00003fce58] nmi_handle at ffffffff89225fa3
> #2 [fffffe00003fceb0] default_do_nmi at ffffffff8922642e
> #3 [fffffe00003fced0] do_nmi at ffffffff8922660d
> #4 [fffffe00003fcef0] end_repeat_nmi at ffffffff89c01663
> [exception RIP: io_serial_in+20]
> RIP: ffffffff89792594 RSP: ffffa655314979e8 RFLAGS: 00000002
> RAX: ffffffff89792500 RBX: ffffffff8af428a0 RCX: 0000000000000000
> RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff8af428a0
> RBP: 0000000000002710 R8: 0000000000000004 R9: 000000000000000f
> R10: 0000000000000000 R11: ffffffff8acbf64f R12: 0000000000000020
> R13: ffffffff8acbf698 R14: 0000000000000058 R15: 0000000000000000
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #5 [ffffa655314979e8] io_serial_in at ffffffff89792594
> #6 [ffffa655314979e8] wait_for_xmitr at ffffffff89793470
> #7 [ffffa65531497a08] serial8250_console_putchar at ffffffff897934f6
> #8 [ffffa65531497a20] uart_console_write at ffffffff8978b605
> #9 [ffffa65531497a48] serial8250_console_write at ffffffff89796558
> #10 [ffffa65531497ac8] console_unlock at ffffffff89316124
> #11 [ffffa65531497b10] vprintk_emit at ffffffff89317c07
> #12 [ffffa65531497b68] printk at ffffffff89318306
> #13 [ffffa65531497bc8] print_hex_dump at ffffffff89650765
> #14 [ffffa65531497ca8] tun_do_read at ffffffffc0b06c27 [tun]
> #15 [ffffa65531497d38] tun_recvmsg at ffffffffc0b06e34 [tun]
> #16 [ffffa65531497d68] handle_rx at ffffffffc0c5d682 [vhost_net]
> #17 [ffffa65531497ed0] vhost_worker at ffffffffc0c644dc [vhost]
> #18 [ffffa65531497f10] kthread at ffffffff892d2e72
> #19 [ffffa65531497f50] ret_from_fork at ffffffff89c0022f
>
> Fixes: ef3db4a59542 (\"tun: avoid BUG, dump packet on GSO errors\")
> Signed-off-by: Lei Chen <[email protected]>
> Reviewed-by: Willem de Bruijn <[email protected]>
> Suggested-by: Paolo Abeni <[email protected]>
> Acked-by: Jason Wang <[email protected]>
You should insert the '---' separator here, just after the tag area, so
that the changelog below will not be included into the commit message.
Additionally, I have a terrible memory, but I don't recall suggesting
this patch !?? ;) You can drop my tag, thanks.
Paolo