2015-05-02 00:23:58

by Wolfgang Walter

[permalink] [raw]
Subject: kernel >= 4.0: crashes when using traceroute6 with isatap

Hello,

kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
isatap-tunnel.

I took an image of the message I got. It is not complete as my vt has not
enough lines.

3.19.3 works fine.

Regards,
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts


Attachments:
crash.jpeg (49.84 kB)

2015-05-02 12:29:42

by Wolfgang Walter

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> Hello,
>
> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
> isatap-tunnel.

I did some further tests. To trigger the crash you need

* isatap-tunnel (probably any sit-tunnel will do it)
* raw-socket
* udp

Using icmpv6 or tcp i.e. does not trigger it.

>
> I took an image of the message I got. It is not complete as my vt has not
> enough lines.
>
> 3.19.3 works fine.
>
> Regards,

Regards,
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts

2015-05-06 18:15:22

by Cong Wang

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

(Cc'ing netdev.)

On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
>> Hello,
>>
>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
>> isatap-tunnel.
>
> I did some further tests. To trigger the crash you need
>
> * isatap-tunnel (probably any sit-tunnel will do it)
> * raw-socket
> * udp
>
> Using icmpv6 or tcp i.e. does not trigger it.
>

Do you have a script to reproduce it?


Thanks for the bug report!

2015-05-06 19:10:05

by Eric Dumazet

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On Wed, 2015-05-06 at 11:15 -0700, Cong Wang wrote:
> (Cc'ing netdev.)
>
> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> > Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> >> Hello,
> >>
> >> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
> >> isatap-tunnel.
> >
> > I did some further tests. To trigger the crash you need
> >
> > * isatap-tunnel (probably any sit-tunnel will do it)
> > * raw-socket
> > * udp
> >
> > Using icmpv6 or tcp i.e. does not trigger it.
> >
>
> Do you have a script to reproduce it?
>
>
> Thanks for the bug report!
> --

Please Wolfgang try to revert 32dce968dd987adfb0c00946d78dad9154f64759
("ipv6: Allow for partial checksums on non-ufo packets")

2015-05-06 22:04:43

by Wolfgang Walter

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

Am Mittwoch, 6. Mai 2015, 12:10:00 schrieb Eric Dumazet:
> On Wed, 2015-05-06 at 11:15 -0700, Cong Wang wrote:
> > (Cc'ing netdev.)
> >
> > On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> > > Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> > >> Hello,
> > >>
> > >> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with
> > >> an
> > >> isatap-tunnel.
> > >
> > > I did some further tests. To trigger the crash you need
> > >
> > > * isatap-tunnel (probably any sit-tunnel will do it)
> > > * raw-socket
> > > * udp
> > >
> > > Using icmpv6 or tcp i.e. does not trigger it.
> >
> > Do you have a script to reproduce it?
> >
> >
> > Thanks for the bug report!
> > --
>
> Please Wolfgang try to revert 32dce968dd987adfb0c00946d78dad9154f64759
> ("ipv6: Allow for partial checksums on non-ufo packets")

Indeed, that fixes the problem.

Regards
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts

2015-05-06 22:11:25

by Wolfgang Walter

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

Am Mittwoch, 6. Mai 2015, 11:15:18 schrieben Sie:
> (Cc'ing netdev.)
>
> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> > Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> >> Hello,
> >>
> >> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
> >> isatap-tunnel.
> >
> > I did some further tests. To trigger the crash you need
> >
> > * isatap-tunnel (probably any sit-tunnel will do it)
> > * raw-socket
> > * udp
> >
> > Using icmpv6 or tcp i.e. does not trigger it.
>
> Do you have a script to reproduce it?
>
>
> Thanks for the bug report!

You need a isatap-server with say ipv4-address $X

Then, on host with 4.0, start isatapd: isatapd --mtu 1280 $X

then do

traceroute6 http://www.google.de

Regards,
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts

2015-05-06 22:42:42

by Eric Dumazet

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On Thu, 2015-05-07 at 00:04 +0200, Wolfgang Walter wrote:
> Am Mittwoch, 6. Mai 2015, 12:10:00 schrieb Eric Dumazet:
> > On Wed, 2015-05-06 at 11:15 -0700, Cong Wang wrote:
> > > (Cc'ing netdev.)
> > >
> > > On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> > > > Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> > > >> Hello,
> > > >>
> > > >> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with
> > > >> an
> > > >> isatap-tunnel.
> > > >
> > > > I did some further tests. To trigger the crash you need
> > > >
> > > > * isatap-tunnel (probably any sit-tunnel will do it)
> > > > * raw-socket
> > > > * udp
> > > >
> > > > Using icmpv6 or tcp i.e. does not trigger it.
> > >
> > > Do you have a script to reproduce it?
> > >
> > >
> > > Thanks for the bug report!
> > > --
> >
> > Please Wolfgang try to revert 32dce968dd987adfb0c00946d78dad9154f64759
> > ("ipv6: Allow for partial checksums on non-ufo packets")
>
> Indeed, that fixes the problem.

Yes, setting skb->csum to 0 is clearly wrong for CHECKSUM_PARTIAL

Would you try :

diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 7fde1f265c90e90f16291e6c861b6e242111c25b..694ae630e1ca67e25ab1e5f6dd0b3597db3669b0 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1416,6 +1416,10 @@ alloc_new_skb:
data += fragheaderlen;
skb->transport_header = (skb->network_header +
fragheaderlen);
+ if (csummode == CHECKSUM_PARTIAL) {
+ skb->csum_start = skb_transport_header(skb) - skb->head;
+ skb->csum_offset = offsetof(struct udphdr, check);
+ }
if (fraggap) {
skb->csum = skb_copy_and_csum_bits(
skb_prev, maxfraglen,


2015-05-07 00:04:51

by Wolfgang Walter

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

Am Mittwoch, 6. Mai 2015, 15:42:38 schrieben Sie:
> On Thu, 2015-05-07 at 00:04 +0200, Wolfgang Walter wrote:
> > Am Mittwoch, 6. Mai 2015, 12:10:00 schrieb Eric Dumazet:
> > > On Wed, 2015-05-06 at 11:15 -0700, Cong Wang wrote:
> > > > (Cc'ing netdev.)
> > > >
> > > > On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> > > > > Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> > > > >> Hello,
> > > > >>
> > > > >> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6
> > > > >> with
> > > > >> an
> > > > >> isatap-tunnel.
> > > > >
> > > > > I did some further tests. To trigger the crash you need
> > > > >
> > > > > * isatap-tunnel (probably any sit-tunnel will do it)
> > > > > * raw-socket
> > > > > * udp
> > > > >
> > > > > Using icmpv6 or tcp i.e. does not trigger it.
> > > >
> > > > Do you have a script to reproduce it?
> > > >
> > > >
> > > > Thanks for the bug report!
> > > > --
> > >
> > > Please Wolfgang try to revert 32dce968dd987adfb0c00946d78dad9154f64759
> > > ("ipv6: Allow for partial checksums on non-ufo packets")
> >
> > Indeed, that fixes the problem.
>
> Yes, setting skb->csum to 0 is clearly wrong for CHECKSUM_PARTIAL
>
> Would you try :
>
> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index
> 7fde1f265c90e90f16291e6c861b6e242111c25b..694ae630e1ca67e25ab1e5f6dd0b3597d
> b3669b0 100644 --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c
> @@ -1416,6 +1416,10 @@ alloc_new_skb:
> data += fragheaderlen;
> skb->transport_header = (skb->network_header +
> fragheaderlen);
> + if (csummode == CHECKSUM_PARTIAL) {
> + skb->csum_start = skb_transport_header(skb) - skb->head;
> + skb->csum_offset = offsetof(struct udphdr, check);
> + }
> if (fraggap) {
> skb->csum = skb_copy_and_csum_bits(
> skb_prev, maxfraglen,

This patch fixes 4.0.1 (and 4.0.2).

Thanks Eric,
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts

2015-05-12 16:34:07

by Vlad Yasevich

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On 05/06/2015 06:42 PM, Eric Dumazet wrote:
> On Thu, 2015-05-07 at 00:04 +0200, Wolfgang Walter wrote:
>> Am Mittwoch, 6. Mai 2015, 12:10:00 schrieb Eric Dumazet:
>>> On Wed, 2015-05-06 at 11:15 -0700, Cong Wang wrote:
>>>> (Cc'ing netdev.)
>>>>
>>>> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
>>>>> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
>>>>>> Hello,
>>>>>>
>>>>>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with
>>>>>> an
>>>>>> isatap-tunnel.
>>>>>
>>>>> I did some further tests. To trigger the crash you need
>>>>>
>>>>> * isatap-tunnel (probably any sit-tunnel will do it)
>>>>> * raw-socket
>>>>> * udp
>>>>>
>>>>> Using icmpv6 or tcp i.e. does not trigger it.
>>>>
>>>> Do you have a script to reproduce it?
>>>>
>>>>
>>>> Thanks for the bug report!
>>>> --
>>>
>>> Please Wolfgang try to revert 32dce968dd987adfb0c00946d78dad9154f64759
>>> ("ipv6: Allow for partial checksums on non-ufo packets")
>>
>> Indeed, that fixes the problem.
>
> Yes, setting skb->csum to 0 is clearly wrong for CHECKSUM_PARTIAL
>
> Would you try :
>
> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index 7fde1f265c90e90f16291e6c861b6e242111c25b..694ae630e1ca67e25ab1e5f6dd0b3597db3669b0 100644
> --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c
> @@ -1416,6 +1416,10 @@ alloc_new_skb:
> data += fragheaderlen;
> skb->transport_header = (skb->network_header +
> fragheaderlen);
> + if (csummode == CHECKSUM_PARTIAL) {
> + skb->csum_start = skb_transport_header(skb) - skb->head;
> + skb->csum_offset = offsetof(struct udphdr, check);
> + }
> if (fraggap) {
> skb->csum = skb_copy_and_csum_bits(
> skb_prev, maxfraglen,
>
>
>

So why is this not an issue in __ip_append_data()?

-vlad

2015-05-12 20:18:59

by Vlad Yasevich

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On 05/06/2015 06:11 PM, Wolfgang Walter wrote:
> Am Mittwoch, 6. Mai 2015, 11:15:18 schrieben Sie:
>> (Cc'ing netdev.)
>>
>> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
>>> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
>>>> Hello,
>>>>
>>>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
>>>> isatap-tunnel.
>>>
>>> I did some further tests. To trigger the crash you need
>>>
>>> * isatap-tunnel (probably any sit-tunnel will do it)
>>> * raw-socket
>>> * udp
>>>
>>> Using icmpv6 or tcp i.e. does not trigger it.
>>
>> Do you have a script to reproduce it?
>>
>>
>> Thanks for the bug report!
>
> You need a isatap-server with say ipv4-address $X
>
> Then, on host with 4.0, start isatapd: isatapd --mtu 1280 $X
>
> then do
>
> traceroute6 http://www.google.de
>
> Regards,
>

Hi Walter

Could you try this patch. Looks like raw passes transhdrlen
of 0 on the first packet and that makes IPv4 behave correctly,
but not IPv6.


diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 7fde1f2..fd9c079 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1278,7 +1278,7 @@ emsgsize:
/* If this is the first and only packet and device
* supports checksum offloading, let's use it.
*/
- if (!skb && sk->sk_protocol == IPPROTO_UDP &&
+ if (transhdrlen && sk->sk_protocol == IPPROTO_UDP &&
length + fragheaderlen < mtu &&
rt->dst.dev->features & NETIF_F_V6_CSUM &&
!exthdrlen)

2015-05-12 20:36:36

by Eric Dumazet

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On Tue, 2015-05-12 at 16:18 -0400, Vlad Yasevich wrote:
> On 05/06/2015 06:11 PM, Wolfgang Walter wrote:
> > Am Mittwoch, 6. Mai 2015, 11:15:18 schrieben Sie:
> >> (Cc'ing netdev.)
> >>
> >> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> >>> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> >>>> Hello,
> >>>>
> >>>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
> >>>> isatap-tunnel.
> >>>
> >>> I did some further tests. To trigger the crash you need
> >>>
> >>> * isatap-tunnel (probably any sit-tunnel will do it)
> >>> * raw-socket
> >>> * udp
> >>>
> >>> Using icmpv6 or tcp i.e. does not trigger it.
> >>
> >> Do you have a script to reproduce it?
> >>
> >>
> >> Thanks for the bug report!
> >
> > You need a isatap-server with say ipv4-address $X
> >
> > Then, on host with 4.0, start isatapd: isatapd --mtu 1280 $X
> >
> > then do
> >
> > traceroute6 http://www.google.de
> >
> > Regards,
> >
>
> Hi Walter
>
> Could you try this patch. Looks like raw passes transhdrlen
> of 0 on the first packet and that makes IPv4 behave correctly,
> but not IPv6.
>
>
> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index 7fde1f2..fd9c079 100644
> --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c
> @@ -1278,7 +1278,7 @@ emsgsize:
> /* If this is the first and only packet and device
> * supports checksum offloading, let's use it.
> */
> - if (!skb && sk->sk_protocol == IPPROTO_UDP &&
> + if (transhdrlen && sk->sk_protocol == IPPROTO_UDP &&
> length + fragheaderlen < mtu &&
> rt->dst.dev->features & NETIF_F_V6_CSUM &&
> !exthdrlen)

And make sure the checksum is correct ;)

Vlad, can you tell where skb->cum_start and skb->csum_offset are set ?


2015-05-12 20:46:35

by Vlad Yasevich

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

On 05/12/2015 04:36 PM, Eric Dumazet wrote:
> On Tue, 2015-05-12 at 16:18 -0400, Vlad Yasevich wrote:
>> On 05/06/2015 06:11 PM, Wolfgang Walter wrote:
>>> Am Mittwoch, 6. Mai 2015, 11:15:18 schrieben Sie:
>>>> (Cc'ing netdev.)
>>>>
>>>> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
>>>>> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
>>>>>> Hello,
>>>>>>
>>>>>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with an
>>>>>> isatap-tunnel.
>>>>>
>>>>> I did some further tests. To trigger the crash you need
>>>>>
>>>>> * isatap-tunnel (probably any sit-tunnel will do it)
>>>>> * raw-socket
>>>>> * udp
>>>>>
>>>>> Using icmpv6 or tcp i.e. does not trigger it.
>>>>
>>>> Do you have a script to reproduce it?
>>>>
>>>>
>>>> Thanks for the bug report!
>>>
>>> You need a isatap-server with say ipv4-address $X
>>>
>>> Then, on host with 4.0, start isatapd: isatapd --mtu 1280 $X
>>>
>>> then do
>>>
>>> traceroute6 http://www.google.de
>>>
>>> Regards,
>>>
>>
>> Hi Walter
>>
>> Could you try this patch. Looks like raw passes transhdrlen
>> of 0 on the first packet and that makes IPv4 behave correctly,
>> but not IPv6.
>>
>>
>> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
>> index 7fde1f2..fd9c079 100644
>> --- a/net/ipv6/ip6_output.c
>> +++ b/net/ipv6/ip6_output.c
>> @@ -1278,7 +1278,7 @@ emsgsize:
>> /* If this is the first and only packet and device
>> * supports checksum offloading, let's use it.
>> */
>> - if (!skb && sk->sk_protocol == IPPROTO_UDP &&
>> + if (transhdrlen && sk->sk_protocol == IPPROTO_UDP &&
>> length + fragheaderlen < mtu &&
>> rt->dst.dev->features & NETIF_F_V6_CSUM &&
>> !exthdrlen)
>
> And make sure the checksum is correct ;)
>
> Vlad, can you tell where skb->cum_start and skb->csum_offset are set ?
>
>

For udp, udp6_hwcsum_outgoing(), but with the above patch, the check above will return
false, and we'll fallback to using CHECKSUM_NONE.

Before, the !skb was true since there there was no skb on the queue. Now, that
we are checking transhdr and raw passing in 0, the check will be false. This
is what's making IPv4 work correctly in this case.

-vlad

2015-05-14 00:32:29

by Wolfgang Walter

[permalink] [raw]
Subject: Re: kernel >= 4.0: crashes when using traceroute6 with isatap

Am Dienstag, 12. Mai 2015, 16:18:54 schrieb Vlad Yasevich:
> On 05/06/2015 06:11 PM, Wolfgang Walter wrote:
> > Am Mittwoch, 6. Mai 2015, 11:15:18 schrieben Sie:
> >> (Cc'ing netdev.)
> >>
> >> On Sat, May 2, 2015 at 5:29 AM, Wolfgang Walter <[email protected]> wrote:
> >>> Am Samstag, 2. Mai 2015, 02:16:36 schrieb Wolfgang Walter:
> >>>> Hello,
> >>>>
> >>>> kernel 4.0 (and 4.0.1) crashes immediately when I use traceroute6 with
> >>>> an
> >>>> isatap-tunnel.
> >>>
> >>> I did some further tests. To trigger the crash you need
> >>>
> >>> * isatap-tunnel (probably any sit-tunnel will do it)
> >>> * raw-socket
> >>> * udp
> >>>
> >>> Using icmpv6 or tcp i.e. does not trigger it.
> >>
> >> Do you have a script to reproduce it?
> >>
> >>
> >> Thanks for the bug report!
> >
> > You need a isatap-server with say ipv4-address $X
> >
> > Then, on host with 4.0, start isatapd: isatapd --mtu 1280 $X
> >
> > then do
> >
> > traceroute6 http://www.google.de
> >
> > Regards,
>
> Hi Walter
>
> Could you try this patch. Looks like raw passes transhdrlen
> of 0 on the first packet and that makes IPv4 behave correctly,
> but not IPv6.
>
>
> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index 7fde1f2..fd9c079 100644
> --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c
> @@ -1278,7 +1278,7 @@ emsgsize:
> /* If this is the first and only packet and device
> * supports checksum offloading, let's use it.
> */
> - if (!skb && sk->sk_protocol == IPPROTO_UDP &&
> + if (transhdrlen && sk->sk_protocol == IPPROTO_UDP &&
> length + fragheaderlen < mtu &&
> rt->dst.dev->features & NETIF_F_V6_CSUM &&
> !exthdrlen)

Ok, that works as fix. No crash and checksums are ok.

Regards,
--
Wolfgang Walter
Studentenwerk M?nchen
Anstalt des ?ffentlichen Rechts