IPv6 packets without NEXTHDR_NONE extension header can make continuous
__skb_pull() until pskb_may_pull() failed in ipv6_gso_pull_exthdrs().
That results in a big value of skb_gro_offset(), and after __skb_push()
in ipv6_gro_receive(), skb->data will less than skb->head, an out of
bounds memory bug occurs. That will trigger the problem as following:
==================================================================
BUG: KASAN: use-after-free in eth_type_trans+0x100/0x260
...
Call trace:
dump_backtrace+0xd8/0x130
show_stack+0x1c/0x50
dump_stack_lvl+0x64/0x7c
print_address_description.constprop.0+0xbc/0x2e8
print_report+0x100/0x1e4
kasan_report+0x80/0x120
__asan_load8+0x78/0xa0
eth_type_trans+0x100/0x260
napi_gro_frags+0x164/0x550
tun_get_user+0xda4/0x1270
tun_chr_write_iter+0x74/0x130
do_iter_readv_writev+0x130/0x1ec
do_iter_write+0xbc/0x1e0
vfs_writev+0x13c/0x26c
Add comparison between skb->data - skb_gro_offset() and skb->head
and exception handler before __skb_push() to fix the bug.
Fixes: 86911732d399 ("gro: Avoid copying headers of unmerged packets")
Signed-off-by: Ziyang Xuan <[email protected]>
---
net/ipv6/ip6_offload.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
index 3ee345672849..6659ccf25387 100644
--- a/net/ipv6/ip6_offload.c
+++ b/net/ipv6/ip6_offload.c
@@ -237,6 +237,10 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
proto = ipv6_gso_pull_exthdrs(skb, proto);
skb_gro_pull(skb, -skb_transport_offset(skb));
skb_reset_transport_header(skb);
+ if (unlikely(skb_headroom(skb) < skb_gro_offset(skb))) {
+ kfree_skb(skb);
+ return ERR_PTR(-EINPROGRESS);
+ }
__skb_push(skb, skb_gro_offset(skb));
ops = rcu_dereference(inet6_offloads[proto]);
--
2.25.1
On Thu, Oct 27, 2022 at 3:25 AM Ziyang Xuan
<[email protected]> wrote:
>
> IPv6 packets without NEXTHDR_NONE extension header can make continuous
> __skb_pull() until pskb_may_pull() failed in ipv6_gso_pull_exthdrs().
> That results in a big value of skb_gro_offset(), and after __skb_push()
> in ipv6_gro_receive(), skb->data will less than skb->head, an out of
> bounds memory bug occurs. That will trigger the problem as following:
>
> ==================================================================
> BUG: KASAN: use-after-free in eth_type_trans+0x100/0x260
> ...
> Call trace:
> dump_backtrace+0xd8/0x130
> show_stack+0x1c/0x50
> dump_stack_lvl+0x64/0x7c
> print_address_description.constprop.0+0xbc/0x2e8
> print_report+0x100/0x1e4
> kasan_report+0x80/0x120
> __asan_load8+0x78/0xa0
> eth_type_trans+0x100/0x260
Crash happens from eth_type_trans() , this should happen before
ipv6_gro_receive() ?
It seems your patch is unrelated.
Please provide a repro.
> napi_gro_frags+0x164/0x550
> tun_get_user+0xda4/0x1270
> tun_chr_write_iter+0x74/0x130
> do_iter_readv_writev+0x130/0x1ec
> do_iter_write+0xbc/0x1e0
> vfs_writev+0x13c/0x26c
>
> Add comparison between skb->data - skb_gro_offset() and skb->head
> and exception handler before __skb_push() to fix the bug.
>
> Fixes: 86911732d399 ("gro: Avoid copying headers of unmerged packets")
> Signed-off-by: Ziyang Xuan <[email protected]>
> ---
> net/ipv6/ip6_offload.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
> index 3ee345672849..6659ccf25387 100644
> --- a/net/ipv6/ip6_offload.c
> +++ b/net/ipv6/ip6_offload.c
> @@ -237,6 +237,10 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
> proto = ipv6_gso_pull_exthdrs(skb, proto);
> skb_gro_pull(skb, -skb_transport_offset(skb));
> skb_reset_transport_header(skb);
> + if (unlikely(skb_headroom(skb) < skb_gro_offset(skb))) {
This makes no sense to me.
If there is a bug, it should be fixed earlier.
> + kfree_skb(skb);
> + return ERR_PTR(-EINPROGRESS);
> + }
> __skb_push(skb, skb_gro_offset(skb));
>
> ops = rcu_dereference(inet6_offloads[proto]);
> --
> 2.25.1
>
> On Thu, Oct 27, 2022 at 3:25 AM Ziyang Xuan
> <[email protected]> wrote:
>>
>> IPv6 packets without NEXTHDR_NONE extension header can make continuous
>> __skb_pull() until pskb_may_pull() failed in ipv6_gso_pull_exthdrs().
>> That results in a big value of skb_gro_offset(), and after __skb_push()
>> in ipv6_gro_receive(), skb->data will less than skb->head, an out of
>> bounds memory bug occurs. That will trigger the problem as following:
>>
>> ==================================================================
>> BUG: KASAN: use-after-free in eth_type_trans+0x100/0x260
>> ...
>> Call trace:
>> dump_backtrace+0xd8/0x130
>> show_stack+0x1c/0x50
>> dump_stack_lvl+0x64/0x7c
>> print_address_description.constprop.0+0xbc/0x2e8
>> print_report+0x100/0x1e4
>> kasan_report+0x80/0x120
>> __asan_load8+0x78/0xa0
>> eth_type_trans+0x100/0x260
>
> Crash happens from eth_type_trans() , this should happen before
> ipv6_gro_receive() ?
>
> It seems your patch is unrelated.
>
> Please provide a repro.
C repro put in attachment.
>
>
>> napi_gro_frags+0x164/0x550
>> tun_get_user+0xda4/0x1270
>> tun_chr_write_iter+0x74/0x130
>> do_iter_readv_writev+0x130/0x1ec
>> do_iter_write+0xbc/0x1e0
>> vfs_writev+0x13c/0x26c
>>
>> Add comparison between skb->data - skb_gro_offset() and skb->head
>> and exception handler before __skb_push() to fix the bug.
>>
>> Fixes: 86911732d399 ("gro: Avoid copying headers of unmerged packets")
>> Signed-off-by: Ziyang Xuan <[email protected]>
>> ---
>> net/ipv6/ip6_offload.c | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
>> index 3ee345672849..6659ccf25387 100644
>> --- a/net/ipv6/ip6_offload.c
>> +++ b/net/ipv6/ip6_offload.c
>> @@ -237,6 +237,10 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
>> proto = ipv6_gso_pull_exthdrs(skb, proto);
>> skb_gro_pull(skb, -skb_transport_offset(skb));
>> skb_reset_transport_header(skb);
>> + if (unlikely(skb_headroom(skb) < skb_gro_offset(skb))) {
>
> This makes no sense to me.
>
> If there is a bug, it should be fixed earlier.
Maybe it is good to validate IPv6 packet earlier in ipv6_gro_receive() or more earlier?
>
>> + kfree_skb(skb);
>> + return ERR_PTR(-EINPROGRESS);
>> + }
>> __skb_push(skb, skb_gro_offset(skb));
>>
>> ops = rcu_dereference(inet6_offloads[proto]);
>> --
>> 2.25.1
>>
> .
>
On Thu, Oct 27, 2022 at 6:01 AM Ziyang Xuan (William)
<[email protected]> wrote:
>
> > On Thu, Oct 27, 2022 at 3:25 AM Ziyang Xuan
> > <[email protected]> wrote:
> >>
> >> IPv6 packets without NEXTHDR_NONE extension header can make continuous
> >> __skb_pull() until pskb_may_pull() failed in ipv6_gso_pull_exthdrs().
> >> That results in a big value of skb_gro_offset(), and after __skb_push()
> >> in ipv6_gro_receive(), skb->data will less than skb->head, an out of
> >> bounds memory bug occurs. That will trigger the problem as following:
> >>
> >> ==================================================================
> >> BUG: KASAN: use-after-free in eth_type_trans+0x100/0x260
> >> ...
> >> Call trace:
> >> dump_backtrace+0xd8/0x130
> >> show_stack+0x1c/0x50
> >> dump_stack_lvl+0x64/0x7c
> >> print_address_description.constprop.0+0xbc/0x2e8
> >> print_report+0x100/0x1e4
> >> kasan_report+0x80/0x120
> >> __asan_load8+0x78/0xa0
> >> eth_type_trans+0x100/0x260
> >
> > Crash happens from eth_type_trans() , this should happen before
> > ipv6_gro_receive() ?
> >
> > It seems your patch is unrelated.
> >
> > Please provide a repro.
>
> C repro put in attachment.
This seems to be a bug in tun device.
Please take more time to root cause this issue, instead of adding work
arounds all over the place.
Thanks.
>
> >
> >
> >> napi_gro_frags+0x164/0x550
> >> tun_get_user+0xda4/0x1270
> >> tun_chr_write_iter+0x74/0x130
> >> do_iter_readv_writev+0x130/0x1ec
> >> do_iter_write+0xbc/0x1e0
> >> vfs_writev+0x13c/0x26c
> >>
> >> Add comparison between skb->data - skb_gro_offset() and skb->head
> >> and exception handler before __skb_push() to fix the bug.
> >>
> >> Fixes: 86911732d399 ("gro: Avoid copying headers of unmerged packets")
> >> Signed-off-by: Ziyang Xuan <[email protected]>
> >> ---
> >> net/ipv6/ip6_offload.c | 4 ++++
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
> >> index 3ee345672849..6659ccf25387 100644
> >> --- a/net/ipv6/ip6_offload.c
> >> +++ b/net/ipv6/ip6_offload.c
> >> @@ -237,6 +237,10 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
> >> proto = ipv6_gso_pull_exthdrs(skb, proto);
> >> skb_gro_pull(skb, -skb_transport_offset(skb));
> >> skb_reset_transport_header(skb);
> >> + if (unlikely(skb_headroom(skb) < skb_gro_offset(skb))) {
> >
> > This makes no sense to me.
> >
> > If there is a bug, it should be fixed earlier.
>
> Maybe it is good to validate IPv6 packet earlier in ipv6_gro_receive() or more earlier?
>
> >
> >> + kfree_skb(skb);
> >> + return ERR_PTR(-EINPROGRESS);
> >> + }
> >> __skb_push(skb, skb_gro_offset(skb));
> >>
> >> ops = rcu_dereference(inet6_offloads[proto]);
> >> --
> >> 2.25.1
> >>
> > .
> >
> On Thu, Oct 27, 2022 at 6:01 AM Ziyang Xuan (William)
> <[email protected]> wrote:
>>
>>> On Thu, Oct 27, 2022 at 3:25 AM Ziyang Xuan
>>> <[email protected]> wrote:
>>>>
>>>> IPv6 packets without NEXTHDR_NONE extension header can make continuous
>>>> __skb_pull() until pskb_may_pull() failed in ipv6_gso_pull_exthdrs().
>>>> That results in a big value of skb_gro_offset(), and after __skb_push()
>>>> in ipv6_gro_receive(), skb->data will less than skb->head, an out of
>>>> bounds memory bug occurs. That will trigger the problem as following:
>>>>
>>>> ==================================================================
>>>> BUG: KASAN: use-after-free in eth_type_trans+0x100/0x260
>>>> ...
>>>> Call trace:
>>>> dump_backtrace+0xd8/0x130
>>>> show_stack+0x1c/0x50
>>>> dump_stack_lvl+0x64/0x7c
>>>> print_address_description.constprop.0+0xbc/0x2e8
>>>> print_report+0x100/0x1e4
>>>> kasan_report+0x80/0x120
>>>> __asan_load8+0x78/0xa0
>>>> eth_type_trans+0x100/0x260
>>>
>>> Crash happens from eth_type_trans() , this should happen before
>>> ipv6_gro_receive() ?
>>>
>>> It seems your patch is unrelated.
>>>
>>> Please provide a repro.
>>
>> C repro put in attachment.
>
> This seems to be a bug in tun device.
>
> Please take more time to root cause this issue, instead of adding work
> arounds all over the place.
Hi Eric,
Thank you for your suggestion.
I have analyzed the problem more deeply. The odd IPv6 packet and
big packet length value(IPv6 payload length more than 65535)
together cause the problem.
skb->network_header and skb->transport_header are all u16 type.
They would occuer overflow errors during ipv6_gro_receive() processing.
That cause the value error for __skb_push(skb, value).
So the problem is a bug in tun device.
I will combine my previous problem "net: tun: limit first seg size to avoid oversized linearization"
together to give the fix patch later.
Thanks.
>
> Thanks.
>
>>
>>>
>>>
>>>> napi_gro_frags+0x164/0x550
>>>> tun_get_user+0xda4/0x1270
>>>> tun_chr_write_iter+0x74/0x130
>>>> do_iter_readv_writev+0x130/0x1ec
>>>> do_iter_write+0xbc/0x1e0
>>>> vfs_writev+0x13c/0x26c
>>>>
>>>> Add comparison between skb->data - skb_gro_offset() and skb->head
>>>> and exception handler before __skb_push() to fix the bug.
>>>>
>>>> Fixes: 86911732d399 ("gro: Avoid copying headers of unmerged packets")
>>>> Signed-off-by: Ziyang Xuan <[email protected]>
>>>> ---
>>>> net/ipv6/ip6_offload.c | 4 ++++
>>>> 1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
>>>> index 3ee345672849..6659ccf25387 100644
>>>> --- a/net/ipv6/ip6_offload.c
>>>> +++ b/net/ipv6/ip6_offload.c
>>>> @@ -237,6 +237,10 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
>>>> proto = ipv6_gso_pull_exthdrs(skb, proto);
>>>> skb_gro_pull(skb, -skb_transport_offset(skb));
>>>> skb_reset_transport_header(skb);
>>>> + if (unlikely(skb_headroom(skb) < skb_gro_offset(skb))) {
>>>
>>> This makes no sense to me.
>>>
>>> If there is a bug, it should be fixed earlier.
>>
>> Maybe it is good to validate IPv6 packet earlier in ipv6_gro_receive() or more earlier?
>>
>>>
>>>> + kfree_skb(skb);
>>>> + return ERR_PTR(-EINPROGRESS);
>>>> + }
>>>> __skb_push(skb, skb_gro_offset(skb));
>>>>
>>>> ops = rcu_dereference(inet6_offloads[proto]);
>>>> --
>>>> 2.25.1
>>>>
>>> .
>>>
> .
>
On Fri, Oct 28, 2022 at 3:11 AM Ziyang Xuan (William)
<[email protected]> wrote:
> Hi Eric,
>
> Thank you for your suggestion.
>
> I have analyzed the problem more deeply. The odd IPv6 packet and
> big packet length value(IPv6 payload length more than 65535)
> together cause the problem.
>
> skb->network_header and skb->transport_header are all u16 type.
> They would occuer overflow errors during ipv6_gro_receive() processing.
> That cause the value error for __skb_push(skb, value).
>
> So the problem is a bug in tun device.
>
> I will combine my previous problem "net: tun: limit first seg size to avoid oversized linearization"
> together to give the fix patch later.
SGTM, thanks !