2019-10-16 08:04:41

by David Miller

[permalink] [raw]
Subject: Re: [PATCH v2 net-next] net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

From: Alexander Lobakin <[email protected]>
Date: Mon, 14 Oct 2019 11:00:33 +0300

> Commit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORMAL
> skbs") made use of listified skb processing for the users of
> napi_gro_frags().
> The same technique can be used in a way more common napi_gro_receive()
> to speed up non-merged (GRO_NORMAL) skbs for a wide range of drivers
> including gro_cells and mac80211 users.
> This slightly changes the return value in cases where skb is being
> dropped by the core stack, but it seems to have no impact on related
> drivers' functionality.
> gro_normal_batch is left untouched as it's very individual for every
> single system configuration and might be tuned in manual order to
> achieve an optimal performance.
>
> Signed-off-by: Alexander Lobakin <[email protected]>
> Acked-by: Edward Cree <[email protected]>

Applied, thank you.


2019-10-16 12:14:17

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v2 net-next] net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

David Miller wrote 16.10.2019 04:16:
> From: Alexander Lobakin <[email protected]>
> Date: Mon, 14 Oct 2019 11:00:33 +0300
>
>> Commit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORMAL
>> skbs") made use of listified skb processing for the users of
>> napi_gro_frags().
>> The same technique can be used in a way more common napi_gro_receive()
>> to speed up non-merged (GRO_NORMAL) skbs for a wide range of drivers
>> including gro_cells and mac80211 users.
>> This slightly changes the return value in cases where skb is being
>> dropped by the core stack, but it seems to have no impact on related
>> drivers' functionality.
>> gro_normal_batch is left untouched as it's very individual for every
>> single system configuration and might be tuned in manual order to
>> achieve an optimal performance.
>>
>> Signed-off-by: Alexander Lobakin <[email protected]>
>> Acked-by: Edward Cree <[email protected]>
>
> Applied, thank you.

David, Edward, Eric, Ilias,
thank you for your time.

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

2019-11-25 07:33:45

by Nicholas Johnson

[permalink] [raw]
Subject: Re: [PATCH v2 net-next] net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

Hi,

On Wed, Oct 16, 2019 at 10:31:31AM +0300, Alexander Lobakin wrote:
> David Miller wrote 16.10.2019 04:16:
> > From: Alexander Lobakin <[email protected]>
> > Date: Mon, 14 Oct 2019 11:00:33 +0300
> >
> > > Commit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORMAL
> > > skbs") made use of listified skb processing for the users of
> > > napi_gro_frags().
> > > The same technique can be used in a way more common napi_gro_receive()
> > > to speed up non-merged (GRO_NORMAL) skbs for a wide range of drivers
> > > including gro_cells and mac80211 users.
> > > This slightly changes the return value in cases where skb is being
> > > dropped by the core stack, but it seems to have no impact on related
> > > drivers' functionality.
> > > gro_normal_batch is left untouched as it's very individual for every
> > > single system configuration and might be tuned in manual order to
> > > achieve an optimal performance.
> > >
> > > Signed-off-by: Alexander Lobakin <[email protected]>
> > > Acked-by: Edward Cree <[email protected]>
> >
> > Applied, thank you.
>
> David, Edward, Eric, Ilias,
> thank you for your time.
>
> Regards,
> ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

I am very sorry to be the bearer of bad news. It appears that this
commit is causing a regression in Linux 5.4.0-rc8-next-20191122,
preventing me from connecting to Wi-Fi networks. I have a Dell XPS 9370
(Intel Core i7-8650U) with Intel Wireless 8265 [8086:24fd].

I did a bisect, and this commit was named the culprit. I then applied
the reverse patch on another clone of Linux next-20191122, and it
started working.

6570bc79c0dfff0f228b7afd2de720fb4e84d61d
net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

You can see more at the bug report I filed at [0].

[0]
https://bugzilla.kernel.org/show_bug.cgi?id=205647

I called on others at [0] to try to reproduce this - you should not pull
a patch because of a single reporter - as I could be wrong.

Please let me know if you want me to give more debugging information or
test any potential fixes. I am happy to help to fix this. :)

Kind regards,
Nicholas Johnson

2019-11-25 07:57:22

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v2 net-next] net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

Nicholas Johnson wrote 25.11.2019 10:29:
> Hi,
>
> On Wed, Oct 16, 2019 at 10:31:31AM +0300, Alexander Lobakin wrote:
>> David Miller wrote 16.10.2019 04:16:
>> > From: Alexander Lobakin <[email protected]>
>> > Date: Mon, 14 Oct 2019 11:00:33 +0300
>> >
>> > > Commit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORMAL
>> > > skbs") made use of listified skb processing for the users of
>> > > napi_gro_frags().
>> > > The same technique can be used in a way more common napi_gro_receive()
>> > > to speed up non-merged (GRO_NORMAL) skbs for a wide range of drivers
>> > > including gro_cells and mac80211 users.
>> > > This slightly changes the return value in cases where skb is being
>> > > dropped by the core stack, but it seems to have no impact on related
>> > > drivers' functionality.
>> > > gro_normal_batch is left untouched as it's very individual for every
>> > > single system configuration and might be tuned in manual order to
>> > > achieve an optimal performance.
>> > >
>> > > Signed-off-by: Alexander Lobakin <[email protected]>
>> > > Acked-by: Edward Cree <[email protected]>
>> >
>> > Applied, thank you.
>>
>> David, Edward, Eric, Ilias,
>> thank you for your time.
>>
>> Regards,
>> ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ
>
> I am very sorry to be the bearer of bad news. It appears that this
> commit is causing a regression in Linux 5.4.0-rc8-next-20191122,
> preventing me from connecting to Wi-Fi networks. I have a Dell XPS 9370
> (Intel Core i7-8650U) with Intel Wireless 8265 [8086:24fd].

Hi!

It's a bit strange as this commit doesn't directly affect the packet
flow. I don't have any iwlwifi hardware at the moment, so let's see if
anyone else will be able to reproduce this (for now, it is the first
report in a ~6 weeks after applying to net-next).
Anyway, I'll investigate iwlwifi's Rx processing -- maybe I could find
something driver-specific that might produce this.

Thank you for the report.

> I did a bisect, and this commit was named the culprit. I then applied
> the reverse patch on another clone of Linux next-20191122, and it
> started working.
>
> 6570bc79c0dfff0f228b7afd2de720fb4e84d61d
> net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()
>
> You can see more at the bug report I filed at [0].
>
> [0]
> https://bugzilla.kernel.org/show_bug.cgi?id=205647
>
> I called on others at [0] to try to reproduce this - you should not
> pull
> a patch because of a single reporter - as I could be wrong.
>
> Please let me know if you want me to give more debugging information or
> test any potential fixes. I am happy to help to fix this. :)
>
> Kind regards,
> Nicholas Johnson

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

2019-11-25 08:27:37

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v2 net-next] net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()

Alexander Lobakin wrote 25.11.2019 10:54:
> Nicholas Johnson wrote 25.11.2019 10:29:
>> Hi,
>>
>> On Wed, Oct 16, 2019 at 10:31:31AM +0300, Alexander Lobakin wrote:
>>> David Miller wrote 16.10.2019 04:16:
>>> > From: Alexander Lobakin <[email protected]>
>>> > Date: Mon, 14 Oct 2019 11:00:33 +0300
>>> >
>>> > > Commit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORMAL
>>> > > skbs") made use of listified skb processing for the users of
>>> > > napi_gro_frags().
>>> > > The same technique can be used in a way more common napi_gro_receive()
>>> > > to speed up non-merged (GRO_NORMAL) skbs for a wide range of drivers
>>> > > including gro_cells and mac80211 users.
>>> > > This slightly changes the return value in cases where skb is being
>>> > > dropped by the core stack, but it seems to have no impact on related
>>> > > drivers' functionality.
>>> > > gro_normal_batch is left untouched as it's very individual for every
>>> > > single system configuration and might be tuned in manual order to
>>> > > achieve an optimal performance.
>>> > >
>>> > > Signed-off-by: Alexander Lobakin <[email protected]>
>>> > > Acked-by: Edward Cree <[email protected]>
>>> >
>>> > Applied, thank you.
>>>
>>> David, Edward, Eric, Ilias,
>>> thank you for your time.
>>>
>>> Regards,
>>> ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ
>>
>> I am very sorry to be the bearer of bad news. It appears that this
>> commit is causing a regression in Linux 5.4.0-rc8-next-20191122,
>> preventing me from connecting to Wi-Fi networks. I have a Dell XPS
>> 9370
>> (Intel Core i7-8650U) with Intel Wireless 8265 [8086:24fd].
>
> Hi!
>
> It's a bit strange as this commit doesn't directly affect the packet
> flow. I don't have any iwlwifi hardware at the moment, so let's see if
> anyone else will be able to reproduce this (for now, it is the first
> report in a ~6 weeks after applying to net-next).
> Anyway, I'll investigate iwlwifi's Rx processing -- maybe I could find
> something driver-specific that might produce this.
>
> Thank you for the report.
>
>> I did a bisect, and this commit was named the culprit. I then applied
>> the reverse patch on another clone of Linux next-20191122, and it
>> started working.
>>
>> 6570bc79c0dfff0f228b7afd2de720fb4e84d61d
>> net: core: use listified Rx for GRO_NORMAL in napi_gro_receive()
>>
>> You can see more at the bug report I filed at [0].
>>
>> [0]
>> https://bugzilla.kernel.org/show_bug.cgi?id=205647
>>
>> I called on others at [0] to try to reproduce this - you should not
>> pull
>> a patch because of a single reporter - as I could be wrong.
>>
>> Please let me know if you want me to give more debugging information
>> or
>> test any potential fixes. I am happy to help to fix this. :)

And you can also set /proc/sys/net/core/gro_normal_batch to the value
of 1 and see if there are any changes. This value makes GRO stack to
behave just like without the patch.

>> Kind regards,
>> Nicholas Johnson
>
> Regards,
> ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ