2013-08-06 13:51:37

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 8:36 AM, Stephen Rothwell <[email protected]> wrote:
> Hi all,
>
> Changes since 20130805:
>
> The ext4 tree still has its build failure so I used the version from
> next-20130726.
>
> The akpm tree gained a conflict against the drm-intel tree.
>
> ----------------------------------------------------------------------------
>

Just FYI:
I cannot connect to my WLAN-AP, not sure what's wrong.

I pulled in latest (for-john) mac80211(-next) and iwlwifi-(fixes|next)
into next-20130806 which did not help.

OK, I did not try with debugging enabled and/or w/o Network-Manager.

If you know of an issue, please let me know.

- Sedat -


Attachments:
dmesg_3.11.0-rc4-next20130806-2-iniza-small.txt (108.08 kB)
config-3.11.0-rc4-next20130806-2-iniza-small (112.86 kB)
Download all attachments

2013-08-06 21:58:52

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 23:45 +0200, Johannes Berg wrote:

> I think skb->protocol is probably getting set up wrong, and just putting
> back the last two lines
>
> skb->protocol = proto;
> skb->dev = dev;
>
> is probably sufficient to fix wifi. If skb->protocol isn't set to
> ETH_P_PAE, then we'd drop the packet in the wifi stack - might be worth
> printing out what it's set to at the point where the skb->protocol
> assignment above was removed.
>
> I'm trying to wrap my head around all this right now but I don't yet see
> how the code after the patch would not get skb->protocol correct.

Actually, I think that's not it, but the code now behaves totally
differently?

Say this is the frame data with two points marked:

(1) (2)
| dst | src | ethtype | ... |

As I understand it (in my admittedly rather tired state), previously we
had

skb_network_header() == (1)
skb_mac_header() == (1)
skb->data == (1)

After calling eth_type_trans(), we get

skb_network_header() == (1)
skb_mac_header() == (1)
skb->data == (2)

I think? Maybe I'm totally confused though.

johannes


2013-08-06 19:32:28

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 9:18 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 21:14 +0200, Sedat Dilek wrote:
>> On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
>> > On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
>> >
>> >> Attached is a diff comparing all new commits in next-20130805.
>> >> If one of the commits smells bad to you, please let me know.
>> >
>> > Out of that list, only the af_packet changes would seem to have any
>> > impact on wireless at all.
>> >
>>
>> git-bisecting... 2 steps to go...
>>
>> This one is bad... "af_packet: simplify VLAN frame check in packet_snd"
>>
>> http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc
>
> That seems weird, does reverting it fix it?
>

[ TO Phil Sutter ]

This was 3/3 of af_packet patches :-).

So, the culprit commit is...

0f75b09c798ed00c30d7d5551b896be883bc2aeb is the first bad commit
commit 0f75b09c798ed00c30d7d5551b896be883bc2aeb
Author: Phil Sutter <[email protected]>
Date: Fri Aug 2 11:37:39 2013 +0200

af_packet: when sending ethernet frames, parse header for skb->protocol

This may be necessary when the SKB is passed to other layers on the go,
which check the protocol field on their own. An example is a VLAN packet
sent out using AF_PACKET on a bridge interface. The bridging code checks
the SKB size, accounting for any VLAN header only if the protocol field
is set accordingly.

Note that eth_type_trans() sets skb->dev to the passed argument, so this
can be skipped in packet_snd() for ethernet frames, as well.

Signed-off-by: Phil Sutter <[email protected]>
Signed-off-by: David S. Miller <[email protected]>

:040000 040000 af403a20a321517f6cfb51d2e22c17ca5a60e947
1f302ebd62a87b9e874a3e61203499e17d6fce3c M net

- Sedat -


Attachments:
git-bisect-log.txt (1.79 kB)
git-bisect-visualize.txt (786.00 B)
Download all attachments

2013-08-06 19:18:47

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 21:14 +0200, Sedat Dilek wrote:
> On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
> > On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
> >
> >> Attached is a diff comparing all new commits in next-20130805.
> >> If one of the commits smells bad to you, please let me know.
> >
> > Out of that list, only the af_packet changes would seem to have any
> > impact on wireless at all.
> >
>
> git-bisecting... 2 steps to go...
>
> This one is bad... "af_packet: simplify VLAN frame check in packet_snd"
>
> http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc

That seems weird, does reverting it fix it?

johannes


2013-08-06 18:26:55

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 6:51 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 6:07 PM, Sedat Dilek <[email protected]> wrote:
>> On Tue, Aug 6, 2013 at 6:03 PM, Johannes Berg <[email protected]> wrote:
>>> [reducing mailing lists - Stephen if you want off let us know]
>>>
>>> On Tue, 2013-08-06 at 17:58 +0200, Sedat Dilek wrote:
>>>
>>>> $ cat /proc/version
>>>> Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
>>>> (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
>>>> 17:52:45 CEST 2013
>>>>
>>>> This kernel boots fine and has no WiFi problems.
>>>>
>>>> > You assume a breakage in a different area than wireless or network in general?
>>>> >
>>>>
>>>> Any wild guess?
>>>
>>> Not really, but maybe you can pull the other trees on top again and see
>>> if those broke it? I don't really think/hope so but it's possible that
>>> one of my pending pull requests broke it, I guess?
>>>
>>
>> Hmm, I pulled in latest for-john of mac80211(-next) and
>> iwlwifi-(fixes|next) which did not help.
>>
>> I am testing next-20130805 and next-20130802 to identify which
>> Linux-next release is BROKEN.
>>
>
> GOOD: next-20130801 and next-20130802
>
> BROKEN: next-20130805 and next-20130806
>
> Comparing diffs between next-20130802 and next-20130805.
>

[ To Dave and CC netdev ]

[ Full thread see [1] ]

The diff was not very helpful.

$ cd linux-next-20130805

$ git log --oneline v3.11-rc4.. | grep "Merge remote-tracking" | egrep
'net-next|slave-dma|wireless'
7f84622 Merge remote-tracking branch 'net-next/master' <--- BAD
(before slave-dma/next)
3580d24 Merge remote-tracking branch 'slave-dma/next' <--- GOOD (after
net-next/master)
88825c7 Merge remote-tracking branch 'wireless/master' <--- GOOD

So, the culprit commit seems to be in the merge of 'net-next/master'.

( BTW, disabling IPv6 in NM/wifi-setup did not change behaviour -
still disconnected to AP. )

- Sedat -

[1] http://marc.info/?t=137579712800008&r=1&w=2


Attachments:
list-of-commits-from-next-next-in-next-20130805.txt (10.40 kB)

2013-08-06 15:09:53

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 4:59 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 4:57 PM, Johannes Berg <[email protected]> wrote:
>> On Tue, 2013-08-06 at 16:56 +0200, Sedat Dilek wrote:
>>> On Tue, Aug 6, 2013 at 4:50 PM, Johannes Berg <[email protected]> wrote:
>>> > On Tue, 2013-08-06 at 16:46 +0200, Sedat Dilek wrote:
>>> >
>>> > Heh, the interesting messages are always cut out of the log:
>>> >
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: wlan0: WPA: RX message 1 of 4-Way Handshake from 00:04:0e:e4:00:3d (ver=2)
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: RSN: msg 1/4 key data - hexdump(len=0):
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: Get randomness: len=32 entropy=4
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from os_get_random - hexdump(len=32): [REMOVED]
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from internal pool - hexdump(len=16): [REMOVED]
>>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
>>> > Aug 6 16:30:34 fambox rsyslogd-2177: imuxsock begins to drop messages from pid 2533 due to rate-limiting
>>> > [...]
>>> >
>>>
>>> Do you know how I can workaround the rsyslogd rate-limiting?
>>
>> Me? no.
>>
>
> Hmmm, I thought about reducing from -ddd to -d or install a newer rsyslog.
>

I have sent you a v2 of my logs (with -d and rsyslog-7.x).

- Sedat -

> - Sedat -
>
>> johannes
>>

2013-08-06 19:47:04

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 9:32 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 9:18 PM, Johannes Berg <[email protected]> wrote:
>> On Tue, 2013-08-06 at 21:14 +0200, Sedat Dilek wrote:
>>> On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
>>> > On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
>>> >
>>> >> Attached is a diff comparing all new commits in next-20130805.
>>> >> If one of the commits smells bad to you, please let me know.
>>> >
>>> > Out of that list, only the af_packet changes would seem to have any
>>> > impact on wireless at all.
>>> >
>>>
>>> git-bisecting... 2 steps to go...
>>>
>>> This one is bad... "af_packet: simplify VLAN frame check in packet_snd"
>>>
>>> http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc
>>
>> That seems weird, does reverting it fix it?
>>
>
> [ TO Phil Sutter ]
>
> This was 3/3 of af_packet patches :-).
>
> So, the culprit commit is...
>
> 0f75b09c798ed00c30d7d5551b896be883bc2aeb is the first bad commit
> commit 0f75b09c798ed00c30d7d5551b896be883bc2aeb
> Author: Phil Sutter <[email protected]>
> Date: Fri Aug 2 11:37:39 2013 +0200
>
> af_packet: when sending ethernet frames, parse header for skb->protocol
>
> This may be necessary when the SKB is passed to other layers on the go,
> which check the protocol field on their own. An example is a VLAN packet
> sent out using AF_PACKET on a bridge interface. The bridging code checks
> the SKB size, accounting for any VLAN header only if the protocol field
> is set accordingly.
>
> Note that eth_type_trans() sets skb->dev to the passed argument, so this
> can be skipped in packet_snd() for ethernet frames, as well.
>
> Signed-off-by: Phil Sutter <[email protected]>
> Signed-off-by: David S. Miller <[email protected]>
>
> :040000 040000 af403a20a321517f6cfb51d2e22c17ca5a60e947
> 1f302ebd62a87b9e874a3e61203499e17d6fce3c M net
>
> - Sedat -

[ net/packet/af_packet.c ]
...
#include <linux/if_arp.h>

$ find include/ -name if_arp.h
include/uapi/linux/if_arp.h
include/linux/if_arp.h

$ LC_ALL=C ll include/uapi/linux/if_arp.h include/linux/if_arp.h
-rw-r--r-- 1 wearefam wearefam 1560 Jul 11 19:42 include/linux/if_arp.h
-rw-r--r-- 1 wearefam wearefam 6344 Jul 26 12:36 include/uapi/linux/if_arp.h

$ grep ARPHRD_ETHER include/linux/if_arp.h include/uapi/linux/if_arp.h
include/uapi/linux/if_arp.h:#define ARPHRD_ETHER 1
/* Ethernet 10Mbps */

Wrong include?

- Sedat -

2013-08-06 23:14:40

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Wed, Aug 7, 2013 at 1:07 AM, Hannes Frederic Sowa
<[email protected]> wrote:
> On Wed, Aug 07, 2013 at 12:55:33AM +0200, Sedat Dilek wrote:
>> No, tested only with iwlwifi.
>> Can you try the patch from [1]?
>>
>> - Sedat -
>>
>> [1] http://marc.info/?l=linux-netdev&m=137582524017840&w=2
>
> Fixed the problem with virtio_net, too.
>
> Thanks,
>

Cool!
I retested with latest next-20130806 which is also fine with this patch.

- Sedat -

> Hannes
>

2013-08-06 14:02:28

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 3:54 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 15:51 +0200, Sedat Dilek wrote:
>> OK, I did not try with debugging enabled and/or w/o Network-Manager.
>>
>> If you know of an issue, please let me know.
>
> No, but the dmesg looks like security problems - check wpa_supplicant
> log
>

Which line is this?

This one:

[ 27.992248] wlan0: deauthenticated from 00:04:0e:e4:00:3d (Reason: 1)

- Sedat -

> johannes
>

2013-08-06 21:37:08

by Phil Sutter

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 06, 2013 at 09:47:01PM +0200, Sedat Dilek wrote:
> On Tue, Aug 6, 2013 at 9:32 PM, Sedat Dilek <[email protected]> wrote:
> > On Tue, Aug 6, 2013 at 9:18 PM, Johannes Berg <[email protected]> wrote:
> >> On Tue, 2013-08-06 at 21:14 +0200, Sedat Dilek wrote:
> >>> On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
> >>> > On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
> >>> >
> >>> >> Attached is a diff comparing all new commits in next-20130805.
> >>> >> If one of the commits smells bad to you, please let me know.
> >>> >
> >>> > Out of that list, only the af_packet changes would seem to have any
> >>> > impact on wireless at all.
> >>> >
> >>>
> >>> git-bisecting... 2 steps to go...
> >>>
> >>> This one is bad... "af_packet: simplify VLAN frame check in packet_snd"
> >>>
> >>> http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc
> >>
> >> That seems weird, does reverting it fix it?
> >>
> >
> > [ TO Phil Sutter ]
> >
> > This was 3/3 of af_packet patches :-).
> >
> > So, the culprit commit is...
> >
> > 0f75b09c798ed00c30d7d5551b896be883bc2aeb is the first bad commit
> > commit 0f75b09c798ed00c30d7d5551b896be883bc2aeb
> > Author: Phil Sutter <[email protected]>
> > Date: Fri Aug 2 11:37:39 2013 +0200
> >
> > af_packet: when sending ethernet frames, parse header for skb->protocol
> >
> > This may be necessary when the SKB is passed to other layers on the go,
> > which check the protocol field on their own. An example is a VLAN packet
> > sent out using AF_PACKET on a bridge interface. The bridging code checks
> > the SKB size, accounting for any VLAN header only if the protocol field
> > is set accordingly.
> >
> > Note that eth_type_trans() sets skb->dev to the passed argument, so this
> > can be skipped in packet_snd() for ethernet frames, as well.
> >
> > Signed-off-by: Phil Sutter <[email protected]>
> > Signed-off-by: David S. Miller <[email protected]>
> >
> > :040000 040000 af403a20a321517f6cfb51d2e22c17ca5a60e947
> > 1f302ebd62a87b9e874a3e61203499e17d6fce3c M net
> >
> > - Sedat -
>
> [ net/packet/af_packet.c ]
> ...
> #include <linux/if_arp.h>
>
> $ find include/ -name if_arp.h
> include/uapi/linux/if_arp.h
> include/linux/if_arp.h
>
> $ LC_ALL=C ll include/uapi/linux/if_arp.h include/linux/if_arp.h
> -rw-r--r-- 1 wearefam wearefam 1560 Jul 11 19:42 include/linux/if_arp.h
> -rw-r--r-- 1 wearefam wearefam 6344 Jul 26 12:36 include/uapi/linux/if_arp.h
>
> $ grep ARPHRD_ETHER include/linux/if_arp.h include/uapi/linux/if_arp.h
> include/uapi/linux/if_arp.h:#define ARPHRD_ETHER 1
> /* Ethernet 10Mbps */
>
> Wrong include?

Nope, <linux/if_arp.h> includes <uapi/linux/if_arp.h>. I suppose there
is a semantical problem here.

Did you verify your bisect by reverting just the three patches?

Does the problem occur on client or server side? AFAICT, hostapd as well
as wpa_supplicant use AF_PACKET.

The tricky thing is, these patches are meant to *loosen* the
restrictions in af_packet.c, so *should* not be harmful. So either my
patches create a side effect I did not foresee, or it's something nasty
(too much delay introduced by calling eth_type_trans() or so).

Could you please provide steps on how to reproduce the faulty behaviour?

Best wishes, Phil

2013-08-06 21:56:59

by Hannes Frederic Sowa

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 06, 2013 at 11:45:38PM +0200, Johannes Berg wrote:
> On Tue, 2013-08-06 at 23:40 +0200, Sedat Dilek wrote:
>
> > > Does the problem occur on client or server side? AFAICT, hostapd as well
> > > as wpa_supplicant use AF_PACKET.
> > >
> > > The tricky thing is, these patches are meant to *loosen* the
> > > restrictions in af_packet.c, so *should* not be harmful. So either my
> > > patches create a side effect I did not foresee, or it's something nasty
> > > (too much delay introduced by calling eth_type_trans() or so).
>
> > By reverting the culprit commit my network/wifi is fine, again.
> > See also attached patch with changelog.
>
> I think skb->protocol is probably getting set up wrong, and just putting
> back the last two lines
>
> skb->protocol = proto;
> skb->dev = dev;
>
> is probably sufficient to fix wifi. If skb->protocol isn't set to
> ETH_P_PAE, then we'd drop the packet in the wifi stack - might be worth
> printing out what it's set to at the point where the skb->protocol
> assignment above was removed.
>
> I'm trying to wrap my head around all this right now but I don't yet see
> how the code after the patch would not get skb->protocol correct.

Has anybody tested plain ethernet? I have a malfunctioning dhclient on
ethernet since the weekend(it seems to not receive any packet). I did not
look after it because have other patches on my todo list currently. Maybe
it is the same error?

Greetings,

Hannes


2013-08-06 15:27:25

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 5:20 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 5:18 PM, Johannes Berg <[email protected]> wrote:
>>
>>
>>> I have sent you a v2 of my logs (with -d and rsyslog-7.x).
>>
>> Hmm, that looks different? There no longer is anything about "PSK might
>> be wrong", but now just disconnections by the AP, which unfortunately
>> indicate no reason.
>>
>> When was it working last?
>>
>
> next-20130801 was OK here.
>

What about these lines?

Aug 6 17:03:37 fambox kernel: [ 134.575120] wlan0: deauthenticated
from 00:04:0e:e4:00:3d (Reason: 1)
Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Event DEAUTH (12) received
Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Deauthentication
notification
Aug 6 17:03:37 fambox wpa_supplicant[2424]: Deauthentication frame
IE(s) - hexdump(len=0): [NULL]
Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: WPA: 4-Way
Handshake failed - pre-shared key may be incorrect
Aug 6 17:03:37 fambox NetworkManager[2431]: <info> (wlan0):
supplicant interface state: 4-way handshake -> disconnected

- Sedat -

2013-08-06 16:07:33

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 6:03 PM, Johannes Berg <[email protected]> wrote:
> [reducing mailing lists - Stephen if you want off let us know]
>
> On Tue, 2013-08-06 at 17:58 +0200, Sedat Dilek wrote:
>
>> $ cat /proc/version
>> Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
>> (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
>> 17:52:45 CEST 2013
>>
>> This kernel boots fine and has no WiFi problems.
>>
>> > You assume a breakage in a different area than wireless or network in general?
>> >
>>
>> Any wild guess?
>
> Not really, but maybe you can pull the other trees on top again and see
> if those broke it? I don't really think/hope so but it's possible that
> one of my pending pull requests broke it, I guess?
>

Hmm, I pulled in latest for-john of mac80211(-next) and
iwlwifi-(fixes|next) which did not help.

I am testing next-20130805 and next-20130802 to identify which
Linux-next release is BROKEN.

- Sedat -

2013-08-06 19:08:58

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:

> Attached is a diff comparing all new commits in next-20130805.
> If one of the commits smells bad to you, please let me know.

Out of that list, only the af_packet changes would seem to have any
impact on wireless at all.

johannes


2013-08-06 14:56:10

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 4:50 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 16:46 +0200, Sedat Dilek wrote:
>
> Heh, the interesting messages are always cut out of the log:
>
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: wlan0: WPA: RX message 1 of 4-Way Handshake from 00:04:0e:e4:00:3d (ver=2)
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: RSN: msg 1/4 key data - hexdump(len=0):
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: Get randomness: len=32 entropy=4
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from os_get_random - hexdump(len=32): [REMOVED]
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from internal pool - hexdump(len=16): [REMOVED]
> Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
> Aug 6 16:30:34 fambox rsyslogd-2177: imuxsock begins to drop messages from pid 2533 due to rate-limiting
> [...]
>

Do you know how I can workaround the rsyslogd rate-limiting?

- Sedat -

> johannes
>

2013-08-06 18:35:08

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 8:26 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 6:51 PM, Sedat Dilek <[email protected]> wrote:
>> On Tue, Aug 6, 2013 at 6:07 PM, Sedat Dilek <[email protected]> wrote:
>>> On Tue, Aug 6, 2013 at 6:03 PM, Johannes Berg <[email protected]> wrote:
>>>> [reducing mailing lists - Stephen if you want off let us know]
>>>>
>>>> On Tue, 2013-08-06 at 17:58 +0200, Sedat Dilek wrote:
>>>>
>>>>> $ cat /proc/version
>>>>> Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
>>>>> (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
>>>>> 17:52:45 CEST 2013
>>>>>
>>>>> This kernel boots fine and has no WiFi problems.
>>>>>
>>>>> > You assume a breakage in a different area than wireless or network in general?
>>>>> >
>>>>>
>>>>> Any wild guess?
>>>>
>>>> Not really, but maybe you can pull the other trees on top again and see
>>>> if those broke it? I don't really think/hope so but it's possible that
>>>> one of my pending pull requests broke it, I guess?
>>>>
>>>
>>> Hmm, I pulled in latest for-john of mac80211(-next) and
>>> iwlwifi-(fixes|next) which did not help.
>>>
>>> I am testing next-20130805 and next-20130802 to identify which
>>> Linux-next release is BROKEN.
>>>
>>
>> GOOD: next-20130801 and next-20130802
>>
>> BROKEN: next-20130805 and next-20130806
>>
>> Comparing diffs between next-20130802 and next-20130805.
>>
>
> [ To Dave and CC netdev ]
>
> [ Full thread see [1] ]
>
> The diff was not very helpful.
>
> $ cd linux-next-20130805
>
> $ git log --oneline v3.11-rc4.. | grep "Merge remote-tracking" | egrep
> 'net-next|slave-dma|wireless'
> 7f84622 Merge remote-tracking branch 'net-next/master' <--- BAD
> (before slave-dma/next)
> 3580d24 Merge remote-tracking branch 'slave-dma/next' <--- GOOD (after
> net-next/master)
> 88825c7 Merge remote-tracking branch 'wireless/master' <--- GOOD
>
> So, the culprit commit seems to be in the merge of 'net-next/master'.
>
> ( BTW, disabling IPv6 in NM/wifi-setup did not change behaviour -
> still disconnected to AP. )
>
> - Sedat -
>
> [1] http://marc.info/?t=137579712800008&r=1&w=2

Attached is a diff comparing all new commits in next-20130805.
If one of the commits smells bad to you, please let me know.

- Sedat -


Attachments:
list-of-commits-from-net-next_next-20130802-VS-next-20130805.diff (1.81 kB)

2013-08-06 15:43:28

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 5:29 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 17:27 +0200, Sedat Dilek wrote:
>> On Tue, Aug 6, 2013 at 5:20 PM, Sedat Dilek <[email protected]> wrote:
>> > On Tue, Aug 6, 2013 at 5:18 PM, Johannes Berg <[email protected]> wrote:
>> >>
>> >>
>> >>> I have sent you a v2 of my logs (with -d and rsyslog-7.x).
>> >>
>> >> Hmm, that looks different? There no longer is anything about "PSK might
>> >> be wrong", but now just disconnections by the AP, which unfortunately
>> >> indicate no reason.
>> >>
>> >> When was it working last?
>> >>
>> >
>> > next-20130801 was OK here.
>> >
>>
>> What about these lines?
>>
>> Aug 6 17:03:37 fambox kernel: [ 134.575120] wlan0: deauthenticated
>> from 00:04:0e:e4:00:3d (Reason: 1)
>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Event DEAUTH (12) received
>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Deauthentication
>> notification
>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: Deauthentication frame
>> IE(s) - hexdump(len=0): [NULL]
>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: WPA: 4-Way
>> Handshake failed - pre-shared key may be incorrect
>> Aug 6 17:03:37 fambox NetworkManager[2431]: <info> (wlan0):
>> supplicant interface state: 4-way handshake -> disconnected
>
> Yeah but before you had a local deauth request, this time it's the AP
> kicking you off.
>
> I really don't know. Can you try something like wireless-testing maybe?
> -next could have any kind of breakage I guess ...
>

I pulled in master-2013-08-05 tag on top of Linux v3.11-rc4... Building...

You assume a breakage in a different area than wireless or network in general?

- Sedat -


> johannes
>

2013-08-06 21:45:47

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 23:40 +0200, Sedat Dilek wrote:

> > Does the problem occur on client or server side? AFAICT, hostapd as well
> > as wpa_supplicant use AF_PACKET.
> >
> > The tricky thing is, these patches are meant to *loosen* the
> > restrictions in af_packet.c, so *should* not be harmful. So either my
> > patches create a side effect I did not foresee, or it's something nasty
> > (too much delay introduced by calling eth_type_trans() or so).

> By reverting the culprit commit my network/wifi is fine, again.
> See also attached patch with changelog.

I think skb->protocol is probably getting set up wrong, and just putting
back the last two lines

skb->protocol = proto;
skb->dev = dev;

is probably sufficient to fix wifi. If skb->protocol isn't set to
ETH_P_PAE, then we'd drop the packet in the wifi stack - might be worth
printing out what it's set to at the point where the skb->protocol
assignment above was removed.

I'm trying to wrap my head around all this right now but I don't yet see
how the code after the patch would not get skb->protocol correct.

johannes


2013-08-06 16:51:30

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 6:07 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 6:03 PM, Johannes Berg <[email protected]> wrote:
>> [reducing mailing lists - Stephen if you want off let us know]
>>
>> On Tue, 2013-08-06 at 17:58 +0200, Sedat Dilek wrote:
>>
>>> $ cat /proc/version
>>> Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
>>> (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
>>> 17:52:45 CEST 2013
>>>
>>> This kernel boots fine and has no WiFi problems.
>>>
>>> > You assume a breakage in a different area than wireless or network in general?
>>> >
>>>
>>> Any wild guess?
>>
>> Not really, but maybe you can pull the other trees on top again and see
>> if those broke it? I don't really think/hope so but it's possible that
>> one of my pending pull requests broke it, I guess?
>>
>
> Hmm, I pulled in latest for-john of mac80211(-next) and
> iwlwifi-(fixes|next) which did not help.
>
> I am testing next-20130805 and next-20130802 to identify which
> Linux-next release is BROKEN.
>

GOOD: next-20130801 and next-20130802

BROKEN: next-20130805 and next-20130806

Comparing diffs between next-20130802 and next-20130805.

- Sedat -

2013-08-06 15:29:58

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 17:27 +0200, Sedat Dilek wrote:
> On Tue, Aug 6, 2013 at 5:20 PM, Sedat Dilek <[email protected]> wrote:
> > On Tue, Aug 6, 2013 at 5:18 PM, Johannes Berg <[email protected]> wrote:
> >>
> >>
> >>> I have sent you a v2 of my logs (with -d and rsyslog-7.x).
> >>
> >> Hmm, that looks different? There no longer is anything about "PSK might
> >> be wrong", but now just disconnections by the AP, which unfortunately
> >> indicate no reason.
> >>
> >> When was it working last?
> >>
> >
> > next-20130801 was OK here.
> >
>
> What about these lines?
>
> Aug 6 17:03:37 fambox kernel: [ 134.575120] wlan0: deauthenticated
> from 00:04:0e:e4:00:3d (Reason: 1)
> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Event DEAUTH (12) received
> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Deauthentication
> notification
> Aug 6 17:03:37 fambox wpa_supplicant[2424]: Deauthentication frame
> IE(s) - hexdump(len=0): [NULL]
> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: WPA: 4-Way
> Handshake failed - pre-shared key may be incorrect
> Aug 6 17:03:37 fambox NetworkManager[2431]: <info> (wlan0):
> supplicant interface state: 4-way handshake -> disconnected

Yeah but before you had a local deauth request, this time it's the AP
kicking you off.

I really don't know. Can you try something like wireless-testing maybe?
-next could have any kind of breakage I guess ...

johannes


2013-08-06 16:03:27

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

[reducing mailing lists - Stephen if you want off let us know]

On Tue, 2013-08-06 at 17:58 +0200, Sedat Dilek wrote:

> $ cat /proc/version
> Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
> (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
> 17:52:45 CEST 2013
>
> This kernel boots fine and has no WiFi problems.
>
> > You assume a breakage in a different area than wireless or network in general?
> >
>
> Any wild guess?

Not really, but maybe you can pull the other trees on top again and see
if those broke it? I don't really think/hope so but it's possible that
one of my pending pull requests broke it, I guess?

johannes


2013-08-06 13:54:29

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 15:51 +0200, Sedat Dilek wrote:
> OK, I did not try with debugging enabled and/or w/o Network-Manager.
>
> If you know of an issue, please let me know.

No, but the dmesg looks like security problems - check wpa_supplicant
log

johannes


2013-08-06 21:40:33

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 11:29 PM, Phil Sutter <[email protected]> wrote:
> On Tue, Aug 06, 2013 at 09:47:01PM +0200, Sedat Dilek wrote:
>> On Tue, Aug 6, 2013 at 9:32 PM, Sedat Dilek <[email protected]> wrote:
>> > On Tue, Aug 6, 2013 at 9:18 PM, Johannes Berg <[email protected]> wrote:
>> >> On Tue, 2013-08-06 at 21:14 +0200, Sedat Dilek wrote:
>> >>> On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
>> >>> > On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
>> >>> >
>> >>> >> Attached is a diff comparing all new commits in next-20130805.
>> >>> >> If one of the commits smells bad to you, please let me know.
>> >>> >
>> >>> > Out of that list, only the af_packet changes would seem to have any
>> >>> > impact on wireless at all.
>> >>> >
>> >>>
>> >>> git-bisecting... 2 steps to go...
>> >>>
>> >>> This one is bad... "af_packet: simplify VLAN frame check in packet_snd"
>> >>>
>> >>> http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc
>> >>
>> >> That seems weird, does reverting it fix it?
>> >>
>> >
>> > [ TO Phil Sutter ]
>> >
>> > This was 3/3 of af_packet patches :-).
>> >
>> > So, the culprit commit is...
>> >
>> > 0f75b09c798ed00c30d7d5551b896be883bc2aeb is the first bad commit
>> > commit 0f75b09c798ed00c30d7d5551b896be883bc2aeb
>> > Author: Phil Sutter <[email protected]>
>> > Date: Fri Aug 2 11:37:39 2013 +0200
>> >
>> > af_packet: when sending ethernet frames, parse header for skb->protocol
>> >
>> > This may be necessary when the SKB is passed to other layers on the go,
>> > which check the protocol field on their own. An example is a VLAN packet
>> > sent out using AF_PACKET on a bridge interface. The bridging code checks
>> > the SKB size, accounting for any VLAN header only if the protocol field
>> > is set accordingly.
>> >
>> > Note that eth_type_trans() sets skb->dev to the passed argument, so this
>> > can be skipped in packet_snd() for ethernet frames, as well.
>> >
>> > Signed-off-by: Phil Sutter <[email protected]>
>> > Signed-off-by: David S. Miller <[email protected]>
>> >
>> > :040000 040000 af403a20a321517f6cfb51d2e22c17ca5a60e947
>> > 1f302ebd62a87b9e874a3e61203499e17d6fce3c M net
>> >
>> > - Sedat -
>>
>> [ net/packet/af_packet.c ]
>> ...
>> #include <linux/if_arp.h>
>>
>> $ find include/ -name if_arp.h
>> include/uapi/linux/if_arp.h
>> include/linux/if_arp.h
>>
>> $ LC_ALL=C ll include/uapi/linux/if_arp.h include/linux/if_arp.h
>> -rw-r--r-- 1 wearefam wearefam 1560 Jul 11 19:42 include/linux/if_arp.h
>> -rw-r--r-- 1 wearefam wearefam 6344 Jul 26 12:36 include/uapi/linux/if_arp.h
>>
>> $ grep ARPHRD_ETHER include/linux/if_arp.h include/uapi/linux/if_arp.h
>> include/uapi/linux/if_arp.h:#define ARPHRD_ETHER 1
>> /* Ethernet 10Mbps */
>>
>> Wrong include?
>
> Nope, <linux/if_arp.h> includes <uapi/linux/if_arp.h>. I suppose there
> is a semantical problem here.
>
> Did you verify your bisect by reverting just the three patches?
>
> Does the problem occur on client or server side? AFAICT, hostapd as well
> as wpa_supplicant use AF_PACKET.
>
> The tricky thing is, these patches are meant to *loosen* the
> restrictions in af_packet.c, so *should* not be harmful. So either my
> patches create a side effect I did not foresee, or it's something nasty
> (too much delay introduced by calling eth_type_trans() or so).
>
> Could you please provide steps on how to reproduce the faulty behaviour?
>

By reverting the culprit commit my network/wifi is fine, again.
See also attached patch with changelog.

- Sedat -


Attachments:
0001-Revert-af_packet-when-sending-ethernet-frames-parse-.patch (2.25 kB)

2013-08-06 22:55:35

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 11:56 PM, Hannes Frederic Sowa
<[email protected]> wrote:
> On Tue, Aug 06, 2013 at 11:45:38PM +0200, Johannes Berg wrote:
>> On Tue, 2013-08-06 at 23:40 +0200, Sedat Dilek wrote:
>>
>> > > Does the problem occur on client or server side? AFAICT, hostapd as well
>> > > as wpa_supplicant use AF_PACKET.
>> > >
>> > > The tricky thing is, these patches are meant to *loosen* the
>> > > restrictions in af_packet.c, so *should* not be harmful. So either my
>> > > patches create a side effect I did not foresee, or it's something nasty
>> > > (too much delay introduced by calling eth_type_trans() or so).
>>
>> > By reverting the culprit commit my network/wifi is fine, again.
>> > See also attached patch with changelog.
>>
>> I think skb->protocol is probably getting set up wrong, and just putting
>> back the last two lines
>>
>> skb->protocol = proto;
>> skb->dev = dev;
>>
>> is probably sufficient to fix wifi. If skb->protocol isn't set to
>> ETH_P_PAE, then we'd drop the packet in the wifi stack - might be worth
>> printing out what it's set to at the point where the skb->protocol
>> assignment above was removed.
>>
>> I'm trying to wrap my head around all this right now but I don't yet see
>> how the code after the patch would not get skb->protocol correct.
>
> Has anybody tested plain ethernet? I have a malfunctioning dhclient on
> ethernet since the weekend(it seems to not receive any packet). I did not
> look after it because have other patches on my todo list currently. Maybe
> it is the same error?
>

No, tested only with iwlwifi.
Can you try the patch from [1]?

- Sedat -

[1] http://marc.info/?l=linux-netdev&m=137582524017840&w=2

> Greetings,
>
> Hannes
>

2013-08-06 15:20:09

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 5:18 PM, Johannes Berg <[email protected]> wrote:
>
>
>> I have sent you a v2 of my logs (with -d and rsyslog-7.x).
>
> Hmm, that looks different? There no longer is anything about "PSK might
> be wrong", but now just disconnections by the AP, which unfortunately
> indicate no reason.
>
> When was it working last?
>

next-20130801 was OK here.

- Sedat -

> johannes
>

2013-08-06 14:57:45

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 16:56 +0200, Sedat Dilek wrote:
> On Tue, Aug 6, 2013 at 4:50 PM, Johannes Berg <[email protected]> wrote:
> > On Tue, 2013-08-06 at 16:46 +0200, Sedat Dilek wrote:
> >
> > Heh, the interesting messages are always cut out of the log:
> >
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: wlan0: WPA: RX message 1 of 4-Way Handshake from 00:04:0e:e4:00:3d (ver=2)
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: RSN: msg 1/4 key data - hexdump(len=0):
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: Get randomness: len=32 entropy=4
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from os_get_random - hexdump(len=32): [REMOVED]
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from internal pool - hexdump(len=16): [REMOVED]
> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
> > Aug 6 16:30:34 fambox rsyslogd-2177: imuxsock begins to drop messages from pid 2533 due to rate-limiting
> > [...]
> >
>
> Do you know how I can workaround the rsyslogd rate-limiting?

Me? no.

johannes


2013-08-08 21:14:31

by Phil Sutter

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 06, 2013 at 11:58:42PM +0200, Johannes Berg wrote:
> On Tue, 2013-08-06 at 23:45 +0200, Johannes Berg wrote:
>
> > I think skb->protocol is probably getting set up wrong, and just putting
> > back the last two lines
> >
> > skb->protocol = proto;
> > skb->dev = dev;
> >
> > is probably sufficient to fix wifi. If skb->protocol isn't set to
> > ETH_P_PAE, then we'd drop the packet in the wifi stack - might be worth
> > printing out what it's set to at the point where the skb->protocol
> > assignment above was removed.
> >
> > I'm trying to wrap my head around all this right now but I don't yet see
> > how the code after the patch would not get skb->protocol correct.
>
> Actually, I think that's not it, but the code now behaves totally
> differently?
>
> Say this is the frame data with two points marked:
>
> (1) (2)
> | dst | src | ethtype | ... |
>
> As I understand it (in my admittedly rather tired state), previously we
> had
>
> skb_network_header() == (1)
> skb_mac_header() == (1)
> skb->data == (1)
>
> After calling eth_type_trans(), we get
>
> skb_network_header() == (1)
> skb_mac_header() == (1)
> skb->data == (2)
>
> I think? Maybe I'm totally confused though.

Yes, indeed. And it's the explanation for at least some of the problems
that arose with my doubtful change:

Leaving my patch in place and reverting the above pointer by calling
skb_push() after eth_type_trans(), the busybox DHCP client on my testing
machine is functional again.

But as Dave has gone this path already (although in a different way) and
it aparently didn't fix the wifi problem, there may be something else as
well. Interestingly, my skb_push() hack from above fixes wpa_supplicant
for me as well. So I will test Dave's patch myself tomorrow and search
for the difference if I get the same result as Sedat.

Best wishes, Phil

2013-08-06 14:59:14

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 4:57 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 16:56 +0200, Sedat Dilek wrote:
>> On Tue, Aug 6, 2013 at 4:50 PM, Johannes Berg <[email protected]> wrote:
>> > On Tue, 2013-08-06 at 16:46 +0200, Sedat Dilek wrote:
>> >
>> > Heh, the interesting messages are always cut out of the log:
>> >
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: wlan0: WPA: RX message 1 of 4-Way Handshake from 00:04:0e:e4:00:3d (ver=2)
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: RSN: msg 1/4 key data - hexdump(len=0):
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: Get randomness: len=32 entropy=4
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from os_get_random - hexdump(len=32): [REMOVED]
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from internal pool - hexdump(len=16): [REMOVED]
>> > Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
>> > Aug 6 16:30:34 fambox rsyslogd-2177: imuxsock begins to drop messages from pid 2533 due to rate-limiting
>> > [...]
>> >
>>
>> Do you know how I can workaround the rsyslogd rate-limiting?
>
> Me? no.
>

Hmmm, I thought about reducing from -ddd to -d or install a newer rsyslog.

- Sedat -

> johannes
>

2013-08-06 14:46:08

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 3:54 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 15:51 +0200, Sedat Dilek wrote:
>> OK, I did not try with debugging enabled and/or w/o Network-Manager.
>>
>> If you know of an issue, please let me know.
>
> No, but the dmesg looks like security problems - check wpa_supplicant
> log
>

I start a debug-session like this:

# stop network-manager
# kill -9 $(pidof wpa_supplicant)
# /sbin/wpa_supplicant -B -P /run/sendsigs.omit.d/wpasupplicant.pid -u
-s -O /var/run/wpa_supplicant -t -ddd
# start network-manager
# tail -f /var/log/syslog

# egrep 'warn|fail|error' /root/syslog_for-johill.txt
Aug 6 16:29:00 fambox wpa_supplicant[1150]: wlan0: WPA: 4-Way
Handshake failed - pre-shared key may be incorrect
Aug 6 16:29:07 fambox wpa_supplicant[1150]: wlan0: WPA: 4-Way
Handshake failed - pre-shared key may be incorrect
Aug 6 16:29:13 fambox NetworkManager[1064]: <warn> quit request
received, terminating...
Aug 6 16:30:21 fambox NetworkManager[2541]: <warn> (wlan0): driver
supports Access Point (AP) mode
Aug 6 16:30:21 fambox NetworkManager[2541]: <warn> failed to allocate
link cache: (-10) Operation not supported
Aug 6 16:30:21 fambox NetworkManager[2541]: <warn> Trying to remove a
non-existant call id.
Aug 6 16:30:21 fambox wpa_supplicant[2533]: nl80211: Scan trigger
failed: ret=-16 (Device or resource busy)
Aug 6 16:30:27 fambox wpa_supplicant[2533]: EAPOL: External
notification - EAP fail=0
Aug 6 16:30:34 fambox wpa_supplicant[2533]: EAPOL: External
notification - EAP fail=0
Aug 6 16:30:42 fambox wpa_supplicant[2533]: EAPOL: External
notification - EAP fail=0
Aug 6 16:30:48 fambox wpa_supplicant[2533]: EAPOL: External
notification - EAP fail=0
Aug 6 16:31:04 fambox NetworkManager[2541]: <warn> quit request
received, terminating...

I send you the full syslog.txt to your sipsolutions email-address (not
sure if it contains sensitive data).

- Sedat -

> johannes
>

2013-08-06 19:14:41

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 9:08 PM, Johannes Berg <[email protected]> wrote:
> On Tue, 2013-08-06 at 20:35 +0200, Sedat Dilek wrote:
>
>> Attached is a diff comparing all new commits in next-20130805.
>> If one of the commits smells bad to you, please let me know.
>
> Out of that list, only the af_packet changes would seem to have any
> impact on wireless at all.
>

git-bisecting... 2 steps to go...

This one is bad... "af_packet: simplify VLAN frame check in packet_snd"

http://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=c483e02614551e44ced3fe6eedda8e36d3277ccc

- Sedat -

2013-08-06 14:50:23

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, 2013-08-06 at 16:46 +0200, Sedat Dilek wrote:

Heh, the interesting messages are always cut out of the log:

Aug 6 16:30:34 fambox wpa_supplicant[2533]: wlan0: WPA: RX message 1 of 4-Way Handshake from 00:04:0e:e4:00:3d (ver=2)
Aug 6 16:30:34 fambox wpa_supplicant[2533]: RSN: msg 1/4 key data - hexdump(len=0):
Aug 6 16:30:34 fambox wpa_supplicant[2533]: Get randomness: len=32 entropy=4
Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from os_get_random - hexdump(len=32): [REMOVED]
Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
Aug 6 16:30:34 fambox wpa_supplicant[2533]: random from internal pool - hexdump(len=16): [REMOVED]
Aug 6 16:30:34 fambox wpa_supplicant[2533]: random_mix_pool - hexdump(len=20): [REMOVED]
Aug 6 16:30:34 fambox rsyslogd-2177: imuxsock begins to drop messages from pid 2533 due to rate-limiting
[...]

johannes


2013-08-06 15:58:27

by Sedat Dilek

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Tue, Aug 6, 2013 at 5:43 PM, Sedat Dilek <[email protected]> wrote:
> On Tue, Aug 6, 2013 at 5:29 PM, Johannes Berg <[email protected]> wrote:
>> On Tue, 2013-08-06 at 17:27 +0200, Sedat Dilek wrote:
>>> On Tue, Aug 6, 2013 at 5:20 PM, Sedat Dilek <[email protected]> wrote:
>>> > On Tue, Aug 6, 2013 at 5:18 PM, Johannes Berg <[email protected]> wrote:
>>> >>
>>> >>
>>> >>> I have sent you a v2 of my logs (with -d and rsyslog-7.x).
>>> >>
>>> >> Hmm, that looks different? There no longer is anything about "PSK might
>>> >> be wrong", but now just disconnections by the AP, which unfortunately
>>> >> indicate no reason.
>>> >>
>>> >> When was it working last?
>>> >>
>>> >
>>> > next-20130801 was OK here.
>>> >
>>>
>>> What about these lines?
>>>
>>> Aug 6 17:03:37 fambox kernel: [ 134.575120] wlan0: deauthenticated
>>> from 00:04:0e:e4:00:3d (Reason: 1)
>>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Event DEAUTH (12) received
>>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: Deauthentication
>>> notification
>>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: Deauthentication frame
>>> IE(s) - hexdump(len=0): [NULL]
>>> Aug 6 17:03:37 fambox wpa_supplicant[2424]: wlan0: WPA: 4-Way
>>> Handshake failed - pre-shared key may be incorrect
>>> Aug 6 17:03:37 fambox NetworkManager[2431]: <info> (wlan0):
>>> supplicant interface state: 4-way handshake -> disconnected
>>
>> Yeah but before you had a local deauth request, this time it's the AP
>> kicking you off.
>>
>> I really don't know. Can you try something like wireless-testing maybe?
>> -next could have any kind of breakage I guess ...
>>
>
> I pulled in master-2013-08-05 tag on top of Linux v3.11-rc4... Building...
>

$ cat /proc/version
Linux version 3.11.0-rc4-1-wl-20130805 ([email protected]@fambox)
(gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Tue Aug 6
17:52:45 CEST 2013

This kernel boots fine and has no WiFi problems.

> You assume a breakage in a different area than wireless or network in general?
>

Any wild guess?

- Sedat -

2013-08-06 23:07:50

by Hannes Frederic Sowa

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]

On Wed, Aug 07, 2013 at 12:55:33AM +0200, Sedat Dilek wrote:
> No, tested only with iwlwifi.
> Can you try the patch from [1]?
>
> - Sedat -
>
> [1] http://marc.info/?l=linux-netdev&m=137582524017840&w=2

Fixed the problem with virtio_net, too.

Thanks,

Hannes


2013-08-06 15:18:23

by Johannes Berg

[permalink] [raw]
Subject: Re: linux-next: Tree for Aug 6 [ wireless | iwlwifi | mac80211 ? ]



> I have sent you a v2 of my logs (with -d and rsyslog-7.x).

Hmm, that looks different? There no longer is anything about "PSK might
be wrong", but now just disconnections by the AP, which unfortunately
indicate no reason.

When was it working last?

johannes