2011-05-11 16:17:54

by Larry Finger

[permalink] [raw]
Subject: [PATCH] rtlwifi: Fix panic due to memory allocation failure

The PCI routine of this driver is allocating receive buffers of order 2,
which causes an unnecessary fragmentation of memory. To make matters
worse, there are locations that fail to check for allocation failures,
or return success when the allocation actually failed. Kernel panics
result.

Signed-off-by: Larry Finger <[email protected]>
Cc: Stable <[email protected]> [2.6.37 and 2.6.38]
---

John,

This is 2.6.39 material. I hope we make the cutoff.

Larry
---

Index: wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
===================================================================
--- wireless-testing-new.orig/drivers/net/wireless/rtlwifi/pci.c
+++ wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
@@ -732,10 +732,12 @@ static void _rtl_pci_rx_interrupt(struct
false))) {
dev_kfree_skb_any(skb);
} else {
- struct sk_buff *uskb = NULL;
+ struct sk_buff *uskb;
u8 *pdata;
uskb = dev_alloc_skb(skb->len
+ 128);
+ if (!uskb)
+ return;
memcpy(IEEE80211_SKB_RXCB(uskb),
&rx_status,
sizeof(rx_status));
@@ -996,7 +998,7 @@ static void _rtl_pci_init_trx_var(struct
*/
rtlpci->txringcount[BE_QUEUE] = RT_TXDESC_NUM_BE_QUEUE;

- rtlpci->rxbuffersize = 9100; /*2048/1024; */
+ rtlpci->rxbuffersize = 4096;
rtlpci->rxringcount = RTL_PCI_MAX_RX_COUNT; /*64; */
}

@@ -1118,7 +1120,7 @@ static int _rtl_pci_init_rx_ring(struct
dev_alloc_skb(rtlpci->rxbuffersize);
u32 bufferaddress;
if (!skb)
- return 0;
+ return -ENOMEM;
entry = &rtlpci->rx_ring[rx_queue_idx].desc[i];

/*skb->dev = dev; */


2011-05-11 16:36:20

by Larry Finger

[permalink] [raw]
Subject: Re: [PATCH] rtlwifi: Fix panic due to memory allocation failure

On 05/11/2011 11:28 AM, Christian Lamparter wrote:
> On Wednesday 11 May 2011 18:17:51 Larry Finger wrote:
>> The PCI routine of this driver is allocating receive buffers of order 2,
>> which causes an unnecessary fragmentation of memory. To make matters
>> worse, there are locations that fail to check for allocation failures,
>> or return success when the allocation actually failed. Kernel panics
>> result.
>>
>> Signed-off-by: Larry Finger<[email protected]>
>> Cc: Stable<[email protected]> [2.6.37 and 2.6.38]
>> ---
>>
>> John,
>>
>> This is 2.6.39 material. I hope we make the cutoff.
>>
>> Larry
>> ---
>>
>> Index: wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
>> ===================================================================
>> --- wireless-testing-new.orig/drivers/net/wireless/rtlwifi/pci.c
>> +++ wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
>> @@ -996,7 +998,7 @@ static void _rtl_pci_init_trx_var(struct
>> */
>> rtlpci->txringcount[BE_QUEUE] = RT_TXDESC_NUM_BE_QUEUE;
>>
>> - rtlpci->rxbuffersize = 9100; /*2048/1024; */
>> + rtlpci->rxbuffersize = 4096;
>> rtlpci->rxringcount = RTL_PCI_MAX_RX_COUNT; /*64; */
>> }
>>
> Are you sure this change won't break 8k AMSDU rx?
> [or is there some magic that disables disable_amsdu_8k
> for pci devices?]

It is disabled in the newest driver that is not yet submitted, but not in the
two other PCI drivers. I need to rework this patch.

John - please drop it.

Larry

2011-05-11 16:28:31

by Christian Lamparter

[permalink] [raw]
Subject: Re: [PATCH] rtlwifi: Fix panic due to memory allocation failure

On Wednesday 11 May 2011 18:17:51 Larry Finger wrote:
> The PCI routine of this driver is allocating receive buffers of order 2,
> which causes an unnecessary fragmentation of memory. To make matters
> worse, there are locations that fail to check for allocation failures,
> or return success when the allocation actually failed. Kernel panics
> result.
>
> Signed-off-by: Larry Finger <[email protected]>
> Cc: Stable <[email protected]> [2.6.37 and 2.6.38]
> ---
>
> John,
>
> This is 2.6.39 material. I hope we make the cutoff.
>
> Larry
> ---
>
> Index: wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
> ===================================================================
> --- wireless-testing-new.orig/drivers/net/wireless/rtlwifi/pci.c
> +++ wireless-testing-new/drivers/net/wireless/rtlwifi/pci.c
> @@ -996,7 +998,7 @@ static void _rtl_pci_init_trx_var(struct
> */
> rtlpci->txringcount[BE_QUEUE] = RT_TXDESC_NUM_BE_QUEUE;
>
> - rtlpci->rxbuffersize = 9100; /*2048/1024; */
> + rtlpci->rxbuffersize = 4096;
> rtlpci->rxringcount = RTL_PCI_MAX_RX_COUNT; /*64; */
> }
>
Are you sure this change won't break 8k AMSDU rx?
[or is there some magic that disables disable_amsdu_8k
for pci devices?]

Regards,
Chr