2011-05-16 09:41:09

by Emmanuel Grumbach

[permalink] [raw]
Subject: several packets in a single buffer in Rx

Hi,

I am trying to enable a HW feature on a wireless NIC that allows the
HW to put several packets in the same Rx buffer coming from the NIC.
This can save interrupts and traffic on the bus.
Today, the driver allocates pages in which the PCI-e device puts its
data. Then we create an skb and add a fragment that points to the
beginning of the page. The offset of the fragment is set to the offset
of the first byte of the mac header within the buffer. There is only
one packet per page / buffer.

I would like to be able to deliver the same page several times to the
stack without having the stack consume it before the last time I
deliver it.
Of course I would like to avoid cloning it.

I tried to look at shinfo->nr_frags but this looks more like a sgl
related variable
GRO doesn't seem to be related either since as far as I understood, it
is relevant for packets after 802.11 to 802.3 translation.


FWIW: I am talking about Intel's wireless device although I don't
think this fact matters

Thanks

Emmanuel Grumbach
[email protected]


2011-05-16 20:04:55

by Tomas Winkler

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

2011/5/16 Emmanuel Grumbach <[email protected]>:
> 2011/5/16 Michał Mirosław <[email protected]>:
>> W dniu 16 maja 2011 14:59 użytkownik Emmanuel Grumbach
>> <[email protected]> napisał:
>>> 2011/5/16 Michał Mirosław <[email protected]>:
>>>> 2011/5/16 Emmanuel Grumbach <[email protected]>:
>>>>> I would like to be able to deliver the same page several times to the
>>>>> stack without having the stack consume it before the last time I
>>>>> deliver it.
>>>>> Of course I would like to avoid cloning it.
>>>>
>>>> Just do get_page() on the page having another packet in it before
>>>> passing skb up.
>>>>
>>>
>>> I can see the path:
>>> __kfree_skb -> skb_release_all -> skb_release_data -> put_page
>>> put_page will free the page iff the _count variable reaches 0. Of course,
>>> _count is incremented by get_page.
>>>
>>> I will give it try.
>>>
>>> I understand that this will work regardless the order given to
>>> alloc_pages right ?
>>
>> Yes. Remember that if you put a lot of packets in a big-order page
>> then the memory will be freed only after all packets are freed.
>
> Sure. Thanks for the help.

How it is ensured that skb manipulation won't corrupt another packet
on the same page?

Thanks
Tomas

2011-05-16 13:13:59

by Michał Mirosław

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

W dniu 16 maja 2011 14:59 u?ytkownik Emmanuel Grumbach
<[email protected]> napisa?:
> 2011/5/16 Micha? Miros?aw <[email protected]>:
>> 2011/5/16 Emmanuel Grumbach <[email protected]>:
>>> I would like to be able to deliver the same page several times to the
>>> stack without having the stack consume it before the last time I
>>> deliver it.
>>> Of course I would like to avoid cloning it.
>>
>> Just do get_page() on the page having another packet in it before
>> passing skb up.
>>
>
> I can see the path:
> __kfree_skb -> skb_release_all -> skb_release_data -> put_page
> put_page will free the page iff the _count variable reaches 0. Of course,
> _count is incremented by get_page.
>
> I will give it try.
>
> I understand that this will work regardless the order given to
> alloc_pages right ?

Yes. Remember that if you put a lot of packets in a big-order page
then the memory will be freed only after all packets are freed.

Best Regards,
Micha? Miros?aw

2011-05-16 20:15:16

by Ben Hutchings

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

On Mon, 2011-05-16 at 23:04 +0300, Tomas Winkler wrote:
> 2011/5/16 Emmanuel Grumbach <[email protected]>:
> > 2011/5/16 Michał Mirosław <[email protected]>:
> >> W dniu 16 maja 2011 14:59 użytkownik Emmanuel Grumbach
> >> <[email protected]> napisał:
> >>> 2011/5/16 Michał Mirosław <[email protected]>:
> >>>> 2011/5/16 Emmanuel Grumbach <[email protected]>:
> >>>>> I would like to be able to deliver the same page several times to the
> >>>>> stack without having the stack consume it before the last time I
> >>>>> deliver it.
> >>>>> Of course I would like to avoid cloning it.
> >>>>
> >>>> Just do get_page() on the page having another packet in it before
> >>>> passing skb up.
> >>>>
> >>>
> >>> I can see the path:
> >>> __kfree_skb -> skb_release_all -> skb_release_data -> put_page
> >>> put_page will free the page iff the _count variable reaches 0. Of course,
> >>> _count is incremented by get_page.
> >>>
> >>> I will give it try.
> >>>
> >>> I understand that this will work regardless the order given to
> >>> alloc_pages right ?
> >>
> >> Yes. Remember that if you put a lot of packets in a big-order page
> >> then the memory will be freed only after all packets are freed.
> >
> > Sure. Thanks for the help.
>
> How it is ensured that skb manipulation won't corrupt another packet
> on the same page?

Fragments attached to an skb are always treated as read-only. The
headers will be copied into the skb's header buffer and may be modified
there.

Ben.

--
Ben Hutchings, Senior Software Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.


2011-05-16 11:06:29

by Michał Mirosław

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

2011/5/16 Emmanuel Grumbach <[email protected]>:
> I would like to be able to deliver the same page several times to the
> stack without having the stack consume it before the last time I
> deliver it.
> Of course I would like to avoid cloning it.

Just do get_page() on the page having another packet in it before
passing skb up.

Best Regards,
Micha? Miros?aw

2011-05-16 12:59:12

by Emmanuel Grumbach

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

2011/5/16 Micha? Miros?aw <[email protected]>:
> 2011/5/16 Emmanuel Grumbach <[email protected]>:
>> I would like to be able to deliver the same page several times to the
>> stack without having the stack consume it before the last time I
>> deliver it.
>> Of course I would like to avoid cloning it.
>
> Just do get_page() on the page having another packet in it before
> passing skb up.
>

I can see the path:
__kfree_skb -> skb_release_all -> skb_release_data -> put_page
put_page will free the page iff the _count variable reaches 0. Of course,
_count is incremented by get_page.

I will give it try.

I understand that this will work regardless the order given to
alloc_pages right ?

Thanks

2011-05-16 13:36:47

by Emmanuel Grumbach

[permalink] [raw]
Subject: Re: several packets in a single buffer in Rx

2011/5/16 Micha? Miros?aw <[email protected]>:
> W dniu 16 maja 2011 14:59 u?ytkownik Emmanuel Grumbach
> <[email protected]> napisa?:
>> 2011/5/16 Micha? Miros?aw <[email protected]>:
>>> 2011/5/16 Emmanuel Grumbach <[email protected]>:
>>>> I would like to be able to deliver the same page several times to the
>>>> stack without having the stack consume it before the last time I
>>>> deliver it.
>>>> Of course I would like to avoid cloning it.
>>>
>>> Just do get_page() on the page having another packet in it before
>>> passing skb up.
>>>
>>
>> I can see the path:
>> __kfree_skb -> skb_release_all -> skb_release_data -> put_page
>> put_page will free the page iff the _count variable reaches 0. Of course,
>> _count is incremented by get_page.
>>
>> I will give it try.
>>
>> I understand that this will work regardless the order given to
>> alloc_pages right ?
>
> Yes. Remember that if you put a lot of packets in a big-order page
> then the memory will be freed only after all packets are freed.

Sure. Thanks for the help.

>
> Best Regards,
> Micha? Miros?aw
>