Here is the modifications I made:
//skbuff.h
//I added the folloing definitions
#define MAX_POOL_SIZE 4096
unsigned char *skbuff_data_pool[MAX_POOL_SIZE];
int skbPoolHead, skbPoolTail;
//skb_init()
//I add the following codes:
for(i=0;i<MAX_POOL_SIZE;i++)
skbuff_data_pool[i] = NULL;
skbPoolHead = skbPoolTail = 0;
//alloc_skb()
//I made the following changes
if (skbuff_data_pool[skbPoolHead])
{
data = skbuff_data_pool[skbPoolHead];
skbuff_data_pool[skbPoolHead] = NULL;
if (++skbPoolHead == MAX_POOL_SIZE)
skbPoolHead = 0;
}
else{ //Original path
size = SKB_DATA_ALIGN(size);
data = kmalloc(size + sizeof(struct skb_shared_info), gfp_mask);
}
//skb_release_data()
//I made the following changes:
if ((skbPoolHead == skbPoolTail) && (skbuff_data_pool[skbPoolHead] !=
NULL))
//Original path
kfree(skb->head);
else{
if (skbuff_data_pool[skbPoolTail])
panic("Tail pointer must be null");
skbuff_data_pool[skbPoolTail] = skb->head;
if (++skbPoolTail == MAX_POOL_SIZE)
skbPoolTail = 0;
}
----- Original Message -----
From: "Eric Dumazet" <[email protected]>
To: "LeoY" <[email protected]>
Cc: <[email protected]>
Sent: Saturday, October 08, 2005 2:47 PM
Subject: Re: Is this skb recycle buffer helpful to improve Linux network
stack performance?
> LeoY a ?crit :
>> Hi,
>>
>> Motivation: we noticed alloc_skb()/kfree() used lots of clock ticks when
>> handling heavy network traffic. As Linux kernel always need to call
>> kmalloc()/kfree() to allocate and deallocate a skb DATA buffer(not
>> sk_buff)
>> for each incoming/outgoing packet, we try to reduce the frequence of
>> calling
>> these memory functions.
>>
>> I wangt to set up a ring buffer in Linux kernel(skbuff.c) and recycle
>> those
>> skb data buffers. The basic idea is as follows:
>> 1. Create a ring buffer. This ring buffer has a head pointer which points
>> to
>> the virtual address of the data buffer to be reused; It also has a tail
>> pointer, which can be used to store the virutal address of skb data
>> buffer
>> for those transmitted packets.
>> 2. If the ring buffer is full, just use normal kmalloc()/kfree()
>> operation
>> to manager those skb data buffers instead of recycling them.
>> 3. if any DATA buffer is available, Instead of calling kmalloc(), assign
>> a
>> skb data buffer directly from ring buffer to the incoming packets.
>> 4. If ring buffer still has space, Instead of calling kfree(), store the
>> skb
>> data buffer into the ring buffer.
>> 5. if the head and tail pointer overlap and head pointer is not empty,
>> just
>> stop accpeting more DATA buffer until some DATA buffer is used for the
>> incoming packets.
>>
>> I tested my method on the latest Linux kernel 2.6.13.3, it works with the
>> normal traffic; However, the Linux kernel crashed under the heavy network
>> traffic.
>>
>> Any idea to make this ring bufer work under the heavy network traffic?
>
> Your idea seems interesting, but you forgot to post a link to the code you
> wrote. How do you want us to help you ?
>
> Eric
>
LeoY a ?crit :
> Here is the modifications I made:
> //skbuff.h
> //I added the folloing definitions
> #define MAX_POOL_SIZE 4096
> unsigned char *skbuff_data_pool[MAX_POOL_SIZE];
> int skbPoolHead, skbPoolTail;
>
>
> //skb_init()
> //I add the following codes:
> for(i=0;i<MAX_POOL_SIZE;i++)
> skbuff_data_pool[i] = NULL;
> skbPoolHead = skbPoolTail = 0;
>
>
> //alloc_skb()
> //I made the following changes
> if (skbuff_data_pool[skbPoolHead])
> {
> data = skbuff_data_pool[skbPoolHead];
> skbuff_data_pool[skbPoolHead] = NULL;
> if (++skbPoolHead == MAX_POOL_SIZE)
> skbPoolHead = 0;
> }
> else{ //Original path
> size = SKB_DATA_ALIGN(size);
> data = kmalloc(size + sizeof(struct skb_shared_info), gfp_mask);
> }
>
> //skb_release_data()
> //I made the following changes:
> if ((skbPoolHead == skbPoolTail) && (skbuff_data_pool[skbPoolHead] !=
> NULL))
> //Original path
> kfree(skb->head);
> else{
> if (skbuff_data_pool[skbPoolTail])
> panic("Tail pointer must be null");
>
> skbuff_data_pool[skbPoolTail] = skb->head;
> if (++skbPoolTail == MAX_POOL_SIZE)
> skbPoolTail = 0;
> }
>
Hum... Lot of problems I think
Are you aware that skb_alloc() / skb_free() can handle data buffers of
different sizes ?
So if you kmalloc() a small buffer, and store it later in your ring buffer,
you should not give it back to a caller that need a biger buffer.
If you want your 'ring buffer'to work, I suspect you should ignore the 'size'
and let it be the max possible size handled in your machine.
Then, dont forget about SMP and IRQs : I dont see in your code how you protect
against concurrent processors accessing your ring buffers, and how you protect
against IRQ (since a nic handler can runs on IRQ or softirq context)
kmalloc()/kfree() are SMP safe.
> ----- Original Message ----- From: "Eric Dumazet" <[email protected]>
> To: "LeoY" <[email protected]>
> Cc: <[email protected]>
> Sent: Saturday, October 08, 2005 2:47 PM
> Subject: Re: Is this skb recycle buffer helpful to improve Linux network
> stack performance?
>
>
>> LeoY a ?crit :
>>
>>> Hi,
>>>
>>> Motivation: we noticed alloc_skb()/kfree() used lots of clock ticks when
>>> handling heavy network traffic. As Linux kernel always need to call
>>> kmalloc()/kfree() to allocate and deallocate a skb DATA buffer(not
>>> sk_buff)
>>> for each incoming/outgoing packet, we try to reduce the frequence of
>>> calling
>>> these memory functions.
>>>
>>> I wangt to set up a ring buffer in Linux kernel(skbuff.c) and recycle
>>> those
>>> skb data buffers. The basic idea is as follows:
>>> 1. Create a ring buffer. This ring buffer has a head pointer which
>>> points to
>>> the virtual address of the data buffer to be reused; It also has a tail
>>> pointer, which can be used to store the virutal address of skb data
>>> buffer
>>> for those transmitted packets.
>>> 2. If the ring buffer is full, just use normal kmalloc()/kfree()
>>> operation
>>> to manager those skb data buffers instead of recycling them.
>>> 3. if any DATA buffer is available, Instead of calling kmalloc(),
>>> assign a
>>> skb data buffer directly from ring buffer to the incoming packets.
>>> 4. If ring buffer still has space, Instead of calling kfree(), store
>>> the skb
>>> data buffer into the ring buffer.
>>> 5. if the head and tail pointer overlap and head pointer is not
>>> empty, just
>>> stop accpeting more DATA buffer until some DATA buffer is used for the
>>> incoming packets.
>>>
>>> I tested my method on the latest Linux kernel 2.6.13.3, it works with
>>> the
>>> normal traffic; However, the Linux kernel crashed under the heavy
>>> network
>>> traffic.
>>>
>>> Any idea to make this ring bufer work under the heavy network traffic?
>>
>>
>> Your idea seems interesting, but you forgot to post a link to the code
>> you wrote. How do you want us to help you ?
>>
>> Eric
>>
>