It appears from my initial tests on 2.4.20-pre4, that packets sent on
one port, and received on another port on the same machine (via a cross-over
cable), can be re-ordered. I see about 2000 reordered packets per 5,000,000 packets
sent (sending about 70,000 packets-per-second on a dual-port e1000 NIC.)
By re-ordered, I mean that a method called from process_backlog in dev.c
is being handed packets in a different order than they are being poked into
the driver with hard_start_xmit on the other interface. If each CPU can be running the
process_backlog, then I can see how this could be happening.
1) Is this expected behaviour?
2) Is there any standard (ie configurable) way to enforce strict ordering on an
SMP system?
3) If answer to 2 is no, would you all be interested in a patch that
did allow strict ordering (if indeed I can figure out how to write one)?
Thanks,
Ben
--
Ben Greear <[email protected]> <Ben_Greear AT excite.com>
President of Candela Technologies Inc http://www.candelatech.com
ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
Use IRQ affinity settings if you want strict packet receive ordering.
On Sun, 2002-08-25 at 08:18, Ben Greear wrote:
> By re-ordered, I mean that a method called from process_backlog in dev.c
> is being handed packets in a different order than they are being poked into
> the driver with hard_start_xmit on the other interface. If each CPU can be running the
> process_backlog, then I can see how this could be happening.
>
>
> 1) Is this expected behaviour?
Yes
> 2) Is there any standard (ie configurable) way to enforce strict ordering on an
> SMP system?
No
> 3) If answer to 2 is no, would you all be interested in a patch that
> did allow strict ordering (if indeed I can figure out how to write one)?
You should never need it. Ethernet, hubs, switches, routers, internet
backbones etc will all cause packet re-ordering. You should also expect
the percentage of re-ordered frames on the net to rise and rise.
NAPI fixes packet reordering problems.
Could people please post network related questions to netdev please?
I think it even says so in the FAQ Richard Gooch maintains.
Believe it or not, quiet a few people are not subscribed to lk
cheers,
jamal
Alan Cox wrote:
> On Sun, 2002-08-25 at 08:18, Ben Greear wrote:
>
>>By re-ordered, I mean that a method called from process_backlog in dev.c
>>is being handed packets in a different order than they are being poked into
>>the driver with hard_start_xmit on the other interface. If each CPU can be running the
>>process_backlog, then I can see how this could be happening.
>>
>>
>>1) Is this expected behaviour?
>
> Yes
>
>
>>2) Is there any standard (ie configurable) way to enforce strict ordering on an
>> SMP system?
>
> No
>
>
>>3) If answer to 2 is no, would you all be interested in a patch that
>> did allow strict ordering (if indeed I can figure out how to write one)?
>
>
> You should never need it. Ethernet, hubs, switches, routers, internet
> backbones etc will all cause packet re-ordering. You should also expect
> the percentage of re-ordered frames on the net to rise and rise.
I would like to detect the number of pkts that such backbone hardware does
re-order, so if my end machine is also re-ordering, I cannot get valid
numbers.
Thanks,
Ben
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
Ben Greear <[email protected]> <Ben_Greear AT excite.com>
President of Candela Technologies Inc http://www.candelatech.com
ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
On Sun, 2002-08-25 at 18:00, Ben Greear wrote:
> I would like to detect the number of pkts that such backbone hardware does
> re-order, so if my end machine is also re-ordering, I cannot get valid
> numbers.
Good point
From: Alan Cox <[email protected]>
Date: 25 Aug 2002 15:41:13 +0100
> 3) If answer to 2 is no, would you all be interested in a patch that
> did allow strict ordering (if indeed I can figure out how to write one)?
You should never need it. Ethernet, hubs, switches, routers, internet
backbones etc will all cause packet re-ordering. You should also expect
the percentage of re-ordered frames on the net to rise and rise.
But to be honest, NAPI was meant to dramatically reduce the "SMP
specific" causes of packet reordering. When a driver does use NAPI,
the reordering due to SMP goes way down.