Hello Xiongwei,
On Tue, Mar 07, 2023 at 12:11:52PM +0000, Song, Xiongwei wrote:
> ......snip......
> failing SW:
> rx_octets +64
> rx_unicast +1
> rx_frames_below_65_octets +1
> rx_yellow_prio_0 +1
> *drop_yellow_prio_0 +1
> ......snip......
>
> 3). From pcap file(the pcap was collected on the senderside (VM))
>
> Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> Type: 802.1Q Virtual LAN (0x8100)
> 802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
> 110. .... .... .... = Priority: Internetwork Control (6)
> ...0 .... .... .... = DEI: Ineligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: ARP (0x0806)
> Padding: 0000000000000000000000000000
> Trailer: 00000000
>
> Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...0 .... .... .... = DEI: Ineligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: ARP (0x0806)
>
> Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...1 .... .... .... = DEI: Eligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: IPv4 (0x0800)
>
> Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> 000. .... .... .... = Priority: Best Effort (default) (0)
> ...1 .... .... .... = DEI: Eligible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> .... 0011 1101 0101 = ID: 981
> Type: IPv4 (0x0800)
>
> 4). What we've found so far
>
> According binary search, we found out the following commit causes this issue:
> a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
> Without this commit the test case was passed.
>
> Could you please take a look? Please let me know if you need more debug info.
I've marked the DEI values in the message you posted above.
Commit a4ae997adcbd ("net: mscc: ocelot: initialize watermarks to sane defaults")
tells the hardware to not allow frames with DEI=1 consume from the shared switch
resources (buffers / frame references) by default. Drop Eligible Indicator = 1
means "eligible for dropping". The only chance for DEI=1 frames to not be dropped
is to set up a resource reservation for that stream, via the devlink-sb command.
Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
field is part of the intentions of the test? Is there any RFC which says
that IP fragments over VLAN should use DEI=1, or some other reason?
Hello Vladimir,
Thanks for the quick response.
> -----Original Message-----
> From: Vladimir Oltean <[email protected]>
> Sent: Tuesday, March 7, 2023 8:35 PM
> To: Song, Xiongwei <[email protected]>
> Cc: [email protected]; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected]; [email protected]; linux-
> [email protected]; [email protected]
> Subject: Re: Response error to fragmented ICMP echo request
>
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the
> content is safe.
>
> Hello Xiongwei,
>
> On Tue, Mar 07, 2023 at 12:11:52PM +0000, Song, Xiongwei wrote:
> > ......snip......
> > failing SW:
> > rx_octets +64
> > rx_unicast +1
> > rx_frames_below_65_octets +1
> > rx_yellow_prio_0 +1
> > *drop_yellow_prio_0 +1
> > ......snip......
> >
> > 3). From pcap file(the pcap was collected on the senderside (VM))
> >
> > Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> > Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c
> (aa:3a:b3:e7:67:5c)
> > Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
> > Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
> > Type: 802.1Q Virtual LAN (0x8100)
> > 802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
> > 110. .... .... .... = Priority: Internetwork Control (6)
> > ...0 .... .... .... = DEI: Ineligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: ARP (0x0806)
> > Padding: 0000000000000000000000000000
> > Trailer: 00000000
> >
> > Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...0 .... .... .... = DEI: Ineligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: ARP (0x0806)
> >
> > Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...1 .... .... .... = DEI: Eligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: IPv4 (0x0800)
> >
> > Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
> > Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f
> (7c:72:6e:d4:44:5f)
> > 802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
> > 000. .... .... .... = Priority: Best Effort (default) (0)
> > ...1 .... .... .... = DEI: Eligible
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> > .... 0011 1101 0101 = ID: 981
> > Type: IPv4 (0x0800)
> >
> > 4). What we've found so far
> >
> > According binary search, we found out the following commit causes this issue:
> > a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
> > Without this commit the test case was passed.
> >
> > Could you please take a look? Please let me know if you need more debug info.
>
> I've marked the DEI values in the message you posted above.
>
> Commit a4ae997adcbd ("net: mscc: ocelot: initialize watermarks to sane defaults")
> tells the hardware to not allow frames with DEI=1 consume from the shared switch
> resources (buffers / frame references) by default. Drop Eligible Indicator = 1
> means "eligible for dropping". The only chance for DEI=1 frames to not be dropped
> is to set up a resource reservation for that stream, via the devlink-sb command.
Oh.., thanks for the detailed explanation.
>
> Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> field is part of the intentions of the test? Is there any RFC which says
> that IP fragments over VLAN should use DEI=1, or some other reason?
I didn't notice that. Let me check the test why set DEI=1.
Have a good day.
Regards,
Xiongwei
On Tue, Mar 07, 2023 at 12:56:17PM +0000, Song, Xiongwei wrote:
> > Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> > sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> > field is part of the intentions of the test? Is there any RFC which says
> > that IP fragments over VLAN should use DEI=1, or some other reason?
>
> I didn't notice that. Let me check the test why set DEI=1.
Ok. It would be good to have an answer to this, because one of the
assumptions of that patch was that whomever sets DEI=1 doesn't get to
complain that their packets are *actually* dropped :)
FWIW, if you do need to set up a reservation for traffic received on a
port, section 8.6.3.3.15 Buffer reservation watermarks (page 817) should
help with this:
https://www.nxp.com/docs/en/user-guide/LLDPUG_RevL5.15.71-2.2.0.pdf
Hi Vladimir,
> On Tue, Mar 07, 2023 at 12:56:17PM +0000, Song, Xiongwei wrote:
> > > Frames 3 and 4 are sent with DEI=1 and are dropped, frames 1 and 2 are
> > > sent with DEI=0 and are not dropped. I'm not sure if varying the DEI
> > > field is part of the intentions of the test? Is there any RFC which says
> > > that IP fragments over VLAN should use DEI=1, or some other reason?
> >
> > I didn't notice that. Let me check the test why set DEI=1.
>
> Ok. It would be good to have an answer to this, because one of the
> assumptions of that patch was that whomever sets DEI=1 doesn't get to
> complain that their packets are *actually* dropped :)
We found the problem in the test:
=====================
f2=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x00\x00\x00\x00\x00\x00\x00\x00'
f2.frag=1
===>f2.id=1
f2.proto='icmp'
// result of f2.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags =
frag = 1
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x00\x00\x00\x00\x00\x00\x00\x00'
=================
This sets both the L2 & L3 id. L2 id is actually DEI.
We updated the script to only set L3 id and now the test passes.
>
> FWIW, if you do need to set up a reservation for traffic received on a
> port, section 8.6.3.3.15 Buffer reservation watermarks (page 817) should
> help with this:
> https://www.nxp.com/docs/en/user-guide/LLDPUG_RevL5.15.71-2.2.0.pdf
Thank you so much. We don't need to set up a reservation for now, but good
to know this.
Regards,
Xiongwei