On Fri, Oct 01, 2021 at 10:27:12AM -0700, Vinicius Costa Gomes wrote:
> Xiaoliang Yang <[email protected]> writes:
>
> > Hi Vinicius,
> >
> > On Sep 29, 2021 at 6:35:59 +0000, Vinicius Costa Gomes wrote:
> >> > This patch introduce a frer action to implement frame replication and
> >> > elimination for reliability, which is defined in IEEE P802.1CB.
> >> >
> >>
> >> An action seems, to me, a bit too limiting/fine grained for a frame replication
> >> and elimination feature.
> >>
> >> At least I want to hear the reasons that the current hsr/prp support cannot be
> >> extended to support one more tag format/protocol.
> >>
> >> And the current name for the spec is IEEE 802.1CB-2017.
> >>
> > 802.1CB can be set on bridge ports, and need to use bridge forward
> > Function as a relay system. It only works on identified streams,
> > unrecognized flows still need to pass through the bridged network
> > normally.
>
> This ("only on identified streams") is the strongest argument so far to
> have FRER also as an action, in adition to the current hsr netdevice
> approach.
>
> >
> > But current hsr/prp seems only support two ports, and cannot use the
> > ports in bridge. It's hard to implement FRER functions on current HSR
> > driver.
>
> That the hsr netdevice only support two ports, I think is more a bug
> than a design issue. Which will need to get fixed at some point.
What do you mean 'a bug'? HSR and PRP, as protocols, use _two_ ports,
see IEC 62439-3, that's where the "D" (doubly attached node) in DANH and
DANP comes from. There's no TANH/TANH for "triply attached node".
It doesn't scale.
> Speaking of functions, one thing that might be interesting is trying to
> see if it makes sense to make part of the current hsr functionality a
> "library" so it can be used by tc-frer as well. (less duplication of
> bugs).
You mean tc-frer should inherit from the get-go the plethora of bugs
from the unmaintained hsr driver? :)
That would be good for hsr, which is in a pretty poor state, but the
design of the 802.1CB spec isn't really in its favor sadly.
> >
> > You can see chapter "D.2 Example 2: Various stack positions" in IEEE 802.1CB-2017,
> > Protocol stack for relay system is like follows:
> >
> > Stream Transfer Function
> > | |
> > | Sequence generation
> > | Sequence encode/decode
> > Stream identification Active Stream identification
> > | |
> > | Internal LAN---- Relay system forwarding
> > | | |
> > MAC MAC MAC
> >
> > Use port actions to easily implement FRER tag add/delete, split, and
> > recover functions.
> >
> > Current HSR/PRP driver can be used for port HSR/PRP set, and tc-frer
> > Action to be used for stream RTAG/HSR/PRP set and recover.
>
> I am still reading the spec and trying to imagine how things would fit
> together:
> - for which use cases tc-frer would be useful;
> - for which use cases the hsr netdevice would be useful;
> - would it make sense to have them in the same system?
You could use FRER in networks where normally you'd use HSR (aka rings).
In fact the 802.1CB demonstration I have, which uses the NXP tsntool
program with the downstream genetlink tsn interface, does exactly that:
https://github.com/vladimiroltean/tsn-scripts
Basically FRER is IEEE's take on redundancy protocols and more like a
generalization of HSR/PRP, the big changes are:
- not limited to two (or any number of) ports
- more than one type of stream/flow identification function: can look at
source/destination MAC, source/destination IP, VLAN, and most
importantly, there can be passive stream identification functions (don't
modify the packet) and active stream identification functions (do
modify the packet).
Please note that we've already started modeling IEEE 802.1CB stream
identification functions as tc flower filters, since those map nicely on top.
We use these for PSFP (former 802.1Qci) tc-police and tc-gate actions
(yes, tc-police is single-bucket and color-unaware, that needs to be improved).
Basically IEEE 802.1CB is a huge toolbox, the spec gives you the tools
but it doesn't tell you how to use them, that's why the stream
identification functions are so generic and decoupled from the
redundancy protocol itself.
In both HSR and PRP, sequence numbers are kept per source MAC address,
that is absolutely baken into the standard.
But think about this. When the sequence number is kept per source
station, frames sent from node A to multiple destinations (nodes B and C)
will be part of the same stream. So nodes B and C will see
discontinuities in the sequence numbers when node A talks to them.
The opposite is true as well. When sequence numbers are kept per
destination MAC address, then frames sent from multiple talkers (nodes A
and B) to the same destination (node C) will be interpreted as part of
the same stream by the listener. So there will be jumps in sequence
numbers seen by C when A and B are simultaneously transmitting to it.
Which type of stream identification you need depends on the traffic you
need to support, and the topology.
So again, IEEE 802.1CB doesn't tell you what to do, but it gives you the
tools. You can do source MAC based stream identification, and you can
emulate HSR, or you can do something that encompasses both source node
information as well as destination node information.
It's one whole degree of freedom more flexible, plain and simple.
And the topologies are not limited to:
- the rings that HSR supports
- the disjoint IP networks that PRP supports
but are rather generic graphs.
I fully expect there to be hardware out there already that can convert
between the HSR/PRP frame format on one set of ports to 802.1CB frame
format on another set of ports. Maybe that's something that some thought
needs to be put into.
On Fri, Oct 01, 2021 at 08:55:24PM +0300, Vladimir Oltean wrote:
> On Fri, Oct 01, 2021 at 10:27:12AM -0700, Vinicius Costa Gomes wrote:
> > Xiaoliang Yang <[email protected]> writes:
> >
> > > Hi Vinicius,
> > >
> > > On Sep 29, 2021 at 6:35:59 +0000, Vinicius Costa Gomes wrote:
> > >> > This patch introduce a frer action to implement frame replication and
> > >> > elimination for reliability, which is defined in IEEE P802.1CB.
> > >> >
> > >>
> > >> An action seems, to me, a bit too limiting/fine grained for a frame replication
> > >> and elimination feature.
> > >>
> > >> At least I want to hear the reasons that the current hsr/prp support cannot be
> > >> extended to support one more tag format/protocol.
> > >>
> > >> And the current name for the spec is IEEE 802.1CB-2017.
> > >>
> > > 802.1CB can be set on bridge ports, and need to use bridge forward
> > > Function as a relay system. It only works on identified streams,
> > > unrecognized flows still need to pass through the bridged network
> > > normally.
> >
> > This ("only on identified streams") is the strongest argument so far to
> > have FRER also as an action, in adition to the current hsr netdevice
> > approach.
> >
> > >
> > > But current hsr/prp seems only support two ports, and cannot use the
> > > ports in bridge. It's hard to implement FRER functions on current HSR
> > > driver.
> >
> > That the hsr netdevice only support two ports, I think is more a bug
> > than a design issue. Which will need to get fixed at some point.
>
> What do you mean 'a bug'? HSR and PRP, as protocols, use _two_ ports,
> see IEC 62439-3, that's where the "D" (doubly attached node) in DANH and
> DANP comes from. There's no TANH/TANH for "triply attached node".
> It doesn't scale.
>
> > Speaking of functions, one thing that might be interesting is trying to
> > see if it makes sense to make part of the current hsr functionality a
> > "library" so it can be used by tc-frer as well. (less duplication of
> > bugs).
>
> You mean tc-frer should inherit from the get-go the plethora of bugs
> from the unmaintained hsr driver? :)
>
> That would be good for hsr, which is in a pretty poor state, but the
> design of the 802.1CB spec isn't really in its favor sadly.
>
> > >
> > > You can see chapter "D.2 Example 2: Various stack positions" in IEEE 802.1CB-2017,
> > > Protocol stack for relay system is like follows:
> > >
> > > Stream Transfer Function
> > > | |
> > > | Sequence generation
> > > | Sequence encode/decode
> > > Stream identification Active Stream identification
> > > | |
> > > | Internal LAN---- Relay system forwarding
> > > | | |
> > > MAC MAC MAC
> > >
> > > Use port actions to easily implement FRER tag add/delete, split, and
> > > recover functions.
> > >
> > > Current HSR/PRP driver can be used for port HSR/PRP set, and tc-frer
> > > Action to be used for stream RTAG/HSR/PRP set and recover.
> >
> > I am still reading the spec and trying to imagine how things would fit
> > together:
> > - for which use cases tc-frer would be useful;
> > - for which use cases the hsr netdevice would be useful;
> > - would it make sense to have them in the same system?
>
> You could use FRER in networks where normally you'd use HSR (aka rings).
> In fact the 802.1CB demonstration I have, which uses the NXP tsntool
> program with the downstream genetlink tsn interface, does exactly that:
> https://github.com/vladimiroltean/tsn-scripts
>
> Basically FRER is IEEE's take on redundancy protocols and more like a
> generalization of HSR/PRP, the big changes are:
> - not limited to two (or any number of) ports
> - more than one type of stream/flow identification function: can look at
> source/destination MAC, source/destination IP, VLAN, and most
> importantly, there can be passive stream identification functions (don't
> modify the packet) and active stream identification functions (do
> modify the packet).
>
> Please note that we've already started modeling IEEE 802.1CB stream
> identification functions as tc flower filters, since those map nicely on top.
> We use these for PSFP (former 802.1Qci) tc-police and tc-gate actions
> (yes, tc-police is single-bucket and color-unaware, that needs to be improved).
>
> Basically IEEE 802.1CB is a huge toolbox, the spec gives you the tools
> but it doesn't tell you how to use them, that's why the stream
> identification functions are so generic and decoupled from the
> redundancy protocol itself.
>
> In both HSR and PRP, sequence numbers are kept per source MAC address,
> that is absolutely baken into the standard.
>
> But think about this. When the sequence number is kept per source
> station, frames sent from node A to multiple destinations (nodes B and C)
> will be part of the same stream. So nodes B and C will see
> discontinuities in the sequence numbers when node A talks to them.
>
> The opposite is true as well. When sequence numbers are kept per
> destination MAC address, then frames sent from multiple talkers (nodes A
> and B) to the same destination (node C) will be interpreted as part of
> the same stream by the listener. So there will be jumps in sequence
> numbers seen by C when A and B are simultaneously transmitting to it.
>
> Which type of stream identification you need depends on the traffic you
> need to support, and the topology.
>
> So again, IEEE 802.1CB doesn't tell you what to do, but it gives you the
> tools. You can do source MAC based stream identification, and you can
> emulate HSR, or you can do something that encompasses both source node
> information as well as destination node information.
>
> It's one whole degree of freedom more flexible, plain and simple.
> And the topologies are not limited to:
> - the rings that HSR supports
> - the disjoint IP networks that PRP supports
> but are rather generic graphs.
>
> I fully expect there to be hardware out there already that can convert
> between the HSR/PRP frame format on one set of ports to 802.1CB frame
> format on another set of ports. Maybe that's something that some thought
> needs to be put into.
And Xiaoliang, can you PLEASE remove the following email addresses from
further submissions you make:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
You also copied some of them on all 6 submissions for the PSFP offload series.
It gets really annoying to get email bounces from these addresses.
I've removed them from this email thread. Thanks.
Hi,
Vladimir Oltean <[email protected]> writes:
> On Fri, Oct 01, 2021 at 10:27:12AM -0700, Vinicius Costa Gomes wrote:
>> Xiaoliang Yang <[email protected]> writes:
>>
>> > Hi Vinicius,
>> >
>> > On Sep 29, 2021 at 6:35:59 +0000, Vinicius Costa Gomes wrote:
>> >> > This patch introduce a frer action to implement frame replication and
>> >> > elimination for reliability, which is defined in IEEE P802.1CB.
>> >> >
>> >>
>> >> An action seems, to me, a bit too limiting/fine grained for a frame replication
>> >> and elimination feature.
>> >>
>> >> At least I want to hear the reasons that the current hsr/prp support cannot be
>> >> extended to support one more tag format/protocol.
>> >>
>> >> And the current name for the spec is IEEE 802.1CB-2017.
>> >>
>> > 802.1CB can be set on bridge ports, and need to use bridge forward
>> > Function as a relay system. It only works on identified streams,
>> > unrecognized flows still need to pass through the bridged network
>> > normally.
>>
>> This ("only on identified streams") is the strongest argument so far to
>> have FRER also as an action, in adition to the current hsr netdevice
>> approach.
>>
>> >
>> > But current hsr/prp seems only support two ports, and cannot use the
>> > ports in bridge. It's hard to implement FRER functions on current HSR
>> > driver.
>>
>> That the hsr netdevice only support two ports, I think is more a bug
>> than a design issue. Which will need to get fixed at some point.
>
> What do you mean 'a bug'? HSR and PRP, as protocols, use _two_ ports,
> see IEC 62439-3, that's where the "D" (doubly attached node) in DANH and
> DANP comes from. There's no TANH/TANH for "triply attached node".
> It doesn't scale.
First of all, thank you for taking the time to write such detailed
answer, really helpful.
Another spec that I should take some time and read if I want to keep
commenting on this stuff.
>
>> Speaking of functions, one thing that might be interesting is trying to
>> see if it makes sense to make part of the current hsr functionality a
>> "library" so it can be used by tc-frer as well. (less duplication of
>> bugs).
>
> You mean tc-frer should inherit from the get-go the plethora of bugs
> from the unmaintained hsr driver? :)
>
> That would be good for hsr, which is in a pretty poor state, but the
> design of the 802.1CB spec isn't really in its favor sadly.
>
Fair enough.
So what I am going to suggest is for you folks to write in the RFC how
to use tc-frer (the "toolbox" idea) in "IEC 62439-9 mode", not necessary
to implement it, just to write it down. The idea is that we have a path
forward to better maintained alternatives, as you said, if we stop
recommending people to use/experiment with net/hsr.
>> >
>> > You can see chapter "D.2 Example 2: Various stack positions" in IEEE 802.1CB-2017,
>> > Protocol stack for relay system is like follows:
>> >
>> > Stream Transfer Function
>> > | |
>> > | Sequence generation
>> > | Sequence encode/decode
>> > Stream identification Active Stream identification
>> > | |
>> > | Internal LAN---- Relay system forwarding
>> > | | |
>> > MAC MAC MAC
>> >
>> > Use port actions to easily implement FRER tag add/delete, split, and
>> > recover functions.
>> >
>> > Current HSR/PRP driver can be used for port HSR/PRP set, and tc-frer
>> > Action to be used for stream RTAG/HSR/PRP set and recover.
>>
>> I am still reading the spec and trying to imagine how things would fit
>> together:
>> - for which use cases tc-frer would be useful;
>> - for which use cases the hsr netdevice would be useful;
>> - would it make sense to have them in the same system?
>
> You could use FRER in networks where normally you'd use HSR (aka rings).
> In fact the 802.1CB demonstration I have, which uses the NXP tsntool
> program with the downstream genetlink tsn interface, does exactly that:
> https://github.com/vladimiroltean/tsn-scripts
>
After a very quick look, interesting stuff here. Will take a better
look. (even more reading for the weekend)
> Basically FRER is IEEE's take on redundancy protocols and more like a
> generalization of HSR/PRP, the big changes are:
> - not limited to two (or any number of) ports
> - more than one type of stream/flow identification function: can look at
> source/destination MAC, source/destination IP, VLAN, and most
> importantly, there can be passive stream identification functions (don't
> modify the packet) and active stream identification functions (do
> modify the packet).
>
> Please note that we've already started modeling IEEE 802.1CB stream
> identification functions as tc flower filters, since those map nicely on top.
> We use these for PSFP (former 802.1Qci) tc-police and tc-gate actions
> (yes, tc-police is single-bucket and color-unaware, that needs to be improved).
>
> Basically IEEE 802.1CB is a huge toolbox, the spec gives you the tools
> but it doesn't tell you how to use them, that's why the stream
> identification functions are so generic and decoupled from the
> redundancy protocol itself.
>
> In both HSR and PRP, sequence numbers are kept per source MAC address,
> that is absolutely baken into the standard.
>
> But think about this. When the sequence number is kept per source
> station, frames sent from node A to multiple destinations (nodes B and C)
> will be part of the same stream. So nodes B and C will see
> discontinuities in the sequence numbers when node A talks to them.
>
> The opposite is true as well. When sequence numbers are kept per
> destination MAC address, then frames sent from multiple talkers (nodes A
> and B) to the same destination (node C) will be interpreted as part of
> the same stream by the listener. So there will be jumps in sequence
> numbers seen by C when A and B are simultaneously transmitting to it.
>
> Which type of stream identification you need depends on the traffic you
> need to support, and the topology.
Good insight here. Even if I can imagine those simple stream identification
functions working on simple topologies, I totally get you point.
>
> So again, IEEE 802.1CB doesn't tell you what to do, but it gives you the
> tools. You can do source MAC based stream identification, and you can
> emulate HSR, or you can do something that encompasses both source node
> information as well as destination node information.
>
> It's one whole degree of freedom more flexible, plain and simple.
> And the topologies are not limited to:
> - the rings that HSR supports
> - the disjoint IP networks that PRP supports
> but are rather generic graphs.
>
> I fully expect there to be hardware out there already that can convert
> between the HSR/PRP frame format on one set of ports to 802.1CB frame
> format on another set of ports. Maybe that's something that some thought
> needs to be put into.
In short, I am reasonably satisfied with the proposal that tc-frer
offers a superset of net/hsr can do.
Suggestions for the cover letter:
- Expand a bit on the whole superset/toolbox idea;
- Document how to use the toolbox to emulate HSR/PRP;
Cheers,
--
Vinicius