2001-12-01 00:33:59

by bert hubert

[permalink] [raw]
Subject: Finally, CBQ nearly completely documented

Hi,

After preparing my talk on CBQ/HTB (http://ds9a.nl/cbq-presentation ), I
finally understood how CBQ and filters etc truly work. And I wrote it down.
Check out the Linux Advanced Routing & Shaping HOWTO, it's changed a lot!

Especially this part is very new, please check it for mistakes and
inconsistencies:

http://ds9a.nl/2.4Routing/HOWTO//cvs/2.4routing/output/2.4routing-9.html

I even got 'split' and 'defmap' figured out, which should be a first. There
is not a single other page online that tells you correctly what these do.

One thing - does *anybody* understand how hash tables work in tc filter, and
what they do? Furthermore, I could use some help with the tc filter police
things.

So if you do understand how these work, please drop me a line.

Thanks!

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet


2001-12-03 08:51:48

by Jim Fleming

[permalink] [raw]
Subject: Re: [LARTC] CBQ and all other qdiscs now REALLY completely documented (almost!)

----- Original Message -----
From: "bert hubert" <[email protected]>
>
> The only thing left to document are Policing filters.
>

This may help...
http://www.dot-biz.com/IPv4/Tutorial/


Jim Fleming
http://www.IPv8.info
IPv16....One Better !!

2001-12-03 08:48:37

by bert hubert

[permalink] [raw]
Subject: CBQ and all other qdiscs now REALLY completely documented (almost!)

On Sat, Dec 01, 2001 at 01:33:41AM +0100, bert hubert wrote:

> One thing - does *anybody* understand how hash tables work in tc filter, and
> what they do? Furthermore, I could use some help with the tc filter police
> things.

Thanks to Andreas Steinmetz and David Sauer, tc hash tables are now
documented as well, thanks!

See:

http://ds9a.nl/2.4Routing/HOWTO//cvs/2.4routing/output/2.4routing-12.html

And then 'Hashing filters for very fast massive filtering'.

I also finished documenting all parameters for TBF, CBQ, SFQ, PRIO,
bfifo, pfifo and pfifo_fast. All queues in the Linux kernel are now
described in the Linux Advanced Routing & Shaping HOWTO, which can be found on

http://ds9a.nl/2.4Routing

I want to send this off to the LDP and Freshmeat somewhere next week, I
*would really* like people who are knowledgeable about this subject (this
means you, ANK & Jamal 8) ) to read through this.

This HOWTO is rapidly becoming the perceived authoritative source for
traffic control in linux (google on 'Linux Routing' finds it), it might as
well be right! So if you have any time at all, check the parts you know
about. I expect mistakes.

The parts of the table of contents that document stuff in the kernel not
documented elsewhere:

9. Queueing Disciplines for Bandwidth Management
9.1 Queues and Queueing Disciplines explained
9.2 Simple, classless Queueing Disciplines
9.2.1 pfifo_fast
9.2.1.1 Parameters & usage
9.2.2 Token Bucket Filter
9.2.2.1 Parameters & usage
9.2.2.2 Sample configuration
9.2.3 Stochastic Fairness Queueing
9.2.3.1 Parameters & usage
9.2.3.2 Sample configuration
9.3 Advice for when to use which queue
9.4 Classful Queueing Disciplines
9.4.1 Flow within classful qdiscs & classes
9.4.2 The qdisc family: roots, handles, siblings and parents
9.4.2.1 How filters are used to classify traffic
9.4.2.2 How packets are dequeued to the hardware
9.4.3 The PRIO qdisc
9.4.3.1 PRIO parameters & usage
9.4.3.2 Sample configuration
9.4.4 The famous CBQ qdisc
9.4.4.1 CBQ shaping in detail
9.4.4.2 CBQ classful behaviour
9.4.4.3 CBQ parameters that determine link sharing & borrowing
9.4.4.4 Sample configuration
9.4.4.5 Other CBQ parameters: split & defmap
9.4.5 Hierarchical Token Bucket
9.4.5.1 Sample configuration
9.5 Classifying packets with filters
9.5.1 Some simple filtering examples
9.5.2 All the filtering commands you will normally need

(...)

12. Advanced filters for (re-)classifying packets
12.1 The "u32" classifier
12.1.1 U32 selector
12.1.2 General selectors
12.1.3 Specific selectors
12.2 The "route" classifier
12.3 Policing filters
12.4 Hashing filters for very fast massive filtering

(...)

14. Advanced & less common queueing disciplines
14.1 bfifo/pfifo
14.1.1 Parameters & usage
14.2 Clark-Shenker-Zhang algorithm (CSZ)
14.3 DSMARK
14.3.1 Introduction
14.3.2 What is Dsmark related to?
14.3.3 Differentiated Services guidelines
14.3.4 Working with Dsmark
14.3.5 How SCH_DSMARK works.
14.3.6 TC_INDEX Filter
14.4 Ingress policer qdisc
14.5 Random Early Drop (RED)
14.6 VC/ATM emulation
14.7 Weighted Round Robin (WRR)

The only thing left to document are Policing filters.

Regards,

bert hubert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-08 19:24:18

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented (almost!)



On Mon, 3 Dec 2001, bert hubert wrote:

> On Sat, Dec 01, 2001 at 01:33:41AM +0100, bert hubert wrote:
>
> > One thing - does *anybody* understand how hash tables work in tc filter, and
> > what they do? Furthermore, I could use some help with the tc filter police
> > things.
>
> Thanks to Andreas Steinmetz and David Sauer, tc hash tables are now
> documented as well, thanks!
>
> See:
>
> http://ds9a.nl/2.4Routing/HOWTO//cvs/2.4routing/output/2.4routing-12.html
>
> And then 'Hashing filters for very fast massive filtering'.
>
> I also finished documenting all parameters for TBF, CBQ, SFQ, PRIO,
> bfifo, pfifo and pfifo_fast. All queues in the Linux kernel are now
> described in the Linux Advanced Routing & Shaping HOWTO, which can be found on
>
> http://ds9a.nl/2.4Routing
>
> I want to send this off to the LDP and Freshmeat somewhere next week, I
> *would really* like people who are knowledgeable about this subject (this
> means you, ANK & Jamal 8) ) to read through this.
>
> This HOWTO is rapidly becoming the perceived authoritative source for
> traffic control in linux (google on 'Linux Routing' finds it), it might as
> well be right! So if you have any time at all, check the parts you know
> about. I expect mistakes.
>
> The parts of the table of contents that document stuff in the kernel not
> documented elsewhere:

"not documented elsewhere" comes out rude. Werner and I (and even
Alexey when he was in the mood -- and i have seen some good documentation
by other people as well) have spent numerous hours documenting, presenting
and answering questions on mailing lists at times

Sample docs that i was personally involved in:
ftp://icaftp.epfl.ch/pub/linux/diffserv/misc/dsid-01.txt.gz
You need to introduce the big picture to the user.
and what is wrong with the definitions used in
http://www.davin.ottawa.on.ca/ols/img10.htm that forced you to introduce
your own?
Actually, the big picture is:
http://www.davin.ottawa.on.ca/ols/img9.htm
Also
http://www.linuxjournal.com/article.php?sid=3369
(was written in 98 but got published in 99)

Now despite all the bitching above, i think your efforts are noble.

[My complaints about your style is you often are trying to present facts
by using opinions. For example despite a lot of effort in the past to
explain ingress qdisc to you in the past and, pointing you to very good
documentation from CISCO you still ended using your opinions on what you
thought it should be;->
My scanning of the document shows opinions still posing as miscontrued
facts. It is improving compared to what i saw last when we discussed ingress.
Let me clarify one thing in this email; i'll read what you have later.

Lets start by your description of TC_PRIO and TOS mappings etc:
Your descriptions of these values is insufficient. Consider this a
tutorial and reword it as you wish but please avoid opinions.
Ok here's clarification, this applies to both prio, default fifo 3 band
queueing and CBQ defaultmap classification; applies to both packets being
forwarded as well as locally generated:

First Step:
===========

Define TOS: This is a 4 bit value used as defined in RFC 1349.

0 1 2 3 4 5 6 7
+-----+-----+-----+-----+-----+-----+-----+-----+
| | | |
| PRECEDENCE | TOS | MBZ |
| | | |
+-----+-----+-----+-----+-----+-----+-----+-----+

Then define the values possible as:



1000 -- minimize delay
0100 -- maximize throughput
0010 -- maximize reliability
0001 -- minimize monetary cost
0000 -- normal service

Look at RFC 1349 for typical values used by different applications
Then of course note that RFC 1349 is obsoleted by RFC 2474 (yes, you can
weep);

Having said all that:

Linux remaps packets incoming with different values to some internal
value; the colum "mapped to" shows the internal mapping

8value(hex) TOS(dec) mapped to(dec)
----------------------------------
0x0 0 0
1 7
2 0
3 0
4 2
5 2
6 2
7 2
0x10 8 6
9 6
10 6
11 6
12 2
13 2
14 2
15 2

Fill in the "8value(hex)" column gaps using the bitmap from RFC1349 for
the 8 bits; These are the values ou would see with tcpdump -vvv
I filled the two easiest ones i could compute in my head.

Second step:

Take the default priority map:
1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1
This applies for both default prio and the 3-band FIFO queue.
Note the queue map fitted on the last column

8 but value TOS mapped to queue map
---------------------------------------------
0x0 0 0 1
1 7 2
2 0 2
3 0 2
4 2 1
5 2 2
6 2 0
7 2 0
0x10 8 6 1
9 6 1
10 6 1
11 6 1
12 2 1
13 2 1
14 2 1
15 2 1

Queue 0 gets processed first then queue 1 then queue 2. In the strict
priority processing such as in prio or default 3 band sched, queue 0 is
processed until no more packets are left, then queue1 etc. This could
result in starvation. You could avoid starvation by inserting a TBF
in a prio; limit the size of the fifo in a class or use CBQ configured
as WRR.
I hope the above explains why you have to recreate the priomap everytime
you change the number of bands. You used the word "probably" which is
wrong. The proper word is "MUST".
What i think would be useful for you to do is describe some of the vlaues
used by some applications (RFC 1349 cut-n-paste job would help).

cheers,
jamal

2001-12-08 19:56:03

by bert hubert

[permalink] [raw]
Subject: further CBQ/tc documentation ds9a.nl/lartc/manpages

On Sat, Dec 08, 2001 at 02:20:20PM -0500, jamal wrote:

> > The parts of the table of contents that document stuff in the kernel not
> > documented elsewhere:
>
> "not documented elsewhere" comes out rude. Werner and I (and even
> Alexey when he was in the mood -- and i have seen some good documentation
> by other people as well) have spent numerous hours documenting, presenting
> and answering questions on mailing lists at times

True. I should have worded that better but I lost sight of politeness due to
my great enthusiasm at finally understanding everything. Some parts required
literally *hours* of digging through sources and disembodied slides -
presentations lose something without a speaker.

> Sample docs that i was personally involved in:
> ftp://icaftp.epfl.ch/pub/linux/diffserv/misc/dsid-01.txt.gz

These days I understand this document, but I didn't used to. That might be
because I'm thick, though.

> You need to introduce the big picture to the user.
> and what is wrong with the definitions used in
> http://www.davin.ottawa.on.ca/ols/img10.htm that forced you to introduce
> your own?

I've since moved to this terminology. Please also see the manpages I'm
writing at
http://ds9a.nl/lartc/manpages


> Actually, the big picture is:
> http://www.davin.ottawa.on.ca/ols/img9.htm
> Also
> http://www.linuxjournal.com/article.php?sid=3369
> (was written in 98 but got published in 99)

Google is surely to be praised - I had found all these links already. But to
summarize: stuff is out there.

> [My complaints about your style is you often are trying to present facts
> by using opinions. For example despite a lot of effort in the past to
> explain ingress qdisc to you in the past and, pointing you to very good
> documentation from CISCO you still ended using your opinions on what you
> thought it should be;->

I really didn't understand how everything worked back then, sadly. I do now,
hopefully.

> My scanning of the document shows opinions still posing as miscontrued
> facts. It is improving compared to what i saw last when we discussed ingress.
> Let me clarify one thing in this email; i'll read what you have later.

Some stuff remains from that time, am working on removing it. My current
efforts is writing the manpages and getting them 100% right and devoid of
opinion.

Once they are finished & reviewed, I'm 'backporting' the insight to the
HOWTO, which will then lose a lot of content and instead refer to the
manpages.

> Lets start by your description of TC_PRIO and TOS mappings etc:
> Your descriptions of these values is insufficient. Consider this a
> tutorial and reword it as you wish but please avoid opinions.

Will do, it makes sense now.

> Look at RFC 1349 for typical values used by different applications
> Then of course note that RFC 1349 is obsoleted by RFC 2474 (yes, you can
> weep);

That confused me greatly, yes.

> What i think would be useful for you to do is describe some of the vlaues
> used by some applications (RFC 1349 cut-n-paste job would help).

Thanks. I'm working on making the HOWTO more factual and the manpages 100%
factual. I'm always happy with critiques.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-08 20:47:10

by jamal

[permalink] [raw]
Subject: Re: further CBQ/tc documentation ds9a.nl/lartc/manpages



For starters, i think you need a defintions sections. Look at:
http://www.ietf.org/internet-drafts/draft-ietf-diffserv-model-06.txt

(eg what is a shaper etc and how trhings are placed together). At least
that will ensure that you dont sday things like "Prio cant shape".

It is a good model but may be insufficient given Linux TCs
capabilities. Email me when unsure.

Some other things:
- In your comment "Do not confuse this classless simple qdisc with the
classful PRIO one!". This is misleading:
the default 3 band FIFO queue is conceptually the same as the
default prio qdisc (the priomaps are identical). 3 bands; same
prioritization schemes.
- You really need to fix ingress section:
it works for both forwarding and packets coming in to local sockets.
More importantly, It takes advantages of _all_ filter schemes
available for TC as well as the policing functionality (which sadly seemed
to have been replicated by someone in netfilter, wrongly if i may add ;->).
- You keep saying "reodering" -- dont know what that means. Reordering is
generally considered a Bad Thing(tm).

- your description of the "peakrate" (same in TBF as well as policing)

Well captured. It took ages to get this into peoples heads. This also
applies to CBQ.

- your description of "MTU"
Not very good description:
This is just what it literally says; maximum transmit unit;
A packet larger than this will be dropped. Default is 2K. For ethernet,
MTUs of 1500 bytes, this is fine; however, you should put a cautionary
statement here in regards to people having MTUs smaller than 2K (example
the lo device); they might find that all their packets greater than 2K
being dropped.


More later if dont get distracted.

cheers,
jamal


2001-12-08 23:24:14

by bert hubert

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented (almost!)

On Sat, Dec 08, 2001 at 02:20:20PM -0500, jamal wrote:

> Linux remaps packets incoming with different values to some internal
> value; the colum "mapped to" shows the internal mapping
>
> 8value(hex) TOS(dec) mapped to(dec)
> ----------------------------------
> 0x0 0 0
> 1 7
> 2 0
> 3 0
> 4 2
> 5 2
> 6 2
> 7 2
> 0x10 8 6
> 9 6
> 10 6
> 11 6
> 12 2
> 13 2
> 14 2
> 15 2

I find this tos2prio table in the kernel (2.5.x), which is somewhat
different than your table:

0 TC_PRIO_BESTEFFORT, 0
1 TC_PRIO_(FILLER), 1
2 TC_PRIO_BESTEFFORT, 0
3 TC_PRIO_(BESTEFFORT), 0
4 TC_PRIO_BULK, 2
5 TC_PRIO_(BULK), 2
6 TC_PRIO_BULK, 2
7 TC_PRIO_(BULK), 2
8 TC_PRIO_INTERACTIVE, 6
9 TC_PRIO_(INTERACTIVE), 6
10 TC_PRIO_INTERACTIVE, 6
11 TC_PRIO_(INTERACTIVE), 6
12 TC_PRIO_INTERACTIVE_BULK, 4
13 TC_PRIO_(INTERACTIVE_BULK), 4
14 TC_PRIO_INTERACTIVE_BULK, 4
15 TC_PRIO_(INTERACTIVE_BULK) 4


> Fill in the "8value(hex)" column gaps using the bitmap from RFC1349 for
> the 8 bits; These are the values ou would see with tcpdump -vvv
> I filled the two easiest ones i could compute in my head.
>
> Second step:
>
> Take the default priority map:
> 1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1
> This applies for both default prio and the 3-band FIFO queue.
> Note the queue map fitted on the last column
>
> 8 but value TOS mapped to queue map
> ---------------------------------------------
> 0x0 0 0 1
> 1 7 2
> 2 0 2
> 3 0 2
> 4 2 1
> 5 2 2
> 6 2 0
> 7 2 0
> 0x10 8 6 1
> 9 6 1
> 10 6 1
> 11 6 1
> 12 2 1
> 13 2 1
> 14 2 1
> 15 2 1

I've changed this table to:
TOS Bits Means Linux Priority Band
------------------------------------------------------------
0x0 0 Normal Service 0 Best Effort 1
0x2 1 Minimize Monetary Cost 1 Filler 2
0x4 2 Maximize Reliability 0 Best Effort 1
0x6 3 mmc+mr 0 Best Effort 1
0x8 4 Maximize Throughput 2 Bulk 2
0xa 5 mmc+mt 2 Bulk 2
0xc 6 mr+mt 2 Bulk 2
0xe 7 mmc+mr+mt 2 Bulk 2
0x10 8 Minimize Delay 6 Interactive 0
0x12 9 mmc+md 6 Interactive 0
0x14 10 mr+md 6 Interactive 0
0x16 11 mmc+mr+md 6 Interactive 0
0x18 12 mt+md 4 Int. Bulk 1
0x1a 13 mmc+mt+md 4 Int. Bulk 1
0x1c 14 mr+mt+md 4 Int. Bulk 1
0x1e 15 mmc+mr+mt+md 4 Int. Bulk 1

http://ds9a.nl/lartc/HOWTO/cvs/2.4routing/output/2.4routing-9.html#ss9.2

Your table appears to imply that a Maximum Reliability, Mininum Delay
packet, TOS bits=9, gets mapped to band 1, not 0, which would not make
sense.

Laying it out like this, which does appear how it works, does mean that you
can specify priorities in the priomap which do not correspond to possible
TOS values.

Is it possible at all to set skb->priority from userspace without going
through the tos2prio mapping?

CBQ can use the skb->priority to classify:

/*
* Step 1. If skb->priority points to one of our classes, use it.
*/
if (TC_H_MAJ(prio^sch->handle) == 0 &&
(cl = cbq_class_lookup(q, prio)) != NULL)
return cl;

But to do this, you would need to be able to set skb->priority to a 32bit
number:

include/linux/pkt_sched.h:#define TC_H_MAJ_MASK (0xFFFF0000U)
include/linux/pkt_sched.h:#define TC_H_MAJ(h) ((h)&TC_H_MAJ_MASK)

I can't find where you would do this, any clues?

Thanks again for taking the time to help me.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-09 01:18:05

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented (almost!)



On Sun, 9 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 02:20:20PM -0500, jamal wrote:
>
> > Linux remaps packets incoming with different values to some internal
> > value; the colum "mapped to" shows the internal mapping
> >
> > 8value(hex) TOS(dec) mapped to(dec)
> > ----------------------------------
> > 0x0 0 0
> > 1 7
> > 2 0
> > 3 0
> > 4 2
> > 5 2
> > 6 2
> > 7 2
> > 0x10 8 6
> > 9 6
> > 10 6
> > 11 6
> > 12 2
> > 13 2
> > 14 2
> > 15 2
>
> I find this tos2prio table in the kernel (2.5.x), which is somewhat
> different than your table:
>
> 0 TC_PRIO_BESTEFFORT, 0
> 1 TC_PRIO_(FILLER), 1
> 2 TC_PRIO_BESTEFFORT, 0
> 3 TC_PRIO_(BESTEFFORT), 0
> 4 TC_PRIO_BULK, 2
> 5 TC_PRIO_(BULK), 2
> 6 TC_PRIO_BULK, 2
> 7 TC_PRIO_(BULK), 2
> 8 TC_PRIO_INTERACTIVE, 6
> 9 TC_PRIO_(INTERACTIVE), 6
> 10 TC_PRIO_INTERACTIVE, 6
> 11 TC_PRIO_(INTERACTIVE), 6
> 12 TC_PRIO_INTERACTIVE_BULK, 4
> 13 TC_PRIO_(INTERACTIVE_BULK), 4
> 14 TC_PRIO_INTERACTIVE_BULK, 4
> 15 TC_PRIO_(INTERACTIVE_BULK) 4
>
>
> > Fill in the "8value(hex)" column gaps using the bitmap from RFC1349 for
> > the 8 bits; These are the values ou would see with tcpdump -vvv
> > I filled the two easiest ones i could compute in my head.
> >
> > Second step:
> >
> > Take the default priority map:
> > 1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1
> > This applies for both default prio and the 3-band FIFO queue.
> > Note the queue map fitted on the last column
> >
> > 8 but value TOS mapped to queue map
> > ---------------------------------------------
> > 0x0 0 0 1
> > 1 7 2
> > 2 0 2
> > 3 0 2
> > 4 2 1
> > 5 2 2
> > 6 2 0
> > 7 2 0
> > 0x10 8 6 1
> > 9 6 1
> > 10 6 1
> > 11 6 1
> > 12 2 1
> > 13 2 1
> > 14 2 1
> > 15 2 1
>
> I've changed this table to:
> TOS Bits Means Linux Priority Band
> ------------------------------------------------------------
> 0x0 0 Normal Service 0 Best Effort 1
> 0x2 1 Minimize Monetary Cost 1 Filler 2
> 0x4 2 Maximize Reliability 0 Best Effort 1
> 0x6 3 mmc+mr 0 Best Effort 1
> 0x8 4 Maximize Throughput 2 Bulk 2
> 0xa 5 mmc+mt 2 Bulk 2
> 0xc 6 mr+mt 2 Bulk 2
> 0xe 7 mmc+mr+mt 2 Bulk 2
> 0x10 8 Minimize Delay 6 Interactive 0
> 0x12 9 mmc+md 6 Interactive 0
> 0x14 10 mr+md 6 Interactive 0
> 0x16 11 mmc+mr+md 6 Interactive 0
> 0x18 12 mt+md 4 Int. Bulk 1
> 0x1a 13 mmc+mt+md 4 Int. Bulk 1
> 0x1c 14 mr+mt+md 4 Int. Bulk 1
> 0x1e 15 mmc+mr+mt+md 4 Int. Bulk 1
>


Yes, sorry the last 4 are int_bulk (value 4) and not just bulk (2). good
eye. You are still abusing the word TOS. Thats only 4 bits not 8;
Use the terminology from RFC1349 at least.

> http://ds9a.nl/lartc/HOWTO/cvs/2.4routing/output/2.4routing-9.html#ss9.2
>
> Your table appears to imply that a Maximum Reliability, Mininum Delay
> packet, TOS bits=9, gets mapped to band 1, not 0, which would not make
> sense.
>

This is the priomap: 1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1
It says 1 is the right value

> Laying it out like this, which does appear how it works, does mean that you
> can specify priorities in the priomap which do not correspond to possible
> TOS values.
>

You cant remap the 3 band scheduler trivially, but you should be able to
replace it with a default prio qdisc get exactly the same behavior and use
whatever map you want (eg your 0 to 1 substitution for TOS 1001)

> Is it possible at all to set skb->priority from userspace without going
> through the tos2prio mapping?
>

SO_PRIORITY socket option is doable; you have to be root.

> CBQ can use the skb->priority to classify:

so do prio and pfifo_fast (as i am sure you are aware)

> /*
> * Step 1. If skb->priority points to one of our classes, use it.
> */
> if (TC_H_MAJ(prio^sch->handle) == 0 &&
> (cl = cbq_class_lookup(q, prio)) != NULL)
> return cl;
>
> But to do this, you would need to be able to set skb->priority to a 32bit
> number:
>

Cant think of a straight way to do this .... Alexey would know,

cheers,
jamal


2001-12-09 01:30:55

by bert hubert

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented (almost!)

On Sat, Dec 08, 2001 at 08:14:10PM -0500, jamal wrote:

> Yes, sorry the last 4 are int_bulk (value 4) and not just bulk (2). good
> eye. You are still abusing the word TOS. Thats only 4 bits not 8;
> Use the terminology from RFC1349 at least.

Will do.

> > http://ds9a.nl/lartc/HOWTO/cvs/2.4routing/output/2.4routing-9.html#ss9.2
> >
> > Your table appears to imply that a Maximum Reliability, Mininum Delay
> > packet, TOS bits=9, gets mapped to band 1, not 0, which would not make
> > sense.
>
> This is the priomap: 1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1
> It says 1 is the right value

AFAICT, the priomap maps skb->priority to band. So the translation is as
follows:

Type of Service octet, which is fed to:
skb->priority = rt_tos2priority(iph->tos);


To extract the four TOS bits, and to translate to prio:
static inline char rt_tos2priority(u8 tos)
{
return ip_tos2prio[IPTOS_TOS(tos)>>1];
}

----

__u8 ip_tos2prio[16] = {
TC_PRIO_BESTEFFORT,
ECN_OR_COST(FILLER),
TC_PRIO_BESTEFFORT,
ECN_OR_COST(BESTEFFORT),
TC_PRIO_BULK,
ECN_OR_COST(BULK),
TC_PRIO_BULK,
ECN_OR_COST(BULK),
TC_PRIO_INTERACTIVE,
ECN_OR_COST(INTERACTIVE),
TC_PRIO_INTERACTIVE,
ECN_OR_COST(INTERACTIVE),
TC_PRIO_INTERACTIVE_BULK,
ECN_OR_COST(INTERACTIVE_BULK),
TC_PRIO_INTERACTIVE_BULK,
ECN_OR_COST(INTERACTIVE_BULK)
};

---

#define TC_PRIO_BESTEFFORT 0
#define TC_PRIO_FILLER 1
#define TC_PRIO_BULK 2
#define TC_PRIO_INTERACTIVE_BULK 4
#define TC_PRIO_INTERACTIVE 6
#define TC_PRIO_CONTROL 7
#define TC_PRIO_MAX 15

net/sched/sched_generic.c:

static const u8 prio2band[TC_PRIO_MAX+1] =
{ 1, 2, 2, 2, 1, 2, 0, 0 , 1, 1, 1, 1, 1, 1, 1, 1 };

list = ((struct sk_buff_head*)qdisc->data) +
prio2band[skb->priority&TC_PRIO_MAX];

> > CBQ can use the skb->priority to classify:
>
> so do prio and pfifo_fast (as i am sure you are aware)

Of course, but only CBQ (& HTB, by the way) can extract a classid directly
from it, without a priomap. Devik is planning to learn HTB to extract a
classid directly from the fwmark, to skip a layer of indirection.

Regards,

bert hubert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-09 02:14:38

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented (almost!)



On Sun, 9 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 08:14:10PM -0500, jamal wrote:
>
> AFAICT, the priomap maps skb->priority to band. So the translation is as
> follows:
>

yes ;->

> >
> > so do prio and pfifo_fast (as i am sure you are aware)
>
> Of course, but only CBQ (& HTB, by the way) can extract a classid directly
> from it, without a priomap. Devik is planning to learn HTB to extract a
> classid directly from the fwmark, to skip a layer of indirection.
>

I am not sure if this is such a nice hack. Whats wrong with with using the
fwmark classifier to select classes?

cheers,
jamal

2001-12-09 18:15:32

by Alexey Kuznetsov

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented

Hello!

> > But to do this, you would need to be able to set skb->priority to a 32bit
> > number:
> >
>
> Cant think of a straight way to do this .... Alexey would know,

SO_PRIORITY. Or I did not follow you?

Alexey

2001-12-09 18:18:32

by bert hubert

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented

On Sun, Dec 09, 2001 at 09:14:46PM +0300, [email protected] wrote:

> > > But to do this, you would need to be able to set skb->priority to a 32bit
> > > number:
> > Cant think of a straight way to do this .... Alexey would know,
>
> SO_PRIORITY. Or I did not follow you?

Ah yes, thanks, that sets sk->priority which later sets skb->priority.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-09 21:49:04

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented



On Sun, 9 Dec 2001 [email protected] wrote:

> Hello!
>
> > > But to do this, you would need to be able to set skb->priority to a 32bit
> > > number:
> > >
> >
> > Cant think of a straight way to do this .... Alexey would know,
>
> SO_PRIORITY. Or I did not follow you?
>

So priority limits the size of skb->priority to be from 0..6; this wont
work with that check in cbq.

cheers,
jamal

2001-12-09 21:54:06

by bert hubert

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented

On Sun, Dec 09, 2001 at 04:45:01PM -0500, jamal wrote:
> > > Cant think of a straight way to do this .... Alexey would know,
> >
> > SO_PRIORITY. Or I did not follow you?
>
> So priority limits the size of skb->priority to be from 0..6; this wont
> work with that check in cbq.

No, only IP_TOS does so.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-09 22:11:06

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented



On Sun, 9 Dec 2001, bert hubert wrote:

> On Sun, Dec 09, 2001 at 04:45:01PM -0500, jamal wrote:
> > > > Cant think of a straight way to do this .... Alexey would know,
> > >
> > > SO_PRIORITY. Or I did not follow you?
> >
> > So priority limits the size of skb->priority to be from 0..6; this wont
> > work with that check in cbq.
>
> No, only IP_TOS does so.
>

probaly ip precedence. Have you tried this or you are following what the
man pages say?

cheers,
jamal

2001-12-09 22:14:06

by bert hubert

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented

On Sun, Dec 09, 2001 at 05:07:03PM -0500, jamal wrote:
> > > So priority limits the size of skb->priority to be from 0..6; this wont
> > > work with that check in cbq.
> >
> > No, only IP_TOS does so.
>
> probaly ip precedence. Have you tried this or you are following what the
> man pages say?

I have been living in the source for quite a while now - see ip_setsockopt()
in net/ipv4/ip_sockglue.c.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-10 00:42:14

by bert hubert

[permalink] [raw]
Subject: CBQ MANPAGE: I hear the theme of '2001, A Space Odyssey'

... to the sound of 'Also sprach Zarathustra':

After weeks of social deprivation and much digging through heaps of code, I
bring you

tc-cbq.8

The CBQ manpage. Nearly 2500 words, 8 printed pages, of nearly
unintelligible gobledygook, explaining mostly how CBQ works.

It is part of the Linux Advanced Routing & Traffic Control documentation
project which contains a HOWTO, a mailinglist, an IRC channel and now
manpages:

http://ds9a.nl/lartc

I want to thank Jamal for stubbornly straightening me out when I use messy
language and explaining how things work. The errors are mine though.

I *implore* ANK and others to read through this. I'm about exhausted and
running out of time (need to get on with work), and have a hard time
figuring out the exact details of the CBQ link sharing algorithm. I need
help, so to speak. The manpage indicates where.

Thanks for your attention. Please find tc-cbq.8 attached.

Regards,

bert hubert


--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet


Attachments:
(No filename) (1.27 kB)
tc-cbq.8 (13.74 kB)
Download all attachments

2001-12-10 01:02:45

by jamal

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented



On Sun, 9 Dec 2001, bert hubert wrote:

> On Sun, Dec 09, 2001 at 05:07:03PM -0500, jamal wrote:
> > > > So priority limits the size of skb->priority to be from 0..6; this wont
> > > > work with that check in cbq.
> > >
> > > No, only IP_TOS does so.
> >
> > probaly ip precedence. Have you tried this or you are following what the
> > man pages say?
>
> I have been living in the source for quite a while now - see ip_setsockopt()
> in net/ipv4/ip_sockglue.c.
>

Thats the wrong place to look. Look instead at:
net/core/sock.c
I got it; non root is limited to 0..6; root can set the full 32 bit range.

cheers,
jamal

2001-12-10 01:08:35

by jamal

[permalink] [raw]
Subject: Re: CBQ MANPAGE: I hear the theme of '2001, A Space Odyssey'


Sorry didnt read it; did the 30 sec scan ..
If this is meant to be for users, why are you talking about skb->priority?
Isnt it sufficient to just call it prioirity?
Also, if you think that Alexeys imp. is based on Floyd only, you are
highly mistaken;
Going back to high latency response mode ...

cheers,
jamal

On Mon, 10 Dec 2001, bert hubert wrote:

> ... to the sound of 'Also sprach Zarathustra':
>
> After weeks of social deprivation and much digging through heaps of code, I
> bring you
>
> tc-cbq.8
>
> The CBQ manpage. Nearly 2500 words, 8 printed pages, of nearly
> unintelligible gobledygook, explaining mostly how CBQ works.
>
> It is part of the Linux Advanced Routing & Traffic Control documentation
> project which contains a HOWTO, a mailinglist, an IRC channel and now
> manpages:
>
> http://ds9a.nl/lartc
>
> I want to thank Jamal for stubbornly straightening me out when I use messy
> language and explaining how things work. The errors are mine though.
>
> I *implore* ANK and others to read through this. I'm about exhausted and
> running out of time (need to get on with work), and have a hard time
> figuring out the exact details of the CBQ link sharing algorithm. I need
> help, so to speak. The manpage indicates where.
>
> Thanks for your attention. Please find tc-cbq.8 attached.
>
> Regards,
>
> bert hubert
>
>
> --
> http://www.PowerDNS.com Versatile DNS Software & Services
> Trilab The Technology People
> Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
> 'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
>

2001-12-10 01:14:18

by bert hubert

[permalink] [raw]
Subject: Re: CBQ MANPAGE: I hear the theme of '2001, A Space Odyssey'

On Sun, Dec 09, 2001 at 08:04:42PM -0500, jamal wrote:

> Sorry didnt read it; did the 30 sec scan ..
> If this is meant to be for users, why are you talking about skb->priority?
> Isnt it sufficient to just call it prioirity?

It's not done yet and may need some readability tuning. Note however that
skb->priority is a bit overloaded. It can contain a priority, but also a
32bit encoded classid. These are different things, so they deserve different
mention.

> Also, if you think that Alexeys imp. is based on Floyd only, you are
> highly mistaken;

I just copied the attribution from the kernel, am glad to rectify things.

> Going back to high latency response mode ...

Thanks for reviewing.

Regards,

bert

--
http://www.PowerDNS.com Versatile DNS Software & Services
Trilab The Technology People
Netherlabs BV / Rent-a-Nerd.nl - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

2001-12-10 17:04:57

by Alexey Kuznetsov

[permalink] [raw]
Subject: Re: CBQ and all other qdiscs now REALLY completely documented

Hello!

> So priority limits the size of skb->priority to be from 0..6; this wont
> work with that check in cbq.

No, it does not. Values different of "low prio" defaults (0..6)
are not allowed to user without privileges by evident reasons.
User with correspoding capability may direct traffic to any class.

Alexey