From:   Vladimir Oltean <vladimir.oltean@nxp.com>
To:     netdev@vger.kernel.org
Cc:     "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Jamal Hadi Salim <jhs@mojatatu.com>,
        Cong Wang <xiyou.wangcong@gmail.com>,
        Jiri Pirko <jiri@resnulli.us>,
        Vinicius Costa Gomes <vinicius.gomes@intel.com>,
        Kurt Kanzenbach <kurt@linutronix.de>,
        Jacob Keller <jacob.e.keller@intel.com>,
        Gerhard Engleder <gerhard@engleder-embedded.com>,
        Jesse Brandeburg <jesse.brandeburg@intel.com>,
        Tony Nguyen <anthony.l.nguyen@intel.com>,
        intel-wired-lan@lists.osuosl.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2 net-next 06/15] net/sched: taprio: calculate tc gate durations
Date:   Tue,  7 Feb 2023 15:54:31 +0200
Message-Id: <20230207135440.1482856-7-vladimir.oltean@nxp.com>
In-Reply-To: <20230207135440.1482856-1-vladimir.oltean@nxp.com>
References: <20230207135440.1482856-1-vladimir.oltean@nxp.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MjicDT4IJHk8QZCBgAHlnJUTeqsStLllOeYiCHu9FB3alLr7mrTM1BPedi3R?=
 =?us-ascii?Q?PwR3Kk3nDxMJDpDz2g0GSo8Pwq3OOvHw9GD0SEmnRb0TqzG0ba76yuvQNmfM?=
 =?us-ascii?Q?4+i6j50CHea0TyJLDrJub/N9bjVsZ69HxkFtS9tYVKDlm+y3i0pMA7EgJn+v?=
 =?us-ascii?Q?7r5Zd0ieBQuUm3uZvRloW8l/UrQRbbOP4YPP2pm8TCHrp0aS//4s3b1YM8FN?=
 =?us-ascii?Q?0JCLY9PD40WaxWiO3yyDTeLHa0h6OwP9qknwrV/72i/VUEP1blFRVAaU+mc3?=
 =?us-ascii?Q?Yb4sBsMIY2f+tWuRbzIQ3uCXRbXC8bD3w59wU0d+puPmsa3KCANO76l1tlgY?=
 =?us-ascii?Q?58UXV7gEL3B7ThYKCCuWQ7J9U+dmPs/byfTXZHag7iICtIxyaSfuvjwZSWlk?=
 =?us-ascii?Q?grPvb+dS4gwFU6C9b67ZIDyWItv6k54rs595dC/NX3Z871BH3YCgXwsceiJx?=
 =?us-ascii?Q?2Dm7i6Wch+JybdW+0lomFNHoKStXQEiSYgP+Ri5aZVtG2DYOY1MGW8tZtHQf?=
 =?us-ascii?Q?eMWkc+RevYNe6zDzgzBO9b/LO1LtxisWUccT+bt2fmaqgwExnAbYMSuATu1p?=
 =?us-ascii?Q?oVEqc6htUE9F9i9iNBzYL9fq9T569B3AY257hde5bgK3S6kcXg8K/UpfTq2x?=
 =?us-ascii?Q?5hCl0FowUamNotzUc/52vGqipfDh49fxUMtWEIuQwVM5Rvyb+e0G6lmQRBmH?=
 =?us-ascii?Q?PSpXoiAX4XzhNm2NsqFjVI7jRuM9e/9fH3oj6btLX0icl37VIXCb3VW9CDwZ?=
 =?us-ascii?Q?uwRm7mbfagsZnAZ5f98uwxdBGBeT+VBGuvrER4U9R06ElzRDGzDrPoweisOI?=
 =?us-ascii?Q?MOymQh13umrHZ3hQq1+BGq+D4+tll4BB3otbpIPCesIsYqO/FWlHXeodMBBT?=
 =?us-ascii?Q?Fb4xgelYcsyzvSbkX1geiIu3yUeTkGR5GtsKOl6TubmKg9bbGoX19pXBJ05F?=
 =?us-ascii?Q?33Z7ED71sgvJPMvPEnNSTrTgJfUGe5RRddK9/NQHyxMu0q0uIjR6oY1kbqxw?=
 =?us-ascii?Q?7BFWz8z+f1SZ8Ot1nEqhAwDwjE76ClUtSJI6dCaDwpbuXqpVaaa0vbshVPEe?=
 =?us-ascii?Q?g3DwMZE/4wgN18FK4XG3VXSLIz2YH0OWfH4v/X5LbPjqIsBX/M1W45+eprDw?=
 =?us-ascii?Q?qwG2fGyDo5FJ6u9yArtBSHvRwpEialwAvo1nWvqhOx1fxjnpYH/Uvzt4FYJG?=
 =?us-ascii?Q?4GuB2QoHDo2BCJQoPr7SjZrCVYyF3u0UzqgcK5PqkFd1vqZdndI/s/cPukEf?=
 =?us-ascii?Q?q0gyWULDzAKvji89wMcT/4VgOmz5+Edylu5XlVX3JlDubflQGesP9QDX7WoQ?=
 =?us-ascii?Q?QpejzUxbaDxhpwK4h1e75Q/1ruEg0lSSnNWwPwfjX+qP2fRlrHtydyx2W3t+?=
 =?us-ascii?Q?BXxwgQsjnZIR4NEruCACsRDbLLVucdoc1Bn0Cblf7IKymBusPhr4nimTU64W?=
 =?us-ascii?Q?QjM0lkQtwt1s0yF/zfj6ul7abImAtREFU1oRPxCYnqxiIBMdLwNBV8o2TdpC?=
 =?us-ascii?Q?OiuyXiACOrbzFP308wvy1UiAP1bTkwnm1aDP2c+Xp5z6e5bOnfm/v04tqdFg?=
 =?us-ascii?Q?nGS7a63R/YmmGt9B5qFEiVhdxBnk9lxQFlYeurVbHO24aX47FBT31PEUb+pE?=
 =?us-ascii?Q?zA=3D=3D?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a74c2a09-2596-415d-c2b0-08db0912f019
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5136.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2023 13:55:15.0806
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1VFoOv6hbqakzEWnyhITby6R8W/enJKg6OEb/3we2ptNT52OsnFvU09SLg9mof6oELzoGoq+N1/BXtQ8u+nbPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB8115
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Current taprio code operates on a very simplistic (and incorrect)
assumption: that egress scheduling for a traffic class can only take
place for the duration of the current interval, or i.o.w., it assumes
that at the end of each schedule entry, there is a "gate close" event
for all traffic classes.

As an example, traffic sent with the schedule below will be jumpy, even
though all 8 TC gates are open, so there is absolutely no "gate close"
event (effectively a transition from BIT(tc)==1 to BIT(tc)==0 in
consecutive schedule entries):

tc qdisc replace dev veth0 parent root taprio \
	num_tc 2 \
	map 0 1 \
	queues 1@0 1@1 \
	base-time 0 \
	sched-entry S 0xff 4000000000 \
	clockid CLOCK_TAI \
	flags 0x0

This qdisc simply does not have what it takes in terms of logic to
*actually* compute the durations of traffic classes. Also, it does not
recognize the need to use this information on a per-traffic-class basis:
it always looks at entry->interval and entry->close_time.

This change proposes that each schedule entry has an array called
tc_gate_duration[tc]. This holds the information: "for how long will
this traffic class gate remain open, starting from *this* schedule
entry". If the traffic class gate is always open, that value is equal to
the cycle time of the schedule.

We'll also need to keep track, for the purpose of queueMaxSDU[tc]
calculation, what is the maximum time duration for a traffic class
having an open gate. This gives us directly what is the maximum sized
packet that this traffic class will have to accept. For everything else
it has to qdisc_drop() it in qdisc_enqueue().

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Kurt Kanzenbach <kurt@linutronix.de>
---
v1->v2: rename "tc_gate" to just "gate" to reduce line length

 net/sched/sch_taprio.c | 55 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 5f57dcfafffd..7d897bbd48ca 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -39,6 +39,10 @@ static struct static_key_false taprio_have_working_mqprio;
 #define TAPRIO_FLAGS_INVALID U32_MAX
 
 struct sched_entry {
+	/* Durations between this GCL entry and the GCL entry where the
+	 * respective traffic class gate closes
+	 */
+	u64 gate_duration[TC_MAX_QUEUE];
 	struct list_head list;
 
 	/* The instant that this entry "closes" and the next one
@@ -55,6 +59,10 @@ struct sched_entry {
 };
 
 struct sched_gate_list {
+	/* Longest non-zero contiguous gate durations per traffic class,
+	 * or 0 if a traffic class gate never opens during the schedule.
+	 */
+	u64 max_open_gate_duration[TC_MAX_QUEUE];
 	struct rcu_head rcu;
 	struct list_head entries;
 	size_t num_entries;
@@ -95,6 +103,51 @@ struct __tc_taprio_qopt_offload {
 	struct tc_taprio_qopt_offload offload;
 };
 
+static void taprio_calculate_gate_durations(struct taprio_sched *q,
+					    struct sched_gate_list *sched)
+{
+	struct net_device *dev = qdisc_dev(q->root);
+	int num_tc = netdev_get_num_tc(dev);
+	struct sched_entry *entry, *cur;
+	int tc;
+
+	list_for_each_entry(entry, &sched->entries, list) {
+		u32 gates_still_open = entry->gate_mask;
+
+		/* For each traffic class, calculate each open gate duration,
+		 * starting at this schedule entry and ending at the schedule
+		 * entry containing a gate close event for that TC.
+		 */
+		cur = entry;
+
+		do {
+			if (!gates_still_open)
+				break;
+
+			for (tc = 0; tc < num_tc; tc++) {
+				if (!(gates_still_open & BIT(tc)))
+					continue;
+
+				if (cur->gate_mask & BIT(tc))
+					entry->gate_duration[tc] += cur->interval;
+				else
+					gates_still_open &= ~BIT(tc);
+			}
+
+			cur = list_next_entry_circular(cur, &sched->entries, list);
+		} while (cur != entry);
+
+		/* Keep track of the maximum gate duration for each traffic
+		 * class, taking care to not confuse a traffic class which is
+		 * temporarily closed with one that is always closed.
+		 */
+		for (tc = 0; tc < num_tc; tc++)
+			if (entry->gate_duration[tc] &&
+			    sched->max_open_gate_duration[tc] < entry->gate_duration[tc])
+				sched->max_open_gate_duration[tc] = entry->gate_duration[tc];
+	}
+}
+
 static ktime_t sched_base_time(const struct sched_gate_list *sched)
 {
 	if (!sched)
@@ -953,6 +1006,8 @@ static int parse_taprio_schedule(struct taprio_sched *q, struct nlattr **tb,
 		new->cycle_time = cycle;
 	}
 
+	taprio_calculate_gate_durations(q, new);
+
 	return 0;
 }
 
-- 
2.34.1