Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 5 Jun 2023 19:53:53 +0300
From:   Vladimir Oltean <vladimir.oltean@nxp.com>
To:     Jamal Hadi Salim <jhs@mojatatu.com>
Cc:     netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Cong Wang <xiyou.wangcong@gmail.com>,
        Jiri Pirko <jiri@resnulli.us>,
        Vinicius Costa Gomes <vinicius.gomes@intel.com>,
        linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org,
        Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>,
        Peilin Ye <yepeilin.cs@gmail.com>,
        Pedro Tammela <pctammela@mojatatu.com>
Subject: Re: [PATCH RESEND net-next 0/5] Improve the taprio qdisc's
 relationship with its children
Message-ID: <20230605165353.tnjrwa7gbcp4qhim@skbuf>
References: <20230602103750.2290132-1-vladimir.oltean@nxp.com>
 <CAM0EoMnqscw=OfWzyEKV10qFW5+EFMd5JWZxPSPCod3TvqpnuQ@mail.gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAM0EoMnqscw=OfWzyEKV10qFW5+EFMd5JWZxPSPCod3TvqpnuQ@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rZxzD9zQO5yNdHgeR/xDG++OsMHDqEUUAfjcHDPHYohSezIOGqugLtSfveRl?=
 =?us-ascii?Q?1V7VyK3UBsFhuNdouDToVvzXZwJA38ukZL2q59w6yTDyETYUm9rfh25bveTx?=
 =?us-ascii?Q?s8joCyR4s47+vCi1+PmvFHZmlAHIfELMJ2sRztW0Zk4S1P3m/ZsEc/3bTucr?=
 =?us-ascii?Q?PdV4tU8VnS33AIHsUgJbveCfTR66lrVadJzY0GyDFnKJchUZzPze/JBi0uYy?=
 =?us-ascii?Q?iCHhGbnN0WdWPxfiUnuLHNQnsN+o1YVgkMDsIubj3T+aYHBzmzlEANCbnZeg?=
 =?us-ascii?Q?Mxq4aB3FLzMBqb3lZND6BLfObFic7SCSvtr4gYatIiWO7F7a9zHSjufrwEtc?=
 =?us-ascii?Q?sUbtKLFW+95XLzBo2mkWSPNznNI2CQ+jUujVnMXXoNAhUfE5ZJqXFecHPMEX?=
 =?us-ascii?Q?+TPznAiePzqBeF4LkZOOysbYQ+K6wSCX1krN7gnVwCzl+qO7DdypWVMgW0ZB?=
 =?us-ascii?Q?H7ZoHiGTB6+4JRfT+AFkfxFmfKS7Ci930g+yVE/06ubn7NFPS9ZuPF6/S+mZ?=
 =?us-ascii?Q?CH6xwag9dcwg+KtoDpf0KvOTcrauuz2GZDdKGIMU7NActZE2gTyRgUL3Jb9n?=
 =?us-ascii?Q?o3pdd/jjBerIDYKATbQ8IpYN3H2L2F5DeJ6IUkcSTXX3mbpdLJuQxlYDU7zR?=
 =?us-ascii?Q?J+Zbcn3fA6MgkH51tQLeaWF6/kXrhwG3vL8WxzNgTC25kS6dcrTdyyW9Lq8V?=
 =?us-ascii?Q?Am0HUgCYSbzTfRkSzSSo1Ph/hMnELtpvtYV2wyRMRbRnF0nDZBOkVI4J4pUm?=
 =?us-ascii?Q?K7f2kAhs//Olf6sakFR36ttcsAWfv6h73JCBmRFGJ0xB6J0m0qtaAmqaew+U?=
 =?us-ascii?Q?miP1SgV8ENrrhuJ76S1Y0DchaieaOURgppBfqSINzIVTcd9npD2yFj3Tc6Ay?=
 =?us-ascii?Q?qGgK5mAkbrkP18CfsP04XMgxAJCVklTYXwQaw/SigsEQgqD2z4e+RUWlrjyK?=
 =?us-ascii?Q?DB+zJ+SVuOHBGVMwC2+ERPkYWpcxG86k9SmQlUFGFLavo5Q3pZmcEurw+cD2?=
 =?us-ascii?Q?In9bw+jJueakEaxdtXJq4FX1BtN1AQibrUKkA4e+BdhggZVhUIgUNF/h8cis?=
 =?us-ascii?Q?sVnhUESJz9dg8brvrKJ0v17uW+WGZjTMZXbtqk+jfYxY9CbOI7wpMysGm8Sa?=
 =?us-ascii?Q?WFGdBW2xc9Uq5R70stw2ldh0mkT/uXnKorUWeJe7Ioa/wit4cpqEvW9SVOSY?=
 =?us-ascii?Q?PiNBI6aZVs8MC7y6K2FXYBK9rt/evHTX4XFGga0sBPX0YtpFxtYqeOpaJt8G?=
 =?us-ascii?Q?ImPaL0UkBCh5cgwJyUhxcoV9OtRcTVV9dCx9eqf8xDjt0G4m2d6IU+YDfySi?=
 =?us-ascii?Q?8PxiPMN/GYsMP12/+I4zY2psymUSyKeA2t5d9hlwdkJuV6BVhdTVst5OYbfs?=
 =?us-ascii?Q?XgHGTqeqjEX6tbHfBh6jVhP42Nz2nhbYh8Wq6bxQjMWZlfoE0vEamBrm5rIv?=
 =?us-ascii?Q?aXYK2WFJaFWbftXqGAaYairtt5Bh0Va+rf08KSik21fXgGi4I5XfvAfqSIUl?=
 =?us-ascii?Q?hnAhkpJ9x4pERZvDRWc6fjEtIpYNMuu1AUa1+lwU9vqq/SRbRtetWGfl6MXK?=
 =?us-ascii?Q?2Ca8YDi2tapRchEeqnXii6W/vqphHxQ7miLwxPChAsB4IMx/8Z7hGRMeMxY3?=
 =?us-ascii?Q?Fg=3D=3D?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a132e6b5-2244-4968-c202-08db65e573c1
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB6452.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2023 16:53:56.8775
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: j87EKMfRCiY8KZ9eqKQBFJY1W6mqR2L/m/b97aYX66PL6alI6Sp46OqyL38isRysvhCsX3X5gbKMZMiqpz++dQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8981
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Jamal,

On Mon, Jun 05, 2023 at 11:44:17AM -0400, Jamal Hadi Salim wrote:
> I havent been following - but if you show me sample intended tc
> configs for both s/w and hardware offloads i can comment.

There is not much difference in usage between the 2 modes. IMO the software
data path logic is only a simulation for demonstrative purposes of what the
shaper is intended to do. If hardware offload is available, it is always
preferable. Otherwise, I'm not sure if anyone uses the pure software
scheduling mode (also without txtime assist) for a real life use case.

I was working with something like this for testing the code paths affected
by these changes:

#!/bin/bash

add_taprio()
{
	local offload=$1
	local extra_flags

	case $offload in
	true)
		extra_flags="flags 0x2"
		;;
	false)
		extra_flags="clockid CLOCK_TAI"
		;;
	esac

	tc qdisc replace dev eno0 handle 8001: parent root stab overhead 24 taprio \
		num_tc 8 \
		map 0 1 2 3 4 5 6 7 \
		queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \
		max-sdu 0 0 0 0 0 200 0 0 \
		base-time 200 \
		sched-entry S 80 20000 \
		sched-entry S a0 20000 \
		sched-entry S 5f 60000 \
		$extra_flags
}

add_cbs()
{
	local offload=$1
	local extra_flags

	case $offload in
	true)
		extra_flags="offload 1"
		;;
	false)
		extra_flags=""
		;;
	esac

	max_frame_size=1500
	data_rate_kbps=20000
	port_transmit_rate_kbps=1000000
	idleslope=$data_rate_kbps
	sendslope=$(($idleslope - $port_transmit_rate_kbps))
	locredit=$(($max_frame_size * $sendslope / $port_transmit_rate_kbps))
	hicredit=$(($max_frame_size * $idleslope / $port_transmit_rate_kbps))
	tc qdisc replace dev eno0 parent 8001:8 cbs \
		idleslope $idleslope \
		sendslope $sendslope \
		hicredit $hicredit \
		locredit $locredit \
		$extra_flags
}

# this should always fail
add_second_taprio()
{
	tc qdisc replace dev eno0 parent 8001:7 taprio \
		num_tc 8 \
		map 0 1 2 3 4 5 6 7 \
		queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \
		max-sdu 0 0 0 0 0 200 0 0 \
		base-time 200 \
		sched-entry S 80 20000 \
		sched-entry S a0 20000 \
		sched-entry S 5f 60000 \
		clockid CLOCK_TAI
}

ip link set eno0 up

echo "Offload:"
add_taprio true
add_cbs true
add_second_taprio
mausezahn eno0 -t ip -b 00:04:9f:05:f6:27 -c 100 -p 60
sleep 5
tc -s class show dev eno0
tc qdisc del dev eno0 root

echo "Software:"
add_taprio false
add_cbs false
add_second_taprio
mausezahn eno0 -t ip -b 00:04:9f:05:f6:27 -c 100 -p 60
sleep 5
tc -s class show dev eno0
tc qdisc del dev eno0 root

> In my cursory look i assumed you wanted to go along the path of mqprio
> where nothing much happens in the s/w datapath other than requeues
> when the tx hardware path is busy (notice it is missing an
> enqueue/deque ops). In that case the hardware selection is essentially
> of a DMA ring based on skb tags. It seems you took it up a notch by
> infact having a choice of whether to have pure s/w or offload path.

Yes. Actually the original taprio design always had the enqueue()/dequeue()
ops involved in the data path, then commit 13511704f8d7 ("net: taprio
offload: enforce qdisc to netdev queue mapping") retrofitted the mqprio
model when using the "flags 0x2" argument.

If you have time to read, the discussion behind that redesign was here:
https://lore.kernel.org/netdev/20210511171829.17181-1-yannick.vignon@oss.nxp.com/