Date:   Tue, 14 Mar 2023 13:03:59 +0100
From:   Simon Horman <simon.horman@corigine.com>
To:     Jason Xing <kerneljasonxing@gmail.com>
Cc:     davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
        pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net,
        hawk@kernel.org, john.fastabend@gmail.com,
        stephen@networkplumber.org, sinquersw@gmail.com,
        bpf@vger.kernel.org, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org, Jason Xing <kernelxing@tencent.com>
Subject: Re: [PATCH v2 net-next 2/2] net: introduce budget_squeeze to help us
 tune rx behavior
Message-ID: <ZBBir/hjHRJz6Laf@corigine.com>
References: <20230314030532.9238-1-kerneljasonxing@gmail.com>
 <20230314030532.9238-3-kerneljasonxing@gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230314030532.9238-3-kerneljasonxing@gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dSIXKf8Gh39yIrp9oTQP2V2wRCGqJ4MWlOXNtThw81rImREMahuho8cAevMO?=
 =?us-ascii?Q?lEuQo8rcPHilMM8tBUGHUJV7Xk24UvBoIecvZMz1wqcRm7HHYSg6724YEnYK?=
 =?us-ascii?Q?zsztFiRVHErVQz9cvNkcFDRnLsY1yA5MqTHeKC5V7TDIm6D4xOKeOHHQN5qH?=
 =?us-ascii?Q?FX0JtX3dqIyPq0LL7XoLGEWcRHOWOOWQp2/aexElneH+GcjHLjKA9Mm0RT9A?=
 =?us-ascii?Q?EVVx3Fjz8QFZw7ba0oYx+eM3Qv0/XtJObEKCIy4hYBdl2Gs2eUYQ7wbnas4w?=
 =?us-ascii?Q?Zz/IqJ31H5olcf2WILFJQvJvSvByENsLBf1uQaw9mRo6W4q2Io0yJi3Um3AR?=
 =?us-ascii?Q?01wcGzf5KCQPzOWCSF2TcOhwr6cDlb6Jw1GY+oLnRB7+gRtPSqkpfNHD12Xr?=
 =?us-ascii?Q?Gs1IEnC6i18m3WVU9q265KbcDC80PU6Y9sIrOuQo7NpKck/pXC7fQtjsxF0H?=
 =?us-ascii?Q?Sm0SXoIKendqglm7Nm7qJAXMpMJ9htPe4nyKnpUQtf2LMk9bqByb0xoWqU/N?=
 =?us-ascii?Q?cmWEPRabHna5EFVPObVHnByKfyDVrjrdd7wkeo9lywfPlzjrX7xGXUwB68xW?=
 =?us-ascii?Q?ePGmQaJhqqeyaU/33vR1vV9m8jd7T+eAZRN8KOVRzoCnsXLBUyQmbdd5bQW2?=
 =?us-ascii?Q?TOamfEymKGNt2Jx39fLJJFcLLW9VYBNSAr9eeBPhlcJf3YmGgwsKGXjSeVl5?=
 =?us-ascii?Q?uHcOX4mZf2T1aUsnsKelwWc0YMrY6CWLf1ItqY9nlied9E2kCIOwJlXNb/WQ?=
 =?us-ascii?Q?2749GwJymuL/84VsyB09+vq/+sepAnoy1F9nCsrHc0OQHiUX31+h+cG5PH2b?=
 =?us-ascii?Q?ra1V18S8mH5gmJF0yHwXISvAtxxIbAe2sWfobxtZgG7zLsqOXd+totOVXJ9F?=
 =?us-ascii?Q?xiwScOWtga2VG3gPJd00DeqiZaTXmXONqlOfCckx4asuSGdn21x6Pq99Cwny?=
 =?us-ascii?Q?M2voMyHaA57H/59HjtyvoQALCpFnVNH/Vl4ugNlrTQI7DKHw12XUeX5N9CAZ?=
 =?us-ascii?Q?JFXrGuXBq4ahVa2QjYeTfGhgFIPSKCphiFkP2l2awbQhZK98OdLzqzVSqx0G?=
 =?us-ascii?Q?JvnAnbSn+ts7aHundqNqaEg9jfmCwLj7DKT/ucX3F1nww/FF7d2JJgn4vK0u?=
 =?us-ascii?Q?genv0SACZ0ieueZyemf00maIiOMaahQEfMkEhroHybBYAEpmibQG1UlhDk93?=
 =?us-ascii?Q?lcmskpix0RinVu1xNBBMObIjJDVBlwQAyBRBp7kLqwHmT0kloUbeghqW3klx?=
 =?us-ascii?Q?1awx+0TQ7zRlbEyBBww66AwN2WNUumxGf009nlV6Obnz7WgQoqmd70Z9FVyk?=
 =?us-ascii?Q?aHRNWIk3fGzpM8FaBSZJOn9Z/r5QPoVfz0FzIH7jHOFfhM1oD/neqmDUQi+X?=
 =?us-ascii?Q?zZ1qTS/7xHb4eRzM75SzyIU6QWsLNT9hUVNzuP7Sgy+NH5Jclu3Iy5y+9uP0?=
 =?us-ascii?Q?4gRU+yVOnyTqrvaiWSeYCRjc49tBlNes9kC7hdORIA3hFHfqFIu+noYiSBPZ?=
 =?us-ascii?Q?/kG3YsnQuqoLdnZlNB/8+YQjSj5ps3Kq4aqVtCKD4F5U7Qy9608ngkx/dmdm?=
 =?us-ascii?Q?R0WL20IaxZqNUffjy+vg3JxUCCCBiK/Sx4ID6XUK5uCtoifKPprJ8FgC022t?=
 =?us-ascii?Q?oF37InDHzR3BDh+1u+OMrgQlHSGUeWBE1eWF2kqaZj9q+3fMDw9UlTjJTFNK?=
 =?us-ascii?Q?BMNgJA=3D=3D?=
X-OriginatorOrg: corigine.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fc8c662-6fab-42de-60fa-08db2484363a
X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2023 12:04:06.8854
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rQJJBYH/vzWFELcoaE8Q07J4LyooMSgtHRwIbbZ7jONOtHjXwBlLjx7edIeydCKthlw69r4ovTEWRbb90TG3gDNvOSFUr60r7oaRVlM6F4c=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0PR13MB4743
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Mar 14, 2023 at 11:05:32AM +0800, Jason Xing wrote:
> From: Jason Xing <kernelxing@tencent.com>
> 
> When we encounter some performance issue and then get lost on how
> to tune the budget limit and time limit in net_rx_action() function,
> we can separately counting both of them to avoid the confusion.
> 
> Signed-off-by: Jason Xing <kernelxing@tencent.com>

As per my comment on patch 1/2, I'd drop the "/* keep it untouched */"
comment.

That notwithstanding:

Reviewed-by: Simon Horman <simon.horman@corigine.com>

> ---
> v2:
> 1) change the coding style suggested by Stephen and Simon
> 2) Keep the display of the old data (time_squeeze) untouched suggested
> by Kui-Feng
> Link: https://lore.kernel.org/lkml/20230311163614.92296-1-kerneljasonxing@gmail.com/
> ---
>  include/linux/netdevice.h |  1 +
>  net/core/dev.c            | 12 ++++++++----
>  net/core/net-procfs.c     |  9 ++++++---
>  3 files changed, 15 insertions(+), 7 deletions(-)
> 
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 6a14b7b11766..5736311a2133 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -3157,6 +3157,7 @@ struct softnet_data {
>  	/* stats */
>  	unsigned int		processed;
>  	unsigned int		time_squeeze;
> +	unsigned int		budget_squeeze;
>  #ifdef CONFIG_RPS
>  	struct softnet_data	*rps_ipi_list;
>  #endif
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 253584777101..1518a366783b 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -6637,6 +6637,7 @@ static __latent_entropy void net_rx_action(struct softirq_action *h)
>  	unsigned long time_limit = jiffies +
>  		usecs_to_jiffies(READ_ONCE(netdev_budget_usecs));
>  	int budget = READ_ONCE(netdev_budget);
> +	bool done = false;
>  	LIST_HEAD(list);
>  	LIST_HEAD(repoll);
>  
> @@ -6644,7 +6645,7 @@ static __latent_entropy void net_rx_action(struct softirq_action *h)
>  	list_splice_init(&sd->poll_list, &list);
>  	local_irq_enable();
>  
> -	for (;;) {
> +	while (!done) {
>  		struct napi_struct *n;
>  
>  		skb_defer_free_flush(sd);
> @@ -6662,10 +6663,13 @@ static __latent_entropy void net_rx_action(struct softirq_action *h)
>  		 * Allow this to run for 2 jiffies since which will allow
>  		 * an average latency of 1.5/HZ.
>  		 */
> -		if (unlikely(budget <= 0 ||
> -			     time_after_eq(jiffies, time_limit))) {
> +		if (unlikely(budget <= 0)) {
> +			sd->budget_squeeze++;
> +			done = true;
> +		}
> +		if (unlikely(time_after_eq(jiffies, time_limit))) {
>  			sd->time_squeeze++;
> -			break;
> +			done = true;
>  		}
>  	}
>  
> diff --git a/net/core/net-procfs.c b/net/core/net-procfs.c
> index 2809b663e78d..25810ee46a04 100644
> --- a/net/core/net-procfs.c
> +++ b/net/core/net-procfs.c
> @@ -179,14 +179,17 @@ static int softnet_seq_show(struct seq_file *seq, void *v)
>  	 */
>  	seq_printf(seq,
>  		   "%08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x "
> -		   "%08x %08x\n",
> -		   sd->processed, sd->dropped, sd->time_squeeze, 0,
> +		   "%08x %08x %08x %08x\n",
> +		   sd->processed, sd->dropped,
> +		   sd->time_squeeze + sd->budget_squeeze, /* keep it untouched */
> +		   0,
>  		   0, 0, 0, 0, /* was fastroute */
>  		   0,	/* was cpu_collision */
>  		   sd->received_rps, flow_limit_count,
>  		   softnet_backlog_len(sd),	/* keep it untouched */
>  		   (int)seq->index,
> -		   softnet_input_pkt_queue_len(sd), softnet_process_queue_len(sd));
> +		   softnet_input_pkt_queue_len(sd), softnet_process_queue_len(sd),
> +		   sd->time_squeeze, sd->budget_squeeze);
>  	return 0;
>  }
>  
> -- 
> 2.37.3
>