Received-SPF: pass (google.com: domain of linux-kernel+bounces-195720-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 30 May 2024 20:14:40 +0300
From: Leon Romanovsky <leon@kernel.org>
To: Anand Khoje <anand.a.khoje@oracle.com>
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
	rama.nichanamatlu@oracle.com, manjunath.b.patil@oracle.com
Subject: Re: [PATCH 1/1] RDMA/mlx5: Release CPU for other processes in
 mlx5_free_cmd_msg()
Message-ID: <20240530171440.GE3884@unreal>
References: <20240522033256.11960-1-anand.a.khoje@oracle.com>
 <20240522033256.11960-2-anand.a.khoje@oracle.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240522033256.11960-2-anand.a.khoje@oracle.com>

On Wed, May 22, 2024 at 09:02:56AM +0530, Anand Khoje wrote:
> In non FLR context, at times CX-5 requests release of ~8 million device pages.
> This needs humongous number of cmd mailboxes, which to be released once
> the pages are reclaimed. Release of humongous number of cmd mailboxes
> consuming cpu time running into many secs, with non preemptable kernels
> is leading to critical process starving on that cpu’s RQ. To alleviate
> this, this patch relinquishes cpu periodically but conditionally.
> 
> Orabug: 36275016
> 
> Signed-off-by: Anand Khoje <anand.a.khoje@oracle.com>
> ---
>  drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
> index 9c21bce..9fbf25d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
> @@ -1336,16 +1336,23 @@ static struct mlx5_cmd_msg *mlx5_alloc_cmd_msg(struct mlx5_core_dev *dev,
>  	return ERR_PTR(err);
>  }
>  
> +#define RESCHED_MSEC 2
>  static void mlx5_free_cmd_msg(struct mlx5_core_dev *dev,
>  			      struct mlx5_cmd_msg *msg)
>  {
>  	struct mlx5_cmd_mailbox *head = msg->next;
>  	struct mlx5_cmd_mailbox *next;
> +	unsigned long start_time = jiffies;
>  
>  	while (head) {
>  		next = head->next;
>  		free_cmd_box(dev, head);

Did you consider to make this function asynchronous and parallel?

Thanks

>  		head = next;
> +		if (time_after(jiffies, start_time + msecs_to_jiffies(RESCHED_MSEC))) {
> +			mlx5_core_warn_rl(dev, "Spent more than %d msecs, yielding CPU\n", RESCHED_MSEC);
> +			cond_resched();
> +			start_time = jiffies;
> +		}
>  	}
>  	kfree(msg);
>  }
> -- 
> 1.8.3.1
> 
>