Received-SPF: pass (google.com: domain of linux-kernel+bounces-185721-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Anand Khoje <anand.a.khoje@oracle.com>
To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: anand.a.khoje@oracle.com, rama.nichanamatlu@oracle.com,
        manjunath.b.patil@oracle.com
Subject: [PATCH 0/1] RDMA/mlx5: Release CPU for other processes in mlx5_free_cmd_msg() 
Date: Wed, 22 May 2024 09:02:55 +0530
Message-Id: <20240522033256.11960-1-anand.a.khoje@oracle.com>
Precedence: bulk
MIME-Version: 1.0 Content-Type:
Content-Transfer-Encoding: 8bit

FW pages are allocated and reclaimed through a worker pages_handler().  This
worker allocates mlx5 mailbox messages to populate meta-data associated with the
pages being allocated or reclaimed.  During reclaim of pages, after getting the
meta-data of the pages to reclaim and releasing the pages, the dma pool
associated with the mailbox message is freed using dma_pool_free(), where it
tried to find the dma_page of this dma_pool by walking the page_list.  This is a
slow approach and if the number of pages reclaimed is high, it takes a lot of
time in execution of one work.  As a result, other critical processes are
starved of CPU.

This patch checks if time spent in mlx5_free_cmd_msg() is more than 2 msec, it
yields the CPU for other processes to use.

In our tests, we were able to allocate around 3.4 million FW pages and tried to
deallocate all of them at once, this resulted in the worker thread to yield the
CPU many times.

May 21 04:39:28 kernel: mlx5_core 0000:17:00.0:
mlx5_free_cmd_msg:1352:(pid 327407): Spent more than 2 msecs, yielding CPU
May 21 04:39:28 kernel: mlx5_core 0000:17:00.0:
mlx5_free_cmd_msg:1352:(pid 327407): Spent more than 2 msecs, yielding CPU
May 21 04:39:29 kernel: mlx5_core 0000:17:00.0:
mlx5_free_cmd_msg:1352:(pid 327407): Spent more than 2 msecs, yielding CPU
May 21 04:39:29 kernel: mlx5_core 0000:17:00.0:
mlx5_free_cmd_msg:1352:(pid 327407): Spent more than 2 msecs, yielding CPU


Anand Khoje (1): RDMA/mlx5: Release CPU for other processes in
mlx5_free_cmd_msg()

 drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 7 +++++++ 1 file changed, 7
insertions(+)

-- 1.8.3.1