2021-04-03 04:54:39

by Praveen Kumar Kannoju

Subject: [PATCH v3] IB/mlx5: Reduce max order of memory allocated for xlt update

To update xlt (during mlx5_ib_reg_user_mr()), the driver can request up to
1 MB (order-8) memory, depending on the size of the MR. This costly
allocation can sometimes take very long to return (a few seconds). This
causes the calling application to hang for a long time, especially when the
system is fragmented. To avoid these long latency spikes, the calls the
higher order allocations need to fail faster in case they are not
available. In order to acheive this we need __GFP_NORETRY flag in the
gfp_mask before during fetching the free pages. This patch adds this flag
to the mask.

Signed-off-by: Praveen Kumar Kannoju <[email protected]>
---
drivers/infiniband/hw/mlx5/mr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index db05b0e..429e7aa6 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1028,7 +1028,7 @@ static void *mlx5_ib_alloc_xlt(size_t *nents, size_t ent_size, gfp_t gfp_mask)
*/
might_sleep();

- gfp_mask |= __GFP_ZERO;
+ gfp_mask |= __GFP_ZERO | __GFP_NORETRY;

/*
* If the system already has a suitable high order page then just use
--
1.8.3.1

2021-04-04 10:46:36

by Leon Romanovsky

Subject: Re: [PATCH v3] IB/mlx5: Reduce max order of memory allocated for xlt update

On Sat, Apr 03, 2021 at 04:53:55AM +0000, Praveen Kumar Kannoju wrote:
> To update xlt (during mlx5_ib_reg_user_mr()), the driver can request up to
> 1 MB (order-8) memory, depending on the size of the MR. This costly
> allocation can sometimes take very long to return (a few seconds). This
> causes the calling application to hang for a long time, especially when the
> system is fragmented. To avoid these long latency spikes, the calls the
> higher order allocations need to fail faster in case they are not
> available. In order to acheive this we need __GFP_NORETRY flag in the
> gfp_mask before during fetching the free pages. This patch adds this flag
> to the mask.
>
> Signed-off-by: Praveen Kumar Kannoju <[email protected]>
> ---
> drivers/infiniband/hw/mlx5/mr.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>

Thanks a lot,
Acked-by: Leon Romanovsky <[email protected]>

2021-04-07 23:33:08

by Jason Gunthorpe

Subject: Re: [PATCH v3] IB/mlx5: Reduce max order of memory allocated for xlt update

On Sat, Apr 03, 2021 at 04:53:55AM +0000, Praveen Kumar Kannoju wrote:
> To update xlt (during mlx5_ib_reg_user_mr()), the driver can request up to
> 1 MB (order-8) memory, depending on the size of the MR. This costly
> allocation can sometimes take very long to return (a few seconds). This
> causes the calling application to hang for a long time, especially when the
> system is fragmented. To avoid these long latency spikes, the calls the
> higher order allocations need to fail faster in case they are not
> available. In order to acheive this we need __GFP_NORETRY flag in the
> gfp_mask before during fetching the free pages. This patch adds this flag
> to the mask.
>
> Signed-off-by: Praveen Kumar Kannoju <[email protected]>
> Acked-by: Leon Romanovsky <[email protected]>
> ---
> drivers/infiniband/hw/mlx5/mr.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)

Applied to for-next, thanks

Jason