2013-01-07 19:41:36

by Joe Perches

[permalink] [raw]
Subject: [PATCH] sunrpc: verbs: Avoid 1kb stack

16 * 64 is a bit much.
Use kmalloc_array instead.

Signed-off-by: Joe Perches <[email protected]>
---
net/sunrpc/xprtrdma/verbs.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index 745973b..9cfebb4 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -1736,8 +1736,13 @@ rpcrdma_register_default_external(struct rpcrdma_mr_seg *seg,
int mem_priv = (writing ? IB_ACCESS_REMOTE_WRITE :
IB_ACCESS_REMOTE_READ);
struct rpcrdma_mr_seg *seg1 = seg;
- struct ib_phys_buf ipb[RPCRDMA_MAX_DATA_SEGS];
int len, i, rc = 0;
+ struct ib_phys_buf *ipb = kmalloc_array(RPCRDMA_MAX_DATA_SEGS,
+ sizeof(*ipb),
+ GFP_KERNEL);
+
+ if (!ipb)
+ return -ENOMEM;

if (*nsegs > RPCRDMA_MAX_DATA_SEGS)
*nsegs = RPCRDMA_MAX_DATA_SEGS;
@@ -1770,6 +1775,9 @@ rpcrdma_register_default_external(struct rpcrdma_mr_seg *seg,
seg1->mr_len = len;
}
*nsegs = i;
+
+ kfree(ipb);
+
return rc;
}





2013-01-08 14:46:09

by J. Bruce Fields

[permalink] [raw]
Subject: Re: [PATCH] sunrpc: verbs: Avoid 1kb stack

On Mon, Jan 07, 2013 at 11:41:36AM -0800, Joe Perches wrote:
> 16 * 64 is a bit much.
> Use kmalloc_array instead.

I thought there was some reason we didn't do this.

Grepping up through the callers.... It looks like the result is
xprt_rdma_send_request returns -EIO, and as far as I can tell that gets
passed up to the application on the client. That doesn't sound right.

--b.

>
> Signed-off-by: Joe Perches <[email protected]>
> ---
> net/sunrpc/xprtrdma/verbs.c | 10 +++++++++-
> 1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
> index 745973b..9cfebb4 100644
> --- a/net/sunrpc/xprtrdma/verbs.c
> +++ b/net/sunrpc/xprtrdma/verbs.c
> @@ -1736,8 +1736,13 @@ rpcrdma_register_default_external(struct rpcrdma_mr_seg *seg,
> int mem_priv = (writing ? IB_ACCESS_REMOTE_WRITE :
> IB_ACCESS_REMOTE_READ);
> struct rpcrdma_mr_seg *seg1 = seg;
> - struct ib_phys_buf ipb[RPCRDMA_MAX_DATA_SEGS];
> int len, i, rc = 0;
> + struct ib_phys_buf *ipb = kmalloc_array(RPCRDMA_MAX_DATA_SEGS,
> + sizeof(*ipb),
> + GFP_KERNEL);
> +
> + if (!ipb)
> + return -ENOMEM;
>
> if (*nsegs > RPCRDMA_MAX_DATA_SEGS)
> *nsegs = RPCRDMA_MAX_DATA_SEGS;
> @@ -1770,6 +1775,9 @@ rpcrdma_register_default_external(struct rpcrdma_mr_seg *seg,
> seg1->mr_len = len;
> }
> *nsegs = i;
> +
> + kfree(ipb);
> +
> return rc;
> }
>
>
>

2013-01-08 22:57:48

by Tom Talpey

[permalink] [raw]
Subject: RE: [PATCH] sunrpc: verbs: Avoid 1kb stack

> -----Original Message-----
> From: Joe Perches [mailto:[email protected]]
> Sent: Tuesday, January 8, 2013 1:10 PM
> To: J. Bruce Fields
> Cc: Trond Myklebust; David S. Miller; [email protected];
> [email protected]; [email protected]; Tom Tucker;
> [email protected]; Tom Talpey
> Subject: Re: [PATCH] sunrpc: verbs: Avoid 1kb stack
>
> On Tue, 2013-01-08 at 09:46 -0500, J. Bruce Fields wrote:
> > On Mon, Jan 07, 2013 at 11:41:36AM -0800, Joe Perches wrote:
> > > 16 * 64 is a bit much.
> > > Use kmalloc_array instead.
> >
> > I thought there was some reason we didn't do this.

The value of RPCRDMA_MAX_DATA_SEGS appears to have been increased to allow 256KB operations (64 pages). Not all requests will need this, so you may want to consider dynamically allocating the array only when it's relatively large (*nsegs is >8, perhaps).

IIRC, the ib_phys_buf array is only built because it's passed to the FastRegister verb as a parameter. It's not needed by the xprtrdma layer.

> >
> > Grepping up through the callers.... It looks like the result is
> > xprt_rdma_send_request returns -EIO, and as far as I can tell that gets
> > passed up to the application on the client. That doesn't sound right.
>
> No worries, it was just a warning I noticed when I did an allmodconfig
> compilation.
>
> Perhaps a comment there might be appropriate instead.
>


2013-01-08 18:09:35

by Joe Perches

[permalink] [raw]
Subject: Re: [PATCH] sunrpc: verbs: Avoid 1kb stack

On Tue, 2013-01-08 at 09:46 -0500, J. Bruce Fields wrote:
> On Mon, Jan 07, 2013 at 11:41:36AM -0800, Joe Perches wrote:
> > 16 * 64 is a bit much.
> > Use kmalloc_array instead.
>
> I thought there was some reason we didn't do this.
>
> Grepping up through the callers.... It looks like the result is
> xprt_rdma_send_request returns -EIO, and as far as I can tell that gets
> passed up to the application on the client. That doesn't sound right.

No worries, it was just a warning I noticed when I
did an allmodconfig compilation.

Perhaps a comment there might be appropriate instead.