Svcrdma currently advertises 1MB, which is too large. The correct value
is the minimum of RPCSVC_MAXPAYLOAD and the max scatter-gather allowed
in an NFSRDMA IO chunk * the host page size. This bug is usually benign
because the Linux X64 NFSRDMA client correctly limits the payload size to
the correct value (64*4096 = 256KB). But if the Linux client is PPC64
with a 64KB page size, then the client will indeed use a payload size
that will overflow the server.
Signed-off-by: Steve Wise <[email protected]>
---
net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +-
net/sunrpc/xprtrdma/xprt_rdma.h | 4 ++++
2 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 374feb4..4e61880 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -91,7 +91,7 @@ struct svc_xprt_class svc_rdma_class = {
.xcl_name = "rdma",
.xcl_owner = THIS_MODULE,
.xcl_ops = &svc_rdma_ops,
- .xcl_max_payload = RPCSVC_MAXPAYLOAD_TCP,
+ .xcl_max_payload = RPCSVC_MAXPAYLOAD_RDMA,
.xcl_ident = XPRT_TRANSPORT_RDMA,
};
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index c419498..a9cf5c3 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
/* Workqueue created in svc_rdma.c */
extern struct workqueue_struct *svc_rdma_wq;
+#define RPCSVC_MAXPAYLOAD_RDMA \
+ (RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+ RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+
#endif /* _LINUX_SUNRPC_XPRT_RDMA_H */
> -----Original Message-----
> From: 'J. Bruce Fields' [mailto:[email protected]]
> Sent: Tuesday, September 23, 2014 2:48 PM
> To: Steve Wise
> Cc: 'Chuck Lever'; [email protected]; [email protected]
> Subject: Re: [PATCH V2] svcrdma: Advertise the correct max payload
>
> On Tue, Sep 23, 2014 at 02:42:34PM -0500, Steve Wise wrote:
> >
> > > > diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > index c419498..a9cf5c3 100644
> > > > --- a/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > +++ b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > @@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
> > > > /* Workqueue created in svc_rdma.c */
> > > > extern struct workqueue_struct *svc_rdma_wq;
> > > >
> > > > +#define RPCSVC_MAXPAYLOAD_RDMA \
> > > > + (RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
> > > > + RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
> > > > +
> > >
> > > Couldn't you use:
> > >
> > > #if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > > #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
> > > #else
> > > #define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > > #endif
> > >
> > > That seems more idiomatic.
> >
> > Sure. That makes it easier to read in my opinion too.
> >
> > I'll send out V3 with this change.
>
> While we're bikeshedding, why not use min()?
>
> --b.
I tried that initially. But min() and min_t() don't work because of the way we use the #define. With it defined thusly:
#define RPCSVC_MAXPAYLOAD_RDMA min(RPC_MAXPAYLOAD, RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
I see this error:
CC [M] net/sunrpc/xprtrdma/svc_rdma_transport.o
net/sunrpc/xprtrdma/svc_rdma_transport.c:94: error: braced-group within expression allowed only inside a function
make[3]: *** [net/sunrpc/xprtrdma/svc_rdma_transport.o] Error 1
make[2]: *** [net/sunrpc/xprtrdma] Error 2
make[1]: *** [net/sunrpc] Error 2
make: *** [net] Error 2
min() looks like this:
#define min(x, y) ({ \
typeof(x) _min1 = (x); \
typeof(y) _min2 = (y); \
(void) (&_min1 == &_min2); \
_min1 < _min2 ? _min1 : _min2; })
On Tue, Sep 23, 2014 at 02:42:34PM -0500, Steve Wise wrote:
>
> > > diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > index c419498..a9cf5c3 100644
> > > --- a/net/sunrpc/xprtrdma/xprt_rdma.h
> > > +++ b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > @@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
> > > /* Workqueue created in svc_rdma.c */
> > > extern struct workqueue_struct *svc_rdma_wq;
> > >
> > > +#define RPCSVC_MAXPAYLOAD_RDMA \
> > > + (RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
> > > + RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
> > > +
> >
> > Couldn't you use:
> >
> > #if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
> > #else
> > #define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > #endif
> >
> > That seems more idiomatic.
>
> Sure. That makes it easier to read in my opinion too.
>
> I'll send out V3 with this change.
While we're bikeshedding, why not use min()?
--b.
On Tue, Sep 23, 2014 at 02:53:28PM -0500, Steve Wise wrote:
>
>
> > -----Original Message-----
> > From: 'J. Bruce Fields' [mailto:[email protected]]
> > Sent: Tuesday, September 23, 2014 2:48 PM
> > To: Steve Wise
> > Cc: 'Chuck Lever'; [email protected]; [email protected]
> > Subject: Re: [PATCH V2] svcrdma: Advertise the correct max payload
> >
> > On Tue, Sep 23, 2014 at 02:42:34PM -0500, Steve Wise wrote:
> > >
> > > > > diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > > index c419498..a9cf5c3 100644
> > > > > --- a/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > > +++ b/net/sunrpc/xprtrdma/xprt_rdma.h
> > > > > @@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
> > > > > /* Workqueue created in svc_rdma.c */
> > > > > extern struct workqueue_struct *svc_rdma_wq;
> > > > >
> > > > > +#define RPCSVC_MAXPAYLOAD_RDMA \
> > > > > + (RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
> > > > > + RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
> > > > > +
> > > >
> > > > Couldn't you use:
> > > >
> > > > #if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > > > #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
> > > > #else
> > > > #define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> > > > #endif
> > > >
> > > > That seems more idiomatic.
> > >
> > > Sure. That makes it easier to read in my opinion too.
> > >
> > > I'll send out V3 with this change.
> >
> > While we're bikeshedding, why not use min()?
> >
> > --b.
>
> I tried that initially. But min() and min_t() don't work because of the way we use the #define. With it defined thusly:
OK, OK. Whatever works.--b.
>
> #define RPCSVC_MAXPAYLOAD_RDMA min(RPC_MAXPAYLOAD, RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
>
> I see this error:
>
> CC [M] net/sunrpc/xprtrdma/svc_rdma_transport.o
> net/sunrpc/xprtrdma/svc_rdma_transport.c:94: error: braced-group within expression allowed only inside a function
> make[3]: *** [net/sunrpc/xprtrdma/svc_rdma_transport.o] Error 1
> make[2]: *** [net/sunrpc/xprtrdma] Error 2
> make[1]: *** [net/sunrpc] Error 2
> make: *** [net] Error 2
>
> min() looks like this:
>
> #define min(x, y) ({ \
> typeof(x) _min1 = (x); \
> typeof(y) _min2 = (y); \
> (void) (&_min1 == &_min2); \
> _min1 < _min2 ? _min1 : _min2; })
>
> > diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
> > index c419498..a9cf5c3 100644
> > --- a/net/sunrpc/xprtrdma/xprt_rdma.h
> > +++ b/net/sunrpc/xprtrdma/xprt_rdma.h
> > @@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
> > /* Workqueue created in svc_rdma.c */
> > extern struct workqueue_struct *svc_rdma_wq;
> >
> > +#define RPCSVC_MAXPAYLOAD_RDMA \
> > + (RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
> > + RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
> > +
>
> Couldn't you use:
>
> #if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
> #else
> #define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
> #endif
>
> That seems more idiomatic.
Sure. That makes it easier to read in my opinion too.
I'll send out V3 with this change.
Thanks,
Steve.