2007-04-20 14:43:50

by Jeff Layton

[permalink] [raw]
Subject: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

I think I've finally chased down a bug that I noticed at connectathon
this year. When a client mounts a Linux NFS server using NFSv2 or 3 with
sec=krb5i, the server gets a ton of messages like this in the logs:

RPC request reserved 80 but used 124

The sizes vary here, but usually the "used" is 44 bytes off from what
was reserved (sometimes a little less). The issue seems to be that after
writing out the response header, svc_process() calls this:

/* un-reserve some of the out-queue now that we have a
* better idea of reply size
*/
if (procp->pc_xdrressize)
svc_reserve(rqstp, procp->pc_xdrressize<<2);

...but, this does not take into account the size of the checksum at the
end of the response. This is also a problem in the svc_reserve calls
made from nfsd3_proc_read and nfsd_proc_read.

While I haven't checked it, my guess is that this is also a problem for
krb5p and for spkm3i too. We don't seem to see this issue with NFSv4,
but I think that's because the reserved size for a compound packet is
likely large enough to paper over this problem.

My thinking is that we need a wrapper of some sort around these calls to
svc_reserve that increases the "space" parameter by the amount of the
expected checksum. My question is -- is there a simple way to compute
the expected length of the checksum, given a svc_rqst?

Thanks,
Jeff


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2007-04-20 17:47:48

by J. Bruce Fields

[permalink] [raw]
Subject: Re: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

On Fri, Apr 20, 2007 at 10:43:43AM -0400, Jeff Layton wrote:
> I think I've finally chased down a bug that I noticed at connectathon
> this year. When a client mounts a Linux NFS server using NFSv2 or 3 with
> sec=krb5i, the server gets a ton of messages like this in the logs:
>
> RPC request reserved 80 but used 124

Great, thanks!

> My thinking is that we need a wrapper of some sort around these calls to
> svc_reserve that increases the "space" parameter by the amount of the
> expected checksum. My question is -- is there a simple way to compute
> the expected length of the checksum, given a svc_rqst?

You don't need to know the exact amount, just an upper bound, right?

In which case I'd be tempted to just look at some krb5i and krb5p
traffic in wireshark, figure out how much it adds (it should always be
the same, except that krb5p pads the arguments to the nearest 8-byte
boundary, which will add padding that varies between 1 and 8 bytes.)

That might not be right for spkm3 (or any other mechanism), but if this
isn't critical then maybe we can ignore that problem for now.

Computing the exact length in general is hard. (For example, I think
spkm3 encodes the checksum using some wacky asn1 encoding that means
it's shorter if it happens to have enough consecutive zeros in it.... So
you really can't know the exactly length till you've already computed
it.)

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2007-04-20 17:57:56

by Kevin Coffman

[permalink] [raw]
Subject: Re: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

On 4/20/07, J. Bruce Fields <[email protected]> wrote:
>
> You don't need to know the exact amount, just an upper bound, right?
>
> In which case I'd be tempted to just look at some krb5i and krb5p
> traffic in wireshark, figure out how much it adds (it should always be
> the same, except that krb5p pads the arguments to the nearest 8-byte
> boundary, which will add padding that varies between 1 and 8 bytes.)

Is the length going to depend on the Kerberos encryption-type as well?

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2007-04-20 18:11:47

by J. Bruce Fields

[permalink] [raw]
Subject: Re: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

On Fri, Apr 20, 2007 at 01:57:55PM -0400, Kevin Coffman wrote:
> On 4/20/07, J. Bruce Fields <[email protected]> wrote:
> >
> >You don't need to know the exact amount, just an upper bound, right?
> >
> >In which case I'd be tempted to just look at some krb5i and krb5p
> >traffic in wireshark, figure out how much it adds (it should always be
> >the same, except that krb5p pads the arguments to the nearest 8-byte
> >boundary, which will add padding that varies between 1 and 8 bytes.)
>
> Is the length going to depend on the Kerberos encryption-type as well?

Yep. But for now we can just hard-code a constant that works for
des-cbc-whatever-it-is, with a note that some day we should increase the
constant or (if we really need it to be a tight bound on the length),
replace it by a call to the gssapi code.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2007-04-20 18:33:48

by Jeff Layton

[permalink] [raw]
Subject: Re: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

J. Bruce Fields wrote:
> On Fri, Apr 20, 2007 at 01:57:55PM -0400, Kevin Coffman wrote:
>> On 4/20/07, J. Bruce Fields <[email protected]> wrote:
>>> You don't need to know the exact amount, just an upper bound, right?
>>>
>>> In which case I'd be tempted to just look at some krb5i and krb5p
>>> traffic in wireshark, figure out how much it adds (it should always be
>>> the same, except that krb5p pads the arguments to the nearest 8-byte
>>> boundary, which will add padding that varies between 1 and 8 bytes.)
>> Is the length going to depend on the Kerberos encryption-type as well?
>
> Yep. But for now we can just hard-code a constant that works for
> des-cbc-whatever-it-is, with a note that some day we should increase the
> constant or (if we really need it to be a tight bound on the length),
> replace it by a call to the gssapi code.
>
> --b.

Thanks for the info. I'll shoot for just having the wrapper add a
constant value to it for now (though I'll at least get the auth flavor
and set that value to 0 for auth_null and auth_unix).

I tested krb5p and it seems to be off by 52 bytes, but I was using
des-cbc-crc:normal. Got a suggestion of what kerberos enc type I should
use to try to maximize the length?

Thanks,
Jeff



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2007-04-20 18:38:42

by J. Bruce Fields

[permalink] [raw]
Subject: Re: svc_process and nfsd_proc_read not taking checksum into account when calling svc_reserve

On Fri, Apr 20, 2007 at 02:33:45PM -0400, Jeff Layton wrote:
> Thanks for the info. I'll shoot for just having the wrapper add a
> constant value to it for now (though I'll at least get the auth flavor
> and set that value to 0 for auth_null and auth_unix).
>
> I tested krb5p and it seems to be off by 52 bytes, but I was using
> des-cbc-crc:normal. Got a suggestion of what kerberos enc type I should
> use to try to maximize the length?

That's all we've got in mainline right now--the others are only in
experimental code.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs