On 08/29/2013 03:30 PM, J. Bruce Fields wrote:
> On Thu, Aug 29, 2013 at 11:57:45AM +0200, richard -rw- weinberger wrote:
>> On Wed, Aug 28, 2013 at 7:21 PM, Toralf Förster <[email protected]> wrote:
>>> On 08/27/2013 08:06 PM, J. Bruce Fields wrote:
>>>> On Tue, Aug 13, 2013 at 05:53:14PM -0400, bfields wrote:
>>>>> On Mon, Aug 12, 2013 at 04:36:40PM +0200, Jan Kara wrote:
>>>>>> On Sun 11-08-13 11:48:49, Toralf Förster wrote:
>>>>>>> so that the server either crashes (if it is a user mode linux image) or at least its reboot functionality got broken
>>>>>>> - if the NFS server is hammered with scary NFS calls using a fuzzy tool running at a remote NFS client under a non-privileged user id.
>>>>>>>
>>>>>>> It can re reproduced, if
>>>>>>> - the NFS share is an EXT3 or EXT4 directory
>>>>>>> - and it is created at file located at tempfs and mounted via loop device
>>>>>>> - and the NFS server is forced to umount the NFS share
>>>>>>> - and the server forced to restart the NSF service afterwards
>>>>>>> - and trinity is used
>>>>>>>
>>>>>>> I could find a scenario for an automated bisect. 2 times it brought this commit
>>>>>>> commit 68a3396178e6688ad7367202cdf0af8ed03c8727
>>>>>>> Author: J. Bruce Fields <[email protected]>
>>>>>>> Date: Thu Mar 21 11:21:50 2013 -0400
>>>>>>>
>>>>>>> nfsd4: shut down more of delegation earlier
>>>>>
>>>>> Thanks for the report. I think I see the problem--after this commit
>>>>> nfs4_set_delegation() failures result in nfs4_put_delegation being
>>>>> called, but nfs4_put_delegation doesn't free the nfs4_file that has
>>>>> already been set by alloc_init_deleg().
>>>>>
>>>>> Let me think about how to fix that....
>>>>
>>>> Sorry for the slow response--can you check whether this fixes the
>>>> problem?
>>>>
>>> Yes.
>>>
>>> With the attached patch the problem can't be reproduced any longer with
>>> the prepared test case and current git kernels.
>>
>> BTW: Is nobody else fuzz testing NFS?
>
> I don't know. Toralf's reports are the only ones I recall off the top
> of my head, but I may have forgotten others.
>
well, 7255e71 and 3c50ba8 I'd say.
>> Or are these bugs just more likely to hit on UML?
This definitely not. I observed at a real system EXT4 corruptions/
issues but reported them to the EXT4 mailing list.
It just took me a longer time to figure out a reliable configuration
with 2 UML machiens to automatic bisect it.
> That's also possible.
>
>> This is not the first NFS issue found by Toralf using UML and Trinity.
>
> Yep. The testing is definitely appreciated.
Thx - in the mean while although my UML bisect scripts are working fine
and trinity is stable enough even in UML environments to be trust worth.
>
> --b.
>
--
MfG/Sincerely
Toralf Förster
pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
On Fri, Aug 30, 2013 at 04:10:42PM +0200, Toralf F?rster wrote:
> On 08/29/2013 03:30 PM, J. Bruce Fields wrote:
> > On Thu, Aug 29, 2013 at 11:57:45AM +0200, richard -rw- weinberger wrote:
> >> On Wed, Aug 28, 2013 at 7:21 PM, Toralf F?rster <[email protected]> wrote:
> >>> On 08/27/2013 08:06 PM, J. Bruce Fields wrote:
> >>>> On Tue, Aug 13, 2013 at 05:53:14PM -0400, bfields wrote:
> >>>>> On Mon, Aug 12, 2013 at 04:36:40PM +0200, Jan Kara wrote:
> >>>>>> On Sun 11-08-13 11:48:49, Toralf F?rster wrote:
> >>>>>>> so that the server either crashes (if it is a user mode linux image) or at least its reboot functionality got broken
> >>>>>>> - if the NFS server is hammered with scary NFS calls using a fuzzy tool running at a remote NFS client under a non-privileged user id.
> >>>>>>>
> >>>>>>> It can re reproduced, if
> >>>>>>> - the NFS share is an EXT3 or EXT4 directory
> >>>>>>> - and it is created at file located at tempfs and mounted via loop device
> >>>>>>> - and the NFS server is forced to umount the NFS share
> >>>>>>> - and the server forced to restart the NSF service afterwards
> >>>>>>> - and trinity is used
> >>>>>>>
> >>>>>>> I could find a scenario for an automated bisect. 2 times it brought this commit
> >>>>>>> commit 68a3396178e6688ad7367202cdf0af8ed03c8727
> >>>>>>> Author: J. Bruce Fields <[email protected]>
> >>>>>>> Date: Thu Mar 21 11:21:50 2013 -0400
> >>>>>>>
> >>>>>>> nfsd4: shut down more of delegation earlier
> >>>>>
> >>>>> Thanks for the report. I think I see the problem--after this commit
> >>>>> nfs4_set_delegation() failures result in nfs4_put_delegation being
> >>>>> called, but nfs4_put_delegation doesn't free the nfs4_file that has
> >>>>> already been set by alloc_init_deleg().
> >>>>>
> >>>>> Let me think about how to fix that....
> >>>>
> >>>> Sorry for the slow response--can you check whether this fixes the
> >>>> problem?
> >>>>
> >>> Yes.
> >>>
> >>> With the attached patch the problem can't be reproduced any longer with
> >>> the prepared test case and current git kernels.
> >>
> >> BTW: Is nobody else fuzz testing NFS?
> >
> > I don't know. Toralf's reports are the only ones I recall off the top
> > of my head, but I may have forgotten others.
> >
>
> well, 7255e71 and 3c50ba8 I'd say.
I don't know any 3c50ba8. 0c7c3e67 "nfsd4: don't close read-write opens
too soon" and 64a817cf "nfsd4: reject "negative" acl lengths" are two
other serious bugs found by your testing. I don't recall fuzz testing
results from anyone else, but as I say I may have forgotten.
--b.
Am 30.08.2013 16:10, schrieb Toralf Förster:
> On 08/29/2013 03:30 PM, J. Bruce Fields wrote:
>> On Thu, Aug 29, 2013 at 11:57:45AM +0200, richard -rw- weinberger wrote:
>>> On Wed, Aug 28, 2013 at 7:21 PM, Toralf Förster <[email protected]> wrote:
>>>> On 08/27/2013 08:06 PM, J. Bruce Fields wrote:
>>>>> On Tue, Aug 13, 2013 at 05:53:14PM -0400, bfields wrote:
>>>>>> On Mon, Aug 12, 2013 at 04:36:40PM +0200, Jan Kara wrote:
>>>>>>> On Sun 11-08-13 11:48:49, Toralf Förster wrote:
>>>>>>>> so that the server either crashes (if it is a user mode linux image) or at least its reboot functionality got broken
>>>>>>>> - if the NFS server is hammered with scary NFS calls using a fuzzy tool running at a remote NFS client under a non-privileged user id.
>>>>>>>>
>>>>>>>> It can re reproduced, if
>>>>>>>> - the NFS share is an EXT3 or EXT4 directory
>>>>>>>> - and it is created at file located at tempfs and mounted via loop device
>>>>>>>> - and the NFS server is forced to umount the NFS share
>>>>>>>> - and the server forced to restart the NSF service afterwards
>>>>>>>> - and trinity is used
>>>>>>>>
>>>>>>>> I could find a scenario for an automated bisect. 2 times it brought this commit
>>>>>>>> commit 68a3396178e6688ad7367202cdf0af8ed03c8727
>>>>>>>> Author: J. Bruce Fields <[email protected]>
>>>>>>>> Date: Thu Mar 21 11:21:50 2013 -0400
>>>>>>>>
>>>>>>>> nfsd4: shut down more of delegation earlier
>>>>>>
>>>>>> Thanks for the report. I think I see the problem--after this commit
>>>>>> nfs4_set_delegation() failures result in nfs4_put_delegation being
>>>>>> called, but nfs4_put_delegation doesn't free the nfs4_file that has
>>>>>> already been set by alloc_init_deleg().
>>>>>>
>>>>>> Let me think about how to fix that....
>>>>>
>>>>> Sorry for the slow response--can you check whether this fixes the
>>>>> problem?
>>>>>
>>>> Yes.
>>>>
>>>> With the attached patch the problem can't be reproduced any longer with
>>>> the prepared test case and current git kernels.
>>>
>>> BTW: Is nobody else fuzz testing NFS?
>>
>> I don't know. Toralf's reports are the only ones I recall off the top
>> of my head, but I may have forgotten others.
>>
>
> well, 7255e71 and 3c50ba8 I'd say.
>
>>> Or are these bugs just more likely to hit on UML?
>
> This definitely not. I observed at a real system EXT4 corruptions/
> issues but reported them to the EXT4 mailing list.
> It just took me a longer time to figure out a reliable configuration
> with 2 UML machiens to automatic bisect it.
>
>
>> That's also possible.
>>
>>> This is not the first NFS issue found by Toralf using UML and Trinity.
>>
>> Yep. The testing is definitely appreciated.
>
> Thx - in the mean while although my UML bisect scripts are working fine
> and trinity is stable enough even in UML environments to be trust worth.
That's good to know.
Thanks you and trinity we got rid of some nasty UML bugs.
Thanks,
//richard