On Aug 17, 2009 10:55 -0400, Nick Dokos wrote:
> There were two disk errors encountered that resulted in short reads, but
> the patched ll_ver_fs continued on (patch attached). So two chunks (1MB
> each) were not completely verified (only the part that was read
> successfully was), but the rest of the fs checked out OK. Luckily, the
> fsck did not find any errors: both disk errors were in file data.
> We have replaced the disks but are not planning to repeat the test: it's
> not clear that it would tells us anything more at this point.
> write File name: /mnt/dir00725/file011
> write complete
>
> read File name: /mnt/dir00725/file010
> read complete
Nick, thanks for the patch. I'm incorporating the fixes upstream,
but one question that was raised is that (in essence) this allows
IO errors to be hit, yet and the return code from llverfs is 0.
The llverdev/llverfs tools are used not only for finding software
data corruption bugs, but also to verify the underlying media.
It was definitely a bug in the original code that there was no
error reported during the write phase if there was a short write,
but this was at least caught during the read phase because the
data would be incorrect.
What I've done is to count errors hit during read and write, and
then exit with a non-zero value if there were any IO errors hit
(as happened in your case), even if the rest of the data was
verified correctly. This allows scanning the whole disk in a
single pass (if there are not too many underlying errors) but
still ensuring there is no false sense of security because the
program exited with 0.
The current patch can be gotten at:
https://bugzilla.lustre.org/attachment.cgi?id=25407&action=edit
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
> Nick, thanks for the patch. I'm incorporating the fixes upstream,
> but one question that was raised is that (in essence) this allows
> IO errors to be hit, yet and the return code from llverfs is 0.
> The llverdev/llverfs tools are used not only for finding software
> data corruption bugs, but also to verify the underlying media.
>
> It was definitely a bug in the original code that there was no
> error reported during the write phase if there was a short write,
> but this was at least caught during the read phase because the
> data would be incorrect.
>
> What I've done is to count errors hit during read and write, and
> then exit with a non-zero value if there were any IO errors hit
> (as happened in your case), even if the rest of the data was
> verified correctly. This allows scanning the whole disk in a
> single pass (if there are not too many underlying errors) but
> still ensuring there is no false sense of security because the
> program exited with 0.
>
> The current patch can be gotten at:
>
> https://bugzilla.lustre.org/attachment.cgi?id=25407&action=edit
>
Thanks! It looks good at first glance: I'll be trying it out before
long and will let you know if I come up against any problems.
Nick