All,
On the mdraid list, there was a recent thread about using raid
functionality to detect / repair silent corruption.
The issues brought up were that a lot of silent data corruption occurs
when cables, controllers, power supplies, ram, cache, etc. goes bad.
It made me think about another option for detecting silent corruption
I have not seen discussed, but maybe I missed it.
Aiui, the ATA spec allows for the reading of a long sector as well as
the normal 512 byte sector. When you get a long sector you also get
the CRC (or whatever checksum data there is on the disk that allows
the drive itself to detect media errors).
I don't have any idea how easy or hard it would be to do, but I would
like to see the entire block subsystem enhanced to optionally allow
long sector reads to be used in a "paranoid" fashion.
Effectively it would be:
1) Read long sector from drive: verify CRC in kernel. This tests
most everything on the i/o path.
2) maintain CRC type information in block subsystem. Verify no
corruption just before handing off to userspace. This would
potentially identify CPU/cache/RAM failures.
Mark Lord has implemented long sector reads via hdparm. Mark can you
comment on the feasibility of this idea?
Thanks
Greg
--
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
Greg Freemyer wrote:
> All,
>
> On the mdraid list, there was a recent thread about using raid
> functionality to detect / repair silent corruption.
>
> The issues brought up were that a lot of silent data corruption occurs
> when cables, controllers, power supplies, ram, cache, etc. goes bad.
>
> It made me think about another option for detecting silent corruption
> I have not seen discussed, but maybe I missed it.
>
> Aiui, the ATA spec allows for the reading of a long sector as well as
> the normal 512 byte sector. When you get a long sector you also get
> the CRC (or whatever checksum data there is on the disk that allows
> the drive itself to detect media errors).
>
> I don't have any idea how easy or hard it would be to do, but I would
> like to see the entire block subsystem enhanced to optionally allow
> long sector reads to be used in a "paranoid" fashion.
>
> Effectively it would be:
>
> 1) Read long sector from drive: verify CRC in kernel. This tests
> most everything on the i/o path.
>
> 2) maintain CRC type information in block subsystem. Verify no
> corruption just before handing off to userspace. This would
> potentially identify CPU/cache/RAM failures.
Even if the drive supports those commands the problem is the CRC/ECC
data is in a vendor-specific format, so it couldn't be processed
generically.
> -----Original Message-----
> From: [email protected] [mailto:linux-raid-
> [email protected]] On Behalf Of Robert Hancock
> Sent: Friday, December 26, 2008 4:16 PM
> To: [email protected]
> Cc: [email protected]
> Subject: Re: RFC: detection of silent corruption via ATA long sector
> reads
>
> Greg Freemyer wrote:
> > All,
> >
> > On the mdraid list, there was a recent thread about using raid
> > functionality to detect / repair silent corruption.
> >
> > The issues brought up were that a lot of silent data corruption
> occurs
> > when cables, controllers, power supplies, ram, cache, etc. goes bad.
> >
> > It made me think about another option for detecting silent
corruption
> > I have not seen discussed, but maybe I missed it.
> >
> > Aiui, the ATA spec allows for the reading of a long sector as well
as
> > the normal 512 byte sector. When you get a long sector you also get
> > the CRC (or whatever checksum data there is on the disk that allows
> > the drive itself to detect media errors).
> >
> > I don't have any idea how easy or hard it would be to do, but I
would
> > like to see the entire block subsystem enhanced to optionally allow
> > long sector reads to be used in a "paranoid" fashion.
> >
> > Effectively it would be:
> >
> > 1) Read long sector from drive: verify CRC in kernel. This tests
> > most everything on the i/o path.
> >
> > 2) maintain CRC type information in block subsystem. Verify no
> > corruption just before handing off to userspace. This would
> > potentially identify CPU/cache/RAM failures.
>
> Even if the drive supports those commands the problem is the CRC/ECC
> data is in a vendor-specific format, so it couldn't be processed
> generically.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Many of the RAID appliance/subsystem vendors format the disks to 520 or
528
Bytes/sector, but expose 512-byte blocks to the user. The ECC logic is
done
by the firmware ... or if this ever gets implemented, would be done by
the
LINUX kernel. True there are some issues with many of the cheap
consumer
class drives not supporting anything but 512-byte blocks, but we
shouldn't
code to lowest common denominator.
With 1TB SATA disks selling for $99, then it isn't as if the extra 8-16
bytes
for ECC on the disk drive is going to be a problem.
David
Greg Freemyer wrote:
> All,
>
> On the mdraid list, there was a recent thread about using raid
> functionality to detect / repair silent corruption.
>
> The issues brought up were that a lot of silent data corruption occurs
> when cables, controllers, power supplies, ram, cache, etc. goes bad.
>
> It made me think about another option for detecting silent corruption
> I have not seen discussed, but maybe I missed it.
>
> Aiui, the ATA spec allows for the reading of a long sector as well as
> the normal 512 byte sector. When you get a long sector you also get
> the CRC (or whatever checksum data there is on the disk that allows
> the drive itself to detect media errors).
>
> I don't have any idea how easy or hard it would be to do, but I would
> like to see the entire block subsystem enhanced to optionally allow
> long sector reads to be used in a "paranoid" fashion.
>
> Effectively it would be:
>
> 1) Read long sector from drive: verify CRC in kernel. This tests
> most everything on the i/o path.
>
> 2) maintain CRC type information in block subsystem. Verify no
> corruption just before handing off to userspace. This would
> potentially identify CPU/cache/RAM failures.
>
> Mark Lord has implemented long sector reads via hdparm. Mark can you
> comment on the feasibility of this idea?
..
The ATA READ/WRITE LONG commands have been obsoleted in the past few ATA specs,
even though most drives continue to implement them.
But not a good avenue.
There's a separate effort, involving drive vendors and kernel hackers,
to provide end-to-end CRC protection of data. I forget what it was called,
but that's the future of this stuff for high-reliability requirements.
Cheers
> Is there a more comprehensive write-up / resource that describes the
> current status of the overall INTEGRITY support is, especially as it
> relates to ATA devices?
Did you check the kernel notes on kernelnewbies when the feature went in -
http://kernelnewbies.org/Linux_2_6_27 ?
On Fri, Jan 2, 2009 at 3:43 PM, Sitsofe Wheeler <[email protected]> wrote:
>> Is there a more comprehensive write-up / resource that describes the
>> current status of the overall INTEGRITY support is, especially as it
>> relates to ATA devices?
>
>
> Did you check the kernel notes on kernelnewbies when the feature went in -
> http://kernelnewbies.org/Linux_2_6_27 ?
Interesting read, but it does not really answer the questions I posed.
I did look through the 2.6.27 source I have handy and the only call to
blk_integrity_register() is in./drivers/scsi/sd_dif.c.
That leaves me with the impression that there are not any ATA devices
claiming support yet.
Greg
--
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com