2008-12-18 22:14:54

by Alan Stern

[permalink] [raw]
Subject: Checking a USB drive's capacity

Jens:

As you may know, there are plenty of USB mass-storage devices that
respond incorrectly to READ CAPACITY: They report the total number of
blocks instead of the highest block number. As a result, the kernel
thinks the drive has one more sector than it really does.

So far we have dealt with this problem by means of a blacklist, but
this gets more and more unsatisfactory all the time. We haven't been
able to find any other way to cope, since a few devices are so badly
behaved that they crash hard when you try to access the nonexistent
"last" sector, requiring a replug or power cycle.

My new idea is to keep in the blacklist only those devices which do
crash -- a relatively small number. Everything else we should be able
to detect safely at runtime, by testing if it's possible to read the
last sector.

The question is, how and where? The logical place for testing the
capacity is near the end of sd_read_capacity(). However this code runs
before any media accesses have occurred; the drive might not even be
spun up yet. Not to mention that it seems strange to read the last
sector before reading the first!

It would make more sense to carry out this test after the first sector
has been read. That's done by the partition management code in the
gendisk layer. Would near the end of check_partition() be an
appropriate place to make the test? And could I essentially copy the
code in scsi_ioctl.c:__blk_send_generic()?

Alan Stern


2008-12-18 22:44:25

by Alan

[permalink] [raw]
Subject: Re: Checking a USB drive's capacity

> spun up yet. Not to mention that it seems strange to read the last
> sector before reading the first!

The SCSI CD code deals with this by spotting the error is close to the
end and of particular types then interpreting it and adjusting the volume
size. This is done because the size value for a CD-R/RW is imprecise.

2008-12-19 04:33:20

by Alan Stern

[permalink] [raw]
Subject: Re: Checking a USB drive's capacity

On Thu, 18 Dec 2008, Alan Cox wrote:

> > spun up yet. Not to mention that it seems strange to read the last
> > sector before reading the first!
>
> The SCSI CD code deals with this by spotting the error is close to the
> end and of particular types then interpreting it and adjusting the volume
> size. This is done because the size value for a CD-R/RW is imprecise.

Although it may be okay for CDs, I suspect that approach won't work for
disks.

For one thing, this would cause the volume size to change after the
partition table had been read and interpreted, which doesn't seem like
a good idea.

For another, it would cause the location of the "last" sector to change
in mid-flight, which might well cause problems for programs like vol_id
and hald_probe_storage -- they need to read the last sector to
determine if the volume is part of a RAID array.

Now maybe these objections won't matter; the code in each case might be
resilient enough to handle such changes. Do you think it could work?

Alan Stern

2008-12-21 14:45:16

by Alan

[permalink] [raw]
Subject: Re: Checking a USB drive's capacity

On Thu, 18 Dec 2008 23:33:09 -0500 (EST)
Alan Stern <[email protected]> wrote:

> On Thu, 18 Dec 2008, Alan Cox wrote:
>
> > > spun up yet. Not to mention that it seems strange to read the last
> > > sector before reading the first!
> >
> > The SCSI CD code deals with this by spotting the error is close to the
> > end and of particular types then interpreting it and adjusting the volume
> > size. This is done because the size value for a CD-R/RW is imprecise.
>
> Although it may be okay for CDs, I suspect that approach won't work for
> disks.

Some of the partition formats do indeed go looking for the end of disk so
it won't work in that case.

2008-12-23 04:51:57

by Robert Hancock

[permalink] [raw]
Subject: Re: Checking a USB drive's capacity

Alan Stern wrote:
> Jens:
>
> As you may know, there are plenty of USB mass-storage devices that
> respond incorrectly to READ CAPACITY: They report the total number of
> blocks instead of the highest block number. As a result, the kernel
> thinks the drive has one more sector than it really does.
>
> So far we have dealt with this problem by means of a blacklist, but
> this gets more and more unsatisfactory all the time. We haven't been
> able to find any other way to cope, since a few devices are so badly
> behaved that they crash hard when you try to access the nonexistent
> "last" sector, requiring a replug or power cycle.
>
> My new idea is to keep in the blacklist only those devices which do
> crash -- a relatively small number. Everything else we should be able
> to detect safely at runtime, by testing if it's possible to read the
> last sector.
>
> The question is, how and where? The logical place for testing the
> capacity is near the end of sd_read_capacity(). However this code runs
> before any media accesses have occurred; the drive might not even be
> spun up yet. Not to mention that it seems strange to read the last
> sector before reading the first!

It seems to me that the safest solution would be to mark USB storage
devices as unsafe for last-sector partition table probing, like some
kind of CAPACITY_DUBIOUS flag, causing it to be skipped on these
devices, at least by default. After all, it would seem quite unusual to
see anything other than a DOS-style partition table on a USB storage
device (or else no partition table at all).

2008-12-23 12:32:34

by Oliver Neukum

[permalink] [raw]
Subject: Re: Checking a USB drive's capacity

Am Dienstag, 23. Dezember 2008 05:51:39 schrieb Robert Hancock:
> It seems to me that the safest solution would be to mark USB storage
> devices as unsafe for last-sector partition table probing, like some
> kind of CAPACITY_DUBIOUS flag, causing it to be skipped on these
> devices, at least by default. After all, it would seem quite unusual to
> see anything other than a DOS-style partition table on a USB storage
> device (or else no partition table at all).

That would not help if user space (especially udev helpers) go for the end
of the device.

Regards
Oliver