Ok,
This is about stupid users (including me) unplugging USB devices whilst
still mounted, and expecting sane semantics.
This has generally not been the 'Unix' or even 'Linux' way, but people
expect it to work. I also see no clear automated and robust solution from
userspace. "Don't do that then" is a pretty weak answer, especially since we
want to work on the desktop.
The expected behaviour is that on forceably unplugging an USB memory stick,
the created SCSI device should vanish, along with the mounts based on it.
When the user plugs in the device again, people expect to see it get the
first available name, and be available for remount, possible automated.
What actually happens with 2.6.9-rc4 is this:
[plug in the memory stick]
usb 1-2: new high speed USB device using address 9
scsi7 : SCSI emulation for USB Mass Storage devices
Vendor: M-Sys Model: DiskOnKey Rev: 4.20
Type: Direct-Access ANSI SCSI revision: 02
Attached scsi removable disk sda at scsi7, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi7, channel 0, id 0, lun 0, type 0
Note that the numbers 7 and 9 increase which each unplug/replug event, even
when no mount was attempted. This may be a bad thing in and of itself.
Now we mount:
# mount /dev/sda1 /keychain
# grep /keychain /proc/mounts
/dev/sda1 /keychain vfat rw,nodiratime,fmask=0033,dmask=0033 0 0
dmesg reports:
USB Mass Storage device found at 8
SCSI device sda: 499712 512-byte hdwr sectors (256 MB)
sda: Write Protect is off
sda: Mode Sense: 45 00 00 08
sda: assuming drive cache: write through
SCSI device sda: 499712 512-byte hdwr sectors (256 MB)
sda: Write Protect is off
sda: Mode Sense: 45 00 00 08
sda: assuming drive cache: write through
sda: sda1
Note the duplication.
Now we unplug the memory stick:
usb 1-2: USB disconnect, address 9
# grep /keychain /proc/mounts
/dev/sda1 /keychain vfat rw,nodiratime,fmask=0033,dmask=0033 0 0
# ls /keychain
#
No errors reported by ls, dmesg is filled with:
scsi7 (0:0): rejecting I/O to dead device
FAT: Directory bread(block 517) failed
scsi7 (0:0): rejecting I/O to dead device
FAT: Directory bread(block 518) failed
scsi7 (0:0): rejecting I/O to dead device
FAT: Directory bread(block 519) failed
scsi7 (0:0): rejecting I/O to dead device
FAT: Directory bread(block 520) failed
We can unmount /keychain, this reports in dmesg:
SCSI error: host 7 id 0 lun 0 return code = 4000000
Sense class 0, sense error 0, extended sense 0
# grep /keychain /proc/mounts
#
On reinsert, we can mount again:
usb 1-2: new high speed USB device using address 10
scsi8 : SCSI emulation for USB Mass Storage devices
Vendor: M-Sys Model: DiskOnKey Rev: 4.20
Type: Direct-Access ANSI SCSI revision: 02
Attached scsi removable disk sda at scsi8, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi8, channel 0, id 0, lun 0, type 0
Sometimes however, sda appears to be still 'occupied', and higher names are
used.
Now - the perhaps intended behaviour where the user can replug the USB
device when it was disconnected by accident also does not work. When we do
this, things get really out of whack, /dev/sda1 has now become invalid.
Unmounting and unplugging and replugging saves us.
Greg, others, I hope you agree this needs work. I hope we have the
infrastructure to umount based on USB disconnect events, or, alternatively,
will support 'replugging' which at least does part of what people expect.
Thanks.
--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
On Mon, Oct 11, 2004 at 02:07:01PM +0200, bert hubert wrote:
> Ok,
>
> This is about stupid users (including me) unplugging USB devices whilst
> still mounted, and expecting sane semantics.
>
> This has generally not been the 'Unix' or even 'Linux' way, but people
> expect it to work. I also see no clear automated and robust solution from
> userspace. "Don't do that then" is a pretty weak answer, especially since we
> want to work on the desktop.
>
> The expected behaviour is that on forceably unplugging an USB memory stick,
> the created SCSI device should vanish, along with the mounts based on it.
That is clearly bejond the scope of the kernel or hotplug. This policy
belongs to some other device management software. We are currently working on
HAL as one example, to make that happen.
> When the user plugs in the device again, people expect to see it get the
> first available name, and be available for remount, possible automated.
...
> Sometimes however, sda appears to be still 'occupied', and higher names are
> used.
>
> Now - the perhaps intended behaviour where the user can replug the USB
> device when it was disconnected by accident also does not work. When we do
> this, things get really out of whack, /dev/sda1 has now become invalid.
Forget about the kernel device names, these are "cookies" to access the
device and have no other meaning. Never rely on that longer as your
device session lasts! You may use a udev rule to create a stable name for
your device based on some unique device property and that will work.
Btw: With HAL we even don't care about the /dev-name, all volumes are
recognized by uuid or fslabel.
> Unmounting and unplugging and replugging saves us.
>
> Greg, others, I hope you agree this needs work. I hope we have the
> infrastructure to umount based on USB disconnect events, or, alternatively,
> will support 'replugging' which at least does part of what people expect.
Yes, we need to make the unplug of mounted devices more safe, especially
with sync mount, but don't expect the kernel or hotplug to do anything
like that. It's up to some policy software higher in the stack.
Thanks,
Kay
On Monday 11 October 2004 8:37 am, Kay Sievers wrote:
> On Mon, Oct 11, 2004 at 02:07:01PM +0200, bert hubert wrote:
> >
> > The expected behaviour is that on forceably unplugging an USB
> > memory stick,
> > the created SCSI device should vanish, along with the mounts based on it.
>
> That is clearly bejond the scope of the kernel or hotplug. This policy
> belongs to some other device management software
I've got to disagree with the "clearly", if not that entire claim.
"Clearly" would imply there's some other sane default policy
that doesn't amount to "wedge the system".
- Dave
On Mon, Oct 11, 2004 at 05:37:19PM +0200, Kay Sievers wrote:
> > The expected behaviour is that on forceably unplugging an USB memory stick,
> > the created SCSI device should vanish, along with the mounts based on it.
>
> That is clearly bejond the scope of the kernel or hotplug. This policy
> belongs to some other device management software. We are currently working on
> HAL as one example, to make that happen.
There is no way for userspace to do this. I thank you for your kneejerk 'it
is a userspace problem' report though - somebody has to do it.
Not very productive however. The kernel does not perform minimal services,
and gets confused to boot.
Now I'm all for offloading hotplug &c to userspace but when a device is
gone, people expect the associated devices to vanish as well, not linger
around generating invisible errors in dmesg.
> Yes, we need to make the unplug of mounted devices more safe, especially
> with sync mount, but don't expect the kernel or hotplug to do anything
> like that. It's up to some policy software higher in the stack.
Pray tell how I can do this from 'higher in the stack'. At least something
needs to happen, because now there is the possibility of lingering devices,
and I'm pretty sure we could get un-umountable mounts pointing there.
Thanks.
--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
There really is a 90% solution all in userspace, which is at the
bottom. The rest of this message is mostly an argument for why the
"user expecations" aren't really supportable.
bert hubert wrote:
>This is about stupid users (including me) unplugging USB devices whilst
>still mounted, and expecting sane semantics.
>
>This has generally not been the 'Unix' or even 'Linux' way, but people
>expect it to work. I also see no clear automated and robust solution from
>userspace. "Don't do that then" is a pretty weak answer, especially since we
>want to work on the desktop.
>
>
How do you expect writing to a device followed by a forced dismount to
work if you aren't using a data journaled file system, and you don't
tell it needs to clean its caches? Our lab has been using memory sticks
for embedded development for 5 years and we've managed to teach people
"don't do that" pretty well (in fact I just taught another person
today). It is of course a mistake everyone makes a few times while
learning. However the last time I used Windows for this, you had to
click on the taskbar to shut down the USB storage device, which is
basically the same thing as unmounting. Not doing so often required a
reboot.
With *nix, most data only gets written at unmount, so the only way this
can "sanely" work is for mounts you haven't written to. That case is of
course not currently handled very well, but writing would be damn near
impossible to unmount well. In order to keep the device consistent, the
only thing you can do is wait for the user to reinsert the device and
then clear your caches. However they might have modified the storage in
the meantime on another device, so you'd need some sort of consistency
check and a mirror of pretty much everything in order to do that check.
See how this is gets complicated real quickly?
In the days of DOS, you could just cut the power to the computer to turn
it off; eventually users were educated not to do that, for much the same
reason (unwritten data to storage devices). I think with a well
designed UI, most of these errors can be eliminated.
>The expected behaviour is that on forceably unplugging an USB memory stick,
>the created SCSI device should vanish, along with the mounts based on it.
>
>
Along with any data that hasen't yet been written to the drive. You're
quite likely to corrupt a fragile FS such as FAT.
>When the user plugs in the device again, people expect to see it get the
>first available name, and be available for remount, possible automated.
>
>
Automated mounting with special fixed names can already be done, this
has little to do with forced dismounting. Use something like udev for
this part.
># mount /dev/sda1 /keychain
># grep /keychain /proc/mounts
>/dev/sda1 /keychain vfat rw,nodiratime,fmask=0033,dmask=0033 0 0
>
>
Read-write VFAT without being mounted sync will pretty much never work
for forced-dismount if you have written anything.
>Unmounting and unplugging and replugging saves us.
>
>
You're likely to have corruption too if you did any writing.
>Greg, others, I hope you agree this needs work. I hope we have the
>infrastructure to umount based on USB disconnect events, or, alternatively,
>will support 'replugging' which at least does part of what people expect.
>
>
That infrastructure would have to include knowing when to clear caches
before the user ever disconnects the device. In other words, not
possible, unless you force it to sync constantly which is not very
healthy for a flash device (limited number or writes before it dies).
Replugging is the only possibility that could ever support writes. The
only case that could really be *solved* is the "read-only or no-writes"
condition, which is only 50% of the time for something like flash used
to transfer files. The only thing would could reasonably expect with
writing is to be able to clear the pinned resources somehow.
Well, what are we to do then when new university students have to use
the use the system for a class? Simply wrap copies in a script like the
following:
copy-to-memstick:
if(!mounted) mount /memstick
rsync $arg1 /memstick
umount /memstick
All I ever expect the kernel to eventually support is forced dismount of
devices that haven't been written to. I think from there its up to
userspace to sync whenever it thinks its done copying, or perhaps even
to speculatively unmount something that hasen't been used in a while. A
common data-journaled file system for use on flash could change things,
but I'm not holding my breath for other devices or OSes to support
something like that.
If you have an idea how your proposed behavior could be implemented with
details sorted out, such as writing and where the data in caches goes,
then please prove me wrong. That would make our students happier anyway...
- Jim Bruce
> With *nix, most data only gets written at unmount, so the only way this
> can "sanely" work is for mounts you haven't written to. That case is of
This is not a law of nature. You can mount sync as well. That, of course,
sucks in terms of performance and wear. A reasonable compromise
would be to do sync on close.
Supermount did this years ago.
> course not currently handled very well, but writing would be damn near
> impossible to unmount well. In order to keep the device consistent, the
> only thing you can do is wait for the user to reinsert the device and
> then clear your caches. However they might have modified the storage in
You cannot. That's giving mlock() to everybody.
[..]
> Automated mounting with special fixed names can already be done, this
> has little to do with forced dismounting. Use something like udev for
> this part.
Exactly.
[..]
> All I ever expect the kernel to eventually support is forced dismount of
> devices that haven't been written to. I think from there its up to
Devices break. You have to cope with devices going away suddenly.
You are not required to ensure data integrity in all cases, but the system
must not suffer. To allow that you must be able to get rid of the mounts
even if users do not cooperate.
Regards
Oliver
On Tue, Oct 12, 2004 at 12:24:44PM +0200, Oliver Neukum wrote:
> Devices break. You have to cope with devices going away suddenly.
> You are not required to ensure data integrity in all cases, but the system
> must not suffer. To allow that you must be able to get rid of the mounts
> even if users do not cooperate.
Well, in retrospect, the kernel appears to offer the following semantics,
perhaps unintentionally:
When a device goes away for any reason, but there are mounts that
refer to it, the device nominally stays around and an umount will
always succeed, removing the vestiges of the device with it.
This would in fact allow something in userspace listening to hotplug events
to umount on a disconnect event from USB. Except that I'm not sure if the
semantics above are guaranteed - they may just be an accident.
Things get more complicated if we have logical volumes or raid partitions
which ultimately depend on a device that is removed. In this case, userspace
should be aware of all dependencies in order to know which mountpoints to
umount. This might even include loopback mounts.
The kernel knows the dependencies implicitly and might be in a better
position to know what is invalidated by a disconnect, and which devices
disappear because of dependencies on it.
I'm hoping either Greg or Al will chime in - it appears as if part of the
infrastructure is there, but not quite developed.
Thanks.
--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
On Tue, Oct 12, 2004 at 12:24:44PM +0200, Oliver Neukum was heard to remark:
>
> > With *nix, most data only gets written at unmount, so the only way this
> > can "sanely" work is for mounts you haven't written to. That case is of
>
> This is not a law of nature. You can mount sync as well. That, of course,
> sucks in terms of performance and wear. A reasonable compromise
> would be to do sync on close.
> Supermount did this years ago.
As a practical matter, sync-on-file-close should solve most
of the practical problem of data corruption if the device is
yanked before being onmounted. However, when I read
'man 2 open' there is no O_SYNC_ON_CLOSE.
Similarly 'man 8 mount' doesn't list any option -o synconclose
It sure would be nice to be able to set up a sync-on-file-close
in the hotplug equiv of /etc/fstab for USB devices ...
(When I think of sync-on-file-close, I don't mean 'global sync
when the file is closed', I mean 'sync only that file's data and
metadata only when the file is closed'. That way, you don't slow
down systems doing a lot of i/o on other, unrelated files)
--linas