Hi,
extending files by ftruncate(2) runs very slow on VFAT file
systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
My question is: is this problem on the conceptual level (ie there is
no way of extending files on FAT that doesn't involve many disk
operations) or is the current Linux fs driver suboptimal in this
respect?
The reason for asking is that I run Samba which service files on USB
devices (w/ VFAT for portability) to Windows XP clients. When copying
files to the Samba server, Microsoft SMB clients seem to extend the
file before actually starting to copy the data. This results in
sluggishness and timeouts when copying large files to VFAT
filesystems.
Thanks,
Mattias
On Tue, 2006-09-05 at 15:52, Mattias R?nnblom wrote:
> Hi,
>
> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
>
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?
>
> The reason for asking is that I run Samba which service files on USB
> devices (w/ VFAT for portability) to Windows XP clients. When copying
> files to the Samba server, Microsoft SMB clients seem to extend the
> file before actually starting to copy the data. This results in
> sluggishness and timeouts when copying large files to VFAT
> filesystems.
Is your USB stick mounted -o sync ? If that's the case, the truncate()
and write() won't be merged so they will take twice as long. -o sync
generally kills performance on USB sticks.
Xav
Mattias R?nnblom <[email protected]> writes:
> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
>
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?
Unfortunately FAT doesn't support sparse file, so ftruncate(2) which
extend size needs to fill all clusters with zero, and write data.
This is limitation of FAT filesystem.
--
OGAWA Hirofumi <[email protected]>
Xavier Bestel <[email protected]> writes:
> On Tue, 2006-09-05 at 15:52, Mattias R?nnblom wrote:
> > Hi,
> >
> > extending files by ftruncate(2) runs very slow on VFAT file
> > systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> > empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
> >
> > My question is: is this problem on the conceptual level (ie there is
> > no way of extending files on FAT that doesn't involve many disk
> > operations) or is the current Linux fs driver suboptimal in this
> > respect?
> >
> > The reason for asking is that I run Samba which service files on USB
> > devices (w/ VFAT for portability) to Windows XP clients. When copying
> > files to the Samba server, Microsoft SMB clients seem to extend the
> > file before actually starting to copy the data. This results in
> > sluggishness and timeouts when copying large files to VFAT
> > filesystems.
>
> Is your USB stick mounted -o sync ? If that's the case, the truncate()
> and write() won't be merged so they will take twice as long. -o sync
> generally kills performance on USB sticks.
>
No, 'async'.
Regards,
Mattias
Mattias R?nnblom <[email protected]> wrote:
> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
>
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?
Linux needs to zero files it truncate-extends because of security guarantees.
You could temporarily ignore the truncate after create if it's followed by
writing the file (defer untill first non-write), but it will be a BAD hack.
It might work.
Default: open(O_WRITE|O_CREAT|O_TRUNC) -> do it, goto State 1
otherwise -> just do it
State 1: ftruncate -> remember offset instead of executing ftruncate,
goto State 2
otherwise -> goto Default
State 2: write -> do it, stay in State 2 unless file size increases
beyond fake size, then goto Default
stat -> return fake size
otherwise -> really do ftruncate, goto Default
It might cause some operations to be slow you'd expect to be fast, and
I'm not sure how it has to deal with concurrent access.
--
Ich danke GMX daf?r, die Verwendung meiner Adressen mittels per SPF
verbreiteten L?gen zu sabotieren.
http://david.woodhou.se/why-not-spf.html