Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
/dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
ext4. /mnt filesystem is only being added to and currently large files
(iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
I could not duplicate the issue and the amount of disk used stabilised
at 182G. The df results, whilst /mnt is mounted as ext4, are below:
1 [29/03 15:05:58] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 317G 555G 37% /mnt
1 [29/03 15:11:05] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 224G 647G 26% /mnt
1 [29/03 15:41:38] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 670G 201G 77% /mnt
1 [29/03 15:48:32] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 246G 626G 29% /mnt
1 [29/03 15:48:36] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 313G 559G 36% /mnt
1 [29/03 15:48:37] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 396G 475G 46% /mnt
1 [29/03 15:48:37] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 615G 256G 71% /mnt
1 [29/03 15:48:39] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 671G 201G 77% /mnt
1 [29/03 15:48:40] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 671G 201G 77% /mnt
1 [29/03 15:48:40] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 215G 656G 25% /mnt
1 [29/03 15:48:41] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 334G 538G 39% /mnt
1 [29/03 15:48:42] >> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 1.3T 1.2T 113G 92% /data/mirror
/dev/sda1 917G 522G 349G 60% /mnt
1 [29/03 15:48:43] >>
--
"A search of his car uncovered pornography, a homemade sex aid, women's
stockings and a Jack Russell terrier."
- http://www.news.com.au/story/0%2C27574%2C24675808-421%2C00.html
CaT wrote:
> Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
> /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
> ext4. /mnt filesystem is only being added to and currently large files
> (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
> I could not duplicate the issue and the amount of disk used stabilised
> at 182G. The df results, whilst /mnt is mounted as ext4, are below:
I would say this is likely speculative allocation due to delalloc
(ext4 has to reserve worst-case metadata amounts to prepare for
delalloc writeback). When the file data actually gets written out,
the worst-case reservation is freed up again. We also flush and
switch to nodelalloc when the filesystem is pretty close to full.
see ext4_nonda_switch() for example.
This looks pretty severe though, more than I would expect
from that behavior.
Some interesting tests would be to try it again -o nodelalloc
(I expect the behavior to go away) and maybe to copy a single
large file and see how bad the overshoot is before and after
a "sync".
-Eric
> 1 [29/03 15:05:58] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 317G 555G 37% /mnt
> 1 [29/03 15:11:05] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 224G 647G 26% /mnt
> 1 [29/03 15:41:38] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 670G 201G 77% /mnt
> 1 [29/03 15:48:32] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 246G 626G 29% /mnt
> 1 [29/03 15:48:36] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 313G 559G 36% /mnt
> 1 [29/03 15:48:37] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 396G 475G 46% /mnt
> 1 [29/03 15:48:37] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 615G 256G 71% /mnt
> 1 [29/03 15:48:39] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 671G 201G 77% /mnt
> 1 [29/03 15:48:40] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 671G 201G 77% /mnt
> 1 [29/03 15:48:40] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 215G 656G 25% /mnt
> 1 [29/03 15:48:41] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 334G 538G 39% /mnt
> 1 [29/03 15:48:42] >> df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/md5 1.3T 1.2T 113G 92% /data/mirror
> /dev/sda1 917G 522G 349G 60% /mnt
> 1 [29/03 15:48:43] >>
>
[email protected] wrote:
> On Mon, Mar 29, 2010 at 09:48:50AM -0500, Eric Sandeen wrote:
>> CaT wrote:
>>> Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
>>> /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
>>> ext4. /mnt filesystem is only being added to and currently large files
>>> (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
>>> I could not duplicate the issue and the amount of disk used stabilised
>>> at 182G. The df results, whilst /mnt is mounted as ext4, are below:
>> I would say this is likely speculative allocation due to delalloc
>> (ext4 has to reserve worst-case metadata amounts to prepare for
>> delalloc writeback). When the file data actually gets written out,
>> the worst-case reservation is freed up again. We also flush and
>> switch to nodelalloc when the filesystem is pretty close to full.
>> see ext4_nonda_switch() for example.
>>
>> This looks pretty severe though, more than I would expect
>> from that behavior.
>
> This is fixed in commit d330a5bef, which which got merged post
> 2.6.34-rc2, and which we need to get into 2.6.33 stable. If you
> cherry-pick that commit, I think you'll see that it fixes this
> problem.
Oh, right, spaced that one. Thanks for the reminder Ted. :)
And from now on ext3-mounted-as-ext4 is special-cased so you won't get
delalloc, and you won't get this behavior at all (nor will you get
any delalloc benefits, FWIW).
Thanks,
-Eric
> - Ted
On Mon, Mar 29, 2010 at 09:48:50AM -0500, Eric Sandeen wrote:
> CaT wrote:
> > Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
> > /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
> > ext4. /mnt filesystem is only being added to and currently large files
> > (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
> > I could not duplicate the issue and the amount of disk used stabilised
> > at 182G. The df results, whilst /mnt is mounted as ext4, are below:
>
> I would say this is likely speculative allocation due to delalloc
> (ext4 has to reserve worst-case metadata amounts to prepare for
> delalloc writeback). When the file data actually gets written out,
> the worst-case reservation is freed up again. We also flush and
> switch to nodelalloc when the filesystem is pretty close to full.
> see ext4_nonda_switch() for example.
>
> This looks pretty severe though, more than I would expect
> from that behavior.
This is fixed in commit d330a5bef, which which got merged post
2.6.34-rc2, and which we need to get into 2.6.33 stable. If you
cherry-pick that commit, I think you'll see that it fixes this
problem.
- Ted
On Mon, Mar 29, 2010 at 8:01 AM, Eric Sandeen <[email protected]> wrote:
> [email protected] wrote:
>> On Mon, Mar 29, 2010 at 09:48:50AM -0500, Eric Sandeen wrote:
>>> CaT wrote:
>>>> Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
>>>> /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
>>>> ext4. /mnt filesystem is only being added to and currently large files
>>>> (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
>>>> I could not duplicate the issue and the amount of disk used stabilised
>>>> at 182G. The df results, whilst /mnt is mounted as ext4, are below:
>>> I would say this is likely speculative allocation due to delalloc
>>> (ext4 has to reserve worst-case metadata amounts to prepare for
>>> delalloc writeback). ?When the file data actually gets written out,
>>> the worst-case reservation is freed up again. ?We also flush and
>>> switch to nodelalloc when the filesystem is pretty close to full.
>>> see ext4_nonda_switch() for example.
>>>
>>> This looks pretty severe though, more than I would expect
>>> from that behavior.
>>
>> This is fixed in commit d330a5bef, which which got merged post
>> 2.6.34-rc2, and which we need to get into 2.6.33 stable. ?If you
>> cherry-pick that commit, I think you'll see that it fixes this
>> problem.
>
> Oh, right, spaced that one. ?Thanks for the reminder Ted. ?:)
>
> And from now on ext3-mounted-as-ext4 is special-cased so you won't get
> delalloc, and you won't get this behavior at all (nor will you get
> any delalloc benefits, FWIW).
>
Out of curiosity, when you say ext3-mounted-as-ext4, are you referring
to filesystems without ext4 features mounted -t ext4, or filesystems
mounted -t ext3 when CONFIG_EXT4_USE_FOR_EXT23 is defined?
-Justin
Justin Maggard wrote:
> On Mon, Mar 29, 2010 at 8:01 AM, Eric Sandeen <[email protected]> wrote:
>> [email protected] wrote:
>>> On Mon, Mar 29, 2010 at 09:48:50AM -0500, Eric Sandeen wrote:
>>>> CaT wrote:
>>>>> Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
>>>>> /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
>>>>> ext4. /mnt filesystem is only being added to and currently large files
>>>>> (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
>>>>> I could not duplicate the issue and the amount of disk used stabilised
>>>>> at 182G. The df results, whilst /mnt is mounted as ext4, are below:
>>>> I would say this is likely speculative allocation due to delalloc
>>>> (ext4 has to reserve worst-case metadata amounts to prepare for
>>>> delalloc writeback). When the file data actually gets written out,
>>>> the worst-case reservation is freed up again. We also flush and
>>>> switch to nodelalloc when the filesystem is pretty close to full.
>>>> see ext4_nonda_switch() for example.
>>>>
>>>> This looks pretty severe though, more than I would expect
>>>> from that behavior.
>>> This is fixed in commit d330a5bef, which which got merged post
>>> 2.6.34-rc2, and which we need to get into 2.6.33 stable. If you
>>> cherry-pick that commit, I think you'll see that it fixes this
>>> problem.
>> Oh, right, spaced that one. Thanks for the reminder Ted. :)
>>
>> And from now on ext3-mounted-as-ext4 is special-cased so you won't get
>> delalloc, and you won't get this behavior at all (nor will you get
>> any delalloc benefits, FWIW).
>>
>
> Out of curiosity, when you say ext3-mounted-as-ext4, are you referring
> to filesystems without ext4 features mounted -t ext4, or filesystems
> mounted -t ext3 when CONFIG_EXT4_USE_FOR_EXT23 is defined?
hmmm to be honest i've lost track a little ;)
I was referring to:
commit ba69f9ab7df844125898104e854e063b47c26637
Author: Jan Kara <[email protected]>
Date: Wed Mar 24 20:18:37 2010 -0400
ext4: Don't use delayed allocation by default when used instead of ext3
When ext4 driver is used to mount a filesystem instead of the ext3 file
system driver (through CONFIG_EXT4_USE_FOR_EXT23), do not enable delayed
allocation by default since some ext3 users and application writers have
developed unfortunate expectations about the safety of writing files on
systems subject to sudden and violent death without using fsync().
Signed-off-by: Jan Kara <[email protected]>
Signed-off-by: "Theodore Ts'o" <[email protected]>
looking at that commit I guess it's -only- when CONFIG_EXT4_USE_FOR_EXT23
is defined. So in answer to your question above, it's your latter case.
See previous complaint about this all becoming rather confusing....
-Eric
> -Justin
On Mon, Mar 29, 2010 at 10:53:11AM -0400, [email protected] wrote:
> This is fixed in commit d330a5bef, which which got merged post
> 2.6.34-rc2, and which we need to get into 2.6.33 stable. If you
> cherry-pick that commit, I think you'll see that it fixes this
> problem.
As it turns out, that USB drive died soon after so, as soon as I
get a warranty replacement I'll either cherry-pick this or install
the 33 point release with it in. It's kinda wild and wooly with its
speculation. All else fails, I'll use the mount option.
Thanks to both you and Eric for your replies. :)
--
"A search of his car uncovered pornography, a homemade sex aid, women's
stockings and a Jack Russell terrier."
- http://www.news.com.au/story/0%2C27574%2C24675808-421%2C00.html