2010-08-12 09:33:17

by Adam Huffman

[permalink] [raw]
Subject: Problem with 15TB logical volume


Hope it's okay to send "user problems" here.

I've made a 15TB logical volume on a 21TB RAID6 iSCSI array.

When I try to make an ext4 filesystem I see the following error:

mke4fs -E stride=32 stripe-width=384 -i 65536 -j -v \
/dev/mapper/vg2md3000isata-homes2

mke4fs 1.41.9 (22-Aug-2009)
mke4fs: invalid blocks count - /dev/mapper/vg2md3000isata-homes2

Some more details:

--- Physical volume ---
PV Name /dev/mpath/mpath4
VG Name vg2md3000isata
PV Size 21.83 TB / not usable 32.00 MB
Allocatable yes
PE Size (KByte) 32768
Total PE 715205
Free PE 223685
Allocated PE 491520
PV UUID psv5BF-Tkvv-p7uq-ED0k-DCPd-3es1-uNnNSg

--- Logical volume ---
LV Name /dev/vg2md3000isata/homes2
VG Name vg2md3000isata
LV UUID qieiLw-F0he-DlDD-bZeA-v2Ok-DmE5-oS3uy2
LV Write Access read/write
LV Status available
# open 0
LV Size 15.00 TB
Current LE 491520
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:11

--- Volume group ---
VG Name vg2md3000isata
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 21.83 TB
PE Size 32.00 MB
Total PE 715205
Alloc PE / Size 491520 / 15.00 TB
Free PE / Size 223685 / 6.83 TB
VG UUID vLwJLm-z1gj-RZrM-GjXX-hmLo-cf6B-8EGMyR

When i searched briefly yesterday, some documentation stated that the
filesystem limit was now 16TB, and some that the limit was much higher.

The system is running Centos 5.5, e4fsprogs-1.41.9-3.el5.


Adam


2010-08-12 13:43:40

by Eric Sandeen

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

Adam Huffman wrote:
> Hope it's okay to send "user problems" here.

Sure, that's fine.

> I've made a 15TB logical volume on a 21TB RAID6 iSCSI array.
>
> When I try to make an ext4 filesystem I see the following error:
>
> mke4fs -E stride=32 stripe-width=384 -i 65536 -j -v \
> /dev/mapper/vg2md3000isata-homes2
>
> mke4fs 1.41.9 (22-Aug-2009)
> mke4fs: invalid blocks count - /dev/mapper/vg2md3000isata-homes2


You're not specifying the -E options correctly:

[root@host e2fsprogs]# misc/mke2fs.static -t ext4 -E stride=32 stripe-width=384 /mnt/test2/testdir/bigfile
mke2fs 1.41.12 (17-May-2010)
mke2fs.static: invalid blocks count - /mnt/test2/testdir/bigfile

This works - you need comma-separated -E options:

[root@host e2fsprogs]# mke4fs -E stride=32,stripe-width=384 /mnt/test2/testdir/bigfile
mke4fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=32 blocks, Stripe width=384 blocks
1006632960 inodes, 4026531840 blocks
...

Without the comma, mkfs thinks you are trying to mkfs a device called
"stripe-width=384" with a block count of "<devicename>"

Odd eh ;)

Perhaps echoing back the blocks count which it found to be invalid
(in this case the devicename string) would make the failure more
obvious...


> When i searched briefly yesterday, some documentation stated that the
> filesystem limit was now 16TB, and some that the limit was much higher.

Yep that should work.

-Eric

> The system is running Centos 5.5, e4fsprogs-1.41.9-3.el5.



2010-08-12 13:59:18

by Eric Sandeen

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

Eric Sandeen wrote:

...

> Without the comma, mkfs thinks you are trying to mkfs a device called
> "stripe-width=384" with a block count of "<devicename>"
>
> Odd eh ;)
>
> Perhaps echoing back the blocks count which it found to be invalid
> (in this case the devicename string) would make the failure more
> obvious...
>

Oh, actually, that is just what it's doing, but it's not obvious,
since it was the device name, and it looked like it was simply reporting
which device had the error...

-Eric


2010-08-12 14:51:46

by Adam Huffman

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

On Thu, Aug 12, 2010 at 09:43:29AM -0400, Eric Sandeen wrote:
>
> > I've made a 15TB logical volume on a 21TB RAID6 iSCSI array.
> >
> > When I try to make an ext4 filesystem I see the following error:
> >
> > mke4fs -E stride=32 stripe-width=384 -i 65536 -j -v \
> > /dev/mapper/vg2md3000isata-homes2
> >
> > mke4fs 1.41.9 (22-Aug-2009)
> > mke4fs: invalid blocks count - /dev/mapper/vg2md3000isata-homes2
>
>
> You're not specifying the -E options correctly:
>
> [root@host e2fsprogs]# misc/mke2fs.static -t ext4 -E stride=32 stripe-width=384 /mnt/test2/testdir/bigfile
> mke2fs 1.41.12 (17-May-2010)
> mke2fs.static: invalid blocks count - /mnt/test2/testdir/bigfile
>
> This works - you need comma-separated -E options:
>
> [root@host e2fsprogs]# mke4fs -E stride=32,stripe-width=384 /mnt/test2/testdir/bigfile
> mke4fs 1.41.12 (17-May-2010)
> Filesystem label=
> OS type: Linux
> Block size=4096 (log=2)
> Fragment size=4096 (log=2)
> Stride=32 blocks, Stripe width=384 blocks
> 1006632960 inodes, 4026531840 blocks
> ...
>
> Without the comma, mkfs thinks you are trying to mkfs a device called
> "stripe-width=384" with a block count of "<devicename>"
>
> Odd eh ;)
>

Yes, thanks for that. It's happy now.

> Perhaps echoing back the blocks count which it found to be invalid
> (in this case the devicename string) would make the failure more
> obvious...
>

Perhaps it could check for a clearly non-numeric input and report that?
Or maybe it's just me who'd do such a thing...


>
> > When i searched briefly yesterday, some documentation stated that the
> > filesystem limit was now 16TB, and some that the limit was much higher.
>
> Yep that should work.
>

Looking at
https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/521648,
specifically Ted's comment, is it correct that 64-bit blocks support is
still at the early development stage?

Adam


> -Eric
>
> > The system is running Centos 5.5, e4fsprogs-1.41.9-3.el5.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2010-08-12 15:14:14

by Eric Sandeen

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

Adam Huffman wrote:

> Looking at
> https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/521648,
> specifically Ted's comment, is it correct that 64-bit blocks support is
> still at the early development stage?
>
> Adam


the code is in a branch in ted's e2fsprogs git tree but not yet in a release.

-Eric

2010-08-12 17:44:33

by Andreas Dilger

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

On 2010-08-12, at 08:51, Adam Huffman wrote:
>>> mke4fs -E stride=32 stripe-width=384 -i 65536 -j -v \
>>> /dev/mapper/vg2md3000isata-homes2
>>>
>>> mke4fs 1.41.9 (22-Aug-2009)
>>> mke4fs: invalid blocks count - /dev/mapper/vg2md3000isata-homes2

I think if this was reported as:

mke4fs: invalid blocks count '/dev/mapper/vg2md3000isata-homes2'

it would definitely help.

Cheers, Andreas






2010-08-12 18:25:08

by Eric Sandeen

[permalink] [raw]
Subject: Re: Problem with 15TB logical volume

Andreas Dilger wrote:
> On 2010-08-12, at 08:51, Adam Huffman wrote:
>>>> mke4fs -E stride=32 stripe-width=384 -i 65536 -j -v \
>>>> /dev/mapper/vg2md3000isata-homes2
>>>>
>>>> mke4fs 1.41.9 (22-Aug-2009)
>>>> mke4fs: invalid blocks count - /dev/mapper/vg2md3000isata-homes2
>
> I think if this was reported as:
>
> mke4fs: invalid blocks count '/dev/mapper/vg2md3000isata-homes2'
>
> it would definitely help.


maybe even better:

mke4fs: invalid blocks count '/dev/mapper/vg2md3000isata-homes2' on device 'stripe-width=384'

;)

-Eric