2007-06-15 12:34:30

by Christian Schmidt

[permalink] [raw]
Subject: Help needed: Partitioned software raid > 2TB

Hi everyone,

I added a drive to a linux software RAID-5 last night. Now that worked
fine... until I changed the partition table.

Disk /dev/md_d5: 2499.9 GB, 2499990978560 bytes
2 heads, 4 sectors/track, 610349360 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Device Boot Start End Blocks Id System
/dev/md_d5p1 1 244142 976566 83 Linux
/dev/md_d5p2 244143 5126956 19531256 8e Linux LVM
/dev/md_d5p3 5126957 488279488 1932610128 8e Linux LVM

This is how the layout looked before the changes.

Device Boot Start End Blocks Id System
/dev/md_d5p1 1 244142 976566 83 Linux
/dev/md_d5p2 244143 5126956 19531256 8e Linux LVM
/dev/md_d5p3 5126957 73478448 273405968 8e Linux LVM

This is how it looks now - the fdisk tools (fdisk/cfdisk) are unable to
crate a single partition > 2TB. Of course now LVM complains that the PV
is too small:

WARNING: Physical Volume /dev/md_d5p3 is too large for underlying device
PV Size 260.74 GB / not usable 8192.00 EB

I was able to restore a somewhat close partition table now... Close
enough for the LVM to work correctly.

Device Boot Start End Blocks Id System
/dev/md_d5p1 1 244142 976566 83 Linux
/dev/md_d5p2 244143 5126956 19531256 8e Linux LVM
/dev/md_d5p3 5126957 488279495 1932610156 8e Linux LVM
/dev/md_d5p4 488279496 73478448 488279460 8e Linux LVM

Where is the inherent limit? The partitioning software, or partitioning
all by itself?

Regards,
Christian


2007-06-15 13:41:48

by Andi Kleen

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB

Christian Schmidt <[email protected]> writes:
>
> Where is the inherent limit? The partitioning software, or partitioning
> all by itself?

DOS style partitioning don't support more than 2TB. You either need
to use EFI partitions (e.g. using parted) or LVM. Since parted's
user interface is not good for much more than serving
as a bad example I would recommend LVM.

-Andi

2007-06-15 14:03:17

by Christian Schmidt

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB

Hi Andi,

Andi Kleen wrote:
> Christian Schmidt <[email protected]> writes:
>> Where is the inherent limit? The partitioning software, or partitioning
>> all by itself?
>
> DOS style partitioning don't support more than 2TB. You either need
> to use EFI partitions (e.g. using parted) or LVM. Since parted's
> user interface is not good for much more than serving
> as a bad example I would recommend LVM.

Thanks for the clarification. I didn't use LVM on the device on purpose,
as root on LVM requires initrd (which I strongly dislike as
yet-another-point-of-failure). As LVM is on the large partition anyway
I'll just add the second partition for now, and change the system setup
with the next drive migration. Maybe linux even supports root-on-lvm
natively until then ;)

Regards,
Christian

2007-06-15 16:26:01

by Jan Engelhardt

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB


On Jun 15 2007 16:03, Christian Schmidt wrote:
>Hi Andi,
>
>Andi Kleen wrote:
>> Christian Schmidt <[email protected]> writes:
>>> Where is the inherent limit? The partitioning software, or partitioning
>>> all by itself?
>>
>> DOS style partitioning don't support more than 2TB. You either need
>> to use EFI partitions (e.g. using parted) or LVM. Since parted's
>> user interface is not good for much more than serving
>> as a bad example I would recommend LVM.

parted's interface is no worse than that of [sfc]disk. Anyway...

>Thanks for the clarification. I didn't use LVM on the device on purpose,
>as root on LVM requires initrd (which I strongly dislike as
>yet-another-point-of-failure). As LVM is on the large partition anyway
>I'll just add the second partition for now, and change the system setup
>with the next drive migration. Maybe linux even supports root-on-lvm
>natively until then ;)

Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
can't do root-on-LVM, there is at least one who can.


Jan
--

2007-06-16 05:38:17

by Alexander E. Patrakov

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB

Jan Engelhardt wrote:
> On Jun 15 2007 16:03, Christian Schmidt wrote:

>> Thanks for the clarification. I didn't use LVM on the device on purpose,
>> as root on LVM requires initrd (which I strongly dislike as
>> yet-another-point-of-failure). As LVM is on the large partition anyway
>> I'll just add the second partition for now, and change the system setup
>> with the next drive migration. Maybe linux even supports root-on-lvm
>> natively until then ;)
>
> Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
> can't do root-on-LVM, there is at least one who can.

AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires
LILO, doesn't it? Could you please list a few distributions with an easy
method to install LILO (and, of course, root and /boot on LVM on whole disk)
from their default installation media? So far, I only know that Debian can
do it if you run debootstrap by hand, although they say that such setup is a
bug (http://bugs.debian.org/401393).

--
Alexander E. Patrakov

2007-06-16 06:46:52

by Jan Engelhardt

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB


On Jun 16 2007 11:38, Alexander E. Patrakov wrote:
> Jan Engelhardt wrote:
>> On Jun 15 2007 16:03, Christian Schmidt wrote:
>
>> > Thanks for the clarification. I didn't use LVM on the device on purpose,
>> > as root on LVM requires initrd (which I strongly dislike as
>> > yet-another-point-of-failure). As LVM is on the large partition anyway
>> > I'll just add the second partition for now, and change the system setup
>> > with the next drive migration. Maybe linux even supports root-on-lvm
>> > natively until then ;)
>>
>> Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
>> can't do root-on-LVM, there is at least one who can.
>
> AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself)
> requires LILO, doesn't it? Could you please list a few distributions
> with an easy method to install LILO (and, of course, root and /boot on
> LVM on whole disk) from their default installation media? So far, I only
> know that Debian can do it if you run debootstrap by hand, although they
> say that such setup is a bug (http://bugs.debian.org/401393).

I am not sure (would have to check again), but I believe both opensuse and
fedora (the latter of which uses LVM for all partitions by default) have
that working, while still using GRUB. But, what's much more amazing, is
that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps
it's just luck that the needed files are contiguous?

Jan
--

2007-06-16 07:55:58

by Alexander E. Patrakov

[permalink] [raw]
Subject: Re: Help needed: Partitioned software raid > 2TB

Jan Engelhardt wrote:

> I am not sure (would have to check again), but I believe both opensuse and
> fedora (the latter of which uses LVM for all partitions by default) have
> that working, while still using GRUB.

Keyword: partitions. I.e., they partition the hard drive (so that the first
31 sectors are available for GRUB) and use LVM on devices such as /dev/hda2.
But this is not what was needed. I need to use LVM on /dev/hda, without a
partition table.

> But, what's much more amazing, is
> that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps
> it's just luck that the needed files are contiguous?

[wild guess] Maybe they set up a raid1 /boot partition?

--
Alexander E. Patrakov