Hi,
On 2.6.0-testx kernels, I have noticed that there are problems with
GNU Parted. Parted says that the disk geometries reported by the
kernel are incorrect. Here is a sample error message :
---
Error: The partition table on /dev/hdb is inconsistent. There are
many reasons why this might be the case. However, the most likely
reason is that Linux detected the BIOS geometry for /dev/hdb
incorrectly. GNU Parted suspects the real geometry should be
782/128/63 (not 6256/16/63). You should check with your BIOS first,
as this may not be correct. You can inform Linux by adding the
parameter hdb=782,128,63 to the command line. See the LILO or GRUB
documentation for more information. If you think Parted's suggested
geometry is correct, you may select Ignore to continue (and fix Linux
later). Otherwise, select Cancel (and fix Linux and/or the BIOS now).
---
Please let me know if you'll need any additional information.
I am not subscribed to this list so please cc me any replies.
Regards,
- Apurva
On Fri, Nov 28, 2003 at 10:28:54AM +0530, Apurva Mehta wrote:
> On 2.6.0-testx kernels, I have noticed that there are problems with
> GNU Parted. Parted says that the disk geometries reported by the
> kernel are incorrect.
Yes. Parted should be fixed not to complain.
There is no such thing as a "correct" disk geometry.
Roughly speaking the kernel no longer attempts to report anything,
so calling HDIO_GETGEO is useless (for this purpose).
Andries
On Fri, Nov 28, 2003 at 03:24:52PM +0100, Andries Brouwer wrote:
> On Fri, Nov 28, 2003 at 10:28:54AM +0530, Apurva Mehta wrote:
>
> > On 2.6.0-testx kernels, I have noticed that there are problems with
> > GNU Parted. Parted says that the disk geometries reported by the
> > kernel are incorrect.
>
> Yes. Parted should be fixed not to complain.
> There is no such thing as a "correct" disk geometry.
Yes there is. "Correct" is defined by the BIOS. It is important
for boot loaders (in particular Windows). I'm not sure if this
is still a big issue worth worrying about.
Cheers,
Andrew
On Sat, 29 Nov 2003, Andrew Clausen wrote:
> On Fri, Nov 28, 2003 at 03:24:52PM +0100, Andries Brouwer wrote:
> >
> > There is no such thing as a "correct" disk geometry.
>
> Yes there is. "Correct" is defined by the BIOS. It is important
> for boot loaders (in particular Windows).
I suspected the same ... What Windows you mean? DOS (9x/ME/etc) or NT based
(NT4/2000/XP/2003)? All?
> I'm not sure if this is still a big issue worth worrying about.
IMHO it might be. At least I'm getting an increasing number of emails from
people who can't boot Windows anymore after resizing and repartitioning
NTFS on Linux. Everybody thinks it's the Linux NTFS code's fault but so far
it was always about the repartitioning going wrong. I just had to write a
FAQ entry about this issue recently
http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html#troubleshoot
Some users, having problems, did mention the usage of 2.6 kernel. If the
geometry changed during the fdisk, etc process then it could result also
booting problem? It's just a speculation because I've never had enough
information to investigate.
Also, can Parted save/restore the full and exact partition table a
scriptable way? I mean something like this:
sfdisk -d /dev/hda > hda.pt # save
sfdisk /dev/hda < hda.pt # restore
sfdisk can't recover geometry so apparently no one-liner, widely available,
partition table backup/recovery is possible at present on Linux :-o
dd if=/dev/hda of=hda.mbr bs=512 count=1 won't save the logical partitions.
Szaka
On Sat, Nov 29, 2003 at 07:16:31AM +0200, Szakacsits Szabolcs wrote:
>
> On Sat, 29 Nov 2003, Andrew Clausen wrote:
> > On Fri, Nov 28, 2003 at 03:24:52PM +0100, Andries Brouwer wrote:
> > >
> > > There is no such thing as a "correct" disk geometry.
> >
> > Yes there is. "Correct" is defined by the BIOS. It is important
> > for boot loaders (in particular Windows).
>
> I suspected the same ... What Windows you mean? DOS (9x/ME/etc) or NT based
> (NT4/2000/XP/2003)? All?
>
> > I'm not sure if this is still a big issue worth worrying about.
>
> IMHO it might be. At least I'm getting an increasing number of emails from
> people who can't boot Windows anymore after resizing and repartitioning
> NTFS on Linux. Everybody thinks it's the Linux NTFS code's fault but so far
> it was always about the repartitioning going wrong. I just had to write a
> FAQ entry about this issue recently
>
> http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html#troubleshoot
>
> Some users, having problems, did mention the usage of 2.6 kernel. If the
> geometry changed during the fdisk, etc process then it could result also
> booting problem? It's just a speculation because I've never had enough
> information to investigate.
>
> Also, can Parted save/restore the full and exact partition table a
> scriptable way? I mean something like this:
>
> sfdisk -d /dev/hda > hda.pt # save
> sfdisk /dev/hda < hda.pt # restore
>
> sfdisk can't recover geometry so apparently no one-liner, widely available,
> partition table backup/recovery is possible at present on Linux :-o
> dd if=/dev/hda of=hda.mbr bs=512 count=1 won't save the logical partitions.
Yes, that would be a good idea, it would be even nice to automatically
save the partition table the first time parted access the harddisk. The
problem is that it needs to be saved on a separate harddisk though, or
printed or something such.
The partition table saving/restoring would be part of the partition
table specific code, so you could know the logical partitions or
whatever your precise non-mbr partition table mandates.
Friendly,
Sven Luther
On Sat, Nov 29, 2003 at 07:16:31AM +0200, Szakacsits Szabolcs wrote:
> http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html#troubleshoot
>
> Some users, having problems, did mention the usage of 2.6 kernel. If the
> geometry changed during the fdisk, etc process then it could result also
> booting problem?
Let me continue to stress: geometry does not exist.
Consequently, it cannot change.
fdisk does not set any geometry, it writes a partition table.
Start and size of each partition are given twice: in absolute sector
units (LBA) and in CHS units. The former uses 32 bits, and with 512-byte
sectors this works up to 2TB. People are starting to hit that boundary now.
The latter uses 24 bits, which works up to 8GB. Modern systems no longer
use it (but the details are complicated).
Usually booting goes like this: the BIOS reads sector 0 (the MBR)
from the first disk, and starts the code found there. What happens
afterwards is up to that code. If that code uses CHS units to find
a partition, and if the program that wrote the table has different
ideas about those units than the BIOS, booting may fail.
Andries
> > Also, can Parted save/restore the full and exact partition table a
> > scriptable way? I mean something like this:
> >
> > sfdisk -d /dev/hda > hda.pt # save
> > sfdisk /dev/hda < hda.pt # restore
> >
> > sfdisk can't recover geometry so apparently no one-liner, widely available,
> > partition table backup/recovery is possible at present on Linux :-o
> > dd if=/dev/hda of=hda.mbr bs=512 count=1 won't save the logical partitions.
sfdisk has a slightly different option: -O will save all sectors that
you change during an sfdisk operation, so that you can restore them later.
You see, saving the logical partitions is not enough - these sectors
are spread out over the disk, and there used to be something else
where these sectors are written. If the user makes a partitioning mistake
he destroys a sector worth of data. It is that data that sfdisk -O will save
(and -I will restore).
> Let me continue to stress: geometry does not exist.
> Consequently, it cannot change.
> fdisk does not set any geometry, it writes a partition table.
>
> Start and size of each partition are given twice: in absolute sector
> units (LBA) and in CHS units. The former uses 32 bits, and with 512-byte
> sectors this works up to 2TB. People are starting to hit that boundary now.
> The latter uses 24 bits, which works up to 8GB. Modern systems no longer
> use it (but the details are complicated).
Why don't we take the opporunity to make all CHS code configurable out
of the kernel, and define a new, more compact, partition table format
which used LBA exclusively, and allowed more than four partitions in
the main partition table?
I know it sounds pointless to define a new partitioning scheme when
there are so many already in existance, but for dedicated Linux
machines, only being able to define four partitions without resorting
to 'extended' partitions, which store there partitioning data in other
parts of the disk, is a needless limitation. We could also ensure
that there is sufficient magic in the partition table to make
identifying it easy and reliable.
John.
Hi.
> Why don't we take the opporunity to make all CHS code configurable out
> of the kernel, and define a new, more compact, partition table format
> which used LBA exclusively, and allowed more than four partitions in
> the main partition table?
>
> I know it sounds pointless to define a new partitioning scheme when
> there are so many already in existance, but for dedicated Linux
> machines, only being able to define four partitions without resorting
> to 'extended' partitions, which store there partitioning data in other
> parts of the disk, is a needless limitation. We could also ensure
> that there is sufficient magic in the partition table to make
> identifying it easy and reliable.
Then just select a partitioning scheme that fills those features you
request instead, as you say - there are very many out there and
you're bound to find at least ONE that has those features (I can name
a few straight off) :)
// Stefan
On Sat, Nov 29, 2003 at 01:50:00PM +0000, John Bradford wrote:
> > Let me continue to stress: geometry does not exist.
> > Consequently, it cannot change.
> > fdisk does not set any geometry, it writes a partition table.
> >
> > Start and size of each partition are given twice: in absolute sector
> > units (LBA) and in CHS units. The former uses 32 bits, and with 512-byte
> > sectors this works up to 2TB. People are starting to hit that boundary now.
> > The latter uses 24 bits, which works up to 8GB. Modern systems no longer
> > use it (but the details are complicated).
>
> Why don't we take the opporunity to make all CHS code configurable out
> of the kernel, and define a new, more compact, partition table format
> which used LBA exclusively, and allowed more than four partitions in
> the main partition table?
The only problem is that your BIOS will probably not be able to boot
from it, so unless you use some kind of free bios implementation or even
some kind of free OF or something such, you will never be able to boot
from these drives.
For non booting drives, nothing stops you from using alternate partition
tables, the Mac OS on as well as the Amiga partition tables seems nice
to me (maybe others too, but i mostly know these two of the alternative
partition tables) as these are simply linked list of partition entries,
you can have as many as you want (limit to 128 partitions for amiga
partition in the libparted implementation i made for example).
Friendly,
Sven Luther
On Sat, Nov 29, 2003 at 06:01:03PM +0100, Sven Luther wrote:
> The only problem is that your BIOS will probably not be able
> to boot from it
You seem to misunderstand the boot sequence.
The BIOS does not generally do any parsing of partition tables.
On Sat, Nov 29, 2003 at 01:34:51PM +0100, Andries Brouwer wrote:
> On Sat, Nov 29, 2003 at 07:16:31AM +0200, Szakacsits Szabolcs wrote:
>
> > http://mlf.linux.rulez.org/mlf/ezaz/ntfsresize.html#troubleshoot
> >
> > Some users, having problems, did mention the usage of 2.6 kernel. If the
> > geometry changed during the fdisk, etc process then it could result also
> > booting problem?
>
> Let me continue to stress: geometry does not exist.
Let me continue to stress: geometry DOES exist.
It is an abstract construct that is stored in your BIOS that some
configurations use and need for booting.
> Consequently, it cannot change.
The geometry can be changed in many BIOSes.
> fdisk does not set any geometry, it writes a partition table.
But the way it writes the partition table is affected by what it
believes the geometry to be. If its beliefs don't match the BIOS,
there can be trouble. However, since fdisk isn't typically used to
resize Windows partitions, you don't see problems so much.
> Start and size of each partition are given twice: in absolute sector
> units (LBA) and in CHS units. The former uses 32 bits, and with 512-byte
> sectors this works up to 2TB. People are starting to hit that boundary now.
> The latter uses 24 bits, which works up to 8GB. Modern systems no longer
> use it (but the details are complicated).
You mean "modern software". I'm not sure how true this is. Have you
got any evidence?
(i.e. have you got any evidence that, say, that 99.x% of Windows XP
installations use LBA to bootstrap?)
> Usually booting goes like this: the BIOS reads sector 0 (the MBR)
> from the first disk, and starts the code found there. What happens
> afterwards is up to that code. If that code uses CHS units to find
> a partition, and if the program that wrote the table has different
> ideas about those units than the BIOS, booting may fail.
Exactly.
Moreover, booting can fail while reading the file system. I reverse
engineered several of the Microsoft boot loaders (Windows
95/9x/ME/2000/NT). The boot sector understands FAT, and depending
on the configuration, may use CHS to load in info.
I wrote about this in the doc/FAT file in the Parted tarball.
Cheers,
Andrew
On Sat, Nov 29, 2003 at 07:16:31AM +0200, Szakacsits Szabolcs wrote:
> > Yes there is. "Correct" is defined by the BIOS. It is important
> > for boot loaders (in particular Windows).
>
> I suspected the same ... What Windows you mean? DOS (9x/ME/etc) or NT based
> (NT4/2000/XP/2003)? All?
Good question. From 98 up, Windows supports both LBA and CHS. I'm not
sure about XP/2003. The real question is: what is the default install?
How many users have each?
> Also, can Parted save/restore the full and exact partition table a
> scriptable way? I mean something like this:
>
> sfdisk -d /dev/hda > hda.pt # save
> sfdisk /dev/hda < hda.pt # restore
>
> sfdisk can't recover geometry so apparently no one-liner, widely available,
> partition table backup/recovery is possible at present on Linux :-o
> dd if=/dev/hda of=hda.mbr bs=512 count=1 won't save the logical partitions.
Parted can't do it. :/
Cheers,
Andrew
On Sat, Nov 29, 2003 at 01:50:00PM +0000, John Bradford wrote:
> Why don't we take the opporunity to make all CHS code configurable out
> of the kernel, and define a new, more compact, partition table format
> which used LBA exclusively, and allowed more than four partitions in
> the main partition table?
Intel's EFI GPT partition table format seems quite acceptable.
Cheers,
Andrew
On Sat, Nov 29, 2003 at 11:14:24PM +0100, Andries Brouwer wrote:
> On Sat, Nov 29, 2003 at 06:01:03PM +0100, Sven Luther wrote:
>
> > The only problem is that your BIOS will probably not be able
> > to boot from it
>
> You seem to misunderstand the boot sequence.
> The BIOS does not generally do any parsing of partition tables.
A, ok. I am more familiar with open firmware, which does contain
partition table reading code.
That said, i really doubt that a standard BIOS can boot from a not MBR
containing disk, but i may be wrong.
Friendly,
Sven Luther
On Sun, Nov 30, 2003 at 09:27:22AM +1100, Andrew Clausen wrote:
> > > Some users, having problems, did mention the usage of 2.6 kernel. If the
> > > geometry changed during the fdisk, etc process then it could result also
> > > booting problem?
> >
> > Let me continue to stress: geometry does not exist.
> > Consequently, it cannot change.
>
> Let me continue to stress: geometry DOES exist.
Ha, Andrew - you know these things, I know these things - please
do not confuse matters.
My first letter had as essential content: the Linux 2.6 kernel
does not make geometry information available to user space.
Thus, if user space asks the kernel and prints an error message
in case the answer is unexpected, then such user space is broken
under 2.6. There is nothing to gain from asking the kernel.
That was a letter for you - parted should be fixed, otherwise
there will be a long sequence of users that worry that things
might be wrong.
My second letter was for Szaka and affirmed that fdisk cannot
change this non-existent geometry. You still believe in fairies,
I mean, in disk geometry, that is OK, I don't mind, but still,
whatever it is you believe in, fdisk cannot change it.
> It is an abstract construct that is stored in your BIOS that some
> configurations use and need for booting.
I am happy with that description.
"Disk geometry is: some numbers that your BIOS invents".
Of course, details are always more complicated.
What the BIOS invents may be dependent on user settings in its setup.
There are also numbers that certain operating systems or boot managers
invent. Equal to or different from what your BIOS has thought of.
> (i.e. have you got any evidence that, say, that 99.x% of Windows XP
> installations use LBA to bootstrap?)
Just ask yourself this question: does Windows XP require a bootable
partition to start below the 1024 cylinder mark?
Windows NT4 has such a restriction. Not Windows 2000 or XP.
> > Usually booting goes like this: the BIOS reads sector 0 (the MBR)
> > from the first disk, and starts the code found there. What happens
> > afterwards is up to that code. If that code uses CHS units to find
> > a partition, and if the program that wrote the table has different
> > ideas about those units than the BIOS, booting may fail.
>
> Exactly.
Good. We agree.
Andries
On Sat, Nov 29, 2003 at 11:44:11PM +0100, Sven Luther wrote:
> That said, i really doubt that a standard BIOS can boot from a not MBR
> containing disk, but i may be wrong.
The BIOS reads the MBR and jumps to the code loaded from there.
There is no need for any partition table, or, if there is a table,
for any particular format. It is all up to the code that is found
in the MBR.
> The BIOS reads the MBR and jumps to the code loaded from there.
> There is no need for any partition table, or, if there is a table,
> for any particular format. It is all up to the code that is found
> in the MBR.
I found some PC BIOS-es refuse to read the MBR if no active partition is
found in the partition table...
--
=======================================================================
Andrzej M. Krzysztofowicz [email protected]
phone (48)(58) 347 14 61
Faculty of Applied Phys. & Math., Gdansk University of Technology
Andrzej Krzysztofowicz replied to someone:
> > The BIOS reads the MBR and jumps to the code loaded from there.
>
> I found some PC BIOS-es refuse to read the MBR if no active partition is
> found in the partition table...
They read the MBR but refuse the execute the code contained in it. Reading
the MBR is the only way that they get to find out that the partition table
includes zero or two (or more) active partitions and decide not to boot.
SuSE 8.1 had this problem. If you installed grub to a /boot partition but
intended to continue using your existing active partition, the installer
activated the /boot partition. On the next boot, the BIOS detected two
active partitions and refused to boot from the hard disk.
Booting a floppy still works on most machines. The BIOS still reads the MBR
and presents the partition table in a block of information that is visible
to the program that gets booted from the floppy disk. In my experience, the
only machines that refused to do this (becoming 100% unbootable) were the
old NEC 98 architecture.
Andries Brouwer replied to Andrew Clausen:
> I am happy with that description.
> "Disk geometry is: some numbers that your BIOS invents".
I'm happy with that too. Now, since the Linux kernel has no fantasies about
disk geometry, it is fine to refuse to provide such non-existent fantasies
to user space. However, it remains necessary to provide the BIOS's
fantasies to user space. Sometimes user space does something (via the
kernel) that will later be interpreted by the BIOS. User space has to be
able to do it in the manner that the BIOS wants.
> > (i.e. have you got any evidence that, say, that 99.x% of Windows XP
> > installations use LBA to bootstrap?)
>
> Just ask yourself this question: does Windows XP require a bootable
> partition to start below the 1024 cylinder mark?
> Windows NT4 has such a restriction. Not Windows 2000 or XP.
The answer is still yes. Not on sufficiently modern BIOSes but yes in the
way that the booter uses the BIOS to load the kernel. I think that a
computer dating from 1998 is not terribly old to expect Linux to run,
especially when Linux does run and Windows XP does run. I have to keep the
following partitions under the 8 GB mark:
C: (NTLDR, NTDETECT.COM, BOOT.INI, BOOTSECT.LNX, etc.)
D: (WINNT\whatever the names are for kernel, drivers, etc.)
/boot (grub files and vmlinuz-whatever versions)
Windows NT4 SP4 partly overcame the 8GB mark, but of course SP4 was so badly
broken that it is fortunate that the relevant ATAPI.SYS file is downloadable
separately. This ATAPI.SYS (when renamed to C:\NTBOOTDD.SYS) can load a
kernel for NT4, 2000, or XP even past the 8GB mark, but this file itself and
NTLDR and BOOT.INI etc. must remain below the 8GB mark. To do this you have
to keep all of C: below the 8GB mark. If you install Windows 2000 or XP on
such a machine then you have to take a separate download of this specific
version of ATAPI.SYS again, and you have to manually hack BOOT.INI partway
through the installation sequence. If you don't do things right then
Windows 2000 or XP becomes unbootable, sometimes during the installation
process (if lucky), sometimes years after the install (when a file needed
during booting gets updated and moved).
Anyway, regardless of which OS you're running, the OS isn't running until
it's running. The MBR depends on BIOS functions (e.g. the infamous INT13)
to read in the boot loader and the boot loader depends on BIOS functions to
read in the kernel. Yes Dr. Brouwer, I know you know this. The question is
why you think that commands such as parted don't have to know this?
On Sat, 2003-11-29 at 23:31, Andrew Clausen wrote:
> On Sat, Nov 29, 2003 at 01:50:00PM +0000, John Bradford wrote:
> > Why don't we take the opporunity to make all CHS code configurable out
> > of the kernel, and define a new, more compact, partition table format
> > which used LBA exclusively, and allowed more than four partitions in
> > the main partition table?
>
> Intel's EFI GPT partition table format seems quite acceptable.
EFI GPT has some severe downsides (like requiring the last sector on
disk, which in linux may not be accessible if the total number of
sectors is not a multiple of 2, and making dd of one disk to another
impossible if the second one is bigger)
On Sun, 30 Nov 2003, Arjan van de Ven wrote:
> EFI GPT has some severe downsides (like requiring the last sector on
> disk, which in linux may not be accessible if the total number of
> sectors is not a multiple of 2, and making dd of one disk to another
> impossible if the second one is bigger)
Isn't this Linux's shortcoming? NT could always access odd last sectors.
Actually in the majority of cases it stores the backup of its boot sector
in the last sector of the partition for recovery purpose (outside of NTFS).
Anton made a fix for this years ago. It's PITA (and wasting time)
explaining and working around constantly how Linux can (not) access it.
Is it fixed in 2.6?
Szaka
On Sat, 29 Nov 2003 23:14:24 +0100, Andries Brouwer wrote:
> On Sat, Nov 29, 2003 at 06:01:03PM +0100, Sven Luther wrote:
>
>> The only problem is that your BIOS will probably not be able
>> to boot from it
>
> You seem to misunderstand the boot sequence.
> The BIOS does not generally do any parsing of partition tables.
In theory, the BIOS does not need to look in the partition tables.
However, in practice it does this :(
Many BIOSes look at least at the partition table in the MBR to autodetect
the CHS parameters which were used when partitioning the disk. E.g. if
you partition a disk with LBA turned off in the BIOS (so the CHS
parameters will have 16 heads), then notice the mistake and turn LBA on,
in many cases BIOS will still show 16 heads in the CHS parameters.
However, after cleaning the MBR the same BIOS with the same settings will
report 255 heads.
Who knows what such BIOS will do if it will find something other than a
DOS partition table in the first sector...
Quote from Arjan van de Ven <[email protected]>:
> On Sat, 2003-11-29 at 23:31, Andrew Clausen wrote:
> > On Sat, Nov 29, 2003 at 01:50:00PM +0000, John Bradford wrote:
> > > Why don't we take the opporunity to make all CHS code configurable out
> > > of the kernel, and define a new, more compact, partition table format
> > > which used LBA exclusively, and allowed more than four partitions in
> > > the main partition table?
> >
> > Intel's EFI GPT partition table format seems quite acceptable.
>
> EFI GPT has some severe downsides (like requiring the last sector on
> disk, which in linux may not be accessible if the total number of
> sectors is not a multiple of 2, and making dd of one disk to another
> impossible if the second one is bigger)
EFI GPT is also a far more elaborate scheme than is necessary for a
lot of installations.
My 'requirements' are:
* Good magic
We have seen support for not very widely used partitioning schemes
broken in the past when other schemes are checked for ahead of them.
A simple scheme with well defined magic values reduces this risk.
* Simple
The code for some of the partitioning schemes is full of workarounds
for different implementations. Added complexity, and more variations
increase the likelyhood of bugs.
* All partition information stored in one partition table
Linked lists make re-arranging partitions, and backing up the
partition table more difficult.
* Support for more than 4 partitions
This is a common requirement now.
* Support for partitions > 2TB
This will become a standard requirement in a few years.
* No mention of CHS
CHS is so last year. Modern systems don't even need it to boot. We
could allow all of the CHS-related code to be configured out of the
kernel, which would be useful on some embedded systems.
The Ultrix partition code comes fairly close to what I'm thinking of,
but not close enough :-).
John.
On Sun, 30 Nov 2003, Andrew Clausen wrote:
>
> Good question. From 98 up, Windows supports both LBA and CHS. I'm not
> sure about XP/2003.
I don't think it changed. CHS support is needed for backward compatibility
during boot. This is why it would be important not to screw it, if it's
indeed matter in the partition table. Some reading how NT gets/uses drive
geometry,
http://support.microsoft.com/?kbid=98080
> The real question is: what is the default install? How many users have
> each?
Google Zeitgeist says for september at
http://www.google.com/press/zeitgeist/sep03_pie.gif
XP 38%
98 29%
2000 20%
NT 3%
95 1%
XP is growing 1-2% each month at the expense of Win9x (see
http://www.google.com/press/zeitgeist//{...,jun,jul,aug}03_pie.gif)
The majority of NT based uses NTFS. NTFS has its own $Boot file fixed at
sector 0, that is it's the boot sector. I don't know how much it's
different from the one booting from FAT but I guess not much (except of
understanding NTFS instead of FAT during boot, etc).
Szaka
On Sun, Nov 30, 2003 at 10:40:25AM +0000, John Bradford wrote:
> * All partition information stored in one partition table
>
> Linked lists make re-arranging partitions, and backing up the
> partition table more difficult.
I don't agree here. You just follow the linked list and make the backup,
which is one more reason for having the save/restore mechanism in the
per partition table code, which knows how to read/write the partition
table anyway. Also, mostly the linked list is found in a chunk of the
disk which you can easily backup with dd. The amiga scheme has both
information about the number of sectors which can be used in the linked
list, as well as the last used sector.
Also, with a linked list, you can maintain two or more partition tables
on disk, thus making an on-disk backup easy. When you write a new
partition table, you write it on other sectors than the first one, and
then update the root pointer. You can then later go back to the old
partition table by just restoring the root pointer or something such.
Also, it allow you flexibility with the amount of partitions you can
use, as you could have potentially have any number of partitions you
like (upto 2^30 or such).
Friendly,
Sven Luther
On Sun, Nov 30, 2003 at 04:08:22PM +0900, Norman Diamond wrote:
> Andries Brouwer replied to Andrew Clausen:
>
> > I am happy with that description.
> > "Disk geometry is: some numbers that your BIOS invents".
>
> I'm happy with that too. Now, since the Linux kernel has no fantasies about
> disk geometry, it is fine to refuse to provide such non-existent fantasies
> to user space. However, it remains necessary to provide the BIOS's
> fantasies to user space. Sometimes user space does something (via the
> kernel) that will later be interpreted by the BIOS. User space has to be
> able to do it in the manner that the BIOS wants.
The point is just that the Linux kernel has no idea about these BIOS fantasies.
It may have a more or less elaborate system of guesses, but it has no
knowledge. In practice things work better if the kernel never tries to
tell anything to user space, and user space derives the desired BIOS fantasies
from the partition table.
> Anyway, regardless of which OS you're running, the OS isn't running until
> it's running. The MBR depends on BIOS functions (e.g. the infamous INT13)
> to read in the boot loader and the boot loader depends on BIOS functions to
> read in the kernel. Yes Dr. Brouwer, I know you know this. The question is
> why you think that commands such as parted don't have to know this?
You invent a title for me that I never used.
You invent an opinion for me that I never had.
On Sun, 30 Nov 2003, Andries Brouwer wrote:
> Just ask yourself this question: does Windows XP require a bootable
> partition to start below the 1024 cylinder mark?
> Windows NT4 has such a restriction. Not Windows 2000 or XP.
Wrong:
http://support.microsoft.com/default.aspx?scid=kb;en-us;282191
> > > Usually booting goes like this: the BIOS reads sector 0 (the MBR)
> > > from the first disk, and starts the code found there. What happens
> > > afterwards is up to that code. If that code uses CHS units to find
> > > a partition, and if the program that wrote the table has different
> > > ideas about those units than the BIOS, booting may fail.
> > Exactly.
> Good. We agree.
I'm glad also. So what actually [cs]fdisk do with the CHS entries in the
partition table? Ignore them? Might they convert a given partition start to
different CHS units if the partition entry was deleted then recreated at
the same cylinder?
AFAIS, parted tries hard not to break these [IMHO correctly], right Andrew?
Szaka
On Sun, Nov 30, 2003 at 03:22:52AM +0100, Andrzej Krzysztofowicz wrote:
> > The BIOS reads the MBR and jumps to the code loaded from there.
> > There is no need for any partition table, or, if there is a table,
> > for any particular format. It is all up to the code that is found
> > in the MBR.
>
> I found some PC BIOS-es refuse to read the MBR if no active partition is
> found in the partition table...
Yes. We are getting a bit away from disk geometries, but it is true
that there are many broken BIOSes that in some way depend on partition
table format or MBR format.
I recall the report that one BIOS tuned IDE modes by reading the MBR
and seeing whether it ended with 0xaa55. If not it tried a lower speed.
So on a disk without this MBR signature, the I/O would be slow.
BSD used to use an entirely different partition table scheme.
And it was not uncommon to run a whole-disk BSD system, without
any partitioning.
Increasingly often that caused problems with broken BIOSes
that wanted to interpret partition table contents.
The categories of problems that come to mind are:
- BIOS has a virus detection option and checks the MBR
- BIOS inspects the partition table to find the hibernation partition
- BIOS inspects the partition table to find the service partition
- BIOS inspects the partition table to guess what geometry it should report
I recall that certain Thinkpads would not boot FreeBSD even with a DOS-type
partition table because the BIOS did not like the a5 partition ID.
So, yes, you are right, practice is much more complicated than theory.
Andries
On Sun, Nov 30, 2003 at 01:10:36PM +0200, Szakacsits Szabolcs wrote:
>
> On Sun, 30 Nov 2003, Andries Brouwer wrote:
>
> > Just ask yourself this question: does Windows XP require a bootable
> > partition to start below the 1024 cylinder mark?
> > Windows NT4 has such a restriction. Not Windows 2000 or XP.
>
> Wrong:
> http://support.microsoft.com/default.aspx?scid=kb;en-us;282191
"Wrong" - what a pessimism. That URL just confirms what I wrote:
Windows XP has no such restriction. If you explicitly ask Windows XP
to use oldfashioned means, then of course that is your own choice.
> > > > Usually booting goes like this: the BIOS reads sector 0 (the MBR)
> > > > from the first disk, and starts the code found there. What happens
> > > > afterwards is up to that code. If that code uses CHS units to find
> > > > a partition, and if the program that wrote the table has different
> > > > ideas about those units than the BIOS, booting may fail.
> > > Exactly.
> > Good. We agree.
>
> I'm glad also. So what actually [cs]fdisk do with the CHS entries in the
> partition table? Ignore them? Might they convert a given partition start to
> different CHS units if the partition entry was deleted then recreated at
> the same cylinder?
Ha, now we are getting down to business.
*fdisk evolves in time, so the answer is very version dependent.
Let me answer for today's fdisk.
Disk geometry is determined as follows (see fdisk.c:get_geometry())
heads = user_heads ? user_heads :
pt_heads ? pt_heads :
kern_heads ? kern_heads : 255;
sectors = user_sectors ? user_sectors :
pt_sectors ? pt_sectors :
kern_sectors ? kern_sectors : 63;
that is, if the user has specified a geometry on the command line,
then that is what we use; otherwise, if there is a partition
table already and we are able to guess a geometry from that, use that;
otherwise, if the kernel has some idea, use that; finally use */255/63
when no information is available.
Andries
On Sat, 29 Nov 2003, Andries Brouwer wrote:
> You see, saving the logical partitions is not enough - these sectors
> are spread out over the disk, and there used to be something else
> where these sectors are written.
Yes but fdisk, cfdisk and parted doesn't have this feature either. Those
are what people use most often from the command line for interactive
partitioning (use the right tool for the job).
Saving the table with sfdisk is the best but imperfect workaround. sfdisk
can't know what steps will be done later on with a different partitioning
tool.
But at least its data should be enough to diagnose problems in the other
partitioning tools.
Szaka
Quote from Sven Luther <[email protected]>:
> On Sun, Nov 30, 2003 at 10:40:25AM +0000, John Bradford wrote:
> > * All partition information stored in one partition table
> >
> > Linked lists make re-arranging partitions, and backing up the
> > partition table more difficult.
>
> I don't agree here. You just follow the linked list and make the backup,
> which is one more reason for having the save/restore mechanism in the
> per partition table code, which knows how to read/write the partition
> table anyway. Also, mostly the linked list is found in a chunk of the
> disk which you can easily backup with dd. The amiga scheme has both
> information about the number of sectors which can be used in the linked
> list, as well as the last used sector.
I must admit, I haven't looked at every single partitioning scheme we
support in detail.
Also, my method of working may not be typical, in that I don't
generally partition all of the space on a disk, just because it's
there. This came up on LKML a few months ago, in a discussion about
re-sizing filesystems in which I noted that the common case of wanting
to shrink a partition containing a filesytem with free space on it,
just to allow experimentation with a new filesystem, can be completely
avoided in many cases, simply by partitioning only the space you
expect to need from the begining.
Using the standard x86 partitioning system with large numbers of
extended partitions to do all this is not convenient.
I think that maybe I notice it more than most users because I
generally don't have the hassle of resizing partitions before I do
anything anyway. Working with extended partitions might not seem like
much extra effort, but it is certainly less convenient and more
cumbersome than using primary partitions alone.
> Also, with a linked list, you can maintain two or more partition tables
> on disk, thus making an on-disk backup easy. When you write a new
> partition table, you write it on other sectors than the first one, and
> then update the root pointer. You can then later go back to the old
> partition table by just restoring the root pointer or something such.
I can see that linked-list partition tables have uses, but I think
that their disadvantages outweigh their advantages here - if some
partitioning data is stored at non-fixed locations on the disk,
overwriting a partition can destroy partitioning data for several
other partitions, whereas a dedicated area for partition data isn't
vulnerable to this.
> Also, it allow you flexibility with the amount of partitions you can
> use, as you could have potentially have any number of partitions you
> like (upto 2^30 or such).
Again, I can see that large numbers of partitions might have uses, but
in my experience 4 is a real limitation, whereas 8 or 16 wouldn't be.
Where, say > 128 partitions are needed, linked-lists are probably a
win, but my requirements are for a simple, reliable partitioning
scheme which supports large partitions, and allows us to completely
remove CHS code from the kernel. I don't think we currently support
one.
John.
Quote from Andries Brouwer <[email protected]>:
> On Sun, Nov 30, 2003 at 03:22:52AM +0100, Andrzej Krzysztofowicz wrote:
>
> > > The BIOS reads the MBR and jumps to the code loaded from there.
> > > There is no need for any partition table, or, if there is a table,
> > > for any particular format. It is all up to the code that is found
> > > in the MBR.
> >
> > I found some PC BIOS-es refuse to read the MBR if no active partition is
> > found in the partition table...
>
> Yes. We are getting a bit away from disk geometries, but it is true
> that there are many broken BIOSes that in some way depend on partition
> table format or MBR format.
OK, there is broken hardware, but there are also people with
non-broken hardware who want to make better use of it :-). I am not
recommending that everybody moves away from the standard partition
table format, I just want a better partitioning scheme for new
machines I build, (for which I would avoid using known broken
hardware).
> I recall the report that one BIOS tuned IDE modes by reading the MBR
> and seeing whether it ended with 0xaa55. If not it tried a lower speed.
> So on a disk without this MBR signature, the I/O would be slow.
>
> BSD used to use an entirely different partition table scheme.
> And it was not uncommon to run a whole-disk BSD system, without
> any partitioning.
Hmmm, yes, you can use a BSD disk label on a whole disk, as opposed to
putting a BSD disk label on one partition of a disk. I have never
tried to read such a disk on a Linux machine, though - do we support
that correctly?
> Increasingly often that caused problems with broken BIOSes
> that wanted to interpret partition table contents.
>
> The categories of problems that come to mind are:
> - BIOS has a virus detection option and checks the MBR
> - BIOS inspects the partition table to find the hibernation partition
> - BIOS inspects the partition table to find the service partition
> - BIOS inspects the partition table to guess what geometry it should report
>
> I recall that certain Thinkpads would not boot FreeBSD even with a DOS-type
> partition table because the BIOS did not like the a5 partition ID.
>
> So, yes, you are right, practice is much more complicated than theory.
For building new, dedicated Linux machines, though, how much of that
do we have to concern ourselves with?
John.
On Sun, 30 Nov 2003, Andries Brouwer wrote:
> >
> > Wrong:
> > http://support.microsoft.com/default.aspx?scid=kb;en-us;282191
>
> "Wrong" - what a pessimism.
Optimism ends up unbootable systems.
> Windows XP has no such restriction. If you explicitly ask Windows XP
> to use oldfashioned means, then of course that is your own choice.
It's not like that. People get boxes whatever way they were imaged, etc.
Most don't know anything about the internals. And when they want to make it
dualboot with Linux, they might find they can't boot anymore.
Do you also really believe this oldfashioned means is the only way to end
up in problems because of partitioning tools change CHS units?
> if there is a partition table already and we are able to guess a geometry
> from that, use that; otherwise [...]
OK, thanks, the problem is here. Maybe a warning could be added when the
geometry can't be guessed (if the warning isn't there already)?
Szaka
On Sun, Nov 30, 2003 at 01:44:44PM +0200, Szakacsits Szabolcs wrote:
>
> On Sat, 29 Nov 2003, Andries Brouwer wrote:
>
> > You see, saving the logical partitions is not enough - these sectors
> > are spread out over the disk, and there used to be something else
> > where these sectors are written.
> Saving the table with sfdisk is the best but imperfect workaround. sfdisk
> can't know what steps will be done later on with a different partitioning
> tool.
>
> But at least its data should be enough to diagnose problems in the other
> partitioning tools.
You can save the data using "cfdisk -Pr". Slightly more readable versions
are given by "cfdisk -Pt", "cfdisk -Ps", "sfdisk -d", "sfdisk -x -uS -l".
On Sun, Nov 30, 2003 at 02:34:23PM +0200, Szakacsits Szabolcs wrote:
> > if there is a partition table already and we are able to guess a geometry
> > from that, use that; otherwise [...]
>
> OK, thanks, the problem is here. Maybe a warning could be added
The docs and the programs are full of warnings already.
Too many in fact. People worry and get nervous, needlessly.
In a very small percentage of cases there really are problems,
but again these warnings "there might be problems" are not very helpful.
The number of cylinders for this disk is set to 70780.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Does this help anybody?
On Sun, Nov 30, 2003 at 01:48:56PM +0000, John Bradford wrote:
> Quote from Sven Luther <[email protected]>:
> > On Sun, Nov 30, 2003 at 10:40:25AM +0000, John Bradford wrote:
> > > * All partition information stored in one partition table
> > >
> > > Linked lists make re-arranging partitions, and backing up the
> > > partition table more difficult.
> >
> > I don't agree here. You just follow the linked list and make the backup,
> > which is one more reason for having the save/restore mechanism in the
> > per partition table code, which knows how to read/write the partition
> > table anyway. Also, mostly the linked list is found in a chunk of the
> > disk which you can easily backup with dd. The amiga scheme has both
> > information about the number of sectors which can be used in the linked
> > list, as well as the last used sector.
>
> I must admit, I haven't looked at every single partitioning scheme we
> support in detail.
>
> Also, my method of working may not be typical, in that I don't
> generally partition all of the space on a disk, just because it's
> there. This came up on LKML a few months ago, in a discussion about
> re-sizing filesystems in which I noted that the common case of wanting
> to shrink a partition containing a filesytem with free space on it,
> just to allow experimentation with a new filesystem, can be completely
> avoided in many cases, simply by partitioning only the space you
> expect to need from the begining.
Sure, but you don't always know about this at the begining.
> > Also, with a linked list, you can maintain two or more partition tables
> > on disk, thus making an on-disk backup easy. When you write a new
> > partition table, you write it on other sectors than the first one, and
> > then update the root pointer. You can then later go back to the old
> > partition table by just restoring the root pointer or something such.
>
> I can see that linked-list partition tables have uses, but I think
> that their disadvantages outweigh their advantages here - if some
> partitioning data is stored at non-fixed locations on the disk,
> overwriting a partition can destroy partitioning data for several
> other partitions, whereas a dedicated area for partition data isn't
> vulnerable to this.
Well, i don't know about macos, or other linked partition tables, but in
the amiga case, you just use a chunk of the disk at the begining to
store the partition entries in (each partition correspond to a 512 byte
sector). The RDB (the root of the partition tree) contains information
about what sector are allocated to the partition table, and what can be
used for other stuff. In the libparted case, i just made the partition
table not available to put partitions in it, and there can not be
possible overwriting, either by parted or by kernel access, since both
respect the reserved space. Furthermore the RDB can be found in any of
the 16 first sectors of the disk, and i put it in sector 2 by default,
so as to not loose it when someone stupidly writes an MBR on top of it
(like i did when playing with beta debian-installer and evil autopartkit
and a messed up console). Historicaly there were also other stuff stored
in the linked list on amigaos, like a badblock list, as well as
filesystem drivers.
> > Also, it allow you flexibility with the amount of partitions you can
> > use, as you could have potentially have any number of partitions you
> > like (upto 2^30 or such).
>
> Again, I can see that large numbers of partitions might have uses, but
> in my experience 4 is a real limitation, whereas 8 or 16 wouldn't be.
I routinely have more than 8 partitions in any case. With the large
disks we have today, i guess it would be quite easy to have more than 16
of them too, altough i don't remember having them (i had an hda15
though).
> Where, say > 128 partitions are needed, linked-lists are probably a
> win, but my requirements are for a simple, reliable partitioning
> scheme which supports large partitions, and allows us to completely
> remove CHS code from the kernel. I don't think we currently support
> one.
What has linked lists to do with CHS ? These are fully orthogonal
issues.
And consider the simplest of linked list, the one were the next block is
the block immediately after the current one. You would have a root block
with details about the disk, information about the area reserved and
such, and then a simple 0xffffffff (or whatever) terminated array or
something. You could even keep the linked list scheme for flexibility, but
make it linear. Ideally, you would have two lists (or more) for backup,
and the root block would contain the extent of sectors reserved for this
usage (128Ko on larger disks maybe ?) as well as the highest used block
of this, so you can easily back it up with dd or something.
Alternatively you could have replications of this table at regular
intervals of the disks or something.
Friendly,
Sven Luther
On Sun, Nov 30, 2003 at 09:57:56AM +0100, Arjan van de Ven wrote:
> > Intel's EFI GPT partition table format seems quite acceptable.
>
> EFI GPT has some severe downsides (like requiring the last sector on
> disk, which in linux may not be accessible if the total number of
> sectors is not a multiple of 2,
Yeah, this does suck. That ioctl hack isn't pretty.
> and making dd of one disk to another impossible if the second one is
> bigger)
You just lose some of the fault tolerance, until you rerun a partition
tool (or even boot a kernel) that re-does the end of it, adjusting
for the new disk size.
The whole point of having an extra copy of the table is to be fault
tolerant, not to introduce another point of failure!
Cheers,
Andrew
On Sun, Nov 30, 2003 at 10:40:25AM +0000, John Bradford wrote:
> > EFI GPT has some severe downsides (like requiring the last sector on
> > disk, which in linux may not be accessible if the total number of
> > sectors is not a multiple of 2, and making dd of one disk to another
> > impossible if the second one is bigger)
>
> EFI GPT is also a far more elaborate scheme than is necessary for a
> lot of installations.
Is this a problem?
> My 'requirements' are:
>
> * Good magic
>
> We have seen support for not very widely used partitioning schemes
> broken in the past when other schemes are checked for ahead of them.
> A simple scheme with well defined magic values reduces this risk.
I think magic doesn't belong in partition tables. I like probing.
Having the same data stored in two places makes things hairy
if you don't know how to resolve inconsistencies.
> * Simple
>
> The code for some of the partitioning schemes is full of workarounds
> for different implementations. Added complexity, and more variations
> increase the likelyhood of bugs.
If you're not interested in work-arounds, why not use LVM?
> * All partition information stored in one partition table
>
> Linked lists make re-arranging partitions, and backing up the
> partition table more difficult.
I don't think it's very difficult, but I agree that tables are nice
and simple.
Cheers,
Andrew
){kind=link}