2013-07-30 22:26:31

by Eric Sandeen

[permalink] [raw]
Subject: [PATCH] ext3: allow specifying external journal by pathname mount option

It's always been a hassle that if an external journal's
device number changes, the filesystem won't mount.
And since boot-time enumeration can change, device number
changes aren't unusual.

The current mechanism to update the journal location is by
passing in a mount option w/ a new devnum, but that's a hassle;
it's a manual approach, fixing things after the fact.

Adding a mount option, "-o journal_path=/dev/$DEVICE" would
help, since then we can do i.e.

# mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL /mnt

and it'll mount even if the devnum has changed, as shown here:

# losetup /dev/loop0 journalfile
# mke2fs -L mylabel-journal -O journal_dev /dev/loop0
# mkfs.ext3 -L mylabel -J device=/dev/loop0 /dev/sdb1

Change the journal device number:

# losetup -d /dev/loop0
# losetup /dev/loop1 journalfile

And today it will fail:

# mount /dev/sdb1 /mnt/test
mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

# dmesg | tail -n 1
[17343.240702] EXT3-fs (sdb1): error: couldn't read superblock of external journal

But with this new mount option, we can specify the new path:

# mount -o journal_path=/dev/loop1 /dev/sdb1 /mnt/test
#

(which does update the encoded device number, incidentally):

# umount /dev/sdb1
# dumpe2fs -h /dev/sdb1 | grep "Journal device"
dumpe2fs 1.41.12 (17-May-2010)
Journal device: 0x0701

But best of all we can just always mount by journal-path, and
it'll always work:

# mount -o journal_path=/dev/disk/by-label/mylabel-journal /dev/sdb1 /mnt/test
#

So the journal_path option can be specified in fstab, and as long as
the disk is available somewhere, and findable by label (or by UUID),
we can mount.

Signed-off-by: Eric Sandeen <[email protected]>
---

The patch is a little hacky, doing all the work in option parsing,
just to get to a journal devnum like the old option expected, only
to later re-decode it when we really want to open it.

I could clean it up so that both journal-update mount options
find the bdev, rather than ending with an encoded device number,
which must then be decoded & re-opened, if that seems better.

But this was expedient enough to get the idea out on the list.

If we like it, I'll do ext4 as well.

Thanks,
-Eric


diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
index 293855e..7ed0d17 100644
--- a/Documentation/filesystems/ext3.txt
+++ b/Documentation/filesystems/ext3.txt
@@ -26,11 +26,12 @@ journal=inum When a journal already exists, this option is ignored.
Otherwise, it specifies the number of the inode which
will represent the ext3 file system's journal file.

+journal_path=path
journal_dev=devnum When the external journal device's major/minor numbers
- have changed, this option allows the user to specify
+ have changed, these options allow the user to specify
the new journal location. The journal device is
- identified through its new major/minor numbers encoded
- in devnum.
+ identified through either its new major/minor numbers
+ encoded in devnum, or via a path to the device.

norecovery Don't load the journal on mounting. Note that this forces
noload mount of inconsistent filesystem, which can lead to
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index c47f147..cb1dd6e 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -819,6 +819,7 @@ enum {
Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh,
Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev,
+ Opt_journal_path,
Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
Opt_data_err_abort, Opt_data_err_ignore,
Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
@@ -860,6 +861,7 @@ static const match_table_t tokens = {
{Opt_journal_update, "journal=update"},
{Opt_journal_inum, "journal=%u"},
{Opt_journal_dev, "journal_dev=%u"},
+ {Opt_journal_path, "journal_path=%s"},
{Opt_abort, "abort"},
{Opt_data_journal, "data=journal"},
{Opt_data_ordered, "data=ordered"},
@@ -975,6 +977,9 @@ static int parse_options (char *options, struct super_block *sb,
int option;
kuid_t uid;
kgid_t gid;
+ struct block_device *journal_bdev;
+ char *journal_path;
+
#ifdef CONFIG_QUOTA
int qfmt;
#endif
@@ -1129,6 +1134,31 @@ static int parse_options (char *options, struct super_block *sb,
return 0;
*journal_devnum = option;
break;
+ case Opt_journal_path:
+ if (is_remount) {
+ ext3_msg(sb, KERN_ERR, "error: cannot specify "
+ "journal on remount");
+ return 0;
+ }
+ journal_path = match_strdup(&args[0]);
+ if (!journal_path) {
+ ext3_msg(sb, KERN_ERR, "error: could not dup "
+ "journal device string");
+ return 0;
+ }
+ journal_bdev = lookup_bdev(journal_path);
+ if (IS_ERR(journal_bdev)) {
+ ext3_msg(sb, KERN_ERR, "error: could not look "
+ "up journal device: error %d",
+ (int)PTR_ERR(journal_bdev));
+ kfree(journal_path);
+ return 0;
+ }
+
+ *journal_devnum = new_encode_dev(journal_bdev->bd_dev);
+ bdput(journal_bdev);
+ kfree(journal_path);
+ break;
case Opt_noload:
set_opt (sbi->s_mount_opt, NOLOAD);
break;



2013-07-31 14:05:47

by Jan Kara

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option

On Tue 30-07-13 17:26:24, Eric Sandeen wrote:
> It's always been a hassle that if an external journal's
> device number changes, the filesystem won't mount.
> And since boot-time enumeration can change, device number
> changes aren't unusual.
>
> The current mechanism to update the journal location is by
> passing in a mount option w/ a new devnum, but that's a hassle;
> it's a manual approach, fixing things after the fact.
>
> Adding a mount option, "-o journal_path=/dev/$DEVICE" would
> help, since then we can do i.e.
>
> # mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL /mnt
>
> and it'll mount even if the devnum has changed, as shown here:
>
> # losetup /dev/loop0 journalfile
> # mke2fs -L mylabel-journal -O journal_dev /dev/loop0
> # mkfs.ext3 -L mylabel -J device=/dev/loop0 /dev/sdb1
>
> Change the journal device number:
>
> # losetup -d /dev/loop0
> # losetup /dev/loop1 journalfile
>
> And today it will fail:
>
> # mount /dev/sdb1 /mnt/test
> mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
> missing codepage or helper program, or other error
> In some cases useful info is found in syslog - try
> dmesg | tail or so
>
> # dmesg | tail -n 1
> [17343.240702] EXT3-fs (sdb1): error: couldn't read superblock of external journal
>
> But with this new mount option, we can specify the new path:
>
> # mount -o journal_path=/dev/loop1 /dev/sdb1 /mnt/test
> #
>
> (which does update the encoded device number, incidentally):
>
> # umount /dev/sdb1
> # dumpe2fs -h /dev/sdb1 | grep "Journal device"
> dumpe2fs 1.41.12 (17-May-2010)
> Journal device: 0x0701
>
> But best of all we can just always mount by journal-path, and
> it'll always work:
>
> # mount -o journal_path=/dev/disk/by-label/mylabel-journal /dev/sdb1 /mnt/test
> #
>
> So the journal_path option can be specified in fstab, and as long as
> the disk is available somewhere, and findable by label (or by UUID),
> we can mount.
>
> Signed-off-by: Eric Sandeen <[email protected]>
> ---
>
> The patch is a little hacky, doing all the work in option parsing,
> just to get to a journal devnum like the old option expected, only
> to later re-decode it when we really want to open it.
>
> I could clean it up so that both journal-update mount options
> find the bdev, rather than ending with an encoded device number,
> which must then be decoded & re-opened, if that seems better.
>
> But this was expedient enough to get the idea out on the list.
>
> If we like it, I'll do ext4 as well.
Yeah, it looks like a good idea. You could just lookup the path via
kern_path() and then take the device numbers from the inode so you won't
have to do the decode-recode dance. That would look like the cleanest option
to me.

Honza

> diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
> index 293855e..7ed0d17 100644
> --- a/Documentation/filesystems/ext3.txt
> +++ b/Documentation/filesystems/ext3.txt
> @@ -26,11 +26,12 @@ journal=inum When a journal already exists, this option is ignored.
> Otherwise, it specifies the number of the inode which
> will represent the ext3 file system's journal file.
>
> +journal_path=path
> journal_dev=devnum When the external journal device's major/minor numbers
> - have changed, this option allows the user to specify
> + have changed, these options allow the user to specify
> the new journal location. The journal device is
> - identified through its new major/minor numbers encoded
> - in devnum.
> + identified through either its new major/minor numbers
> + encoded in devnum, or via a path to the device.
>
> norecovery Don't load the journal on mounting. Note that this forces
> noload mount of inconsistent filesystem, which can lead to
> diff --git a/fs/ext3/super.c b/fs/ext3/super.c
> index c47f147..cb1dd6e 100644
> --- a/fs/ext3/super.c
> +++ b/fs/ext3/super.c
> @@ -819,6 +819,7 @@ enum {
> Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
> Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh,
> Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev,
> + Opt_journal_path,
> Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
> Opt_data_err_abort, Opt_data_err_ignore,
> Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
> @@ -860,6 +861,7 @@ static const match_table_t tokens = {
> {Opt_journal_update, "journal=update"},
> {Opt_journal_inum, "journal=%u"},
> {Opt_journal_dev, "journal_dev=%u"},
> + {Opt_journal_path, "journal_path=%s"},
> {Opt_abort, "abort"},
> {Opt_data_journal, "data=journal"},
> {Opt_data_ordered, "data=ordered"},
> @@ -975,6 +977,9 @@ static int parse_options (char *options, struct super_block *sb,
> int option;
> kuid_t uid;
> kgid_t gid;
> + struct block_device *journal_bdev;
> + char *journal_path;
> +
> #ifdef CONFIG_QUOTA
> int qfmt;
> #endif
> @@ -1129,6 +1134,31 @@ static int parse_options (char *options, struct super_block *sb,
> return 0;
> *journal_devnum = option;
> break;
> + case Opt_journal_path:
> + if (is_remount) {
> + ext3_msg(sb, KERN_ERR, "error: cannot specify "
> + "journal on remount");
> + return 0;
> + }
> + journal_path = match_strdup(&args[0]);
> + if (!journal_path) {
> + ext3_msg(sb, KERN_ERR, "error: could not dup "
> + "journal device string");
> + return 0;
> + }
> + journal_bdev = lookup_bdev(journal_path);
> + if (IS_ERR(journal_bdev)) {
> + ext3_msg(sb, KERN_ERR, "error: could not look "
> + "up journal device: error %d",
> + (int)PTR_ERR(journal_bdev));
> + kfree(journal_path);
> + return 0;
> + }
> +
> + *journal_devnum = new_encode_dev(journal_bdev->bd_dev);
> + bdput(journal_bdev);
> + kfree(journal_path);
> + break;
> case Opt_noload:
> set_opt (sbi->s_mount_opt, NOLOAD);
> break;
>
--
Jan Kara <[email protected]>
SUSE Labs, CR

2013-07-31 14:16:08

by Eric Sandeen

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option

On 7/31/13 9:05 AM, Jan Kara wrote:
> On Tue 30-07-13 17:26:24, Eric Sandeen wrote:
>> It's always been a hassle that if an external journal's
>> device number changes, the filesystem won't mount.
>> And since boot-time enumeration can change, device number
>> changes aren't unusual.
>>
>> The current mechanism to update the journal location is by
>> passing in a mount option w/ a new devnum, but that's a hassle;
>> it's a manual approach, fixing things after the fact.
>>
>> Adding a mount option, "-o journal_path=/dev/$DEVICE" would
>> help, since then we can do i.e.
>>
>> # mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL /mnt
>>
>> and it'll mount even if the devnum has changed, as shown here:
>>
>> # losetup /dev/loop0 journalfile
>> # mke2fs -L mylabel-journal -O journal_dev /dev/loop0
>> # mkfs.ext3 -L mylabel -J device=/dev/loop0 /dev/sdb1
>>
>> Change the journal device number:
>>
>> # losetup -d /dev/loop0
>> # losetup /dev/loop1 journalfile
>>
>> And today it will fail:
>>
>> # mount /dev/sdb1 /mnt/test
>> mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
>> missing codepage or helper program, or other error
>> In some cases useful info is found in syslog - try
>> dmesg | tail or so
>>
>> # dmesg | tail -n 1
>> [17343.240702] EXT3-fs (sdb1): error: couldn't read superblock of external journal
>>
>> But with this new mount option, we can specify the new path:
>>
>> # mount -o journal_path=/dev/loop1 /dev/sdb1 /mnt/test
>> #
>>
>> (which does update the encoded device number, incidentally):
>>
>> # umount /dev/sdb1
>> # dumpe2fs -h /dev/sdb1 | grep "Journal device"
>> dumpe2fs 1.41.12 (17-May-2010)
>> Journal device: 0x0701
>>
>> But best of all we can just always mount by journal-path, and
>> it'll always work:
>>
>> # mount -o journal_path=/dev/disk/by-label/mylabel-journal /dev/sdb1 /mnt/test
>> #
>>
>> So the journal_path option can be specified in fstab, and as long as
>> the disk is available somewhere, and findable by label (or by UUID),
>> we can mount.
>>
>> Signed-off-by: Eric Sandeen <[email protected]>
>> ---
>>
>> The patch is a little hacky, doing all the work in option parsing,
>> just to get to a journal devnum like the old option expected, only
>> to later re-decode it when we really want to open it.
>>
>> I could clean it up so that both journal-update mount options
>> find the bdev, rather than ending with an encoded device number,
>> which must then be decoded & re-opened, if that seems better.
>>
>> But this was expedient enough to get the idea out on the list.
>>
>> If we like it, I'll do ext4 as well.
> Yeah, it looks like a good idea. You could just lookup the path via
> kern_path() and then take the device numbers from the inode so you won't
> have to do the decode-recode dance. That would look like the cleanest option
> to me.
>
> Honza

Ah, thanks for the suggestion, I'll give that a shot.

-Eric

2013-07-31 17:39:23

by Bill McGonigle

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option


> Adding a mount option, "-o journal_path=/dev/$DEVICE" would
> help, since then we can do i.e.
>
> # mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL /mnt

I came here with a related problem, and I wonder if it would be a more
general solution to this one too.

I've got a machine with root on ext4 on LUKS on md, and while I was
first planning to use flashcache or dm-cache underneath it (for SSD
acceleration) I saw benchmarks that convinced me that an external
journal on ext4 was the better option.

It's getting it mounted at boot time that's the trick.

I saw that tune2fs supports '-J device=UUID=foo-bar-baz', so I setup a
LUKS volume on the SSD, formatted it, passed it in, and it sets up the
journal fine when I'm booted from external media, but when I reboot
under the OS (3.10 in Fedora 19 in this case) it fails to mount because
the device number has changed.

I see the proper UUID listed in 'tune2fs -l' - the "Journal UUID" is
right, but "Journal device" is no longer correct, so boot fails until I
remove the journal again. I think with LUKS, I'm never guaranteed to
get a consistent device ID between boots, so the kernel command line
options don't help either.

So, I was thinking, that if the ext code did:
1) try stored journal device ID
2) on fail, look up the UUID via libblkid
3) perhaps update the stored device ID

it would solve my problem. I wonder if it would be a more robust
solution to the problem posed here as well (having the filesystem
contain its own references is better IMHO). My one handy system that
has an ext3 volume (still on Fedora 16, e2fsprogs-1.41) does not show a
Journal UUID flag via 'tune2fs -l', but I'm unaware of whether the flag
is unavailable/missing/unsupported there or if it could be added.

-Bill

--
Bill McGonigle, Owner
BFC Computing, LLC
http://bfccomputing.com/
Telephone: +1.855.SW.LIBRE
Email, IM, VOIP: [email protected]
VCard: http://bfccomputing.com/vcard/bill.vcf
Social networks: bill_mcgonigle/bill.mcgonigle

2013-07-31 19:33:08

by Eric Sandeen

[permalink] [raw]
Subject: [PATCH V2] ext3: allow specifying external journal by pathname mount option

It's always been a hassle that if an external journal's
device number changes, the filesystem won't mount.
And since boot-time enumeration can change, device number
changes aren't unusual.

The current mechanism to update the journal location is by
passing in a mount option w/ a new devnum, but that's a hassle;
it's a manual approach, fixing things after the fact.

Adding a mount option, "-o journal_path=/dev/$DEVICE" would
help, since then we can do i.e.

# mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL ...

and it'll mount even if the devnum has changed, as shown here:

# losetup /dev/loop0 journalfile
# mke2fs -L mylabel-journal -O journal_dev /dev/loop0
# mkfs.ext3 -L mylabel -J device=/dev/loop0 /dev/sdb1

Change the journal device number:

# losetup -d /dev/loop0
# losetup /dev/loop1 journalfile

And today it will fail:

# mount /dev/sdb1 /mnt/test
mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

# dmesg | tail -n 1
[17343.240702] EXT3-fs (sdb1): error: couldn't read superblock of external journal

But with this new mount option, we can specify the new path:

# mount -o journal_path=/dev/loop1 /dev/sdb1 /mnt/test
#

(which does update the encoded device number, incidentally):

# umount /dev/sdb1
# dumpe2fs -h /dev/sdb1 | grep "Journal device"
dumpe2fs 1.41.12 (17-May-2010)
Journal device: 0x0701

But best of all we can just always mount by journal-path, and
it'll always work:

# mount -o journal_path=/dev/disk/by-label/mylabel-journal /dev/sdb1 /mnt/test
#

So the journal_path option can be specified in fstab, and as long as
the disk is available somewhere, and findable by label (or by UUID),
we can mount.

Signed-off-by: Eric Sandeen <[email protected]>
---

V2: get device out of bdev's inode, rather than grabbing bdev & releasing it

diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
index 293855e..7ed0d17 100644
--- a/Documentation/filesystems/ext3.txt
+++ b/Documentation/filesystems/ext3.txt
@@ -26,11 +26,12 @@ journal=inum When a journal already exists, this option is ignored.
Otherwise, it specifies the number of the inode which
will represent the ext3 file system's journal file.

+journal_path=path
journal_dev=devnum When the external journal device's major/minor numbers
- have changed, this option allows the user to specify
+ have changed, these options allow the user to specify
the new journal location. The journal device is
- identified through its new major/minor numbers encoded
- in devnum.
+ identified through either its new major/minor numbers
+ encoded in devnum, or via a path to the device.

norecovery Don't load the journal on mounting. Note that this forces
noload mount of inconsistent filesystem, which can lead to
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index c47f147..c50c761 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -27,6 +27,7 @@
#include <linux/seq_file.h>
#include <linux/log2.h>
#include <linux/cleancache.h>
+#include <linux/namei.h>

#include <asm/uaccess.h>

@@ -819,6 +820,7 @@ enum {
Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh,
Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev,
+ Opt_journal_path,
Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
Opt_data_err_abort, Opt_data_err_ignore,
Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
@@ -860,6 +862,7 @@ static const match_table_t tokens = {
{Opt_journal_update, "journal=update"},
{Opt_journal_inum, "journal=%u"},
{Opt_journal_dev, "journal_dev=%u"},
+ {Opt_journal_path, "journal_path=%s"},
{Opt_abort, "abort"},
{Opt_data_journal, "data=journal"},
{Opt_data_ordered, "data=ordered"},
@@ -975,6 +978,11 @@ static int parse_options (char *options, struct super_block *sb,
int option;
kuid_t uid;
kgid_t gid;
+ char *journal_path;
+ struct inode *journal_inode;
+ struct path path;
+ int error;
+
#ifdef CONFIG_QUOTA
int qfmt;
#endif
@@ -1129,6 +1137,41 @@ static int parse_options (char *options, struct super_block *sb,
return 0;
*journal_devnum = option;
break;
+ case Opt_journal_path:
+ if (is_remount) {
+ ext3_msg(sb, KERN_ERR, "error: cannot specify "
+ "journal on remount");
+ return 0;
+ }
+
+ journal_path = match_strdup(&args[0]);
+ if (!journal_path) {
+ ext3_msg(sb, KERN_ERR, "error: could not dup "
+ "journal device string");
+ return 0;
+ }
+
+ error = kern_path(journal_path, LOOKUP_FOLLOW, &path);
+ if (error) {
+ ext3_msg(sb, KERN_ERR, "error: could not find "
+ "journal device path: error %d", error);
+ kfree(journal_path);
+ return 0;
+ }
+
+ journal_inode = path.dentry->d_inode;
+ if (!S_ISBLK(journal_inode->i_mode)) {
+ ext3_msg(sb, KERN_ERR, "error: journal path %s "
+ "is not a block device", journal_path);
+ path_put(&path);
+ kfree(journal_path);
+ return 0;
+ }
+
+ *journal_devnum = new_encode_dev(journal_inode->i_rdev);
+ path_put(&path);
+ kfree(journal_path);
+ break;
case Opt_noload:
set_opt (sbi->s_mount_opt, NOLOAD);
break;



2013-07-31 19:45:44

by Eric Sandeen

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option

On 7/31/13 12:28 PM, Bill McGonigle wrote:
>
>> Adding a mount option, "-o journal_path=/dev/$DEVICE" would help,
>> since then we can do i.e.
>>
>> # mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL /mnt
>
> I came here with a related problem, and I wonder if it would be a
> more general solution to this one too.
>
> I've got a machine with root on ext4 on LUKS on md, and while I was
> first planning to use flashcache or dm-cache underneath it (for SSD
> acceleration) I saw benchmarks that convinced me that an external
> journal on ext4 was the better option.
>
> It's getting it mounted at boot time that's the trick.
>
> I saw that tune2fs supports '-J device=UUID=foo-bar-baz', so I setup
> a LUKS volume on the SSD, formatted it, passed it in, and it sets up
> the journal fine when I'm booted from external media, but when I
> reboot under the OS (3.10 in Fedora 19 in this case) it fails to
> mount because the device number has changed.
>
> I see the proper UUID listed in 'tune2fs -l' - the "Journal UUID" is
> right, but "Journal device" is no longer correct, so boot fails until
> I remove the journal again.

Neat eh? ;)

> I think with LUKS, I'm never guaranteed
> to get a consistent device ID between boots, so the kernel command
> line options don't help either.
>
> So, I was thinking, that if the ext code did: 1) try stored journal
> device ID 2) on fail, look up the UUID via libblkid 3) perhaps update
> the stored device ID

well, kernelspace can't use libblkid.

> it would solve my problem. I wonder if it would be a more robust
> solution to the problem posed here as well (having the filesystem
> contain its own references is better IMHO). My one handy system that
> has an ext3 volume (still on Fedora 16, e2fsprogs-1.41) does not show
> a Journal UUID flag via 'tune2fs -l', but I'm unaware of whether the
> flag is unavailable/missing/unsupported there or if it could be
> added.

I think my patch will solve your problems as it is, just add the new
mount option to fstab, remake your init$WHATEVER, and off you'll go.

I'll send the ext4 version in a minute if you want to try it.

-Eric

> -Bill
>


2013-07-31 20:10:39

by Jan Kara

[permalink] [raw]
Subject: Re: [PATCH V2] ext3: allow specifying external journal by pathname mount option

On Wed 31-07-13 14:33:00, Eric Sandeen wrote:
> It's always been a hassle that if an external journal's
> device number changes, the filesystem won't mount.
> And since boot-time enumeration can change, device number
> changes aren't unusual.
>
> The current mechanism to update the journal location is by
> passing in a mount option w/ a new devnum, but that's a hassle;
> it's a manual approach, fixing things after the fact.
>
> Adding a mount option, "-o journal_path=/dev/$DEVICE" would
> help, since then we can do i.e.
>
> # mount -o journal_path=/dev/disk/by-label/$JOURNAL_LABEL ...
>
> and it'll mount even if the devnum has changed, as shown here:
>
> # losetup /dev/loop0 journalfile
> # mke2fs -L mylabel-journal -O journal_dev /dev/loop0
> # mkfs.ext3 -L mylabel -J device=/dev/loop0 /dev/sdb1
>
> Change the journal device number:
>
> # losetup -d /dev/loop0
> # losetup /dev/loop1 journalfile
>
> And today it will fail:
>
> # mount /dev/sdb1 /mnt/test
> mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
> missing codepage or helper program, or other error
> In some cases useful info is found in syslog - try
> dmesg | tail or so
>
> # dmesg | tail -n 1
> [17343.240702] EXT3-fs (sdb1): error: couldn't read superblock of external journal
>
> But with this new mount option, we can specify the new path:
>
> # mount -o journal_path=/dev/loop1 /dev/sdb1 /mnt/test
> #
>
> (which does update the encoded device number, incidentally):
>
> # umount /dev/sdb1
> # dumpe2fs -h /dev/sdb1 | grep "Journal device"
> dumpe2fs 1.41.12 (17-May-2010)
> Journal device: 0x0701
>
> But best of all we can just always mount by journal-path, and
> it'll always work:
>
> # mount -o journal_path=/dev/disk/by-label/mylabel-journal /dev/sdb1 /mnt/test
> #
>
> So the journal_path option can be specified in fstab, and as long as
> the disk is available somewhere, and findable by label (or by UUID),
> we can mount.
>
> Signed-off-by: Eric Sandeen <[email protected]>
Thanks! I've merged the patch to my tree.

Honza

> ---
>
> V2: get device out of bdev's inode, rather than grabbing bdev & releasing it
>
> diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
> index 293855e..7ed0d17 100644
> --- a/Documentation/filesystems/ext3.txt
> +++ b/Documentation/filesystems/ext3.txt
> @@ -26,11 +26,12 @@ journal=inum When a journal already exists, this option is ignored.
> Otherwise, it specifies the number of the inode which
> will represent the ext3 file system's journal file.
>
> +journal_path=path
> journal_dev=devnum When the external journal device's major/minor numbers
> - have changed, this option allows the user to specify
> + have changed, these options allow the user to specify
> the new journal location. The journal device is
> - identified through its new major/minor numbers encoded
> - in devnum.
> + identified through either its new major/minor numbers
> + encoded in devnum, or via a path to the device.
>
> norecovery Don't load the journal on mounting. Note that this forces
> noload mount of inconsistent filesystem, which can lead to
> diff --git a/fs/ext3/super.c b/fs/ext3/super.c
> index c47f147..c50c761 100644
> --- a/fs/ext3/super.c
> +++ b/fs/ext3/super.c
> @@ -27,6 +27,7 @@
> #include <linux/seq_file.h>
> #include <linux/log2.h>
> #include <linux/cleancache.h>
> +#include <linux/namei.h>
>
> #include <asm/uaccess.h>
>
> @@ -819,6 +820,7 @@ enum {
> Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
> Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh,
> Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev,
> + Opt_journal_path,
> Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
> Opt_data_err_abort, Opt_data_err_ignore,
> Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
> @@ -860,6 +862,7 @@ static const match_table_t tokens = {
> {Opt_journal_update, "journal=update"},
> {Opt_journal_inum, "journal=%u"},
> {Opt_journal_dev, "journal_dev=%u"},
> + {Opt_journal_path, "journal_path=%s"},
> {Opt_abort, "abort"},
> {Opt_data_journal, "data=journal"},
> {Opt_data_ordered, "data=ordered"},
> @@ -975,6 +978,11 @@ static int parse_options (char *options, struct super_block *sb,
> int option;
> kuid_t uid;
> kgid_t gid;
> + char *journal_path;
> + struct inode *journal_inode;
> + struct path path;
> + int error;
> +
> #ifdef CONFIG_QUOTA
> int qfmt;
> #endif
> @@ -1129,6 +1137,41 @@ static int parse_options (char *options, struct super_block *sb,
> return 0;
> *journal_devnum = option;
> break;
> + case Opt_journal_path:
> + if (is_remount) {
> + ext3_msg(sb, KERN_ERR, "error: cannot specify "
> + "journal on remount");
> + return 0;
> + }
> +
> + journal_path = match_strdup(&args[0]);
> + if (!journal_path) {
> + ext3_msg(sb, KERN_ERR, "error: could not dup "
> + "journal device string");
> + return 0;
> + }
> +
> + error = kern_path(journal_path, LOOKUP_FOLLOW, &path);
> + if (error) {
> + ext3_msg(sb, KERN_ERR, "error: could not find "
> + "journal device path: error %d", error);
> + kfree(journal_path);
> + return 0;
> + }
> +
> + journal_inode = path.dentry->d_inode;
> + if (!S_ISBLK(journal_inode->i_mode)) {
> + ext3_msg(sb, KERN_ERR, "error: journal path %s "
> + "is not a block device", journal_path);
> + path_put(&path);
> + kfree(journal_path);
> + return 0;
> + }
> +
> + *journal_devnum = new_encode_dev(journal_inode->i_rdev);
> + path_put(&path);
> + kfree(journal_path);
> + break;
> case Opt_noload:
> set_opt (sbi->s_mount_opt, NOLOAD);
> break;
>
>
--
Jan Kara <[email protected]>
SUSE Labs, CR

2013-08-01 06:51:32

by Bill McGonigle

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option

On 07/31/2013 03:45 PM, Eric Sandeen wrote:
> well, kernelspace can't use libblkid.

ah, right. But userspace could, and with your patch, basically do the
same thing and mount using the UUID number stored in the superblock
using the 'by-uuid' path. Sweet.

> I'll send the ext4 version in a minute if you want to try it.

that would be super - I'd be happy to test it soon.

-Bill

--
Bill McGonigle, Owner
BFC Computing, LLC
http://bfccomputing.com/
Telephone: +1.855.SW.LIBRE
Email, IM, VOIP: [email protected]
VCard: http://bfccomputing.com/vcard/bill.vcf
Social networks: bill_mcgonigle/bill.mcgonigle

2013-08-01 17:23:27

by Eric Sandeen

[permalink] [raw]
Subject: Re: [PATCH] ext3: allow specifying external journal by pathname mount option

On 8/1/13 1:51 AM, Bill McGonigle wrote:
> On 07/31/2013 03:45 PM, Eric Sandeen wrote:
>> well, kernelspace can't use libblkid.
>
> ah, right. But userspace could, and with your patch, basically do
> the same thing and mount using the UUID number stored in the
> superblock using the 'by-uuid' path. Sweet.

yeah. Right now mount can't parse i.e. "journal_path=UUID=$UUID" type
options as it can do for the device specification, but if the /dev/disk/by-uuid
paths are set up by the time you mount the fs, that works too.

>> I'll send the ext4 version in a minute if you want to try it.
>
> that would be super - I'd be happy to test it soon.

K, it's on the list now. :)

-Eric

> -Bill
>