2008-10-25 11:22:47

by Vegard Nossum

[permalink] [raw]
Subject: v2.6.28-rc1: Regression in ext3/jbd

[Sorry in advance for the huge Cc, most of it is from the commit.]

Hi,

This commit:

commit 2d7c820e56ce83b23daee9eb5343730fb309418e
Author: Hidehiro Kawai <[email protected]>
Date: Wed Oct 22 14:15:01 2008 -0700

ext3: add checks for errors from jbd

introduces a regression which was discovered by kmemcheck:

WARNING: kmemcheck: Caught 32-bit read from freed memory (f4f1b804)
00b0f1f4fbffffff404439ef008830f20200000097970000ad4eaddeffffffff
i i i i f f f f f f f f f f f f f f f f f f f f f f f f f f f f
^

Pid: 9550, comm: umount Not tainted (2.6.28-rc1 #58) 945P-A
EIP: 0060:[<c05bdf38>] EFLAGS: 00010246 CPU: 0
EIP is at __journal_abort_soft+0x18/0xa0
EAX: f4f1b800 EBX: f4f1b800 ECX: c0462799 EDX: fffffffb
ESI: fffffffb EDI: f4f1a800 EBP: f145dea8 ESP: c25699c8
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
CR0: 8005003b CR2: f6c1d704 CR3: 31448000 CR4: 00000650
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff4ff0 DR7: 00000400
[<c05bdfc8>] journal_abort+0x8/0x10
[<c0589eb5>] ext3_abort+0xb5/0xc0
[<c058a300>] ext3_put_super+0x160/0x230
[<c04ec02a>] generic_shutdown_super+0x5a/0xe0

In particular, this hunk is guilty:

- journal_destroy(sbi->s_journal);
+ if (journal_destroy(sbi->s_journal) < 0)
+ ext3_abort(sb, __func__, "Couldn't clean up the journal");

because journal_destroy() will free the journal regardless of whether
it returned < 0 or not. And then ext3_abort() makes some calls that
dereference the (freed) journal. These are the line numbers for the
backtrace:

addr2line -e vmlinux -i c05bdf38 c05bdfc8 c0589eb5 c058a300 c04ec02a
fs/jbd/journal.c:1502
fs/jbd/journal.c:1560
fs/ext3/super.c:284
fs/ext3/super.c:397
fs/super.c:307

(as of e013e13bf605b9e6b702adffbe2853cfc60e7806 in Linus's tree).

I hope this helps.


Vegard


2008-10-27 07:26:48

by Hidehiro Kawai

[permalink] [raw]
Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Hi Vegard,

Vegard Nossum wrote:

> commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> Author: Hidehiro Kawai <[email protected]>
> Date: Wed Oct 22 14:15:01 2008 -0700
>
> ext3: add checks for errors from jbd
>
> introduces a regression which was discovered by kmemcheck:

Thank you for reporting this regression. It's my fault.
I send patches to fix.

Regards,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory. When
journal has been aborted, ext3_put_super() calls ext3_abort() after
freeing the journal_t object, and then ext3_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <[email protected]>
---
fs/ext3/super.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
sb->s_flags |= MS_RDONLY;
EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
- journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+ if (EXT3_SB(sb)->s_journal)
+ journal_abort(EXT3_SB(sb)->s_journal, -EIO);
}

void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
{
struct ext3_sb_info *sbi = EXT3_SB(sb);
struct ext3_super_block *es = sbi->s_es;
- int i;
+ int i, err;

ext3_xattr_put_super(sb);
- if (journal_destroy(sbi->s_journal) < 0)
+ err = journal_destroy(sbi->s_journal);
+ sbi->s_journal = NULL;
+ if (err < 0)
ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
if (!(sb->s_flags & MS_RDONLY)) {
EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);



2008-10-27 07:31:48

by Hidehiro Kawai

[permalink] [raw]
Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Hi Vegard,

Vegard Nossum wrote:

> commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> Author: Hidehiro Kawai <[email protected]>
> Date: Wed Oct 22 14:15:01 2008 -0700
>
> ext3: add checks for errors from jbd
>
> introduces a regression which was discovered by kmemcheck:

Thank you for reporting this regression. It's my fault.
I send patches to fix.

Regards,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory. When
journal has been aborted, ext3_put_super() calls ext3_abort() after
freeing the journal_t object, and then ext3_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <[email protected]>
---
fs/ext3/super.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
sb->s_flags |= MS_RDONLY;
EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
- journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+ if (EXT3_SB(sb)->s_journal)
+ journal_abort(EXT3_SB(sb)->s_journal, -EIO);
}

void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
{
struct ext3_sb_info *sbi = EXT3_SB(sb);
struct ext3_super_block *es = sbi->s_es;
- int i;
+ int i, err;

ext3_xattr_put_super(sb);
- if (journal_destroy(sbi->s_journal) < 0)
+ err = journal_destroy(sbi->s_journal);
+ sbi->s_journal = NULL;
+ if (err < 0)
ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
if (!(sb->s_flags & MS_RDONLY)) {
EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);





2008-10-27 07:37:33

by Hidehiro Kawai

[permalink] [raw]
Subject: [PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort

Vegard Nossum reported a bug which accesses freed memory. When
journal has been aborted, ext4_put_super() calls ext4_abort() after
freeing the journal_t object, and then ext4_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <[email protected]>
---
fs/ext4/super.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)

Index: linux-2.6.28-rc2/fs/ext4/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext4/super.c
+++ linux-2.6.28-rc2/fs/ext4/super.c
@@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb,
EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
sb->s_flags |= MS_RDONLY;
EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
- jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
+ if (EXT4_SB(sb)->s_journal)
+ jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
}

void ext4_warning(struct super_block *sb, const char *function,
@@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
{
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_super_block *es = sbi->s_es;
- int i;
+ int i, err;

ext4_mb_release(sb);
ext4_ext_release(sb);
ext4_xattr_put_super(sb);
- if (jbd2_journal_destroy(sbi->s_journal) < 0)
- ext4_abort(sb, __func__, "Couldn't clean up the journal");
+ err = jbd2_journal_destroy(sbi->s_journal);
sbi->s_journal = NULL;
+ if (err < 0)
+ ext4_abort(sb, __func__, "Couldn't clean up the journal");
+
if (!(sb->s_flags & MS_RDONLY)) {
EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);



2008-10-27 08:11:10

by Jan Kara

[permalink] [raw]
Subject: Re: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

> Hi Vegard,
>
> Vegard Nossum wrote:
>
> > commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> > Author: Hidehiro Kawai <[email protected]>
> > Date: Wed Oct 22 14:15:01 2008 -0700
> >
> > ext3: add checks for errors from jbd
> >
> > introduces a regression which was discovered by kmemcheck:
>
> Thank you for reporting this regression. It's my fault.
> I send patches to fix.
>
> Regards,
>
>
> Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort
>
> Vegard Nossum reported a bug which accesses freed memory. When
> journal has been aborted, ext3_put_super() calls ext3_abort() after
> freeing the journal_t object, and then ext3_abort() accesses it.
> This patch fix it.
>
> Signed-off-by: Hidehiro Kawai <[email protected]>
The patch looks fine. Acked-by: Jan Kara <[email protected]>

Honza
> ---
> fs/ext3/super.c | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> Index: linux-2.6.28-rc2/fs/ext3/super.c
> ===================================================================
> --- linux-2.6.28-rc2.orig/fs/ext3/super.c
> +++ linux-2.6.28-rc2/fs/ext3/super.c
> @@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
> EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
> sb->s_flags |= MS_RDONLY;
> EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
> - journal_abort(EXT3_SB(sb)->s_journal, -EIO);
> + if (EXT3_SB(sb)->s_journal)
> + journal_abort(EXT3_SB(sb)->s_journal, -EIO);
> }
>
> void ext3_warning (struct super_block * sb, const char * function,
> @@ -390,11 +391,14 @@ static void ext3_put_super (struct super
> {
> struct ext3_sb_info *sbi = EXT3_SB(sb);
> struct ext3_super_block *es = sbi->s_es;
> - int i;
> + int i, err;
>
> ext3_xattr_put_super(sb);
> - if (journal_destroy(sbi->s_journal) < 0)
> + err = journal_destroy(sbi->s_journal);
> + sbi->s_journal = NULL;
> + if (err < 0)
> ext3_abort(sb, __func__, "Couldn't clean up the journal");
> +
> if (!(sb->s_flags & MS_RDONLY)) {
> EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
> es->s_state = cpu_to_le16(sbi->s_mount_state);
>
--
Jan Kara <[email protected]>
SuSE CR Labs

2008-10-27 08:11:55

by Jan Kara

[permalink] [raw]
Subject: Re: [PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort

> Vegard Nossum reported a bug which accesses freed memory. When
> journal has been aborted, ext4_put_super() calls ext4_abort() after
> freeing the journal_t object, and then ext4_abort() accesses it.
> This patch fix it.
>
> Signed-off-by: Hidehiro Kawai <[email protected]>
And this one as well. Acked-by: Jan Kara <[email protected]>
> ---
> fs/ext4/super.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> Index: linux-2.6.28-rc2/fs/ext4/super.c
> ===================================================================
> --- linux-2.6.28-rc2.orig/fs/ext4/super.c
> +++ linux-2.6.28-rc2/fs/ext4/super.c
> @@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb,
> EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
> sb->s_flags |= MS_RDONLY;
> EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
> - jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
> + if (EXT4_SB(sb)->s_journal)
> + jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
> }
>
> void ext4_warning(struct super_block *sb, const char *function,
> @@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
> {
> struct ext4_sb_info *sbi = EXT4_SB(sb);
> struct ext4_super_block *es = sbi->s_es;
> - int i;
> + int i, err;
>
> ext4_mb_release(sb);
> ext4_ext_release(sb);
> ext4_xattr_put_super(sb);
> - if (jbd2_journal_destroy(sbi->s_journal) < 0)
> - ext4_abort(sb, __func__, "Couldn't clean up the journal");
> + err = jbd2_journal_destroy(sbi->s_journal);
> sbi->s_journal = NULL;
> + if (err < 0)
> + ext4_abort(sb, __func__, "Couldn't clean up the journal");
> +
> if (!(sb->s_flags & MS_RDONLY)) {
> EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
> es->s_state = cpu_to_le16(sbi->s_mount_state);
>
--
Jan Kara <[email protected]>
SuSE CR Labs

2008-10-27 10:43:42

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort


* Hidehiro Kawai <[email protected]> wrote:

> Vegard Nossum reported a bug which accesses freed memory. When
> journal has been aborted, ext4_put_super() calls ext4_abort() after
> freeing the journal_t object, and then ext4_abort() accesses it.
> This patch fix it.
>
> Signed-off-by: Hidehiro Kawai <[email protected]>

could you please put the word "kmemcheck" into the commit log? ("found
via kmemcheck" or so) We want to have an easily git-greppable track
record of upstream kernel bugs that were found via kmemcheck. (and
there's already a long list)

Ingo

2008-10-27 11:44:45

by Hidehiro Kawai

[permalink] [raw]
Subject: [PATCH 1/2 take 2] ext3: fix a bug accessing freed memory in ext3_abort

Ingo Molnar wrote:

> * Hidehiro Kawai <[email protected]> wrote:
>
>>Vegard Nossum reported a bug which accesses freed memory. When
>>journal has been aborted, ext4_put_super() calls ext4_abort() after
>>freeing the journal_t object, and then ext4_abort() accesses it.
>>This patch fix it.
>>
>>Signed-off-by: Hidehiro Kawai <[email protected]>
>
> could you please put the word "kmemcheck" into the commit log? ("found
> via kmemcheck" or so) We want to have an easily git-greppable track
> record of upstream kernel bugs that were found via kmemcheck. (and
> there's already a long list)

Ah, OK. I fixed the commit log.

Thanks,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory (found via
kmemcheck). When journal has been aborted, ext3_put_super() calls
ext3_abort() after freeing the journal_t object, and then ext3_abort()
accesses it. This patch fix it.

Signed-off-by: Hidehiro Kawai <[email protected]>
Acked-by: Jan Kara <[email protected]>
---
fs/ext3/super.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
sb->s_flags |= MS_RDONLY;
EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
- journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+ if (EXT3_SB(sb)->s_journal)
+ journal_abort(EXT3_SB(sb)->s_journal, -EIO);
}

void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
{
struct ext3_sb_info *sbi = EXT3_SB(sb);
struct ext3_super_block *es = sbi->s_es;
- int i;
+ int i, err;

ext3_xattr_put_super(sb);
- if (journal_destroy(sbi->s_journal) < 0)
+ err = journal_destroy(sbi->s_journal);
+ sbi->s_journal = NULL;
+ if (err < 0)
ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
if (!(sb->s_flags & MS_RDONLY)) {
EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);



2008-10-27 11:50:26

by Hidehiro Kawai

[permalink] [raw]
Subject: [PATCH 2/2 take 2] ext4: fix a bug accessing freed memory in ext4_abort

Vegard Nossum reported a bug which accesses freed memory (found via
kmemcheck). When journal has been aborted, ext4_put_super() calls
ext4_abort() after freeing the journal_t object, and then ext4_abort()
accesses it. This patch fix it.

Signed-off-by: Hidehiro Kawai <[email protected]>
Acked-by: Jan Kara <[email protected]>
---
fs/ext4/super.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)

Index: linux-2.6.28-rc2/fs/ext4/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext4/super.c
+++ linux-2.6.28-rc2/fs/ext4/super.c
@@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb,
EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
sb->s_flags |= MS_RDONLY;
EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
- jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
+ if (EXT4_SB(sb)->s_journal)
+ jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
}

void ext4_warning(struct super_block *sb, const char *function,
@@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
{
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_super_block *es = sbi->s_es;
- int i;
+ int i, err;

ext4_mb_release(sb);
ext4_ext_release(sb);
ext4_xattr_put_super(sb);
- if (jbd2_journal_destroy(sbi->s_journal) < 0)
- ext4_abort(sb, __func__, "Couldn't clean up the journal");
+ err = jbd2_journal_destroy(sbi->s_journal);
sbi->s_journal = NULL;
+ if (err < 0)
+ ext4_abort(sb, __func__, "Couldn't clean up the journal");
+
if (!(sb->s_flags & MS_RDONLY)) {
EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);



2008-10-27 18:06:12

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [PATCH 2/2 take 2] ext4: fix a bug accessing freed memory in ext4_abort

Many thanks, Kawai-san, it looks good. I'll get this queued for
mainline.

- Ted