Hi all,
There is potential memory leak of journal head in function
jbd2_journal_commit_transaction. The problem is that JBD2 will not
reclaim the journal head of commit record if error occurs or journal is
abotred.
I use the following script to reproduce this issue, on a RHEL6 system. I
found it very easy to reproduce with async commit enabled.
mount /dev/sdb /mnt -o journal_checksum,journal_async_commit
touch /mnt/xxx
echo offline > /sys/block/sdb/device/state
sync
umount /mnt
rmmod ext4
rmmod jbd2
Removal of the jbd2 module will make slab complaining that
"cache `jbd2_journal_head': can't free all objects".
Here is my fix for this issue. The commit record should be reclaimed no
matter error occurs or not.
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
index f3ad159..37a973a 100644
--- a/fs/jbd2/commit.c
+++ b/fs/jbd2/commit.c
@@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal,
int ret;
struct timespec now = current_kernel_time();
+ *cbh = NULL;
+
if (is_journal_aborted(journal))
return 0;
@@ -808,7 +810,7 @@ wait_for_iobuf:
if (err)
__jbd2_journal_abort_hard(journal);
}
- if (!err && !is_journal_aborted(journal))
+ if (cbh)
err = journal_wait_on_commit_record(journal, cbh);
if (JBD2_HAS_INCOMPAT_FEATURE(journal,
JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&
PS: Just out of curiosity, why would journal_submit_commit_record return a
value of 1 instead of an error number if get descriptor buffer is failed.
Zhang Huan
On Tue, 8 Mar 2011, Zhang Huan wrote:
> Hi all,
>
> There is potential memory leak of journal head in function
> jbd2_journal_commit_transaction. The problem is that JBD2 will not
> reclaim the journal head of commit record if error occurs or journal is
> abotred.
>
> I use the following script to reproduce this issue, on a RHEL6 system. I
> found it very easy to reproduce with async commit enabled.
>
> mount /dev/sdb /mnt -o journal_checksum,journal_async_commit
> touch /mnt/xxx
> echo offline > /sys/block/sdb/device/state
> sync
> umount /mnt
> rmmod ext4
> rmmod jbd2
>
> Removal of the jbd2 module will make slab complaining that
> "cache `jbd2_journal_head': can't free all objects".
>
>
> Here is my fix for this issue. The commit record should be reclaimed no
> matter error occurs or not.
>
> diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
> index f3ad159..37a973a 100644
> --- a/fs/jbd2/commit.c
> +++ b/fs/jbd2/commit.c
> @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal,
> int ret;
> struct timespec now = current_kernel_time();
>
> + *cbh = NULL;
> +
> if (is_journal_aborted(journal))
> return 0;
>
> @@ -808,7 +810,7 @@ wait_for_iobuf:
> if (err)
> __jbd2_journal_abort_hard(journal);
> }
> - if (!err && !is_journal_aborted(journal))
> + if (cbh)
Hi,
I wonder if we could do rather this:
if (!err && !is_journal_aborted(journal))
err = journal_wait_on_commit_record(journal, cbh);
else if (cbh) {
put_bh(cbh);
jbd2_journal_put_journal_head(bh2jh(cbh));
}
I think this is more readable...
Thanks!
-Lukas
> err = journal_wait_on_commit_record(journal, cbh);
> if (JBD2_HAS_INCOMPAT_FEATURE(journal,
> JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&
>
>
>
> PS: Just out of curiosity, why would journal_submit_commit_record return a
> value of 1 instead of an error number if get descriptor buffer is failed.
>
>
> Zhang Huan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
On Tue, 8 Mar 2011, Lukas Czerner wrote:
> On Tue, 8 Mar 2011, Zhang Huan wrote:
>
>> Hi all,
>>
>> There is potential memory leak of journal head in function
>> jbd2_journal_commit_transaction. The problem is that JBD2 will not
>> reclaim the journal head of commit record if error occurs or journal is
>> abotred.
>>
>> I use the following script to reproduce this issue, on a RHEL6 system. I
>> found it very easy to reproduce with async commit enabled.
>>
>> mount /dev/sdb /mnt -o journal_checksum,journal_async_commit
>> touch /mnt/xxx
>> echo offline > /sys/block/sdb/device/state
>> sync
>> umount /mnt
>> rmmod ext4
>> rmmod jbd2
>>
>> Removal of the jbd2 module will make slab complaining that
>> "cache `jbd2_journal_head': can't free all objects".
>>
>>
>> Here is my fix for this issue. The commit record should be reclaimed no
>> matter error occurs or not.
>>
>> diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
>> index f3ad159..37a973a 100644
>> --- a/fs/jbd2/commit.c
>> +++ b/fs/jbd2/commit.c
>> @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal,
>> int ret;
>> struct timespec now = current_kernel_time();
>>
>> + *cbh = NULL;
>> +
>> if (is_journal_aborted(journal))
>> return 0;
>>
>> @@ -808,7 +810,7 @@ wait_for_iobuf:
>> if (err)
>> __jbd2_journal_abort_hard(journal);
>> }
>> - if (!err && !is_journal_aborted(journal))
>> + if (cbh)
>
> Hi,
>
> I wonder if we could do rather this:
>
> if (!err && !is_journal_aborted(journal))
> err = journal_wait_on_commit_record(journal, cbh);
> else if (cbh) {
> put_bh(cbh);
> jbd2_journal_put_journal_head(bh2jh(cbh));
> }
>
> I think this is more readable...
Hi,
I don't think it is a good idea. Anyway, you need to wait for buffer to
complete, and then release journal head on it. That is exactly what
journal_wait_on_commit_record does.
>
> Thanks!
> -Lukas
>
>> err = journal_wait_on_commit_record(journal, cbh);
>> if (JBD2_HAS_INCOMPAT_FEATURE(journal,
>> JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&
>>
>>
>>
>> PS: Just out of curiosity, why would journal_submit_commit_record return a
>> value of 1 instead of an error number if get descriptor buffer is failed.
>>
>>
>> Zhang Huan
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
>
Zhang Huan
On Tue, 8 Mar 2011, Zhang Huan wrote:
>
> On Tue, 8 Mar 2011, Lukas Czerner wrote:
>
> > On Tue, 8 Mar 2011, Zhang Huan wrote:
> >
> > > Hi all,
> > >
> > > There is potential memory leak of journal head in function
> > > jbd2_journal_commit_transaction. The problem is that JBD2 will not
> > > reclaim the journal head of commit record if error occurs or journal is
> > > abotred.
> > >
> > > I use the following script to reproduce this issue, on a RHEL6 system. I
> > > found it very easy to reproduce with async commit enabled.
> > >
> > > mount /dev/sdb /mnt -o journal_checksum,journal_async_commit
> > > touch /mnt/xxx
> > > echo offline > /sys/block/sdb/device/state
> > > sync
> > > umount /mnt
> > > rmmod ext4
> > > rmmod jbd2
> > >
> > > Removal of the jbd2 module will make slab complaining that
> > > "cache `jbd2_journal_head': can't free all objects".
> > >
> > >
> > > Here is my fix for this issue. The commit record should be reclaimed no
> > > matter error occurs or not.
> > >
> > > diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
> > > index f3ad159..37a973a 100644
> > > --- a/fs/jbd2/commit.c
> > > +++ b/fs/jbd2/commit.c
> > > @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t
> > > *journal,
> > > int ret;
> > > struct timespec now = current_kernel_time();
> > >
> > > + *cbh = NULL;
> > > +
> > > if (is_journal_aborted(journal))
> > > return 0;
> > >
> > > @@ -808,7 +810,7 @@ wait_for_iobuf:
> > > if (err)
> > > __jbd2_journal_abort_hard(journal);
> > > }
> > > - if (!err && !is_journal_aborted(journal))
> > > + if (cbh)
> >
> > Hi,
> >
> > I wonder if we could do rather this:
> >
> > if (!err && !is_journal_aborted(journal))
> > err = journal_wait_on_commit_record(journal, cbh);
> > else if (cbh) {
> > put_bh(cbh);
> > jbd2_journal_put_journal_head(bh2jh(cbh));
> > }
> >
> > I think this is more readable...
>
> Hi,
>
> I don't think it is a good idea. Anyway, you need to wait for buffer to
> complete, and then release journal head on it. That is exactly what
> journal_wait_on_commit_record does.
Oh, I see. Ok then ignore me :)
Thanks!
-Lukas
>
> >
> > Thanks!
> > -Lukas
> >
> > > err = journal_wait_on_commit_record(journal, cbh);
> > > if (JBD2_HAS_INCOMPAT_FEATURE(journal,
> > > JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&
> > >
> > >
> > >
> > > PS: Just out of curiosity, why would journal_submit_commit_record return a
> > > value of 1 instead of an error number if get descriptor buffer is failed.
> > >
> > >
> > > Zhang Huan
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > > the body of a message to [email protected]
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > >
> >
> > --
> >
>
> Zhang Huan
>
--
Hi,
I would like to know if there is more comment on this?
On Tue, Mar 8, 2011 at 19:40, Zhang Huan <[email protected]> wrote:
> Hi all,
>
> There is potential memory leak of journal head in function
> jbd2_journal_commit_transaction. The problem is that JBD2 will not
> reclaim the journal head of commit record if error occurs or journal is
> abotred.
>
> I use the following script to reproduce this issue, on a RHEL6 system. I
> found it very easy to reproduce with async commit enabled.
>
> mount /dev/sdb /mnt -o journal_checksum,journal_async_commit
> touch /mnt/xxx
> echo offline > /sys/block/sdb/device/state
> sync
> umount /mnt
> rmmod ext4
> rmmod jbd2
>
> Removal of the jbd2 module will make slab complaining that
> "cache `jbd2_journal_head': can't free all objects".
>
>
> Here is my fix for this issue. The commit record should be reclaimed no
> matter error occurs or not.
>
> diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
> index f3ad159..37a973a 100644
> --- a/fs/jbd2/commit.c
> +++ b/fs/jbd2/commit.c
> @@ -105,6 +105,8 @@ static int journal_submit_commit_record(journal_t *journal,
> int ret;
> struct timespec now = current_kernel_time();
>
> + *cbh = NULL;
> +
> if (is_journal_aborted(journal))
> return 0;
>
> @@ -808,7 +810,7 @@ wait_for_iobuf:
> if (err)
> __jbd2_journal_abort_hard(journal);
> }
> - if (!err && !is_journal_aborted(journal))
> + if (cbh)
> err = journal_wait_on_commit_record(journal, cbh);
> if (JBD2_HAS_INCOMPAT_FEATURE(journal,
> JBD2_FEATURE_INCOMPAT_ASYNC_COMMIT) &&
>
>
>
> PS: Just out of curiosity, why would journal_submit_commit_record return a
> value of 1 instead of an error number if get descriptor buffer is failed.
>
>
> Zhang Huan
>
Zhang Huan
Hi Zhang,
Sorry for the delay in looking at it. One thing which is missing from
your patch is a signed-off-by header. From section 12 of the
Documentation/SubmittingPatches file in the Linux source tree:
>The sign-off is a simple line at the end of the explanation for the
>patch, which certifies that you wrote it or otherwise have the right to
>pass it on as a open-source patch. The rules are pretty simple: if you
>can certify the below:
>
> Developer's Certificate of Origin 1.1
>
> By making a contribution to this project, I certify that:
>
> (a) The contribution was created in whole or in part by me and I
> have the right to submit it under the open source license
> indicated in the file; or
>
> (b) The contribution is based upon previous work that, to the best
> of my knowledge, is covered under an appropriate open source
> license and I have the right under that license to submit that
> work with modifications, whether created in whole or in part
> by me, under the same open source license (unless I am
> permitted to submit under a different license), as indicated
> in the file; or
>
> (c) The contribution was provided directly to me by some other
> person who certified (a), (b) or (c) and I have not modified
> it.
>
> (d) I understand and agree that this project and the contribution
> are public and that a record of the contribution (including all
> personal information I submit with it, including my sign-off) is
> maintained indefinitely and may be redistributed consistent with
> this project or the open source license(s) involved.
>
>then you just add a line saying
>
> Signed-off-by: Random J Developer <[email protected]>
>
>using your real name (sorry, no pseudonyms or anonymous contributions.)
If you could send me an acknowledgement that it would be OK to add a
Signed-off-by: Zhang Huan <[email protected]>
line to your patch, it would be much appreciated.
- Ted
On Tue, Apr 5, 2011 at 06:10, Ted Ts'o <[email protected]> wrote:
>
> If you could send me an acknowledgement that it would be OK to add a
>
> Signed-off-by: Zhang Huan <[email protected]>
>
> line to your patch, it would be much appreciated.
>
Yes. please go ahead.
Zhang Huan
On Tue, Apr 05, 2011 at 08:35:35PM +0800, Zhang Huan wrote:
> On Tue, Apr 5, 2011 at 06:10, Ted Ts'o <[email protected]> wrote:
> >
> > If you could send me an acknowledgement that it would be OK to add a
> >
> > Signed-off-by: Zhang Huan <[email protected]>
> >
> > line to your patch, it would be much appreciated.
> >
>
> Yes. please go ahead.
Thanks, I've added your patch to the ext4 patch queue.
- Ted