Subject: ext4 hung tasks in linux-stable 3.0.41

Dear list,
dear ext4 maintainers,

i'm expected every night a hang of my rsync backup process, doing a
backup to an ext4 filesystem. Always throwing Call traces.

The hung task messages are really long so i posted it to pastebin:
http://pastebin.com/raw.php?i=9v7HEvjA

Used kernel: vanilla 3.0.41

Is this problem known? Or is there a fix for 3.0.X available?

Greets,
Stefan


2012-08-31 09:29:02

by Jan Kara

[permalink] [raw]
Subject: Re: ext4 hung tasks in linux-stable 3.0.41

Hello,

On Fri 31-08-12 09:43:37, Stefan Priebe - Profihost AG wrote:
> i'm expected every night a hang of my rsync backup process, doing a
> backup to an ext4 filesystem. Always throwing Call traces.
>
> The hung task messages are really long so i posted it to pastebin:
> http://pastebin.com/raw.php?i=9v7HEvjA
>
> Used kernel: vanilla 3.0.41
>
> Is this problem known? Or is there a fix for 3.0.X available?
From a quick look, processes are waiting for IO so it may be that your
disk is just loaded... Did it start happening after some change? Or do you
experience unexpectedly low performance?

Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR

Subject: Re: ext4 hung tasks in linux-stable 3.0.41

Am 31.08.2012 11:28, schrieb Jan Kara:
> Hello,
>
> On Fri 31-08-12 09:43:37, Stefan Priebe - Profihost AG wrote:
>> i'm expected every night a hang of my rsync backup process, doing a
>> backup to an ext4 filesystem. Always throwing Call traces.
>>
>> The hung task messages are really long so i posted it to pastebin:
>> http://pastebin.com/raw.php?i=9v7HEvjA
>>
>> Used kernel: vanilla 3.0.41
>>
>> Is this problem known? Or is there a fix for 3.0.X available?
> From a quick look, processes are waiting for IO so it may be that your
> disk is just loaded... Did it start happening after some change? Or do you
> experience unexpectedly low performance?

The main disk is XFS. The backup disk is ext4. The backup disk is doing
nothing else than this SINGLE rsync task / process. So the only I/O is
rsync itself.

So i don't expect low performance as the main disk is XFS ;-)

Greets,
Stefan

2012-08-31 17:59:10

by Theodore Ts'o

[permalink] [raw]
Subject: Re: ext4 hung tasks in linux-stable 3.0.41

On Fri, Aug 31, 2012 at 11:28:59AM +0200, Jan Kara wrote:
> >
> > Is this problem known? Or is there a fix for 3.0.X available?
> From a quick look, processes are waiting for IO so it may be that your
> disk is just loaded... Did it start happening after some change? Or do you
> experience unexpectedly low performance?

All of the processes waiting for I/O are waiting for a directory
block. This makes me suspicious that perhaps an interrupt or I/O
request got lost.

Is this a repeatable failure, and is the stack traces always the same?

If so, you might want to try running badblocks on the device and make
sure that hardware issues have been ruled out.

- Ted

Subject: Re: ext4 hung tasks in linux-stable 3.0.41

Am 31.08.2012 19:59, schrieb Theodore Ts'o:
> On Fri, Aug 31, 2012 at 11:28:59AM +0200, Jan Kara wrote:
>>>
>>> Is this problem known? Or is there a fix for 3.0.X available?
>> From a quick look, processes are waiting for IO so it may be that your
>> disk is just loaded... Did it start happening after some change? Or do you
>> experience unexpectedly low performance?
>
> All of the processes waiting for I/O are waiting for a directory
> block. This makes me suspicious that perhaps an interrupt or I/O
> request got lost.
>
> Is this a repeatable failure, and is the stack traces always the same?
>
> If so, you might want to try running badblocks on the device and make
> sure that hardware issues have been ruled out.

But but SMART is OK and i see no I/O errors from AHCI. But i'll try to
replace the disk and see if this fixes it.

Stefan