2003-07-10 15:34:46

by Peter Lojkin

[permalink] [raw]
Subject: 2.4.22-pre3 and reiserfs boot problem

Hello,

I am not on the list so please CC me if replying...

I've found the problem, it's patch with description:

Fix potential IO hangs and increase interactiveness during heavy IO

http://linux.bkbits.net:8080/linux-2.4/user=mason/[email protected]?nav=!-|index.html|stats|!+|index.html|ChangeSet@-7d

After removing all changes from this cset, a had no problems
mounting big reiserfs volumes...


2003-07-10 15:52:06

by Stephan von Krawczynski

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On Thu, 10 Jul 2003 19:49:20 +0400
"Peter Lojkin" <[email protected]> wrote:

> Hello,
>
> I am not on the list so please CC me if replying...
>
> I've found the problem, it's patch with description:
>
> Fix potential IO hangs and increase interactiveness during heavy IO
>
> http://linux.bkbits.net:8080/linux-2.4/user=mason/[email protected]?nav=!-|index.html|stats|!+|index.html|ChangeSet@-7d
>
> After removing all changes from this cset, a had no problems
> mounting big reiserfs volumes...

Hello Marcelo,

can you please send me a separated patch for reversal to verify this.

Hello Chris,

if this is true I am willing to test other versions of the questionable patch
to solve the issue.

Thank you.
Stephan

2003-07-10 17:00:27

by Stephan von Krawczynski

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On Thu, 10 Jul 2003 20:20:02 +0400
"Peter Lojkin" <[email protected]> wrote:

> Hello,
>
> here is exact patch i've used. i made it by cutting pre2-pre3 diff,
> so apply it o top of 2.4.22-pre3 with -R option to patch...

Hello Peter
Hello Marcelo

I can confirm that pre3 works when reversing the attached patch. Thanks very
much, Peter.

Regards,
Stephan



Attachments:
ll_rw_cset.diff.bz2 (6.67 kB)

2003-07-10 17:42:51

by Marcelo Tosatti

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem



On Thu, 10 Jul 2003, Stephan von Krawczynski wrote:

> On Thu, 10 Jul 2003 19:49:20 +0400
> "Peter Lojkin" <[email protected]> wrote:
>
> > Hello,
> >
> > I am not on the list so please CC me if replying...
> >
> > I've found the problem, it's patch with description:
> >
> > Fix potential IO hangs and increase interactiveness during heavy IO
> >
> > http://linux.bkbits.net:8080/linux-2.4/user=mason/[email protected]?nav=!-|index.html|stats|!+|index.html|ChangeSet@-7d
> >
> > After removing all changes from this cset, a had no problems
> > mounting big reiserfs volumes...
>
> Hello Marcelo,
>
> can you please send me a separated patch for reversal to verify this.

Its attached (iostalls). Use patch -R.

Also, when the hang happens, sysrq works right?

If so get us the tasks backtraces with sysrq, ok?

2003-07-10 17:51:40

by Marcelo Tosatti

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem



On Thu, 10 Jul 2003, Stephan von Krawczynski wrote:

> On Thu, 10 Jul 2003 20:20:02 +0400
> "Peter Lojkin" <[email protected]> wrote:
>
> > Hello,
> >
> > here is exact patch i've used. i made it by cutting pre2-pre3 diff,
> > so apply it o top of 2.4.22-pre3 with -R option to patch...
>
> Hello Peter
> Hello Marcelo
>
> I can confirm that pre3 works when reversing the attached patch. Thanks very
> much, Peter.

Fine Stephan. Now can youplease get us the task backtraces from sysrq when
the hang happens?

Andrea, Chris, any idea of why this is happening?


2003-07-10 17:46:04

by Marcelo Tosatti

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem


I forgot to attach the patch. Now its attached, duh.

On Thu, 10 Jul 2003, Marcelo Tosatti wrote:

>
>
> On Thu, 10 Jul 2003, Stephan von Krawczynski wrote:
>
> > On Thu, 10 Jul 2003 19:49:20 +0400
> > "Peter Lojkin" <[email protected]> wrote:
> >
> > > Hello,
> > >
> > > I am not on the list so please CC me if replying...
> > >
> > > I've found the problem, it's patch with description:
> > >
> > > Fix potential IO hangs and increase interactiveness during heavy IO
> > >
> > > http://linux.bkbits.net:8080/linux-2.4/user=mason/[email protected]?nav=!-|index.html|stats|!+|index.html|ChangeSet@-7d
> > >
> > > After removing all changes from this cset, a had no problems
> > > mounting big reiserfs volumes...
> >
> > Hello Marcelo,
> >
> > can you please send me a separated patch for reversal to verify this.
>
> Its attached (iostalls). Use patch -R.
>
> Also, when the hang happens, sysrq works right?
>
> If so get us the tasks backtraces with sysrq, ok?
>
>


Attachments:
iostalls (23.35 kB)

2003-07-11 13:02:21

by Chris Mason

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On Thu, 2003-07-10 at 14:01, Marcelo Tosatti wrote:
> On Thu, 10 Jul 2003, Stephan von Krawczynski wrote:
>
> > On Thu, 10 Jul 2003 20:20:02 +0400
> > "Peter Lojkin" <[email protected]> wrote:
> >
> > > Hello,
> > >
> > > here is exact patch i've used. i made it by cutting pre2-pre3 diff,
> > > so apply it o top of 2.4.22-pre3 with -R option to patch...
> >
> > Hello Peter
> > Hello Marcelo
> >
> > I can confirm that pre3 works when reversing the attached patch. Thanks very
> > much, Peter.
>
> Fine Stephan. Now can youplease get us the task backtraces from sysrq when
> the hang happens?
>
> Andrea, Chris, any idea of why this is happening?

My first guess is that blk_oversized_queue is false but there aren't any
requests left. That will pretty much spin in __get_request_wait with
irqs off, which sounds similar to what he's hitting.

I think we need this hunk even if it doesn't fix his problem.

Stephan, if this patch doesn't help, could you please boot with
nmi_watchdog=1? An earlier email said sysrq wasn't working, so we'll
probably need the nmi_watchdog to get a backtrace.

-chris


Attachments:
io-stalls-11-inc.diff (455.00 B)

2003-07-11 13:18:20

by Stephan von Krawczynski

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On 11 Jul 2003 09:15:21 -0400
Chris Mason <[email protected]> wrote:

> On Thu, 2003-07-10 at 14:01, Marcelo Tosatti wrote:
> > On Thu, 10 Jul 2003, Stephan von Krawczynski wrote:
> >
> > > On Thu, 10 Jul 2003 20:20:02 +0400
> > > "Peter Lojkin" <[email protected]> wrote:
> > >
> > > > Hello,
> > > >
> > > > here is exact patch i've used. i made it by cutting pre2-pre3 diff,
> > > > so apply it o top of 2.4.22-pre3 with -R option to patch...
> > >
> > > Hello Peter
> > > Hello Marcelo
> > >
> > > I can confirm that pre3 works when reversing the attached patch. Thanks
> > > very much, Peter.
> >
> > Fine Stephan. Now can youplease get us the task backtraces from sysrq when
> > the hang happens?
> >
> > Andrea, Chris, any idea of why this is happening?
>
> My first guess is that blk_oversized_queue is false but there aren't any
> requests left. That will pretty much spin in __get_request_wait with
> irqs off, which sounds similar to what he's hitting.

Why is it I am only seeing it on a big device of 320 GB? Even 60GB mount
without problems...

Regards,
Stephan

2003-07-11 13:29:21

by Stephan von Krawczynski

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On 11 Jul 2003 09:15:21 -0400
Chris Mason <[email protected]> wrote:

> > Andrea, Chris, any idea of why this is happening?
>
> My first guess is that blk_oversized_queue is false but there aren't any
> requests left. That will pretty much spin in __get_request_wait with
> irqs off, which sounds similar to what he's hitting.
>
> I think we need this hunk even if it doesn't fix his problem.

Strike!
Your patch solves my problem. I applied it on 2.4.22-pre4 and it now works just
like -pre2 did.
Great Chris, compared to the pretty minimal input I could give ...

Regards,
Stephan

2003-07-11 13:34:45

by Chris Mason

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On Fri, 2003-07-11 at 09:27, Stephan von Krawczynski wrote:
>
> > My first guess is that blk_oversized_queue is false but there aren't any
> > requests left. That will pretty much spin in __get_request_wait with
> > irqs off, which sounds similar to what he's hitting.
>
> Why is it I am only seeing it on a big device of 320 GB? Even 60GB mount
> without problems...

Honestly not sure. I could make a better guess if you got a stack trace
with nmi_watchdog on, so we could see which part of the reiserfs mount
process was causing problems.

The bug would be easier to hit with smaller blocksizes, I'm running
tests now with ext and 1k blocksizes.

-chris


2003-07-11 13:37:52

by Chris Mason

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem

On Fri, 2003-07-11 at 09:38, Stephan von Krawczynski wrote:
> On 11 Jul 2003 09:15:21 -0400
> Chris Mason <[email protected]> wrote:
>
> > > Andrea, Chris, any idea of why this is happening?
> >
> > My first guess is that blk_oversized_queue is false but there aren't any
> > requests left. That will pretty much spin in __get_request_wait with
> > irqs off, which sounds similar to what he's hitting.
> >
> > I think we need this hunk even if it doesn't fix his problem.
>
> Strike!
> Your patch solves my problem. I applied it on 2.4.22-pre4 and it now works just
> like -pre2 did.
> Great Chris, compared to the pretty minimal input I could give ...

Thanks for the quick test, you've been hugely helpful.

-chris


2003-07-11 15:04:08

by Marcelo Tosatti

[permalink] [raw]
Subject: Re: 2.4.22-pre3 and reiserfs boot problem



On Fri, 11 Jul 2003, Stephan von Krawczynski wrote:

> On 11 Jul 2003 09:15:21 -0400
> Chris Mason <[email protected]> wrote:
>
> > > Andrea, Chris, any idea of why this is happening?
> >
> > My first guess is that blk_oversized_queue is false but there aren't any
> > requests left. That will pretty much spin in __get_request_wait with
> > irqs off, which sounds similar to what he's hitting.
> >
> > I think we need this hunk even if it doesn't fix his problem.
>
> Strike!
> Your patch solves my problem. I applied it on 2.4.22-pre4 and it now works just
> like -pre2 did.
> Great Chris, compared to the pretty minimal input I could give ...

The fix is already in my BK tree.

Thanks a lot Chris and Stephan.