LinuxLists.cc - mmotm 2010-12-02-16-34 uploaded

[permalink] [raw]

Subject: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 12/03/2010 01:34 AM, [email protected] wrote:
> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to

Hi, this kernel regresses with respect to resume. It doesn't wake up,
the screen is black with blinking cursor at position [1, 1].
2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
far.

I'm using pm-suspend, any ideas what to test before I start to find the
reason on my own?

regards,
--
js

2010-12-05 13:08:50

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Sunday, December 05, 2010, Jiri Slaby wrote:
> On 12/03/2010 01:34 AM, [email protected] wrote:
> > The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
>
> Hi, this kernel regresses with respect to resume. It doesn't wake up,
> the screen is black with blinking cursor at position [1, 1].
> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
> far.
>
> I'm using pm-suspend, any ideas what to test before I start to find the
> reason on my own?

Please try the pm_test tests for core, processors and devices, eg.:

# echo core > /sys/power/pm_test
# echo mem > /sys/power/state

(this is described in Documentation/power/basic-pm-debugging.txt .

That will tell you if you need to go through the BIOS for the bug to show up
and perhaps at what "level" the problem occurs.

It looks like the problem caused by NX-ing of the kernel code.

Thanks,
Rafael

2010-12-06 17:14:00

by Randy Dunlap

[permalink] [raw]

Subject: Re: mmotm 2010-12-02-16-34 uploaded

On Sat, 4 Dec 2010 11:39:20 +1100 Stephen Rothwell wrote:

> Hi Andrew,
>
> On Thu, 02 Dec 2010 16:34:54 -0800 [email protected] wrote:
> >
> > The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
> >
> > http://userweb.kernel.org/~akpm/mmotm/
>
> Build results here (as they get done): http://kisskb.ellerman.id.au/kisskb/head/3514/
> --

Lots of the errors seem to be toolchain related.

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

2010-12-13 10:04:40

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 12/05/2010 02:08 PM, Rafael J. Wysocki wrote:
> On Sunday, December 05, 2010, Jiri Slaby wrote:
>> On 12/03/2010 01:34 AM, [email protected] wrote:
>>> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
>>
>> Hi, this kernel regresses with respect to resume. It doesn't wake up,
>> the screen is black with blinking cursor at position [1, 1].
>> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
>> far.
>>
>> I'm using pm-suspend, any ideas what to test before I start to find the
>> reason on my own?
>
> Please try the pm_test tests for core, processors and devices, eg.:

Hmm, I haven't seen the issues since then. So it had to be some kind of
coincidence -- I see the issue occasionally for some time already. Now
it triggered twice in a row after I updated the kernel.

regards,
--
js

2010-12-13 21:11:13

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Monday, December 13, 2010, Jiri Slaby wrote:
> On 12/05/2010 02:08 PM, Rafael J. Wysocki wrote:
> > On Sunday, December 05, 2010, Jiri Slaby wrote:
> >> On 12/03/2010 01:34 AM, [email protected] wrote:
> >>> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
> >>
> >> Hi, this kernel regresses with respect to resume. It doesn't wake up,
> >> the screen is black with blinking cursor at position [1, 1].
> >> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
> >> far.
> >>
> >> I'm using pm-suspend, any ideas what to test before I start to find the
> >> reason on my own?
> >
> > Please try the pm_test tests for core, processors and devices, eg.:
>
> Hmm, I haven't seen the issues since then. So it had to be some kind of
> coincidence -- I see the issue occasionally for some time already. Now
> it triggered twice in a row after I updated the kernel.

Hmm. What hardware is that?

Rafael

2010-12-14 22:08:43

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 12/13/2010 10:10 PM, Rafael J. Wysocki wrote:
> On Monday, December 13, 2010, Jiri Slaby wrote:
>> On 12/05/2010 02:08 PM, Rafael J. Wysocki wrote:
>>> On Sunday, December 05, 2010, Jiri Slaby wrote:
>>>> On 12/03/2010 01:34 AM, [email protected] wrote:
>>>>> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
>>>>
>>>> Hi, this kernel regresses with respect to resume. It doesn't wake up,
>>>> the screen is black with blinking cursor at position [1, 1].
>>>> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
>>>> far.
>>>>
>>>> I'm using pm-suspend, any ideas what to test before I start to find the
>>>> reason on my own?
>>>
>>> Please try the pm_test tests for core, processors and devices, eg.:
>>
>> Hmm, I haven't seen the issues since then. So it had to be some kind of
>> coincidence -- I see the issue occasionally for some time already. Now
>> it triggered twice in a row after I updated the kernel.
>
> Hmm. What hardware is that?

What exactly do you want to know about it? It's some specific intel
desktop machine, 3 years old, 2 intel cores, 6G of mem, 2 sata disks in
raid, dvb-t usb receiver (af9015), usb keyboard and mouse.

regards,
--
js

2010-12-14 22:22:54

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Tuesday, December 14, 2010, Jiri Slaby wrote:
> On 12/13/2010 10:10 PM, Rafael J. Wysocki wrote:
> > On Monday, December 13, 2010, Jiri Slaby wrote:
> >> On 12/05/2010 02:08 PM, Rafael J. Wysocki wrote:
> >>> On Sunday, December 05, 2010, Jiri Slaby wrote:
> >>>> On 12/03/2010 01:34 AM, [email protected] wrote:
> >>>>> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
> >>>>
> >>>> Hi, this kernel regresses with respect to resume. It doesn't wake up,
> >>>> the screen is black with blinking cursor at position [1, 1].
> >>>> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
> >>>> far.
> >>>>
> >>>> I'm using pm-suspend, any ideas what to test before I start to find the
> >>>> reason on my own?
> >>>
> >>> Please try the pm_test tests for core, processors and devices, eg.:
> >>
> >> Hmm, I haven't seen the issues since then. So it had to be some kind of
> >> coincidence -- I see the issue occasionally for some time already. Now
> >> it triggered twice in a row after I updated the kernel.
> >
> > Hmm. What hardware is that?
>
> What exactly do you want to know about it? It's some specific intel
> desktop machine, 3 years old, 2 intel cores, 6G of mem, 2 sata disks in
> raid, dvb-t usb receiver (af9015), usb keyboard and mouse.

I'm seeing pretty much the same symptoms on quite different hardware, except
for one thing: USB keyboard and mouse.

Have you checked if writing 0 to /sys/power/pm_async helps?

Rafael

2011-02-04 18:33:57

[permalink] [raw]

Subject: Re: Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 01/31/2011 11:00 PM, Rafael J. Wysocki wrote:
> On Monday, January 31, 2011, Jiri Slaby wrote:
>> On 01/20/2011 10:02 PM, Rafael J. Wysocki wrote:
>>> On Thursday, January 20, 2011, Jiri Slaby wrote:
>>>> On 01/06/2011 03:18 PM, Jiri Slaby wrote:
>>>>> On 12/14/2010 11:22 PM, Rafael J. Wysocki wrote:
>>>>>> On Tuesday, December 14, 2010, Jiri Slaby wrote:
>>>>>>> On 12/13/2010 10:10 PM, Rafael J. Wysocki wrote:
>>>>>>>> On Monday, December 13, 2010, Jiri Slaby wrote:
>>>>>>>>> On 12/05/2010 02:08 PM, Rafael J. Wysocki wrote:
>>>>>>>>>> On Sunday, December 05, 2010, Jiri Slaby wrote:
>>>>>>>>>>> On 12/03/2010 01:34 AM, [email protected] wrote:
>>>>>>>>>>>> The mm-of-the-moment snapshot 2010-12-02-16-34 has been uploaded to
>>>>>>>>>>>
>>>>>>>>>>> Hi, this kernel regresses with respect to resume. It doesn't wake up,
>>>>>>>>>>> the screen is black with blinking cursor at position [1, 1].
>>>>>>>>>>> 2010-11-23-16-12 seemed to be OK. In this one it is 100% reproducible so
>>>>>>>>>>> far.
>>>>>>>>>>>
>>>>>>>>>>> I'm using pm-suspend, any ideas what to test before I start to find the
>>>>>>>>>>> reason on my own?
>>>>>>>>>>
>>>>>>>>>> Please try the pm_test tests for core, processors and devices, eg.:
>>>>>>>>>
>>>>>>>>> Hmm, I haven't seen the issues since then. So it had to be some kind of
>>>>>>>>> coincidence -- I see the issue occasionally for some time already. Now
>>>>>>>>> it triggered twice in a row after I updated the kernel.
>>>>>>>>
>>>>>>>> Hmm. What hardware is that?
>>>>>>>
>>>>>>> What exactly do you want to know about it? It's some specific intel
>>>>>>> desktop machine, 3 years old, 2 intel cores, 6G of mem, 2 sata disks in
>>>>>>> raid, dvb-t usb receiver (af9015), usb keyboard and mouse.
>>>>>>
>>>>>> I'm seeing pretty much the same symptoms on quite different hardware, except
>>>>>> for one thing: USB keyboard and mouse.
>>>>>>
>>>>>> Have you checked if writing 0 to /sys/power/pm_async helps?
>>>>>
>>>>> It's very hard to reproduce (it happens once a week or two), so I can't
>>>>> say yet. As it happened yesterday again, going to test it from now on.
>>>>
>>>> Ok, I haven't seen the issue with 0 in /sys/power/pm_async for two weeks
>>>> now (I added it to boot.local). I would say it's related to that...
>>>
>>> In that case the next step would be to build the kernel with
>>> CONFIG_PM_ADVANCED_DEBUG set and try to switch the async flag for individual
>>> devices (it is located is /sys/devices/.../power/ for each device).
>>>
>>> For example, I wonder if it is sufficient to unset it for all USB devices.
>>
>> It seems to suffice... No hangs still. Should I enable pm_async back
>> again to confirm the issue is still present?
>
> Yes, please, it would be good to know for sure.

Ok, confirmed right now :).

I disabled pm_async for USB again. What do you suggest next?

thanks,
--
js

2011-02-04 19:33:10

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Fri, 4 Feb 2011, Jiri Slaby wrote:

> >> It seems to suffice... No hangs still. Should I enable pm_async back
> >> again to confirm the issue is still present?
> >
> > Yes, please, it would be good to know for sure.
>
> Ok, confirmed right now :).
>
> I disabled pm_async for USB again. What do you suggest next?

What happens if you leave pm_async enabled but unplug all the USB
devices before suspending? If there are any USB devices you can't
unplug, you can get an equivalent result by unconfiguring the root
hubs:

for a in /sys/bus/usb/devices/usb* ; do
echo 0 >$a/bConfigurationValue
done

Alan Stern

2011-02-04 19:38:37

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 02/04/2011 08:33 PM, Alan Stern wrote:
> On Fri, 4 Feb 2011, Jiri Slaby wrote:
>
>>>> It seems to suffice... No hangs still. Should I enable pm_async back
>>>> again to confirm the issue is still present?
>>>
>>> Yes, please, it would be good to know for sure.
>>
>> Ok, confirmed right now :).
>>
>> I disabled pm_async for USB again. What do you suggest next?
>
> What happens if you leave pm_async enabled but unplug all the USB
> devices before suspending?

Then I cannot suspend at all (ENOINPUTDEVICE)... And this would be
painful as the bug occurs once a week or so.

> If there are any USB devices you can't
> unplug, you can get an equivalent result by unconfiguring the root
> hubs:
>
> for a in /sys/bus/usb/devices/usb* ; do
> echo 0 >$a/bConfigurationValue
> done

Well, I can put this into suspend/resume scripts. What's the opposite
operation? 1 > $a/bConfigurationValue?

thanks,
--
js

2011-02-04 20:07:26

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Fri, 4 Feb 2011, Jiri Slaby wrote:

> On 02/04/2011 08:33 PM, Alan Stern wrote:
> > On Fri, 4 Feb 2011, Jiri Slaby wrote:
> >
> >>>> It seems to suffice... No hangs still. Should I enable pm_async back
> >>>> again to confirm the issue is still present?
> >>>
> >>> Yes, please, it would be good to know for sure.
> >>
> >> Ok, confirmed right now :).
> >>
> >> I disabled pm_async for USB again. What do you suggest next?
> >
> > What happens if you leave pm_async enabled but unplug all the USB
> > devices before suspending?
>
> Then I cannot suspend at all (ENOINPUTDEVICE)... And this would be
> painful as the bug occurs once a week or so.

You can suspend using a remote network login. Agreed, this would be
painful.

> > If there are any USB devices you can't
> > unplug, you can get an equivalent result by unconfiguring the root
> > hubs:
> >
> > for a in /sys/bus/usb/devices/usb* ; do
> > echo 0 >$a/bConfigurationValue
> > done
>
> Well, I can put this into suspend/resume scripts. What's the opposite
> operation? 1 > $a/bConfigurationValue?

Yes.

Alan Stern

2011-02-22 08:45:50

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 02/04/2011 08:33 PM, Alan Stern wrote:
> On Fri, 4 Feb 2011, Jiri Slaby wrote:
>
>>>> It seems to suffice... No hangs still. Should I enable pm_async back
>>>> again to confirm the issue is still present?
>>>
>>> Yes, please, it would be good to know for sure.
>>
>> Ok, confirmed right now :).
>>
>> I disabled pm_async for USB again. What do you suggest next?
>
> What happens if you leave pm_async enabled but unplug all the USB
> devices before suspending? If there are any USB devices you can't
> unplug, you can get an equivalent result by unconfiguring the root
> hubs:
>
> for a in /sys/bus/usb/devices/usb* ; do
> echo 0 >$a/bConfigurationValue
> done

This doesn't seem to help. The hang happened 3 times during the 2 weeks.

I have /etc/pm/sleep.d/99usb-debug with:
#!/bin/bash

send() {
for a in /sys/bus/usb/devices/usb* ; do
echo $1 >$a/bConfigurationValue
done
}

case "$1" in
hibernate|suspend)
send 0
;;
thaw|resume)
send 1
;;
*)
;;
esac

exit 0

And it indeed properly enable/disable usb before/after suspend.

regards,
--
js

2011-02-22 15:36:23

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Tue, 22 Feb 2011, Jiri Slaby wrote:

> On 02/04/2011 08:33 PM, Alan Stern wrote:
> > On Fri, 4 Feb 2011, Jiri Slaby wrote:
> >
> >>>> It seems to suffice... No hangs still. Should I enable pm_async back
> >>>> again to confirm the issue is still present?
> >>>
> >>> Yes, please, it would be good to know for sure.
> >>
> >> Ok, confirmed right now :).
> >>
> >> I disabled pm_async for USB again. What do you suggest next?
> >
> > What happens if you leave pm_async enabled but unplug all the USB
> > devices before suspending? If there are any USB devices you can't
> > unplug, you can get an equivalent result by unconfiguring the root
> > hubs:
> >
> > for a in /sys/bus/usb/devices/usb* ; do
> > echo 0 >$a/bConfigurationValue
> > done
>
> This doesn't seem to help. The hang happened 3 times during the 2 weeks.
>
> I have /etc/pm/sleep.d/99usb-debug with:
> #!/bin/bash
>
> send() {
> for a in /sys/bus/usb/devices/usb* ; do
> echo $1 >$a/bConfigurationValue
> done
> }
>
> case "$1" in
> hibernate|suspend)
> send 0
> ;;
> thaw|resume)
> send 1
> ;;
> *)
> ;;
> esac
>
> exit 0
>
> And it indeed properly enable/disable usb before/after suspend.

Strange indeed. It's worth noting that the async stuff affects only
the normal suspend and resume operations, not the late-suspend and
early-resume operations. This means that it all likelihood, the system
crashes either before finishing the suspend or after doing a fair
amount of the resume. And yet that's not consistent with what you see
on the screen.

By the way, are you booting with no_console_suspend? And do you do
"echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
suspending?

What about if you leave async enabled for all the USB devices _except_
usb[1-N]?

Alan Stern

2011-02-23 11:58:54

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 02/22/2011 04:36 PM, Alan Stern wrote:
> By the way, are you booting with no_console_suspend? And do you do
> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> suspending?

Nope.

> What about if you leave async enabled for all the USB devices _except_
> usb[1-N]?

Ok, so now I have:
/sys/bus/usb/devices/usb1/power/async:disabled
/sys/bus/usb/devices/usb2/power/async:disabled
/sys/bus/usb/devices/usb3/power/async:disabled
/sys/bus/usb/devices/usb4/power/async:disabled
/sys/bus/usb/devices/usb5/power/async:disabled
/sys/bus/usb/devices/usb6/power/async:disabled
/sys/bus/usb/devices/usb7/power/async:disabled
/sys/bus/usb/devices/usb8/power/async:disabled
/sys/bus/usb/devices/1-0:1.0/power/async:enabled
/sys/bus/usb/devices/2-0:1.0/power/async:enabled
/sys/bus/usb/devices/2-6/power/async:enabled
/sys/bus/usb/devices/2-6:1.0/power/async:enabled
/sys/bus/usb/devices/2-6:1.1/power/async:enabled
/sys/bus/usb/devices/3-0:1.0/power/async:enabled
/sys/bus/usb/devices/4-0:1.0/power/async:enabled
/sys/bus/usb/devices/4-2/power/async:enabled
/sys/bus/usb/devices/4-2.1/power/async:enabled
/sys/bus/usb/devices/4-2:1.0/power/async:enabled
/sys/bus/usb/devices/4-2.1:1.0/power/async:enabled
/sys/bus/usb/devices/4-2.1:1.1/power/async:enabled
/sys/bus/usb/devices/4-2.2/power/async:enabled
/sys/bus/usb/devices/4-2.2:1.0/power/async:enabled
/sys/bus/usb/devices/5-0:1.0/power/async:enabled
/sys/bus/usb/devices/6-0:1.0/power/async:enabled
/sys/bus/usb/devices/7-0:1.0/power/async:enabled
/sys/bus/usb/devices/8-0:1.0/power/async:enabled

Let's see.

regards,
--
js

2011-02-23 15:59:33

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Wed, 23 Feb 2011, Jiri Slaby wrote:

> On 02/22/2011 04:36 PM, Alan Stern wrote:
> > By the way, are you booting with no_console_suspend? And do you do
> > "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> > suspending?
>
> Nope.

If you do, it's possible you might get some useful information on the
screen the next time one of these hangs occurs.

Alan Stern

2011-02-24 18:41:32

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

2011-03-08 09:13:57

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 02/22/2011 04:36 PM, Alan Stern wrote:
> By the way, are you booting with no_console_suspend? And do you do
> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> suspending?

Unfortunately there are no messages at all.

Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
switch a console in such case? And if yes, to which one?

thanks,
--
js

2011-03-08 19:25:14

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Tuesday, March 08, 2011, Jiri Slaby wrote:
> On 02/22/2011 04:36 PM, Alan Stern wrote:
> > By the way, are you booting with no_console_suspend? And do you do
> > "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> > suspending?
>
> Unfortunately there are no messages at all.
>
> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
> switch a console in such case? And if yes, to which one?

pm-utils don't, but s2ram does, I'm not sure from the top of my head to
which one, though.

If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".

Thanks,
Rafael

2011-03-08 20:31:14

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 03/08/2011 08:25 PM, Rafael J. Wysocki wrote:
> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>> On 02/22/2011 04:36 PM, Alan Stern wrote:
>>> By the way, are you booting with no_console_suspend? And do you do
>>> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
>>> suspending?
>>
>> Unfortunately there are no messages at all.
>>
>> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
>> switch a console in such case? And if yes, to which one?
>
> pm-utils don't, but s2ram does, I'm not sure from the top of my head to
> which one, though.
>
> If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".

Yeah, I meant kernel modesetting, of course. Sorry for the confusion.

I don't have /usr/sbin/s2ram. So I commented the --quirk-no-chvt thing
in a pm-utils script.

thanks,
--
js

2011-03-08 22:12:24

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Tuesday, March 08, 2011, Jiri Slaby wrote:
> On 03/08/2011 08:25 PM, Rafael J. Wysocki wrote:
> > On Tuesday, March 08, 2011, Jiri Slaby wrote:
> >> On 02/22/2011 04:36 PM, Alan Stern wrote:
> >>> By the way, are you booting with no_console_suspend? And do you do
> >>> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> >>> suspending?
> >>
> >> Unfortunately there are no messages at all.
> >>
> >> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
> >> switch a console in such case? And if yes, to which one?
> >
> > pm-utils don't, but s2ram does, I'm not sure from the top of my head to
> > which one, though.
> >
> > If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".
>
> Yeah, I meant kernel modesetting, of course. Sorry for the confusion.
>
> I don't have /usr/sbin/s2ram.

Ah. So it's gone from Factory? Interesting.

2011-03-08 22:14:09

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 03/08/2011 11:12 PM, Rafael J. Wysocki wrote:
> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>> On 03/08/2011 08:25 PM, Rafael J. Wysocki wrote:
>>> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>>>> On 02/22/2011 04:36 PM, Alan Stern wrote:
>>>>> By the way, are you booting with no_console_suspend? And do you do
>>>>> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
>>>>> suspending?
>>>>
>>>> Unfortunately there are no messages at all.
>>>>
>>>> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
>>>> switch a console in such case? And if yes, to which one?
>>>
>>> pm-utils don't, but s2ram does, I'm not sure from the top of my head to
>>> which one, though.
>>>
>>> If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".
>>
>> Yeah, I meant kernel modesetting, of course. Sorry for the confusion.
>>
>> I don't have /usr/sbin/s2ram.
>
> Ah. So it's gone from Factory? Interesting.

I don't know, maybe it's in suspend package which I don't have installed.

--
js

2011-03-08 22:15:47

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Tuesday, March 08, 2011, Jiri Slaby wrote:
> On 03/08/2011 11:12 PM, Rafael J. Wysocki wrote:
> > On Tuesday, March 08, 2011, Jiri Slaby wrote:
> >> On 03/08/2011 08:25 PM, Rafael J. Wysocki wrote:
> >>> On Tuesday, March 08, 2011, Jiri Slaby wrote:
> >>>> On 02/22/2011 04:36 PM, Alan Stern wrote:
> >>>>> By the way, are you booting with no_console_suspend? And do you do
> >>>>> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> >>>>> suspending?
> >>>>
> >>>> Unfortunately there are no messages at all.
> >>>>
> >>>> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
> >>>> switch a console in such case? And if yes, to which one?
> >>>
> >>> pm-utils don't, but s2ram does, I'm not sure from the top of my head to
> >>> which one, though.
> >>>
> >>> If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".
> >>
> >> Yeah, I meant kernel modesetting, of course. Sorry for the confusion.
> >>
> >> I don't have /usr/sbin/s2ram.
> >
> > Ah. So it's gone from Factory? Interesting.
>
> I don't know, maybe it's in suspend package which I don't have installed.

It's there, yes.

2011-03-08 22:16:48

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

2011-03-08 22:26:49

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 03/08/2011 11:16 PM, Rafael J. Wysocki wrote:
> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>> On 03/08/2011 11:12 PM, Rafael J. Wysocki wrote:
>>> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>>>> On 03/08/2011 08:25 PM, Rafael J. Wysocki wrote:
>>>>> On Tuesday, March 08, 2011, Jiri Slaby wrote:
>>>>>> On 02/22/2011 04:36 PM, Alan Stern wrote:
>>>>>>> By the way, are you booting with no_console_suspend? And do you do
>>>>>>> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
>>>>>>> suspending?
>>>>>>
>>>>>> Unfortunately there are no messages at all.
>>>>>>
>>>>>> Rafael, this is a Factory with intel gfx (modesetting). Do pm-utils
>>>>>> switch a console in such case? And if yes, to which one?
>>>>>
>>>>> pm-utils don't, but s2ram does, I'm not sure from the top of my head to
>>>>> which one, though.
>>>>>
>>>>> If you use a KMS driver (I get you do), please just "chmod a-x /usr/sbin/s2ram".
>>>>
>>>> Yeah, I meant kernel modesetting, of course. Sorry for the confusion.
>>>>
>>>> I don't have /usr/sbin/s2ram.
>>>
>>> Ah. So it's gone from Factory? Interesting.
>>
>> I don't know, maybe it's in suspend package which I don't have installed.
>
> Hm. So you use the in-kernel hibernate code?
>
> The one in the suspend package is generally faster.

I don't use hibernation at all. This is a desktop -- suspending to S3.

--
js

2011-03-30 14:11:24

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On 02/22/2011 04:36 PM, Alan Stern wrote:
> On Tue, 22 Feb 2011, Jiri Slaby wrote:
>
>> On 02/04/2011 08:33 PM, Alan Stern wrote:
>>> On Fri, 4 Feb 2011, Jiri Slaby wrote:
>>>
>>>>>> It seems to suffice... No hangs still. Should I enable pm_async back
>>>>>> again to confirm the issue is still present?
>>>>>
>>>>> Yes, please, it would be good to know for sure.
>>>>
>>>> Ok, confirmed right now :).
>>>>
>>>> I disabled pm_async for USB again. What do you suggest next?
>>>
>>> What happens if you leave pm_async enabled but unplug all the USB
>>> devices before suspending? If there are any USB devices you can't
>>> unplug, you can get an equivalent result by unconfiguring the root
>>> hubs:
>>>
>>> for a in /sys/bus/usb/devices/usb* ; do
>>> echo 0 >$a/bConfigurationValue
>>> done
>>
>> This doesn't seem to help. The hang happened 3 times during the 2 weeks.
>>
>> I have /etc/pm/sleep.d/99usb-debug with:
>> #!/bin/bash
>>
>> send() {
>> for a in /sys/bus/usb/devices/usb* ; do
>> echo $1 >$a/bConfigurationValue
>> done
>> }
>>
>> case "$1" in
>> hibernate|suspend)
>> send 0
>> ;;
>> thaw|resume)
>> send 1
>> ;;
>> *)
>> ;;
>> esac
>>
>> exit 0
>>
>> And it indeed properly enable/disable usb before/after suspend.
>
> Strange indeed. It's worth noting that the async stuff affects only
> the normal suspend and resume operations, not the late-suspend and
> early-resume operations. This means that it all likelihood, the system
> crashes either before finishing the suspend or after doing a fair
> amount of the resume. And yet that's not consistent with what you see
> on the screen.
>
> By the way, are you booting with no_console_suspend? And do you do
> "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> suspending?

I'm testing this on my desktop. I found out yesterday, that I had async
completely disabled. So I have no output yet.

I took my dvb-t tuner from desktop and connected it to my notebook. And
got a similar hang during resume right now. It may not be related
though. The trace is:
s2disk D 0000000000000000 0 6125 5902 0x00000004
ffff8800797aba78 0000000000000082 ffff8800797ab9f8 0000000000012ac0
ffff8800797abfd8 0000000000012ac0 ffff8800797abfd8 ffff8800797abfd8
ffff8800777fe7f0 0000000000012ac0 0000000000012ac0 ffff8800797aa000
Call Trace:
[<ffffffff8151e2ed>] schedule_timeout+0x28d/0x310
[<ffffffff8151d410>] wait_for_common+0xc0/0x150
[<ffffffff8133ad22>] _request_firmware+0x132/0x270
[<ffffffffa06df2c7>] dvb_usb_download_firmware+0x37/0xf0 [dvb_usb]
[<ffffffffa06dfbe5>] dvb_usb_device_init+0x165/0x1b0 [dvb_usb]
[<ffffffffa06d6c27>] af9015_usb_probe+0x87/0xa0 [dvb_usb_af9015]
[<ffffffff8138d9d2>] usb_probe_interface+0x142/0x270
[<ffffffff8132f3c0>] really_probe+0x70/0x220
[<ffffffff8132f757>] driver_probe_device+0x47/0xa0
[<ffffffff8132e19c>] bus_for_each_drv+0x5c/0x90
[<ffffffff8132f62f>] device_attach+0x8f/0xb0
[<ffffffff8138d5b0>] usb_rebind_intf+0x60/0xa0
[<ffffffff8138d6c7>] do_unbind_rebind+0x77/0xb0
[<ffffffff8138d76b>] usb_resume+0x6b/0xb0
[<ffffffff8137f96b>] usb_dev_complete+0xb/0x10
[<ffffffff81334d1e>] device_complete+0x8e/0xe0
[<ffffffff81335da1>] dpm_resume_end+0xa1/0x120
[<ffffffff8109d890>] hibernation_snapshot+0xc0/0x120
[<ffffffff810a1cde>] snapshot_ioctl+0x3ae/0x590
[<ffffffff81169794>] do_vfs_ioctl+0x84/0x2f0
[<ffffffff81169a98>] sys_ioctl+0x98/0xa0
[<ffffffff81002ed2>] system_call_fastpath+0x16/0x1b
[<00007fe6c82f8ce7>] 0x7fe6c82f8ce7

The firmware request should time out. I think I waited for longer than
60 s before when I saw the hangs on the desktop, But do you have an idea
why it doesn't load the FW immediately?

The device is:
Bus 001 Device 010: ID 0413:6029 Leadtek Research, Inc. WinFast DTV
Dongle Gold
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x0413 Leadtek Research, Inc.
idProduct 0x6029 WinFast DTV Dongle Gold
bcdDevice 2.00
iManufacturer 1 Leadtek
iProduct 2 WinFast DTV Dongle Gold
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 71
bNumInterfaces 2
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 4
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x84 EP 4 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x85 EP 5 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 3 Human Interface Device
bInterfaceSubClass 0 No Subclass
bInterfaceProtocol 1 Keyboard
iInterface 0
HID Device Descriptor:
bLength 9
bDescriptorType 33
bcdHID 1.01
bCountryCode 0 Not supported
bNumDescriptors 1
bDescriptorType 34 Report
wDescriptorLength 65
Report Descriptors:
** UNAVAILABLE **
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 16
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0000
(Bus Powered)

thanks,
--
js

2011-03-30 15:10:19

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Wed, 30 Mar 2011, Jiri Slaby wrote:

> > Strange indeed. It's worth noting that the async stuff affects only
> > the normal suspend and resume operations, not the late-suspend and
> > early-resume operations. This means that it all likelihood, the system
> > crashes either before finishing the suspend or after doing a fair
> > amount of the resume. And yet that's not consistent with what you see
> > on the screen.
> >
> > By the way, are you booting with no_console_suspend? And do you do
> > "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> > suspending?
>
> I'm testing this on my desktop. I found out yesterday, that I had async
> completely disabled. So I have no output yet.
>
> I took my dvb-t tuner from desktop and connected it to my notebook. And
> got a similar hang during resume right now. It may not be related
> though. The trace is:
> s2disk D 0000000000000000 0 6125 5902 0x00000004
> ffff8800797aba78 0000000000000082 ffff8800797ab9f8 0000000000012ac0
> ffff8800797abfd8 0000000000012ac0 ffff8800797abfd8 ffff8800797abfd8
> ffff8800777fe7f0 0000000000012ac0 0000000000012ac0 ffff8800797aa000
> Call Trace:
> [<ffffffff8151e2ed>] schedule_timeout+0x28d/0x310
> [<ffffffff8151d410>] wait_for_common+0xc0/0x150
> [<ffffffff8133ad22>] _request_firmware+0x132/0x270
> [<ffffffffa06df2c7>] dvb_usb_download_firmware+0x37/0xf0 [dvb_usb]
> [<ffffffffa06dfbe5>] dvb_usb_device_init+0x165/0x1b0 [dvb_usb]
> [<ffffffffa06d6c27>] af9015_usb_probe+0x87/0xa0 [dvb_usb_af9015]
> [<ffffffff8138d9d2>] usb_probe_interface+0x142/0x270
> [<ffffffff8132f3c0>] really_probe+0x70/0x220
> [<ffffffff8132f757>] driver_probe_device+0x47/0xa0
> [<ffffffff8132e19c>] bus_for_each_drv+0x5c/0x90
> [<ffffffff8132f62f>] device_attach+0x8f/0xb0
> [<ffffffff8138d5b0>] usb_rebind_intf+0x60/0xa0
> [<ffffffff8138d6c7>] do_unbind_rebind+0x77/0xb0
> [<ffffffff8138d76b>] usb_resume+0x6b/0xb0
> [<ffffffff8137f96b>] usb_dev_complete+0xb/0x10
> [<ffffffff81334d1e>] device_complete+0x8e/0xe0
> [<ffffffff81335da1>] dpm_resume_end+0xa1/0x120
> [<ffffffff8109d890>] hibernation_snapshot+0xc0/0x120
> [<ffffffff810a1cde>] snapshot_ioctl+0x3ae/0x590
> [<ffffffff81169794>] do_vfs_ioctl+0x84/0x2f0
> [<ffffffff81169a98>] sys_ioctl+0x98/0xa0
> [<ffffffff81002ed2>] system_call_fastpath+0x16/0x1b
> [<00007fe6c82f8ce7>] 0x7fe6c82f8ce7
>
>
> The firmware request should time out. I think I waited for longer than
> 60 s before when I saw the hangs on the desktop, But do you have an idea
> why it doesn't load the FW immediately?

I don't know why the request didn't time out, but this behavior should
at least be reproducible. Your stack trace implies that the firmware
will need to be transferred every time the device resumes... but of
course this leaves open the question of why your desktop crashed only
occasionally.

Probably the firmware wasn't transferred immediately because the kernel
relies on a userspace process to find and load the firmware, but at
that point userspace was still frozen.

Rafael, this does seem to be a general problem. Suppose a new device
gets connected while the system is suspended. How is the driver's
probe method supposed to cope if it gets called while userspace is
still frozen but it needs to load some firmware?

Is the answer that probing should always be delayed until after tasks
are thawed? There must be lots of subsystems which would have trouble
doing that, although we ought to be able to implement delayed probing
from within the driver core easily enough.

Alan Stern

2011-03-30 20:01:29

[permalink] [raw]

Subject: Re: [linux-pm] Resume hangs [was: mmotm 2010-12-02-16-34 uploaded]

On Wednesday, March 30, 2011, Alan Stern wrote:
> On Wed, 30 Mar 2011, Jiri Slaby wrote:
>
> > > Strange indeed. It's worth noting that the async stuff affects only
> > > the normal suspend and resume operations, not the late-suspend and
> > > early-resume operations. This means that it all likelihood, the system
> > > crashes either before finishing the suspend or after doing a fair
> > > amount of the resume. And yet that's not consistent with what you see
> > > on the screen.
> > >
> > > By the way, are you booting with no_console_suspend? And do you do
> > > "echo 8 >/proc/sys/kernel/printk" (or equivalently, Alt-SysRq-8) before
> > > suspending?
> >
> > I'm testing this on my desktop. I found out yesterday, that I had async
> > completely disabled. So I have no output yet.
> >
> > I took my dvb-t tuner from desktop and connected it to my notebook. And
> > got a similar hang during resume right now. It may not be related
> > though. The trace is:
> > s2disk D 0000000000000000 0 6125 5902 0x00000004
> > ffff8800797aba78 0000000000000082 ffff8800797ab9f8 0000000000012ac0
> > ffff8800797abfd8 0000000000012ac0 ffff8800797abfd8 ffff8800797abfd8
> > ffff8800777fe7f0 0000000000012ac0 0000000000012ac0 ffff8800797aa000
> > Call Trace:
> > [<ffffffff8151e2ed>] schedule_timeout+0x28d/0x310
> > [<ffffffff8151d410>] wait_for_common+0xc0/0x150
> > [<ffffffff8133ad22>] _request_firmware+0x132/0x270
> > [<ffffffffa06df2c7>] dvb_usb_download_firmware+0x37/0xf0 [dvb_usb]
> > [<ffffffffa06dfbe5>] dvb_usb_device_init+0x165/0x1b0 [dvb_usb]
> > [<ffffffffa06d6c27>] af9015_usb_probe+0x87/0xa0 [dvb_usb_af9015]
> > [<ffffffff8138d9d2>] usb_probe_interface+0x142/0x270
> > [<ffffffff8132f3c0>] really_probe+0x70/0x220
> > [<ffffffff8132f757>] driver_probe_device+0x47/0xa0
> > [<ffffffff8132e19c>] bus_for_each_drv+0x5c/0x90
> > [<ffffffff8132f62f>] device_attach+0x8f/0xb0
> > [<ffffffff8138d5b0>] usb_rebind_intf+0x60/0xa0
> > [<ffffffff8138d6c7>] do_unbind_rebind+0x77/0xb0
> > [<ffffffff8138d76b>] usb_resume+0x6b/0xb0
> > [<ffffffff8137f96b>] usb_dev_complete+0xb/0x10
> > [<ffffffff81334d1e>] device_complete+0x8e/0xe0
> > [<ffffffff81335da1>] dpm_resume_end+0xa1/0x120
> > [<ffffffff8109d890>] hibernation_snapshot+0xc0/0x120
> > [<ffffffff810a1cde>] snapshot_ioctl+0x3ae/0x590
> > [<ffffffff81169794>] do_vfs_ioctl+0x84/0x2f0
> > [<ffffffff81169a98>] sys_ioctl+0x98/0xa0
> > [<ffffffff81002ed2>] system_call_fastpath+0x16/0x1b
> > [<00007fe6c82f8ce7>] 0x7fe6c82f8ce7
> >
> >
> > The firmware request should time out. I think I waited for longer than
> > 60 s before when I saw the hangs on the desktop, But do you have an idea
> > why it doesn't load the FW immediately?
>
> I don't know why the request didn't time out, but this behavior should
> at least be reproducible. Your stack trace implies that the firmware
> will need to be transferred every time the device resumes... but of
> course this leaves open the question of why your desktop crashed only
> occasionally.
>
> Probably the firmware wasn't transferred immediately because the kernel
> relies on a userspace process to find and load the firmware, but at
> that point userspace was still frozen.
>
> Rafael, this does seem to be a general problem. Suppose a new device
> gets connected while the system is suspended. How is the driver's
> probe method supposed to cope if it gets called while userspace is
> still frozen but it needs to load some firmware?
>
> Is the answer that probing should always be delayed until after tasks
> are thawed? There must be lots of subsystems which would have trouble
> doing that, although we ought to be able to implement delayed probing
> from within the driver core easily enough.

I don't know how to solve that.

First off, the prepare routines of drivers that need to load firmware during
resume probably should preload the firmware for the devices present during the
suspend.

Second, I don't generally think drivers should probe for devices during resume.
I guess the delayed probing would be the cleanest appoach here.

Thanks,
Rafael

2011-03-30 21:20:01