[Please Cc me on any followups]
Hi,
Suspend-to-RAM works fine on my new Dell Latitude D420 (with Core Duo) in
2.6.16, but it broke in 2.6.17 -- the machine suspends just fine, but when it
resumes, the disk never spins up, the screen stays black and it just hangs.
Bisecting shows that the following commit is where it broke:
commit 78eef01b0fae087c5fadbd85dd4fe2918c3a015f
Author: Andrew Morton <[email protected]>
Date: Wed Mar 22 00:08:16 2006 -0800
[PATCH] on_each_cpu(): disable local interrupts
When on_each_cpu() runs the callback on other CPUs, it runs with local
interrupts disabled. So we should run the function with local interrupts
disabled on this CPU, too.
And do the same for UP, so the callback is run in the same environment on both
UP and SMP. (strictly it should do preempt_disable() too, but I think
local_irq_disable is sufficiently equivalent).
Also uninlines on_each_cpu(). softirq.c was the most appropriate file I could
find, but it doesn't seem to justify creating a new file.
Oh, and fix up that comment over (under?) x86's smp_call_function(). It
drives me nuts.
Applying the patch in reverse against 2.6.17 (it doesn't apply cleanly, but
I've done what seems to be the moral equivalent) makes the suspend work
again.
Any ideas? It does not work with the latest git checkout as of today.
/* Steinar */
--
Homepage: http://www.sesse.net/
On Friday 04 August 2006 18:23, Steinar H. Gunderson wrote:
> [Please Cc me on any followups]
>
> Hi,
>
> Suspend-to-RAM works fine on my new Dell Latitude D420 (with Core Duo) in
> 2.6.16, but it broke in 2.6.17 -- the machine suspends just fine, but when it
> resumes, the disk never spins up, the screen stays black and it just hangs.
> Bisecting shows that the following commit is where it broke:
>
> commit 78eef01b0fae087c5fadbd85dd4fe2918c3a015f
> Author: Andrew Morton <[email protected]>
> Date: Wed Mar 22 00:08:16 2006 -0800
>
> [PATCH] on_each_cpu(): disable local interrupts
>
> When on_each_cpu() runs the callback on other CPUs, it runs with local
> interrupts disabled. So we should run the function with local interrupts
> disabled on this CPU, too.
>
> And do the same for UP, so the callback is run in the same environment on both
> UP and SMP. (strictly it should do preempt_disable() too, but I think
> local_irq_disable is sufficiently equivalent).
>
> Also uninlines on_each_cpu(). softirq.c was the most appropriate file I could
> find, but it doesn't seem to justify creating a new file.
>
> Oh, and fix up that comment over (under?) x86's smp_call_function(). It
> drives me nuts.
>
> Applying the patch in reverse against 2.6.17 (it doesn't apply cleanly, but
> I've done what seems to be the moral equivalent) makes the suspend work
> again.
>
> Any ideas? It does not work with the latest git checkout as of today.
I guess the patch may interfere with the CPU hotplug badly. Could you please
check if you can take CPU1 offline/online?
Rafael
On Fri, 4 Aug 2006 23:27:38 +0200
"Rafael J. Wysocki" <[email protected]> wrote:
> On Friday 04 August 2006 18:23, Steinar H. Gunderson wrote:
> > [Please Cc me on any followups]
> >
> > Hi,
> >
> > Suspend-to-RAM works fine on my new Dell Latitude D420 (with Core Duo) in
> > 2.6.16, but it broke in 2.6.17 -- the machine suspends just fine, but when it
> > resumes, the disk never spins up, the screen stays black and it just hangs.
> > Bisecting shows that the following commit is where it broke:
> >
> > commit 78eef01b0fae087c5fadbd85dd4fe2918c3a015f
> > Author: Andrew Morton <[email protected]>
> > Date: Wed Mar 22 00:08:16 2006 -0800
> >
> > [PATCH] on_each_cpu(): disable local interrupts
> >
> > When on_each_cpu() runs the callback on other CPUs, it runs with local
> > interrupts disabled. So we should run the function with local interrupts
> > disabled on this CPU, too.
> >
> > And do the same for UP, so the callback is run in the same environment on both
> > UP and SMP. (strictly it should do preempt_disable() too, but I think
> > local_irq_disable is sufficiently equivalent).
> >
> > Also uninlines on_each_cpu(). softirq.c was the most appropriate file I could
> > find, but it doesn't seem to justify creating a new file.
> >
> > Oh, and fix up that comment over (under?) x86's smp_call_function(). It
> > drives me nuts.
> >
> > Applying the patch in reverse against 2.6.17 (it doesn't apply cleanly, but
> > I've done what seems to be the moral equivalent) makes the suspend work
> > again.
> >
> > Any ideas? It does not work with the latest git checkout as of today.
>
> I guess the patch may interfere with the CPU hotplug badly.
Why do you think it would do that?
> Could you please
> check if you can take CPU1 offline/online?
If something really wants "disable irqs on the other CPUs but not on this
CPU" semantics then it would need to use smp_call_function and a direct
call. But it would be a strange thing to want to do, surely?
On Saturday 05 August 2006 00:17, Andrew Morton wrote:
> On Fri, 4 Aug 2006 23:27:38 +0200
> "Rafael J. Wysocki" <[email protected]> wrote:
>
> > On Friday 04 August 2006 18:23, Steinar H. Gunderson wrote:
> > > [Please Cc me on any followups]
> > >
> > > Hi,
> > >
> > > Suspend-to-RAM works fine on my new Dell Latitude D420 (with Core Duo) in
> > > 2.6.16, but it broke in 2.6.17 -- the machine suspends just fine, but when it
> > > resumes, the disk never spins up, the screen stays black and it just hangs.
> > > Bisecting shows that the following commit is where it broke:
> > >
> > > commit 78eef01b0fae087c5fadbd85dd4fe2918c3a015f
> > > Author: Andrew Morton <[email protected]>
> > > Date: Wed Mar 22 00:08:16 2006 -0800
> > >
> > > [PATCH] on_each_cpu(): disable local interrupts
> > >
> > > When on_each_cpu() runs the callback on other CPUs, it runs with local
> > > interrupts disabled. So we should run the function with local interrupts
> > > disabled on this CPU, too.
> > >
> > > And do the same for UP, so the callback is run in the same environment on both
> > > UP and SMP. (strictly it should do preempt_disable() too, but I think
> > > local_irq_disable is sufficiently equivalent).
> > >
> > > Also uninlines on_each_cpu(). softirq.c was the most appropriate file I could
> > > find, but it doesn't seem to justify creating a new file.
> > >
> > > Oh, and fix up that comment over (under?) x86's smp_call_function(). It
> > > drives me nuts.
> > >
> > > Applying the patch in reverse against 2.6.17 (it doesn't apply cleanly, but
> > > I've done what seems to be the moral equivalent) makes the suspend work
> > > again.
> > >
> > > Any ideas? It does not work with the latest git checkout as of today.
> >
> > I guess the patch may interfere with the CPU hotplug badly.
>
> Why do you think it would do that?
Because the non-boot CPUs are taken off early, before anything else, and the
system is effectively non-SMP during the entire suspend-resume cycle
(well, almost). If SMP-related things go wrong during the suspend, CPU
hotplug is the first suspect. ;-)
> > Could you please
> > check if you can take CPU1 offline/online?
>
> If something really wants "disable irqs on the other CPUs but not on this
> CPU" semantics then it would need to use smp_call_function and a direct
> call. But it would be a strange thing to want to do, surely?
Yes, it would, but I have a little experience with these things.
Well, looks like on_each_cpu() is run via flush_tlb_all() from
__smp_prepare_cpu() which is called by __cpu_up() and that's used by the CPU
hotplug. Not that I can tell what goes wrong here, if anything.
Greetings,
Rafael
On Sat, Aug 05, 2006 at 01:26:56AM +0200, Rafael J. Wysocki wrote:
> Because the non-boot CPUs are taken off early, before anything else, and the
> system is effectively non-SMP during the entire suspend-resume cycle
> (well, almost). If SMP-related things go wrong during the suspend, CPU
> hotplug is the first suspect. ;-)
Well, it seems to work fine during the suspend phase, at least, and it also
seems to work well during normal use:
fugl:~# echo 0 > /sys/devices/system/cpu/cpu1/online
fugl:~# echo 1 > /sys/devices/system/cpu/cpu1/online
FWIW, there is an error in dmesg afterwards, though:
===
CPU 1 is now offline
SMP alternatives: switching to UP code
SMP alternatives: switching to SMP code
Booting processor 1/1 eip 3000
Initializing CPU#1
Calibrating delay using timer specific routine.. 2394.85 BogoMIPS (lpj=4789709)
CPU: After generic identify, caps: bfe9fbff 00100000 00000000 00000000 0000c1a9 00000000 00000000
CPU: After vendor identify, caps: bfe9fbff 00100000 00000000 00000000 0000c1a9 00000000 00000000
monitor/mwait feature present.
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU: After all inits, caps: bfe9fbff 00100000 00000000 00000140 0000c1a9 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel Genuine Intel(R) CPU U2500 @ 1.20GHz stepping 08
APIC error on CPU1: 00(40)
===
No idea whether it's related. FWIW, resume didn't work with maxcpus=1 on boot
either, so I'm not really sure how related it is.
/* Steinar */
--
Homepage: http://www.sesse.net/
On Saturday 05 August 2006 10:23, Steinar H. Gunderson wrote:
> On Sat, Aug 05, 2006 at 01:26:56AM +0200, Rafael J. Wysocki wrote:
> > Because the non-boot CPUs are taken off early, before anything else, and the
> > system is effectively non-SMP during the entire suspend-resume cycle
> > (well, almost). If SMP-related things go wrong during the suspend, CPU
> > hotplug is the first suspect. ;-)
>
> Well, it seems to work fine during the suspend phase, at least, and it also
> seems to work well during normal use:
>
> fugl:~# echo 0 > /sys/devices/system/cpu/cpu1/online
> fugl:~# echo 1 > /sys/devices/system/cpu/cpu1/online
>
> FWIW, there is an error in dmesg afterwards, though:
>
> ===
> CPU 1 is now offline
> SMP alternatives: switching to UP code
> SMP alternatives: switching to SMP code
> Booting processor 1/1 eip 3000
> Initializing CPU#1
> Calibrating delay using timer specific routine.. 2394.85 BogoMIPS (lpj=4789709)
> CPU: After generic identify, caps: bfe9fbff 00100000 00000000 00000000 0000c1a9 00000000 00000000
> CPU: After vendor identify, caps: bfe9fbff 00100000 00000000 00000000 0000c1a9 00000000 00000000
> monitor/mwait feature present.
> CPU: L1 I cache: 32K, L1 D cache: 32K
> CPU: L2 cache: 2048K
> CPU: Physical Processor ID: 0
> CPU: Processor Core ID: 1
> CPU: After all inits, caps: bfe9fbff 00100000 00000000 00000140 0000c1a9 00000000 00000000
> Intel machine check architecture supported.
> Intel machine check reporting enabled on CPU#1.
> CPU1: Intel Genuine Intel(R) CPU U2500 @ 1.20GHz stepping 08
> APIC error on CPU1: 00(40)
> ===
>
> No idea whether it's related. FWIW, resume didn't work with maxcpus=1 on boot
> either, so I'm not really sure how related it is.
Hm, could you please try it with a non-SMP kernel?
Rafael
On Sat, Aug 05, 2006 at 11:08:00AM +0200, Rafael J. Wysocki wrote:
>> No idea whether it's related. FWIW, resume didn't work with maxcpus=1 on boot
>> either, so I'm not really sure how related it is.
> Hm, could you please try it with a non-SMP kernel?
Compiling straight 2.6.17 with CONFIG_SMP=n suspends and resumes just fine.
/* Steinar */
--
Homepage: http://www.sesse.net/
Hi!
> >> No idea whether it's related. FWIW, resume didn't work with maxcpus=1 on boot
> >> either, so I'm not really sure how related it is.
> > Hm, could you please try it with a non-SMP kernel?
>
> Compiling straight 2.6.17 with CONFIG_SMP=n suspends and resumes just fine.
Interesting... I guess getting some dump where it hangs is not an
option? Does video work for you during resume?
Can you try minimal drivers? It works for other people (smp too), so
it may be driver problem (althrough it looks unlikely)
Pavel
--
Thanks for all the (sleeping) penguins.
On Sun, Aug 06, 2006 at 11:10:13PM +0000, Pavel Machek wrote:
> Interesting... I guess getting some dump where it hangs is not an
> option? Does video work for you during resume?
Define "getting some dump". It doesn't react to anything, so I don't really
think I can easily get anything out.
When the resume works (ie. with 2.6.16, with patched 2.6.17, or with 2.6.17
without SMP), video works, but only with the -s and -p flags to s2ram (sav
VBE state and POST the video card, respectively).
> Can you try minimal drivers? It works for other people (smp too), so
> it may be driver problem (althrough it looks unlikely)
I've tried from init=/bin/bash on an initramfs (which should be about the
minimal you can get); no change in behaviour. (I've already blacklisted
sdhci, since I got a tip it might be bad and I don't really need it; if I get
this to work, I'll try whitelisting it again and see if resume works with it,
but that's a different story.)
/* Steinar */
--
Homepage: http://www.sesse.net/
On Sunday 06 August 2006 13:50, Steinar H. Gunderson wrote:
> On Sat, Aug 05, 2006 at 11:08:00AM +0200, Rafael J. Wysocki wrote:
> >> No idea whether it's related. FWIW, resume didn't work with maxcpus=1 on boot
> >> either, so I'm not really sure how related it is.
> > Hm, could you please try it with a non-SMP kernel?
>
> Compiling straight 2.6.17 with CONFIG_SMP=n suspends and resumes just fine.
Sorry for the delay, there have been some problems with -rc3-mm2 recently.
Please apply the appended patch to the SMP kernel and try the following:
(1)
# echo testproc > /sys/power/disk
# echo disk > /sys/power/state
This should turn off the non-boot CPU, freeze all processes, wait for 5
seconds and thaw the processes and the CPU.
(2)
# echo test > /sys/power/disk
# echo disk > /sys/power/state
This should turn off the non-boot CPU, freeze all processes, shrink
memory, suspend all devices, wait for 5 seconds, resume the devices etc.
IOW it does everything that's needed for a suspend except for actually
suspending.
I think (1) will work and (2) will not, but let's see. :-)
Greetings,
Rafael
---
Add some tools for testing the suspend code.
Signed-off-by: Rafael J. Wysocki <[email protected]>
---
include/linux/pm.h | 4 +++-
kernel/power/disk.c | 35 +++++++++++++++++++++++++++--------
2 files changed, 30 insertions(+), 9 deletions(-)
Index: linux-2.6.18-rc2-mm1/include/linux/pm.h
===================================================================
--- linux-2.6.18-rc2-mm1.orig/include/linux/pm.h 2006-07-28 08:36:24.000000000 +0200
+++ linux-2.6.18-rc2-mm1/include/linux/pm.h 2006-08-07 09:07:12.000000000 +0200
@@ -116,7 +116,9 @@ typedef int __bitwise suspend_disk_metho
#define PM_DISK_PLATFORM ((__force suspend_disk_method_t) 2)
#define PM_DISK_SHUTDOWN ((__force suspend_disk_method_t) 3)
#define PM_DISK_REBOOT ((__force suspend_disk_method_t) 4)
-#define PM_DISK_MAX ((__force suspend_disk_method_t) 5)
+#define PM_DISK_TEST ((__force suspend_disk_method_t) 5)
+#define PM_DISK_TESTPROC ((__force suspend_disk_method_t) 6)
+#define PM_DISK_MAX ((__force suspend_disk_method_t) 7)
struct pm_ops {
suspend_disk_method_t pm_disk_mode;
Index: linux-2.6.18-rc2-mm1/kernel/power/disk.c
===================================================================
--- linux-2.6.18-rc2-mm1.orig/kernel/power/disk.c 2006-07-30 16:55:17.000000000 +0200
+++ linux-2.6.18-rc2-mm1/kernel/power/disk.c 2006-08-07 09:37:01.000000000 +0200
@@ -83,6 +83,12 @@ static int prepare_processes(void)
goto thaw;
}
+ if (pm_disk_mode == PM_DISK_TESTPROC) {
+ printk("swsusp debug: Waiting for 5 seconds.\n");
+ mdelay(5000);
+ goto thaw;
+ }
+
/* Free memory before shutting down devices. */
if (!(error = swsusp_shrink_memory()))
return 0;
@@ -119,11 +125,19 @@ int pm_suspend_disk(void)
if (error)
return error;
+ if (pm_disk_mode == PM_DISK_TESTPROC)
+ goto Thaw;
+
error = device_suspend(PMSG_FREEZE);
if (error) {
printk("Some devices failed to suspend\n");
- unprepare_processes();
- return error;
+ goto Thaw;
+ }
+
+ if (pm_disk_mode == PM_DISK_TEST) {
+ printk("swsusp debug: Waiting for 5 seconds.\n");
+ mdelay(5000);
+ goto Done;
}
pr_debug("PM: snapshotting memory.\n");
@@ -139,15 +153,16 @@ int pm_suspend_disk(void)
power_down(pm_disk_mode);
else {
swsusp_free();
- unprepare_processes();
- return error;
+ goto Thaw;
}
- } else
+ } else {
pr_debug("PM: Image restored successfully.\n");
+ }
swsusp_free();
Done:
device_resume();
+ Thaw:
unprepare_processes();
return error;
}
@@ -241,6 +256,8 @@ static const char * const pm_disk_modes[
[PM_DISK_PLATFORM] = "platform",
[PM_DISK_SHUTDOWN] = "shutdown",
[PM_DISK_REBOOT] = "reboot",
+ [PM_DISK_TEST] = "test",
+ [PM_DISK_TESTPROC] = "testproc",
};
/**
@@ -295,17 +312,19 @@ static ssize_t disk_store(struct subsyst
}
}
if (mode) {
- if (mode == PM_DISK_SHUTDOWN || mode == PM_DISK_REBOOT)
+ if (mode == PM_DISK_SHUTDOWN || mode == PM_DISK_REBOOT ||
+ mode == PM_DISK_TEST || mode == PM_DISK_TESTPROC) {
pm_disk_mode = mode;
- else {
+ } else {
if (pm_ops && pm_ops->enter &&
(mode == pm_ops->pm_disk_mode))
pm_disk_mode = mode;
else
error = -EINVAL;
}
- } else
+ } else {
error = -EINVAL;
+ }
pr_debug("PM: suspend-to-disk mode set to '%s'\n",
pm_disk_modes[mode]);
On Tue, Aug 08, 2006 at 04:04:00PM +0200, Rafael J. Wysocki wrote:
> Please apply the appended patch to the SMP kernel and try the following:
>
> [...]
>
> I think (1) will work and (2) will not, but let's see. :-)
Actually, both worked just fine. The first one (testproc) gave me EPERM on
the actual write call according to echo, but I guess that's just a side
effect of sloppy test code :-)
/* Steinar */
--
Homepage: http://www.sesse.net/
On Tuesday 08 August 2006 17:01, Steinar H. Gunderson wrote:
> On Tue, Aug 08, 2006 at 04:04:00PM +0200, Rafael J. Wysocki wrote:
> > Please apply the appended patch to the SMP kernel and try the following:
> >
> > [...]
> >
> > I think (1) will work and (2) will not, but let's see. :-)
>
> Actually, both worked just fine. The first one (testproc) gave me EPERM on
> the actual write call according to echo, but I guess that's just a side
> effect of sloppy test code :-)
Oh, I just forgot to initialize error in kernel/power/disk.c#prepare_processes.c .
Sorry.
However, this means the drivers' suspend and resume routines seem to work fine
and the problem is somehow related to the BIOS black magic that happens
during the "real" suspend.
No idea what to do next. :-(
Rafael
On Tue 2006-08-08 17:41:23, Rafael J. Wysocki wrote:
> On Tuesday 08 August 2006 17:01, Steinar H. Gunderson wrote:
> > On Tue, Aug 08, 2006 at 04:04:00PM +0200, Rafael J. Wysocki wrote:
> > > Please apply the appended patch to the SMP kernel and try the following:
> > >
> > > [...]
> > >
> > > I think (1) will work and (2) will not, but let's see. :-)
> >
> > Actually, both worked just fine. The first one (testproc) gave me EPERM on
> > the actual write call according to echo, but I guess that's just a side
> > effect of sloppy test code :-)
>
> Oh, I just forgot to initialize error in kernel/power/disk.c#prepare_processes.c .
> Sorry.
>
> However, this means the drivers' suspend and resume routines seem to work fine
> and the problem is somehow related to the BIOS black magic that happens
> during the "real" suspend.
>
> No idea what to do next. :-(
(Can we get bugzilla entry? I somehow lost track).
Can you try with method=powerdown or method=reboot? BIOS black magic
is not involved at least in reboot parts...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Wed, Aug 09, 2006 at 02:21:59AM +0200, Pavel Machek wrote:
> Can you try with method=powerdown or method=reboot? BIOS black magic
> is not involved at least in reboot parts...
I lost you here; I've never heard of these options, nor does Google seem to.
Do I specify them on the kernel command line, or something else?
/* Steinar */
--
Homepage: http://www.sesse.net/
Hi!
> > Can you try with method=powerdown or method=reboot? BIOS black magic
> > is not involved at least in reboot parts...
>
> I lost you here; I've never heard of these options, nor does Google seem to.
> Do I specify them on the kernel command line, or something else?
echo reboot > /sys/power/disk
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Wed, Aug 09, 2006 at 11:02:08AM +0200, Pavel Machek wrote:
>> I lost you here; I've never heard of these options, nor does Google seem to.
>> Do I specify them on the kernel command line, or something else?
> echo reboot > /sys/power/disk
But we were discussing suspend-to-RAM, not suspend-to-disk, right?
Suspend-to-disk works just fine.
/* Steinar */
--
Homepage: http://www.sesse.net/
Hi!
> >> I lost you here; I've never heard of these options, nor does Google seem to.
> >> Do I specify them on the kernel command line, or something else?
> > echo reboot > /sys/power/disk
>
> But we were discussing suspend-to-RAM, not suspend-to-disk, right?
> Suspend-to-disk works just fine.
Aha, too bad, I was confused. Thats bugzilla.kernel.org, acpi
category, I guess.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
On Wed, Aug 09, 2006 at 11:08:04AM +0200, Pavel Machek wrote:
> Aha, too bad, I was confused. Thats bugzilla.kernel.org, acpi
> category, I guess.
Entered: http://bugzilla.kernel.org/show_bug.cgi?id=6977
/* Steinar */
--
Homepage: http://www.sesse.net/
Hi!
> > Interesting... I guess getting some dump where it hangs is not an
> > option? Does video work for you during resume?
>
> Define "getting some dump". It doesn't react to anything, so I don't really
> think I can easily get anything out.
With some luck, you'll still get some messages before it hangs.. on
for example serial console.
--
Thanks for all the (sleeping) penguins.