2007-09-08 18:45:04

by Nicolas Capit

[permalink] [raw]
Subject: cpuset trouble after hibernate

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Thank you for your attention,
Nicolas Capit


Note on my system:
- laptop HP dv2000 with a Turion64x2
- kernel : Linux 2.6.22-1-686 #1 SMP Sun Jul 29 14:37:42 UTC
2007 i686 GNU/Linux


2007-09-10 09:45:39

by Simon Derr

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

On Sat, 8 Sep 2007, Nicolas Capit wrote:

> Hello,
>
> This is my situation:
> - I mounted the pseudo cpuset filesystem into /dev/cpuset
> - I created a cpuset named oar with my 2 cpus
>
> cat /dev/cpuset/oar/cpus
> 0-1
>
> - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> - After reboot:
>
> cat /dev/cpuset/oar/cpus
> 0
>
> Why did I lost a cpu?
> Is this a normal behavior???

Hi Nicolas,

I believe this is related to the fact that hibernation uses the hotplug
subsystem to disable all CPUs except the boot CPU.

Thus guarantee_online_cpus() is called on each cpuset and removes all
CPUs, except CPU 0, from all cpusets.

I'm not quite sure about if/how this should be fixed in the kernel,
though. Looks like a very simple user-land workaround would be enough.

Simon.

2007-09-10 09:56:20

by Nicolas Capit

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

On Mon, 10 Sep 2007 11:45:10 +0200 (CEST)
Simon Derr <[email protected]> wrote:

> On Sat, 8 Sep 2007, Nicolas Capit wrote:
>
> > Hello,
> >
> > This is my situation:
> > - I mounted the pseudo cpuset filesystem into /dev/cpuset
> > - I created a cpuset named oar with my 2 cpus
> >
> > cat /dev/cpuset/oar/cpus
> > 0-1
> >
> > - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> > - After reboot:
> >
> > cat /dev/cpuset/oar/cpus
> > 0
> >
> > Why did I lost a cpu?
> > Is this a normal behavior???
>
> Hi Nicolas,
>
> I believe this is related to the fact that hibernation uses the hotplug
> subsystem to disable all CPUs except the boot CPU.
>
> Thus guarantee_online_cpus() is called on each cpuset and removes all
> CPUs, except CPU 0, from all cpusets.
>
> I'm not quite sure about if/how this should be fixed in the kernel,
> though. Looks like a very simple user-land workaround would be enough.
>
> Simon.

Ok, it is not a big deal for me (you don't hibernate computers in a
cluster...) but I wanted to bring you this issue in case of a similar
behavior in another case...

Thanks for your response.

Nicolas Capit.

2007-09-15 08:09:34

by Andrew Morton

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <[email protected]> wrote:

> On Sat, 8 Sep 2007, Nicolas Capit wrote:
>
> > Hello,
> >
> > This is my situation:
> > - I mounted the pseudo cpuset filesystem into /dev/cpuset
> > - I created a cpuset named oar with my 2 cpus
> >
> > cat /dev/cpuset/oar/cpus
> > 0-1
> >
> > - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> > - After reboot:
> >
> > cat /dev/cpuset/oar/cpus
> > 0
> >
> > Why did I lost a cpu?
> > Is this a normal behavior???
>
> Hi Nicolas,
>
> I believe this is related to the fact that hibernation uses the hotplug
> subsystem to disable all CPUs except the boot CPU.
>
> Thus guarantee_online_cpus() is called on each cpuset and removes all
> CPUs, except CPU 0, from all cpusets.
>
> I'm not quite sure about if/how this should be fixed in the kernel,
> though. Looks like a very simple user-land workaround would be enough.
>

Yeah. Bug, surely. But I guess it's always been there.

What are the implications of this for cpusets-via-containers?

2007-09-17 09:41:35

by Pavel Machek

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

Hi!

> This is my situation:
> - I mounted the pseudo cpuset filesystem into /dev/cpuset
> - I created a cpuset named oar with my 2 cpus
>
> cat /dev/cpuset/oar/cpus
> 0-1
>
> - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> - After reboot:
>
> cat /dev/cpuset/oar/cpus
> 0
>
> Why did I lost a cpu?
> Is this a normal behavior???

One of the cpus was unplugged during suspend... perhaps some
save/restore is needed during hotplug/unplug?

Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

2007-09-17 18:04:49

by Paul Menage

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

On 9/15/07, Andrew Morton <[email protected]> wrote:
>
> Yeah. Bug, surely. But I guess it's always been there.
>
> What are the implications of this for cpusets-via-containers?
>

I don't think it should be any different from the previous version - I
tried to avoid touching those bits of cpusets where possible.

Paul

2007-09-17 18:12:23

by Balbir Singh

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

Andrew Morton wrote:
> On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <[email protected]> wrote:
>
>> On Sat, 8 Sep 2007, Nicolas Capit wrote:
>>
>>> Hello,
>>>
>>> This is my situation:
>>> - I mounted the pseudo cpuset filesystem into /dev/cpuset
>>> - I created a cpuset named oar with my 2 cpus
>>>
>>> cat /dev/cpuset/oar/cpus
>>> 0-1
>>>
>>> - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
>>> - After reboot:
>>>
>>> cat /dev/cpuset/oar/cpus
>>> 0
>>>
>>> Why did I lost a cpu?
>>> Is this a normal behavior???
>> Hi Nicolas,
>>
>> I believe this is related to the fact that hibernation uses the hotplug
>> subsystem to disable all CPUs except the boot CPU.
>>
>> Thus guarantee_online_cpus() is called on each cpuset and removes all
>> CPUs, except CPU 0, from all cpusets.
>>
>> I'm not quite sure about if/how this should be fixed in the kernel,
>> though. Looks like a very simple user-land workaround would be enough.
>>
>
> Yeah. Bug, surely. But I guess it's always been there.
>
> What are the implications of this for cpusets-via-containers?
>

I suspect the functionality of cpusets is not affected by containers.

I wonder if containers should become suspend/resume aware and pass
that option on to controllers. I think it's only the bus drivers
and device drivers that do that now.



--
Warm Regards,
Balbir Singh
Linux Technology Center
IBM, ISTL

2007-09-19 01:00:40

by Paul Menage

[permalink] [raw]
Subject: Re: cpuset trouble after hibernate

On 9/9/07, Pavel Machek <[email protected]> wrote:
>
> One of the cpus was unplugged during suspend... perhaps some
> save/restore is needed during hotplug/unplug?

Or else keep track separately in cpusets of

- cpus that the cpuset can run on
- cpus that the admin has specified for the cpu to run on

hotplug/hotunplug events would only affect the former; userspace would
only see/modify the latter. Then when hibernate is over and the CPUs
are hotplugged back in, things would be back as before.

Paul