2008-03-20 20:53:18

by Pavel Machek

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Tue 2008-03-18 21:25:27, Eric W. Biederman wrote:
> Alan Stern <[email protected]> writes:
>
> > On Wed, 19 Mar 2008, Rafael J. Wysocki wrote:
> >
> >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> >> use S5 instead of entering S4, the fan doesn't work correctly after the
> >> resume. Plain and simple.
> >>
> >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> >> but I have no idea what that can be at the moment.
> >
> > IMO it would be worthwhile to track this down. It's a clear indication
> > that something is wrong somewhere.
> >
> > Could it be connected with the way the boot kernel hands control over
> > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > sort of thing during a boot from S5. It would have to be fooled into
> > thinking the two kernels were one and the same.
>
> It should be easy to test if it is a hand over problem, by turning off
> the laptop by placing it in S5 (shutdown -h now) and then booting same
> kernel again.

Feel free to help with testing.

I believe ACPI is simply getting confused by us overwriting memory
with that from old image. I don't see how you can emulate it with
shutdown.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
pomozte zachranit klanovicky les: http://www.ujezdskystrom.info/


2008-03-20 22:46:18

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Thursday, 20 of March 2008, Pavel Machek wrote:
> On Tue 2008-03-18 21:25:27, Eric W. Biederman wrote:
> > Alan Stern <[email protected]> writes:
> >
> > > On Wed, 19 Mar 2008, Rafael J. Wysocki wrote:
> > >
> > >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> > >> use S5 instead of entering S4, the fan doesn't work correctly after the
> > >> resume. Plain and simple.
> > >>
> > >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> > >> but I have no idea what that can be at the moment.
> > >
> > > IMO it would be worthwhile to track this down. It's a clear indication
> > > that something is wrong somewhere.
> > >
> > > Could it be connected with the way the boot kernel hands control over
> > > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > > sort of thing during a boot from S5. It would have to be fooled into
> > > thinking the two kernels were one and the same.
> >
> > It should be easy to test if it is a hand over problem, by turning off
> > the laptop by placing it in S5 (shutdown -h now) and then booting same
> > kernel again.
>
> Feel free to help with testing.
>
> I believe ACPI is simply getting confused by us overwriting memory
> with that from old image. I don't see how you can emulate it with
> shutdown.

Well, in fact ACPI has something called the NVS memory, which we're supposed
to restore during the resume and which we're not doing. The problem may be
related to this.

I have fixing that on my todo list, but frankly there's many different things
in there. :-)

Thanks,
Rafael

2008-03-20 23:02:13

by Alan Stern

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Thu, 20 Mar 2008, Rafael J. Wysocki wrote:

> > > >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> > > >> use S5 instead of entering S4, the fan doesn't work correctly after the
> > > >> resume. Plain and simple.
> > > >>
> > > >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> > > >> but I have no idea what that can be at the moment.
> > > >
> > > > IMO it would be worthwhile to track this down. It's a clear indication
> > > > that something is wrong somewhere.
> > > >
> > > > Could it be connected with the way the boot kernel hands control over
> > > > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > > > sort of thing during a boot from S5. It would have to be fooled into
> > > > thinking the two kernels were one and the same.
> > >
> > > It should be easy to test if it is a hand over problem, by turning off
> > > the laptop by placing it in S5 (shutdown -h now) and then booting same
> > > kernel again.
> >
> > Feel free to help with testing.
> >
> > I believe ACPI is simply getting confused by us overwriting memory
> > with that from old image. I don't see how you can emulate it with
> > shutdown.
>
> Well, in fact ACPI has something called the NVS memory, which we're supposed
> to restore during the resume and which we're not doing. The problem may be
> related to this.

No, it can't be. ACPI won't expect the NVS memory to be restored
following an S5-shutdown. In fact, as far as ACPI is concerned,
resuming from an S5-type hibernation should not be considered a resume
at all but just an ordinary reboot. All ACPI-related memory areas
in the boot kernel should be passed directly through to the image
kernel.

Alan Stern

2008-03-20 23:21:41

by Pavel Machek

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Thu 2008-03-20 19:01:56, Alan Stern wrote:
> On Thu, 20 Mar 2008, Rafael J. Wysocki wrote:
>
> > > > >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> > > > >> use S5 instead of entering S4, the fan doesn't work correctly after the
> > > > >> resume. Plain and simple.
> > > > >>
> > > > >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> > > > >> but I have no idea what that can be at the moment.
> > > > >
> > > > > IMO it would be worthwhile to track this down. It's a clear indication
> > > > > that something is wrong somewhere.
> > > > >
> > > > > Could it be connected with the way the boot kernel hands control over
> > > > > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > > > > sort of thing during a boot from S5. It would have to be fooled into
> > > > > thinking the two kernels were one and the same.
> > > >
> > > > It should be easy to test if it is a hand over problem, by turning off
> > > > the laptop by placing it in S5 (shutdown -h now) and then booting same
> > > > kernel again.
> > >
> > > Feel free to help with testing.
> > >
> > > I believe ACPI is simply getting confused by us overwriting memory
> > > with that from old image. I don't see how you can emulate it with
> > > shutdown.
> >
> > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > to restore during the resume and which we're not doing. The problem may be
> > related to this.
>
> No, it can't be. ACPI won't expect the NVS memory to be restored
> following an S5-shutdown. In fact, as far as ACPI is concerned,
> resuming from an S5-type hibernation should not be considered a resume
> at all but just an ordinary reboot. All ACPI-related memory areas
> in the boot kernel should be passed directly through to the image
> kernel.

How can we pass interpretter state? I do not think we do this kind of
passing.

If it was enough to pass some static area, we could just mark it
nosave...

Len: Is ACPI AML permitted to allocate memory (like in ACPI_ALLOC or
something)? Could we easily identify BIOS data so we could mark them
nosave?
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
pomozte zachranit klanovicky les: http://www.ujezdskystrom.info/

2008-03-20 23:41:27

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Friday, 21 of March 2008, Pavel Machek wrote:
> On Thu 2008-03-20 19:01:56, Alan Stern wrote:
> > On Thu, 20 Mar 2008, Rafael J. Wysocki wrote:
> >
> > > > > >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> > > > > >> use S5 instead of entering S4, the fan doesn't work correctly after the
> > > > > >> resume. Plain and simple.
> > > > > >>
> > > > > >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> > > > > >> but I have no idea what that can be at the moment.
> > > > > >
> > > > > > IMO it would be worthwhile to track this down. It's a clear indication
> > > > > > that something is wrong somewhere.
> > > > > >
> > > > > > Could it be connected with the way the boot kernel hands control over
> > > > > > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > > > > > sort of thing during a boot from S5. It would have to be fooled into
> > > > > > thinking the two kernels were one and the same.
> > > > >
> > > > > It should be easy to test if it is a hand over problem, by turning off
> > > > > the laptop by placing it in S5 (shutdown -h now) and then booting same
> > > > > kernel again.
> > > >
> > > > Feel free to help with testing.
> > > >
> > > > I believe ACPI is simply getting confused by us overwriting memory
> > > > with that from old image. I don't see how you can emulate it with
> > > > shutdown.
> > >
> > > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > > to restore during the resume and which we're not doing. The problem may be
> > > related to this.
> >
> > No, it can't be. ACPI won't expect the NVS memory to be restored
> > following an S5-shutdown. In fact, as far as ACPI is concerned,
> > resuming from an S5-type hibernation should not be considered a resume
> > at all but just an ordinary reboot.

I agree here.

> > All ACPI-related memory areas in the boot kernel should be passed directly
> > through to the image kernel.

However, the image kernel is supposed to restore the NVS area (from the
image) before executing _WAK.

> How can we pass interpretter state? I do not think we do this kind of
> passing.

The interpreter state is passed withing the image. The platform state is not.

> If it was enough to pass some static area, we could just mark it
> nosave...
>
> Len: Is ACPI AML permitted to allocate memory (like in ACPI_ALLOC or
> something)? Could we easily identify BIOS data so we could mark them
> nosave?

This wouldn't work even if we could (at least on x86-64).

In fact I'm going to remove the 'nosave' section in the future (another
thing on the todo list).

Thanks,
Rafael

2008-03-21 00:37:33

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Friday, 21 of March 2008, Rafael J. Wysocki wrote:
> On Friday, 21 of March 2008, Pavel Machek wrote:
> > On Thu 2008-03-20 19:01:56, Alan Stern wrote:
> > > On Thu, 20 Mar 2008, Rafael J. Wysocki wrote:
> > >
> > > > > > >> Well, I've been saying that for I-don't-remember-how-long: on my box, if you
> > > > > > >> use S5 instead of entering S4, the fan doesn't work correctly after the
> > > > > > >> resume. Plain and simple.
> > > > > > >>
> > > > > > >> Perhaps there's a problem with our ACPI drivers that causes this to happen,
> > > > > > >> but I have no idea what that can be at the moment.
> > > > > > >
> > > > > > > IMO it would be worthwhile to track this down. It's a clear indication
> > > > > > > that something is wrong somewhere.
> > > > > > >
> > > > > > > Could it be connected with the way the boot kernel hands control over
> > > > > > > to the image kernel? Presumably ACPI isn't prepared to deal with that
> > > > > > > sort of thing during a boot from S5. It would have to be fooled into
> > > > > > > thinking the two kernels were one and the same.
> > > > > >
> > > > > > It should be easy to test if it is a hand over problem, by turning off
> > > > > > the laptop by placing it in S5 (shutdown -h now) and then booting same
> > > > > > kernel again.
> > > > >
> > > > > Feel free to help with testing.
> > > > >
> > > > > I believe ACPI is simply getting confused by us overwriting memory
> > > > > with that from old image. I don't see how you can emulate it with
> > > > > shutdown.
> > > >
> > > > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > > > to restore during the resume and which we're not doing. The problem may be
> > > > related to this.
> > >
> > > No, it can't be. ACPI won't expect the NVS memory to be restored
> > > following an S5-shutdown. In fact, as far as ACPI is concerned,
> > > resuming from an S5-type hibernation should not be considered a resume
> > > at all but just an ordinary reboot.
>
> I agree here.
>
> > > All ACPI-related memory areas in the boot kernel should be passed directly
> > > through to the image kernel.
>
> However, the image kernel is supposed to restore the NVS area (from the
> image) before executing _WAK.
>
> > How can we pass interpretter state? I do not think we do this kind of
> > passing.
>
> The interpreter state is passed withing the image. The platform state is not.
>
> > If it was enough to pass some static area, we could just mark it
> > nosave...
> >
> > Len: Is ACPI AML permitted to allocate memory (like in ACPI_ALLOC or
> > something)? Could we easily identify BIOS data so we could mark them
> > nosave?
>
> This wouldn't work even if we could (at least on x86-64).

Ah, I misunderstood your comment, sorry.

The regions used by ACPI are registered as 'nosave' by the arch code and
we don't save them. However, the ACPI NVS area is exceptional in that we
are supposed to save and restore it. The problem is to restore it at the right
time and it's quite hard to figure out from the spec what time is the right
one (the only thing it says is we should do that before calling _WAK).

Thanks,
Rafael

2008-03-21 00:53:16

by Alan Stern

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Fri, 21 Mar 2008, Rafael J. Wysocki wrote:

> > > > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > > > to restore during the resume and which we're not doing. The problem may be
> > > > related to this.
> > >
> > > No, it can't be. ACPI won't expect the NVS memory to be restored
> > > following an S5-shutdown. In fact, as far as ACPI is concerned,
> > > resuming from an S5-type hibernation should not be considered a resume
> > > at all but just an ordinary reboot.
>
> I agree here.
>
> > > All ACPI-related memory areas in the boot kernel should be passed directly
> > > through to the image kernel.
>
> However, the image kernel is supposed to restore the NVS area (from the
> image) before executing _WAK.

It's supposed to do that when resuming from an S4 hibernation, not
when resuming from an S5 hibernation.

> > How can we pass interpretter state? I do not think we do this kind of
> > passing.
>
> The interpreter state is passed withing the image. The platform state is not.

For an S5 hibernation, the interpreter state within the image is wrong.
The image kernel needs to have the interpreter state from the boot
kernel -- I don't know if this is possible.

Alan Stern

2008-03-21 22:05:54

by Nigel Cunningham

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

Hi.

On Thu, 2008-03-20 at 20:52 -0400, Alan Stern wrote:
> For an S5 hibernation, the interpreter state within the image is wrong.
> The image kernel needs to have the interpreter state from the boot
> kernel -- I don't know if this is possible.

It's possible.

1) When hibernating, allocate a page (or pages if one isn't enough) for
the data to end up in after the atomic restore.
2) Put the location(s) in the image header.
3) At resume time, allocate an equivalent number of extra 'safe' pages
and set up extra pbes for the atomic restore to copy data from the extra
pages to the ones allocated when hibernating.
4) At the appropriate point in time, copy the NVS data to the extra
'safe' pages allocated in step 3.

The data will then be available to the resumed kernel post-resume.

I've been using this method to pass data from the boot kernel to the
resumed kernel for a while now. (I'm using it for I/O speed statistics
and state preservation).

Regards,

Nigel

2008-03-22 16:21:19

by Pavel Machek

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

> On Fri, 21 Mar 2008, Rafael J. Wysocki wrote:
>
> > > > > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > > > > to restore during the resume and which we're not doing. The problem may be
> > > > > related to this.
> > > >
> > > > No, it can't be. ACPI won't expect the NVS memory to be restored
> > > > following an S5-shutdown. In fact, as far as ACPI is concerned,
> > > > resuming from an S5-type hibernation should not be considered a resume
> > > > at all but just an ordinary reboot.
> >
> > I agree here.
> >
> > > > All ACPI-related memory areas in the boot kernel should be passed directly
> > > > through to the image kernel.
> >
> > However, the image kernel is supposed to restore the NVS area (from the
> > image) before executing _WAK.
>
> It's supposed to do that when resuming from an S4 hibernation, not
> when resuming from an S5 hibernation.
>
> > > How can we pass interpretter state? I do not think we do this kind of
> > > passing.
> >
> > The interpreter state is passed withing the image. The platform state is not.
>
> For an S5 hibernation, the interpreter state within the image is wrong.
> The image kernel needs to have the interpreter state from the boot
> kernel -- I don't know if this is possible.

yes, nosave pages could be used to do this passing -- if we can put
interpretter state into pre-allocated memory block.

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

2008-03-22 17:45:42

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Saturday, 22 of March 2008, Pavel Machek wrote:
> > On Fri, 21 Mar 2008, Rafael J. Wysocki wrote:
> >
> > > > > > Well, in fact ACPI has something called the NVS memory, which we're supposed
> > > > > > to restore during the resume and which we're not doing. The problem may be
> > > > > > related to this.
> > > > >
> > > > > No, it can't be. ACPI won't expect the NVS memory to be restored
> > > > > following an S5-shutdown. In fact, as far as ACPI is concerned,
> > > > > resuming from an S5-type hibernation should not be considered a resume
> > > > > at all but just an ordinary reboot.
> > >
> > > I agree here.
> > >
> > > > > All ACPI-related memory areas in the boot kernel should be passed directly
> > > > > through to the image kernel.
> > >
> > > However, the image kernel is supposed to restore the NVS area (from the
> > > image) before executing _WAK.
> >
> > It's supposed to do that when resuming from an S4 hibernation, not
> > when resuming from an S5 hibernation.
> >
> > > > How can we pass interpretter state? I do not think we do this kind of
> > > > passing.
> > >
> > > The interpreter state is passed withing the image. The platform state is not.
> >
> > For an S5 hibernation, the interpreter state within the image is wrong.
> > The image kernel needs to have the interpreter state from the boot
> > kernel -- I don't know if this is possible.
>
> yes, nosave pages could be used to do this passing -- if we can put
> interpretter state into pre-allocated memory block.

On x86-64 there's no guarantee that the "nosave" pages will be at the same
locations in both the image kernel and the boot kernel. What we could do
is to pass the data in the image header, preallocate some "safe" pages from
the boot kernel, put the data in there and pass a pointer to them to the
image kernel.

However, as far as the ACPI NVS area is concerned, this is probably not
necessary, because the spec wants us to restore the ACPI NVS before calling
_WAK, which is just after the image kernel gets the control back. So, in
theory, the ACPI NVS data could be stored in the image and restored by
the image kernel from a location known to it (the procedure may be to copy
the ACPI NVS data into a region of regular RAM before creating the image and
copy them back into the ACPI NVS area in platform->leave(), for example), but
I suspect that for this to work we'll have to switch ACPI off in the boot
kernel, just prior to passing control back to the image kernel.

Thanks,
Rafael

2008-03-22 20:50:21

by Alan Stern

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Sat, 22 Mar 2008, Rafael J. Wysocki wrote:

> However, as far as the ACPI NVS area is concerned, this is probably not
> necessary, because the spec wants us to restore the ACPI NVS before calling
> _WAK, which is just after the image kernel gets the control back. So, in
> theory, the ACPI NVS data could be stored in the image and restored by
> the image kernel from a location known to it (the procedure may be to copy
> the ACPI NVS data into a region of regular RAM before creating the image and
> copy them back into the ACPI NVS area in platform->leave(), for example), but
> I suspect that for this to work we'll have to switch ACPI off in the boot
> kernel, just prior to passing control back to the image kernel.

That sounds by far the simplest solution. If the boot kernel can tell
(by looking at some header field in the image or any other way) that
the hibernation used S5 instead of S4, then it should just turn off
ACPI before passing control to the image kernel. Then the image kernel
can turn ACPI back on and all should be well. If you do this, does the
NVS region still need to be preserved?

Alan Stern

2008-03-22 21:30:35

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [linux-pm] [PATCH -mm] kexec jump -v9

On Saturday, 22 of March 2008, Alan Stern wrote:
> On Sat, 22 Mar 2008, Rafael J. Wysocki wrote:
>
> > However, as far as the ACPI NVS area is concerned, this is probably not
> > necessary, because the spec wants us to restore the ACPI NVS before calling
> > _WAK, which is just after the image kernel gets the control back. So, in
> > theory, the ACPI NVS data could be stored in the image and restored by
> > the image kernel from a location known to it (the procedure may be to copy
> > the ACPI NVS data into a region of regular RAM before creating the image and
> > copy them back into the ACPI NVS area in platform->leave(), for example), but
> > I suspect that for this to work we'll have to switch ACPI off in the boot
> > kernel, just prior to passing control back to the image kernel.
>
> That sounds by far the simplest solution. If the boot kernel can tell
> (by looking at some header field in the image or any other way) that
> the hibernation used S5 instead of S4, then it should just turn off
> ACPI before passing control to the image kernel. Then the image kernel
> can turn ACPI back on and all should be well. If you do this, does the
> NVS region still need to be preserved?

The spec doesn't say much about that, so we'll need to carry out some
experiments.

Still, as far as I can figure out what the spec authors _might_ mean, I think
that it would be inappropriate to restore the ACPI NVS area if S5 was entered
on "power off". The idea seems to be that the restoration of the ACPI NVS area
should complement whatever has been preserved by the platform over the
hibernation/resume cycle.

IMO, if S5 was entered on "powe off", there are two possible ways to go.
Either ACPI is initialized by the boot kernel, in which case the image kernel
should not touch things like _WAK and similar, just throw away whatever
ACPI-related state it got from the image and try to rebuild the ACPI-related
data from scratch. Or the boot kernel doesn't touch ACPI and the image kernel
initializes it in the same way as during a fresh boot (that might be difficult,
though).

Thanks,
Rafael