I'm trying to "glue" hibernation and UPS control
together, and have a question.
When the system power comes off an UPS (Uninterruptable
Power Supply I mean), it's probably a good idea to turn
the UPS off when shutting the system down or hibernating.
Even with shutdown (not related to hibernating) there's
an interesting twist: modern systems can remember the
previous on/off status when the power gets cut and
restores - BIOS-controlled option that allows the
system to automatically start when the input power
returns. Now, when we shutting down a system, we
need to turn the UPS *before* powering down the
system but *after* the shutdown procedure has been
completed. This is done by replacing poweroff with
halt, and ordering the UPS to turn itself off after
a small delay (needed for the poweroff command to
complete) - this way, system will be on but in a
safe-to-powercut mode, and in order to turn it
back on only the UPS has to be turned on.
But the question comes as how to control the UPS when
we replace shutdown with hibernation. Ideally I want
to send some commands to the UPS after hibernation has
been completed (since I don't know how much time it
will require to hibernate), but before the machine
will be turned off by the kernel. Or at the very least,
to be able to stop the kernel turning the machine off
after hibernation process is complete - this way,
I can order the UPS to turn the power off after some
delay and hope hibernation process will finish when
the UPS will finally cut the power (not all UPSes
are able to do timely shutdown like this).
What's my way here?
Thanks!
/mjt
P.S. Surprisingly, there's NO software that works
with UPSes and implements even the basic shutdown
process completely (not even mentioning hibernation).
Most common is just to turn the system off without
touching the UPS. Yes it will work (till the UPS
will run out of battery), but how about servers which
should be up-n-running, ie. which should restart
automatically?.. Oh well.
On Wednesday 30 January 2008 20:18:40 you wrote:
> I'm trying to "glue" hibernation and UPS control
> together, and have a question.
>
> When the system power comes off an UPS (Uninterruptable
> Power Supply I mean), it's probably a good idea to turn
> the UPS off when shutting the system down or hibernating.
This depends on how you can turn the UPS back on...
If pushing the button manually is not an issue or the UPS
can still be controlled remotely (e.g. via network) when
being off it doesn't matter.
> Even with shutdown (not related to hibernating) there's
> an interesting twist: modern systems can remember the
> previous on/off status when the power gets cut and
> restores - BIOS-controlled option that allows the
> system to automatically start when the input power
> returns. Now, when we shutting down a system, we
> need to turn the UPS *before* powering down the
> system but *after* the shutdown procedure has been
> completed. This is done by replacing poweroff with
> halt, and ordering the UPS to turn itself off after
> a small delay (needed for the poweroff command to
> complete) - this way, system will be on but in a
> safe-to-powercut mode, and in order to turn it
> back on only the UPS has to be turned on.
All systems I've seen that provide such an option in the BIOS
list three possible actions on power-back:
- remain off
- go back to previous state
- boot
In your case it's possibly easier to choose the option to
always boot when power comes back.
> But the question comes as how to control the UPS when
> we replace shutdown with hibernation. Ideally I want
> to send some commands to the UPS after hibernation has
> been completed (since I don't know how much time it
> will require to hibernate), but before the machine
> will be turned off by the kernel. Or at the very least,
> to be able to stop the kernel turning the machine off
> after hibernation process is complete - this way,
> I can order the UPS to turn the power off after some
> delay and hope hibernation process will finish when
> the UPS will finally cut the power (not all UPSes
> are able to do timely shutdown like this).
Can you configure your UPS to poweroff when load on its output
drops below a given threshold? (I know that such a feature
exists on master-slave power outlets where all slave outlets
are powered only when the master outlet has a sufficient load)
Another path may be to dig into hibernate via kexec:
http://kerneltrap.org/node/11756
> What's my way here?
>
> Thanks!
>
> /mjt
>
> P.S. Surprisingly, there's NO software that works
> with UPSes and implements even the basic shutdown
> process completely (not even mentioning hibernation).
> Most common is just to turn the system off without
> touching the UPS. Yes it will work (till the UPS
> will run out of battery), but how about servers which
> should be up-n-running, ie. which should restart
> automatically?.. Oh well.
Can't the UPS software trigger scripts or be controlled
using scripts?
Regards,
Bruno
Bruno Pr?mont wrote:
> On Wednesday 30 January 2008 20:18:40 you wrote:
>> I'm trying to "glue" hibernation and UPS control
>> together, and have a question.
>>
>> When the system power comes off an UPS (Uninterruptable
>> Power Supply I mean), it's probably a good idea to turn
>> the UPS off when shutting the system down or hibernating.
>
> This depends on how you can turn the UPS back on...
Most UPSes has an option to turn themselves on when the
power returns (except of very dumb ones).
> If pushing the button manually is not an issue or the UPS
> can still be controlled remotely (e.g. via network) when
> being off it doesn't matter.
It does not matter really.
[]
>> returns. Now, when we shutting down a system, we
>> need to turn the UPS *before* powering down the
>> system but *after* the shutdown procedure has been
>> completed. This is done by replacing poweroff with
>> halt, and ordering the UPS to turn itself off after
>> a small delay (needed for the poweroff command to
>> complete) - this way, system will be on but in a
>> safe-to-powercut mode, and in order to turn it
>> back on only the UPS has to be turned on.
>
> All systems I've seen that provide such an option in the BIOS
> list three possible actions on power-back:
> - remain off
> - go back to previous state
> - boot
Or sometimes a subset of the 3 - like "always off or return to
previous state". But this does not matter again - I'm not
asking about how to control the power-on feature, or how to
control a particular UPS, -- instead, I'm asking how to hook
up something into the hibernate path, to the very end of it,
to do whatever is needed, and/or how to control whether after
the hibernate is complete the kernel should turn the system
off or not.
> In your case it's possibly easier to choose the option to
> always boot when power comes back.
Whatever works and is most suitable given the particular
situation. It doesn't really matter here.
[]
> Can you configure your UPS to poweroff when load on its output
> drops below a given threshold? (I know that such a feature
> exists on master-slave power outlets where all slave outlets
> are powered only when the master outlet has a sufficient load)
Certain models can be configured this way. But the question
stands still, because many models can't be programmed this way.
What's needed is to run something at the end of hibernate
process, nothing more nothing less.
> Another path may be to dig into hibernate via kexec:
> http://kerneltrap.org/node/11756
Well.. that might work after all. It doesn't work
with vanilla kernel, does it? I mean, I can get
*my* system to work as I like it to work, but it
can't work on another Linux system...
>> P.S. Surprisingly, there's NO software that works
>> with UPSes and implements even the basic shutdown
>> process completely (not even mentioning hibernation).
>> Most common is just to turn the system off without
>> touching the UPS. Yes it will work (till the UPS
>> will run out of battery), but how about servers which
>> should be up-n-running, ie. which should restart
>> automatically?.. Oh well.
>
> Can't the UPS software trigger scripts or be controlled
> using scripts?
In some cases it can, but usually in a very limited way.
It's just like no one wrote good software about this
topic... ;) Or good scripts, for that matter. That's
exactly what I'm trying to do, and certain questions
(like this one) pops up.
Thanks.
/mjt
On Wednesday, 30 of January 2008, Michael Tokarev wrote:
> I'm trying to "glue" hibernation and UPS control
> together, and have a question.
>
> When the system power comes off an UPS (Uninterruptable
> Power Supply I mean), it's probably a good idea to turn
> the UPS off when shutting the system down or hibernating.
>
> Even with shutdown (not related to hibernating) there's
> an interesting twist: modern systems can remember the
> previous on/off status when the power gets cut and
> restores - BIOS-controlled option that allows the
> system to automatically start when the input power
> returns. Now, when we shutting down a system, we
> need to turn the UPS *before* powering down the
> system but *after* the shutdown procedure has been
> completed. This is done by replacing poweroff with
> halt, and ordering the UPS to turn itself off after
> a small delay (needed for the poweroff command to
> complete) - this way, system will be on but in a
> safe-to-powercut mode, and in order to turn it
> back on only the UPS has to be turned on.
>
> But the question comes as how to control the UPS when
> we replace shutdown with hibernation. Ideally I want
> to send some commands to the UPS after hibernation has
> been completed (since I don't know how much time it
> will require to hibernate), but before the machine
> will be turned off by the kernel. Or at the very least,
> to be able to stop the kernel turning the machine off
> after hibernation process is complete - this way,
> I can order the UPS to turn the power off after some
> delay and hope hibernation process will finish when
> the UPS will finally cut the power (not all UPSes
> are able to do timely shutdown like this).
>
> What's my way here?
If your box hibernates and resumes correctly in the shutdown mode (ie.
basically without ACPI assistance), you can use s2disk for hibernation and
modify it to switch off the UPS instead of powering off the box.
Greetings,
Rafael
Hi.
Michael Tokarev wrote:
> I'm trying to "glue" hibernation and UPS control
> together, and have a question.
>
> When the system power comes off an UPS (Uninterruptable
> Power Supply I mean), it's probably a good idea to turn
> the UPS off when shutting the system down or hibernating.
>
> Even with shutdown (not related to hibernating) there's
> an interesting twist: modern systems can remember the
> previous on/off status when the power gets cut and
> restores - BIOS-controlled option that allows the
> system to automatically start when the input power
> returns. Now, when we shutting down a system, we
> need to turn the UPS *before* powering down the
> system but *after* the shutdown procedure has been
> completed. This is done by replacing poweroff with
> halt, and ordering the UPS to turn itself off after
> a small delay (needed for the poweroff command to
> complete) - this way, system will be on but in a
> safe-to-powercut mode, and in order to turn it
> back on only the UPS has to be turned on.
>
> But the question comes as how to control the UPS when
> we replace shutdown with hibernation. Ideally I want
> to send some commands to the UPS after hibernation has
> been completed (since I don't know how much time it
> will require to hibernate), but before the machine
> will be turned off by the kernel. Or at the very least,
> to be able to stop the kernel turning the machine off
> after hibernation process is complete - this way,
> I can order the UPS to turn the power off after some
> delay and hope hibernation process will finish when
> the UPS will finally cut the power (not all UPSes
> are able to do timely shutdown like this).
>
> What's my way here?
That should be doable. How is your UPS connected? Presumably, with some
modifications to the appropriate driver, we could send the commands when
we're ready to shutdown. It would probably be useful whether or not your
hibernating (if not, sending the commands could always be made an option).
Regards,
Nigel
Rafael J. Wysocki wrote:
> On Wednesday, 30 of January 2008, Michael Tokarev wrote:
>> I'm trying to "glue" hibernation and UPS control
>> together, and have a question.
[]
> If your box hibernates and resumes correctly in the shutdown mode (ie.
Ohh-well.. :)
I was thinking about more or less general solution to this.
Yes, my boxes counts as primary targets, because we've seen
so many various failures due to power loss and because I want
to finally stop it somehow. The thing basically works with
plain shutdown (no suspend is involved), but it turned out
that many esp. cheap UPSes (the ones used by "masses") don't
check the status of their batteries up until they lose AC
power, when it's too late... The result is that when you
lose AC power, you discover that you've only a few secs to
perform shutdown, and hence hibernate is the only option,
as it usually is much faster than to try to shut down stuff
(esp. when things like squid or a big database are on the
way, with their looong time to shutdown cleanly).
> basically without ACPI assistance), you can use s2disk for hibernation and
> modify it to switch off the UPS instead of powering off the box.
Heh. First, I didn't know about s2disk up until now. I've
seen uswsusp utils some years before, but I thought it's
about some patch for vanilla kernel (some time ago it was
indeed the case) and that the utils aren't needed for current
kernels (I always used "echo disk > /sys/power/state" to
suspend).
Now, and I hate to say that, I found that this my test
box is totally unsuitable for testing this all. Because
it all just Does Not Work. Because:
o 2.6.24 kernel fails to suspend properly. It "saves pages",
prints "Suspending console", when prints "Sl" and nothing
more happens. At this time, I can manually poweroff the
machine, and resume works. The same happens when running
32bits or 64bits kernel (it's an amd x2-64 system).
o uswsusp is unaware of 64bits kernel and 32bits userland.
Looking at the source, that's probably (and for sure in
certain places) more than just missing compat_ioctl wrappers
for those:
ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(400c330d){t:'3';sz:12} arg(ffa4578c) on /dev/snapshot
ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(4004330a){t:'3';sz:4} arg(00000805) on /dev/snapshot
"probably" is where it reads the swap space - it's just
my guess that s2disk (and resume) will do some wrong here.
"for sure" is vbe code - since 32bits vbetools package
fails badly with 64bit kernel, it seems that it is looking
at the wrong place in /dev/mem.
o when running 32bits 2.6.24 kernel, s2disk fails the same
way as pure in-kernel solution fails, above. And it
fails to resume, too, -- unlike pure in-kernel solution.
I know kernel-only way worked with 2.6.23 (both 32 and 64
bits) - compiling that now...
It's sad when while working on something you discover that
"underlying" tools you use also don't work, and start fixing
those, when discover that something else fails, and so on,
until you finally gave up and abandom the whole initial
idea... Oh well, but c'est la vie...
Back to the original question and a proposed solution.
I'm looking at the uswsusp source (while the kernel compiles),
and have a question here. Is it possible to call some external
application (typically a shell script) to do the final work after
when the image has been written? I mean in principle - I
understand there are some limitations here, but I don't know
which exactly.
For the given task (send some command to the damn UPS),
it typically involves writing/reading something to/from
a given serial port (/dev/ttySxx), or to an USB device,
or to a network -- depending on the UPS in use. There
are so many different UPSes out there, with so many different
and stupid so-called "protocols", that it's impossible to
teach s2disk about them all.
If such a thing is possible, the next question is obvious --
where exactly it belongs in suspend.c, and again, under
which conditions it can be called (for example, how about
different methods - shutdown/platform/etc?).
Also, if I need the system to stay on while the UPS cuts
the power (in order for it to start up automatically,
provided "restore to previous state on AC power resume"
BIOS option is enabled), should such a script (which is
called from s2disk) just sleep letting the UPS to do the
work, or should it pass information back to s2disk to NOT
to poweroff the machine? I mean, just like in case of
"regular" shutdown when we're running off battery, between
when the UPS control script gets called and when it's
really safe to cut the power off, some more things has to
be done - like stopping disks, "remointing" sw raid arrays
read-only (to mark them as "clean") etc - this is done by
calling reboot(RB_HALT) or reboot(RB_POWEROFF), which, in
turn, is done by halt(8). So when s2disk calls this script,
is it Okay to assume that it's safe to cut the power, or
something more has to be done by s2disk when the script
finishes?
I *hope* it's easy to see where my questions comes from,
and what exactly I'm trying to do... ;)
(kernel compile finished, rebooting to see if s2disk works
with 2.6.23 - 32bits for now, as it definitely will not
work with 64bits kernel...)
/mjt
Nigel Cunningham wrote:
[]
> That should be doable. How is your UPS connected? Presumably, with some
> modifications to the appropriate driver, we could send the commands when
> we're ready to shutdown. It would probably be useful whether or not your
> hibernating (if not, sending the commands could always be made an option).
You mean adding stuff to some KERNEL driver? Like to a serial driver if
the UPS is connected to a COM-port??
I'm afraid either I don't understand what you're talking about here, or,
if I got you right, that YOU don't understand what you're talking about...
Come on, teaching kernel about various idiotic UPSes out there is more
than insane... ;)
/mjt
Hi Michael.
Michael Tokarev wrote:
> Nigel Cunningham wrote:
> []
>> That should be doable. How is your UPS connected? Presumably, with some
>> modifications to the appropriate driver, we could send the commands when
>> we're ready to shutdown. It would probably be useful whether or not your
>> hibernating (if not, sending the commands could always be made an option).
>
> You mean adding stuff to some KERNEL driver? Like to a serial driver if
> the UPS is connected to a COM-port??
>
> I'm afraid either I don't understand what you're talking about here, or,
> if I got you right, that YOU don't understand what you're talking about...
>
> Come on, teaching kernel about various idiotic UPSes out there is more
> than insane... ;)
I wasn't meaning the kernel should have to know about various idiotic
UPSses (yes, I've got experience with them too, so I understand what
you're talking about there). What I was thinking was that maybe it might
be possible to give the kernel some simple (configurable) string to send
out the UPS port when it's time to power off. Of course you're going to
point out that a simple string won't do in every case (though I think it
would do for at least some of the idiotic UPS ones).
I was just wondering aloud if there was a simpler way, but on further
reflection, I guess there's no way around it. Whether this is done with
kexec, uswsusp or TuxOnIce, it's going to involve creating (TuxOnIce /
kexec ) or extending a userspace (uswsusp) binary that's got to somehow
interface with the driver and get the job done.
Nigel
Hi!
> o 2.6.24 kernel fails to suspend properly. It "saves pages",
> prints "Suspending console", when prints "Sl" and nothing
> more happens. At this time, I can manually poweroff the
> machine, and resume works. The same happens when running
> 32bits or 64bits kernel (it's an amd x2-64 system).
Patch welcome, it certainly works here.
> o uswsusp is unaware of 64bits kernel and 32bits userland.
> Looking at the source, that's probably (and for sure in
> certain places) more than just missing compat_ioctl wrappers
> for those:
>
> ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(400c330d){t:'3';sz:12} arg(ffa4578c) on /dev/snapshot
> ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(4004330a){t:'3';sz:4} arg(00000805) on /dev/snapshot
>
> "probably" is where it reads the swap space - it's just
> my guess that s2disk (and resume) will do some wrong here.
> "for sure" is vbe code - since 32bits vbetools package
> fails badly with 64bit kernel, it seems that it is looking
> at the wrong place in /dev/mem.
Hmm.. yes, it would be nice to fix that.
> It's sad when while working on something you discover that
> "underlying" tools you use also don't work, and start fixing
> those, when discover that something else fails, and so on,
> until you finally gave up and abandom the whole initial
> idea... Oh well, but c'est la vie...
:-).
> Back to the original question and a proposed solution.
>
> I'm looking at the uswsusp source (while the kernel compiles),
> and have a question here. Is it possible to call some external
> application (typically a shell script) to do the final work after
> when the image has been written? I mean in principle - I
> understand there are some limitations here, but I don't know
> which exactly.
No, you can't exec() anything. That would write mtime back to disk and
cause badness.
> For the given task (send some command to the damn UPS),
> it typically involves writing/reading something to/from
> a given serial port (/dev/ttySxx), or to an USB device,
> or to a network -- depending on the UPS in use. There
> are so many different UPSes out there, with so many different
> and stupid so-called "protocols", that it's impossible to
> teach s2disk about them all.
Create libups.so, and link s2disk to it?
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek wrote:
[]
>> I'm looking at the uswsusp source (while the kernel compiles),
>> and have a question here. Is it possible to call some external
>> application (typically a shell script) to do the final work after
>> when the image has been written? I mean in principle - I
>> understand there are some limitations here, but I don't know
>> which exactly.
>
> No, you can't exec() anything. That would write mtime back to disk and
> cause badness.
Now that's.. interesting.
s2disk writes to a swap device/file, which should update
mtime of this device node/file. Isn't it something which
also causes the same badness?
Also, if the only concern is mtime update, what's really
wrong with it? I mean, regardless of whether such update
will finally hit the disk or not, there's not much difference
really - it updates just mtime field, and there should be
no harm in that. Unless such update first goes to a
journal (in a journalling filesystem) - which DOES modify
some on-disk structures.
>> it typically involves writing/reading something to/from
>> a given serial port (/dev/ttySxx), or to an USB device,
> Create libups.so, and link s2disk to it?
And what's the difference here again? We'll open a serial
port and write something to it - which, again, will update
mtime of that device node. Unless the said node is on a
tmpfs, exactly the same badness will happen. Not all the
world is udev, after all.
So I don't get the reason why we can't exec something here,
still. (And, for example, call splashy commands as external
processes, instead of linking all this cruft into s2disk and
resume.)
What I'm thinking about here is - s2ram mlock()s its memory.
If it will fork/exec something, that something will obviously
NOT be locked like that. Is it of some concern? Probably
not, because that something will be executed after we've
taken the snapshot.
/mjt
On Friday, 1 of February 2008, Michael Tokarev wrote:
> Pavel Machek wrote:
> []
> >> I'm looking at the uswsusp source (while the kernel compiles),
> >> and have a question here. Is it possible to call some external
> >> application (typically a shell script) to do the final work after
> >> when the image has been written? I mean in principle - I
> >> understand there are some limitations here, but I don't know
> >> which exactly.
> >
> > No, you can't exec() anything. That would write mtime back to disk and
> > cause badness.
>
> Now that's.. interesting.
>
> s2disk writes to a swap device/file, which should update
> mtime of this device node/file. Isn't it something which
> also causes the same badness?
No, please read the source carefully. We create a special tmpfs for that.
> Also, if the only concern is mtime update, what's really
> wrong with it? I mean, regardless of whether such update
> will finally hit the disk or not, there's not much difference
> really - it updates just mtime field, and there should be
> no harm in that. Unless such update first goes to a
> journal (in a journalling filesystem) - which DOES modify
> some on-disk structures.
Yes, we're concerned about the journaling.
> >> it typically involves writing/reading something to/from
> >> a given serial port (/dev/ttySxx), or to an USB device,
>
> > Create libups.so, and link s2disk to it?
>
> And what's the difference here again? We'll open a serial
> port and write something to it - which, again, will update
> mtime of that device node. Unless the said node is on a
> tmpfs, exactly the same badness will happen. Not all the
> world is udev, after all.
We already have a tmpfs for that.
> So I don't get the reason why we can't exec something here,
> still. (And, for example, call splashy commands as external
> processes, instead of linking all this cruft into s2disk and
> resume.)
>
> What I'm thinking about here is - s2ram mlock()s its memory.
> If it will fork/exec something, that something will obviously
> NOT be locked like that. Is it of some concern? Probably
> not, because that something will be executed after we've
> taken the snapshot.
Please don't exec() things from s2disk/s2both. Just link them to
appropriate libraries and modify them to do things you need.
Thanks,
Rafael
On Thu, Jan 31, 2008 at 03:40:30PM +0100, Pavel Machek wrote:
> > Back to the original question and a proposed solution.
> >
> > I'm looking at the uswsusp source (while the kernel compiles),
> > and have a question here. Is it possible to call some external
> > application (typically a shell script) to do the final work after
> > when the image has been written? I mean in principle - I
> > understand there are some limitations here, but I don't know
> > which exactly.
>
> No, you can't exec() anything. That would write mtime back to disk and
> cause badness.
SysRq-U remounts the disks readonly. Why not make the disks readonly
once the kernel state is in swap, why not let any program run and give
an error to writes?
I know, long dead thread, but I too would like to tell the UPS to turn
off once the system state is resting safely in swap. Currently my
modified apcupsd scripts tell the UPS to turn off, then tells the
kernel to hibernate, and hopes the kernel finishes in time. I would
much rather do things the other way, but `echo disk > /sys/power/disk`
doesn't return until power is back on if everything goes well.
--
CC me on replies
David Fries <[email protected]>
http://fries.net/~david/ (PGP encryption key available)
On Sun 2008-07-13 18:26:36, David Fries wrote:
> On Thu, Jan 31, 2008 at 03:40:30PM +0100, Pavel Machek wrote:
> > > Back to the original question and a proposed solution.
> > >
> > > I'm looking at the uswsusp source (while the kernel compiles),
> > > and have a question here. Is it possible to call some external
> > > application (typically a shell script) to do the final work after
> > > when the image has been written? I mean in principle - I
> > > understand there are some limitations here, but I don't know
> > > which exactly.
> >
> > No, you can't exec() anything. That would write mtime back to disk and
> > cause badness.
>
> SysRq-U remounts the disks readonly. Why not make the disks readonly
> once the kernel state is in swap, why not let any program run and give
> an error to writes?
IIRC sysrq-U does not contain all the locking that's neccessary to
make it completely safe. OTOH, it tends to work...
> I know, long dead thread, but I too would like to tell the UPS to turn
> off once the system state is resting safely in swap. Currently my
> modified apcupsd scripts tell the UPS to turn off, then tells the
> kernel to hibernate, and hopes the kernel finishes in time. I would
> much rather do things the other way, but `echo disk > /sys/power/disk`
> doesn't return until power is back on if everything goes well.
If you use s2disk, you can execute C code at the point you want...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html