On Fri, May 23, 2014 at 07:16:29PM +0100, Morten Rasmussen wrote:
> The Energy-aware scheduler implementation is guarded by
> CONFIG_SCHED_ENERGY.
>
> Signed-off-by: Dietmar Eggemann <[email protected]>
> Signed-off-by: Morten Rasmussen <[email protected]>
> ---
> arch/arm/Kconfig | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index ab438cb..bfc3a85 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
Is this going to be duplicate for each architecture enabling this? Why
not make a kernel/Kconfig.energy and link to that from those
architectures using it?
> @@ -1926,6 +1926,11 @@ config XEN
> help
> Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
>
> +config SCHED_ENERGY
> + bool "Energy-aware scheduling (EXPERIMENTAL)"
> + help
> + Highly experimental energy aware task scheduling.
> +
how about adding *slightly* more info here? :) (yes, yes, I know it's an RFC)
"""
Highly experimental energy aware task scheduling.
This will allow the kernel to keep track of energy required for
different capacity levels for a given CPU. That way, the scheduler can
make more informed decisions as to where a newly woken task should be
placed. Heterogenous platform will benefit the most from this option.
Enabling this will add a significant overhead for a task-switch.
If unsure, say N here.
"""
> endmenu
>
> menu "Boot options"
> --
> 1.7.9.5
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Henrik Austad
On Sun, Jun 08, 2014 at 07:03:16AM +0100, Henrik Austad wrote:
> On Fri, May 23, 2014 at 07:16:29PM +0100, Morten Rasmussen wrote:
> > The Energy-aware scheduler implementation is guarded by
> > CONFIG_SCHED_ENERGY.
> >
> > Signed-off-by: Dietmar Eggemann <[email protected]>
> > Signed-off-by: Morten Rasmussen <[email protected]>
> > ---
> > arch/arm/Kconfig | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > index ab438cb..bfc3a85 100644
> > --- a/arch/arm/Kconfig
> > +++ b/arch/arm/Kconfig
>
> Is this going to be duplicate for each architecture enabling this? Why
> not make a kernel/Kconfig.energy and link to that from those
> architectures using it?
kernel/Kconfig.energy is better I think.
>
> > @@ -1926,6 +1926,11 @@ config XEN
> > help
> > Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
> >
> > +config SCHED_ENERGY
> > + bool "Energy-aware scheduling (EXPERIMENTAL)"
> > + help
> > + Highly experimental energy aware task scheduling.
> > +
>
> how about adding *slightly* more info here? :) (yes, yes, I know it's an RFC)
Fair point.
>
> """
> Highly experimental energy aware task scheduling.
>
> This will allow the kernel to keep track of energy required for
> different capacity levels for a given CPU. That way, the scheduler can
> make more informed decisions as to where a newly woken task should be
> placed. Heterogenous platform will benefit the most from this option.
Platforms with hierarchical power domains (for example, having ability
to power off groups of cpus and their caches) should see some benefit as
well.
> Enabling this will add a significant overhead for a task-switch.
The overhead is at task wakeup, task switch (as in task preemption)
should not be affected.
Thanks for the text. I will roll into v2.
Morten
On Mon, Jun 09, 2014 at 11:20:27AM +0100, Morten Rasmussen wrote:
> On Sun, Jun 08, 2014 at 07:03:16AM +0100, Henrik Austad wrote:
> > On Fri, May 23, 2014 at 07:16:29PM +0100, Morten Rasmussen wrote:
> > > The Energy-aware scheduler implementation is guarded by
> > > CONFIG_SCHED_ENERGY.
> > >
> > > Signed-off-by: Dietmar Eggemann <[email protected]>
> > > Signed-off-by: Morten Rasmussen <[email protected]>
> > > ---
> > > arch/arm/Kconfig | 5 +++++
> > > 1 file changed, 5 insertions(+)
> > >
> > > diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > > index ab438cb..bfc3a85 100644
> > > --- a/arch/arm/Kconfig
> > > +++ b/arch/arm/Kconfig
> >
> > Is this going to be duplicate for each architecture enabling this? Why
> > not make a kernel/Kconfig.energy and link to that from those
> > architectures using it?
>
> kernel/Kconfig.energy is better I think.
Well, strictly speaking I'd prefer to not have more sched CONFIG knobs.
Do we really need to have this CONFIG guarded?
On Tue, Jun 10, 2014 at 10:39:43AM +0100, Peter Zijlstra wrote:
> On Mon, Jun 09, 2014 at 11:20:27AM +0100, Morten Rasmussen wrote:
> > On Sun, Jun 08, 2014 at 07:03:16AM +0100, Henrik Austad wrote:
> > > On Fri, May 23, 2014 at 07:16:29PM +0100, Morten Rasmussen wrote:
> > > > The Energy-aware scheduler implementation is guarded by
> > > > CONFIG_SCHED_ENERGY.
> > > >
> > > > Signed-off-by: Dietmar Eggemann <[email protected]>
> > > > Signed-off-by: Morten Rasmussen <[email protected]>
> > > > ---
> > > > arch/arm/Kconfig | 5 +++++
> > > > 1 file changed, 5 insertions(+)
> > > >
> > > > diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > > > index ab438cb..bfc3a85 100644
> > > > --- a/arch/arm/Kconfig
> > > > +++ b/arch/arm/Kconfig
> > >
> > > Is this going to be duplicate for each architecture enabling this? Why
> > > not make a kernel/Kconfig.energy and link to that from those
> > > architectures using it?
> >
> > kernel/Kconfig.energy is better I think.
>
> Well, strictly speaking I'd prefer to not have more sched CONFIG knobs.
>
> Do we really need to have this CONFIG guarded?
How would you like to disable the energy stuff for users for whom
latency is everything?
I mean, we are adding some extra load/utilization tracking. While I
think we should do everything possible to minimize the overhead, I think
it is unrealistic to assume that it will be zero. Is a some extra 'if
(energy_enabled)' acceptable?
I'm open for other suggestions.
On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> How would you like to disable the energy stuff for users for whom
> latency is everything?
>
> I mean, we are adding some extra load/utilization tracking. While I
> think we should do everything possible to minimize the overhead, I think
> it is unrealistic to assume that it will be zero. Is a some extra 'if
> (energy_enabled)' acceptable?
>
> I'm open for other suggestions.
We have the jump-label stuff to do self modifying code ;-) The only
thing we need to be careful with is data-layout.
So I'm _hoping_ we can do all this without more CONFIG knobs, because
{PREEMPT*SMP*CGROUP^3*NUMA^2} is already entirely annoying to
build and run test, not to mention that distro builds will have no other
option than to enable everything anyhow.
On Tue, Jun 10, 2014 at 12:23:53PM +0200, Peter Zijlstra wrote:
> On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> > How would you like to disable the energy stuff for users for whom
> > latency is everything?
> >
> > I mean, we are adding some extra load/utilization tracking. While I
> > think we should do everything possible to minimize the overhead, I think
> > it is unrealistic to assume that it will be zero. Is a some extra 'if
> > (energy_enabled)' acceptable?
> >
> > I'm open for other suggestions.
>
> We have the jump-label stuff to do self modifying code ;-) The only
> thing we need to be careful with is data-layout.
Isn't this asking for trouble?
I do get the point of not introducing more make-ifdeffery, but I'm not
so sure the alternative is much better. Do we really want to spend time
tracing down bugs introduced via a self-modifying process in something
as central as the scheduler?
> So I'm _hoping_ we can do all this without more CONFIG knobs, because
> {PREEMPT*SMP*CGROUP^3*NUMA^2} is already entirely annoying to
> build and run test, not to mention that distro builds will have no other
> option than to enable everything anyhow.
True, but if that is the argument, how is adding this as a dynamic thing
any better, you still end up with a test-matrix of the same size?
Building a kernel isn't _that_ much work and it would make the
test-scripts all the much simpler to maintain if we don't have to rely
on some dynamic tweaking of the core.
Just sayin'
--
Henrik Austad
On Tue, Jun 10, 2014 at 11:23:53AM +0100, Peter Zijlstra wrote:
> On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> > How would you like to disable the energy stuff for users for whom
> > latency is everything?
> >
> > I mean, we are adding some extra load/utilization tracking. While I
> > think we should do everything possible to minimize the overhead, I think
> > it is unrealistic to assume that it will be zero. Is a some extra 'if
> > (energy_enabled)' acceptable?
> >
> > I'm open for other suggestions.
>
> We have the jump-label stuff to do self modifying code ;-) The only
> thing we need to be careful with is data-layout.
Thanks. I can see that it is already used in for various bit in
kernel/sched/*. I didn't catch anything in Documentation/static-keys.txt
related to data-layout caveats. Is there some other
documentation/patches I should read before messing everything up? ;-)
> So I'm _hoping_ we can do all this without more CONFIG knobs, because
> {PREEMPT*SMP*CGROUP^3*NUMA^2} is already entirely annoying to
> build and run test, not to mention that distro builds will have no other
> option than to enable everything anyhow.
Fair enough.
On Tue, Jun 10, 2014 at 01:17:32PM +0200, Henrik Austad wrote:
> On Tue, Jun 10, 2014 at 12:23:53PM +0200, Peter Zijlstra wrote:
> > On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> > > How would you like to disable the energy stuff for users for whom
> > > latency is everything?
> > >
> > > I mean, we are adding some extra load/utilization tracking. While I
> > > think we should do everything possible to minimize the overhead, I think
> > > it is unrealistic to assume that it will be zero. Is a some extra 'if
> > > (energy_enabled)' acceptable?
> > >
> > > I'm open for other suggestions.
> >
> > We have the jump-label stuff to do self modifying code ;-) The only
> > thing we need to be careful with is data-layout.
>
> Isn't this asking for trouble?
>
> I do get the point of not introducing more make-ifdeffery, but I'm not
> so sure the alternative is much better. Do we really want to spend time
> tracing down bugs introduced via a self-modifying process in something
> as central as the scheduler?
Its already chock full of that stuff ;-)
> > So I'm _hoping_ we can do all this without more CONFIG knobs, because
> > {PREEMPT*SMP*CGROUP^3*NUMA^2} is already entirely annoying to
> > build and run test, not to mention that distro builds will have no other
> > option than to enable everything anyhow.
>
> True, but if that is the argument, how is adding this as a dynamic thing
> any better, you still end up with a test-matrix of the same size?
Test-matrix yes, sadly so and there's nothing we can really do about
that, so that sucks.
But it does reduce the coverage of the tests; everything that is not
uber critical fast path we can do unconditionally. So all the
sched_domain wankery gets tested on every boot / hotplug event, which is
tons better than only when that particular option is build in.
So while the total test matrix does suck rocks, the actual code that
needs testing per option can be siginficantly reduced.
> Building a kernel isn't _that_ much work and it would make the
> test-scripts all the much simpler to maintain if we don't have to rely
> on some dynamic tweaking of the core.
its exponential, given that I now already have to build
PREEMPT*SMP*CGROUP^3*NUMA^2 = 2^7 = 128 kernels to cover all options,
adding one more option means I'll have to build another 128 kernels.
Building 128 kernels does take a lot of time, no matter how far you
strip that .config and no matter I can build a kernel in <50 seconds.
On Tue, Jun 10, 2014 at 12:24:03PM +0100, Morten Rasmussen wrote:
> On Tue, Jun 10, 2014 at 11:23:53AM +0100, Peter Zijlstra wrote:
> > On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> > > How would you like to disable the energy stuff for users for whom
> > > latency is everything?
> > >
> > > I mean, we are adding some extra load/utilization tracking. While I
> > > think we should do everything possible to minimize the overhead, I think
> > > it is unrealistic to assume that it will be zero. Is a some extra 'if
> > > (energy_enabled)' acceptable?
> > >
> > > I'm open for other suggestions.
> >
> > We have the jump-label stuff to do self modifying code ;-) The only
> > thing we need to be careful with is data-layout.
>
> Thanks. I can see that it is already used in for various bit in
> kernel/sched/*. I didn't catch anything in Documentation/static-keys.txt
> related to data-layout caveats. Is there some other
> documentation/patches I should read before messing everything up? ;-)
So the data-layout was mostly referring to things like making sure that
struct sched_avg doesn't end up straddling a cacheline somewhere by
accident.
The most expensive part of the per-task accounting nonsense is the
amount of memory we need to touch to do so, the actual instructions come
second, unless of course we go put tons of divisions in there :-)
BTW, are cachelines 64 bytes for you ARM people too?
On Tue, Jun 10, 2014 at 01:24:35PM +0100, Peter Zijlstra wrote:
> On Tue, Jun 10, 2014 at 12:24:03PM +0100, Morten Rasmussen wrote:
> > On Tue, Jun 10, 2014 at 11:23:53AM +0100, Peter Zijlstra wrote:
> > > On Tue, Jun 10, 2014 at 11:06:41AM +0100, Morten Rasmussen wrote:
> > > > How would you like to disable the energy stuff for users for whom
> > > > latency is everything?
> > > >
> > > > I mean, we are adding some extra load/utilization tracking. While I
> > > > think we should do everything possible to minimize the overhead, I think
> > > > it is unrealistic to assume that it will be zero. Is a some extra 'if
> > > > (energy_enabled)' acceptable?
> > > >
> > > > I'm open for other suggestions.
> > >
> > > We have the jump-label stuff to do self modifying code ;-) The only
> > > thing we need to be careful with is data-layout.
> >
> > Thanks. I can see that it is already used in for various bit in
> > kernel/sched/*. I didn't catch anything in Documentation/static-keys.txt
> > related to data-layout caveats. Is there some other
> > documentation/patches I should read before messing everything up? ;-)
>
> So the data-layout was mostly referring to things like making sure that
> struct sched_avg doesn't end up straddling a cacheline somewhere by
> accident.
>
> The most expensive part of the per-task accounting nonsense is the
> amount of memory we need to touch to do so, the actual instructions come
> second, unless of course we go put tons of divisions in there :-)
Make sense.
> BTW, are cachelines 64 bytes for you ARM people too?
Mostly yes, but as with a lot of other things on ARM it is
implementation defined. The cacheline sizes are probeable at runtime,
but for things where we don't know I think 64 bytes is the current
assumption.
Catalin or Will would be able to provide a more detailed answer.