2007-11-19 15:18:30

by Ingo Molnar

[permalink] [raw]
Subject: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


By popular demand, here is release -v24 of the CFS scheduler patch.

It is a full backport of the latest & greatest scheduler code to
v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
downloaded from the usual place:

http://people.redhat.com/mingo/cfs-scheduler/

There's tons of changes since v22 was released:

36 files changed, 2359 insertions(+), 1082 deletions(-)

that's 187 individual commits from 32 authors.

So even if CFS v22 worked well for you, please try this release too and
report regressions (if any).

There are countless improvements in -v24 (see the shortlog further below
for details), but here are a few highlights:

- improved interactivity via Peter Ziljstra's "virtual slices" feature.
As load increases, the scheduler shortens the virtual timeslices that
tasks get, so that applications observe the same constant latency for
getting on the CPU. (This goes on until the slices reach a minimum
granularity value)

- CONFIG_FAIR_USER_SCHED is now available across all backported
kernels and the per user weights are configurable via
/sys/kernel/uids/. Group scheduling got refined all around.

- performance improvements

- bugfixes

99% of these changes are already upstream in Linus's git tree and they
will be released as part of v2.6.24. (there are 4 pending commits that
are in the small 2.6.24-rc3-v24 patch.)

As usual, any sort of feedback, bugreport, fix and suggestion is more
than welcome!

Ingo

------------------>
Adrian Bunk (3):
sched: make kernel/sched.c:account_guest_time() static
sched: proper prototype for kernel/sched.c:migration_init()
sched: make sched_nr_latency static

Alexey Dobriyan (1):
sched: uninline scheduler

Andi Kleen (5):
sched: cleanup: remove unnecessary gotos
sched: cleanup: refactor common code of sleep_on / wait_for_completion
sched: cleanup: refactor normalize_rt_tasks
sched: remove stale comment from sched_group_set_shares()
sched: fix return value of wait_for_completion_interruptible()

Arjan van de Ven (1):
Make scheduler debug file operations const

Balbir Singh (1):
sched: fix delay accounting regression

Christian Borntraeger (1):
sched: fix accounting of interrupts during guest execution on s390

Cliff Wickman (1):
hotplug cpu: migrate a task within its cpuset

Dhaval Giani (1):
sched: group scheduling, sysfs tunables

Dmitry Adamushko (16):
sched: clean up struct load_stat
sched: clean up schedstat block in dequeue_entity()
sched: sched_setscheduler() fix
sched: add set_curr_task() calls
sched: do not keep current in the tree and get rid of sched_entity::fair_key
sched: optimize task_new_fair()
sched: simplify sched_class::yield_task()
sched: rework enqueue/dequeue_entity() to get rid of set_curr_task()
sched: yield fix
sched: fix __pick_next_entity()
sched: tidy up SCHED_RR
sched: cleanup, remove calc_weighted()
sched: cleanup, make dequeue_entity() and update_stats_wait_end() similar
sched: fix group scheduling for SCHED_BATCH
sched: fix __set_task_cpu() SMP race
sched: remove activate_idle_task()

Eric Dumazet (1):
sched: cleanup, use NSEC_PER_MSEC and NSEC_PER_SEC

Eugene Teo (1):
Fix tsk->exit_state usage

Gautham R Shenoy (1):
sched: fix rt ptracer monopolizing CPU

Hiroshi Shimamoto (1):
sched: clean up sched_fork()

Ingo Molnar (80):
sched: fix sysctl_sched_child_runs_first flag
sched: resched task in task_new_fair()
sched: small sched_debug cleanup
sched: debug: track maximum 'slice'
sched: uniform tunings
sched: use constants if !CONFIG_SCHED_DEBUG
sched: remove stat_gran
sched: remove precise CPU load
sched: remove precise CPU load calculations #2
sched: track cfs_rq->curr on !group-scheduling too
sched: cleanup: simplify cfs_rq_curr() methods
sched: uninline __enqueue_entity()/__dequeue_entity()
sched: speed up update_load_add/_sub()
sched: clean up calc_weighted()
sched: introduce se->vruntime
sched: move sched_feat() definitions
sched: optimize vruntime based scheduling
sched: simplify check_preempt() methods
sched: wakeup granularity increase
sched: add se->vruntime debugging
sched: remove SCHED_FEAT_SKIP_INITIAL
sched: add more vruntime statistics
sched: debug: update exec_clock only when SCHED_DEBUG
sched: remove wait_runtime limit
sched: remove wait_runtime fields and features
sched: fix delay accounting performance regression
sched: prettify /proc/sched_debug output
sched: enhance debug output
sched: kernel/sched_fair.c whitespace cleanups
sched debug: BKL usage statistics
sched: remove unneeded tunables
sched debug: print settings
sched debug: more width for parameter printouts
sched: entity_key() fix
sched: remove condition from set_task_cpu()
sched: remove last_min_vruntime effect
sched: undo some of the recent changes
sched: fix sign check error in place_entity()
sched: fix sched_fork()
sched: remove set_leftmost()
sched: clean up schedstats, cnt -> count
sched: cleanup, remove stale comment
sched: mark scheduling classes as const
sched: whitespace cleanups
sched: vslice fixups for non-0 nice levels
sched: optimize schedule() a bit on SMP
sched: tweak wakeup granularity
sched: run sched_domain_debug() if CONFIG_SCHED_DEBUG=y
sched: break out if printing a warning in sched_domain_debug()
sched: style cleanup
sched: kfree(NULL) is valid
sched: cleanup: rename SCHED_FEAT_USE_TREE_AVG to SCHED_FEAT_TREE_AVG
sched: cleanup: rename task_grp to task_group
sched: cleanup: function prototype cleanups
sched: fix: move the CPU check into ->task_new_fair()
sched: update comment
sched: clean up is_migration_thread()
sched: do not normalize kernel threads via SysRq-N
sched: do not wakeup-preempt with SCHED_BATCH tasks
sched: speed up context-switches a bit
sched: reintroduce cache-hot affinity
sched: debug: increase width of debug line
sched: debug, improve migration statistics
sched: allow the immediate migration of cache-cold tasks
sched: affine sync wakeups
sched: sync wakeups preempt too
sched: cleanup, fix spacing
sched: cleanup, make struct rq comments more consistent
sched: add KERN_CONT annotation
sched: fix fastcall mismatch in completion APIs
sched: clean up sched_domain_debug()
sched: fix style of swap() macro in kernel/sched_fair.c
sched: fix style in kernel/sched.c
sched: reintroduce SMP tunings again
sched: turn off PREEMPT_RESTRICT
sched: remove PREEMPT_RESTRICT
sched: wakeup preemption fix
sched: clean up the wakeup preempt check
sched: clean up the wakeup preempt check, #2
sched: reorder SCHED_FEAT_ bits

James Bottomley (1):
sched: fix incorrect assumption that cpu 0 exists

Ken Chen (2):
sched: fix improper load balance across sched domain
sched: reduce schedstat variable overhead a bit

Laurent Vivier (2):
sched: guest CPU accounting: maintain stats in account_system_time()
sched: don't clear PF_VCPU in scheduler

Matthias Kaehlcke (1):
sched: use list_for_each_entry_safe() in __wake_up_common()

Michael Neuling (2):
Add scaled time to taskstats based process accounting
kernel/sched.c: remove bogus comment from account_user_time

Mike Galbraith (3):
sched: fix SMP migration latencies
sched: fix formatting of /proc/sched_debug
sched: prevent wakeup over-scheduling

Milton Miller (7):
sched: domain sysctl fixes: use kcalloc()
sched: domain sysctl fixes: use for_each_online_cpu()
sched: domain sysctl fixes: unregister the sysctl table before domains
sched: domain sysctl fixes: do not crash on allocation failure
sched: domain sysctl fixes: add terminator comment
sched: more robust sd-sysctl entry freeing
sched: fix sched_domain sysctl registration again

Oleg Nesterov (3):
do CPU_DEAD migrating under read_lock(tasklist) instead of write_lock_irq(tasklist)
migration_call(CPU_DEAD): use spin_lock_irq() instead of task_rq_lock()
sched: fix SCHED_FIFO tasks & FAIR_GROUP_SCHED

Paul E. McKenney (1):
sched: export cpu_clock()

Paul Jackson (2):
cpuset: remove sched domain hooks from cpusets
cpuset sched_load_balance flag

Paul Menage (4):
Task Control Groups: example CPU accounting subsystem
Fix cpusets update_cpumask
sched: clean up some control group code
sched: report CPU usage in CFS cgroup directories

Pavel Emelyanov (3):
pid namespaces: changes to show virtual ids to user
Uninline find_task_by_xxx set of functions
Use helpers to obtain task pid in printks

Peter Williams (2):
sched: reduce balance-tasks overhead
sched: isolate SMP balancing code a bit more

Peter Zijlstra (21):
sched: simplify SCHED_FEAT_* code
sched: new task placement for vruntime
sched: simplify adaptive latency
sched: clean up new task placement
sched: add tree based averages
sched: handle vruntime 64-bit overflow
sched: better min_vruntime tracking
sched: add vslice
sched debug: check spread
sched: max_vruntime() simplification
sched: clean up min_vruntime use
sched: speed up and simplify vslice calculations
sched: another wakeup_granularity fix
sched: disable sleeper_fairness on SCHED_BATCH
sched: disable forced preemption by default
sched: activate task_hot() only on fair-scheduled tasks
sched: fix unconditional irq lock
sched: fix vslice
sched: documentation: place_entity() comments
sched: reintroduce the sched_min_granularity tunable
sched: avoid large irq-latencies in smp-balancing

S.Caglar Onur (1):
sched debug: BKL usage statistics, fix

Satyam Sharma (1):
sched: use show_regs() to improve __schedule_bug() output

Srivatsa Vaddagiri (16):
sched: group-scheduler core
sched: revert recent removal of set_curr_task()
sched: fix minor bug in yield
sched: print nr_running and load in /proc/sched_debug
sched: print &rq->cfs stats
sched: clean up code under CONFIG_FAIR_GROUP_SCHED
sched: add fair-user scheduler
sched: group scheduler wakeup latency fix
sched: group scheduler SMP migration fix
sched: group scheduler, fix coding style issues
sched: group scheduler, fix bloat
sched: group scheduler, fix latency
sched: fix new task startup crash
Hook up group scheduler with control groups
sched: move rcu_head to task_group struct
sched: fix copy_namespace() <-> sched_fork() dependency in do_fork

Zou Nan hai (1):
sched: some proc entries are missed in sched_domain sys_ctl debug code


2007-11-19 16:05:25

by Vincent Fortier

[permalink] [raw]
Subject: RE: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8,v2.6.22.13, v2.6.21.7

> -----Message d'origine-----
> De : [email protected]
> [mailto:[email protected]] De la part de Ingo Molnar
>
> By popular demand, here is release -v24 of the CFS scheduler patch.
>
> It is a full backport of the latest & greatest scheduler code
> to v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches
> can be downloaded from the usual place:
>
> http://people.redhat.com/mingo/cfs-scheduler/
>
> There's tons of changes since v22 was released:
>
> 36 files changed, 2359 insertions(+), 1082 deletions(-)
>
> that's 187 individual commits from 32 authors.
>
> So even if CFS v22 worked well for you, please try this
> release too and report regressions (if any).
>

Hi Ingo,

Thnx a lot for theses backports...

Ran into this while compiling a 2.6.22.13 with CFS v24

CC kernel/sched.o
kernel/sched.c: In function 'cpu_to_core_group':
kernel/sched.c:6056: error: 'per_cpu__cpu_sibling_map' undeclared (first
use in this function)
kernel/sched.c:6056: error: (Each undeclared identifier is reported only
once
kernel/sched.c:6056: error: for each function it appears in.)
kernel/sched.c:6056: warning: type defaults to 'int' in declaration of
'type name'
kernel/sched.c:6056: error: invalid type argument of 'unary *'
kernel/sched.c: In function 'build_sched_domains':
kernel/sched.c:6319: error: 'per_cpu__cpu_sibling_map' undeclared (first
use in this function)
kernel/sched.c:6319: warning: type defaults to 'int' in declaration of
'type name'
kernel/sched.c:6319: error: invalid type argument of 'unary *'
kernel/sched.c:6330: warning: type defaults to 'int' in declaration of
'type name'
kernel/sched.c:6330: error: invalid type argument of 'unary *'
make[2]: *** [kernel/sched.o] Error 1
make[1]: *** [kernel] Error 2
make[1]: Leaving directory
`/usr/src/linux-headers-2.6.22.13-cfs-sarge-686-envcan'
make: *** [debian/stamp-build-kernel] Error 2

- vin

2007-11-19 18:47:00

by Erik Andersen

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

On Mon Nov 19, 2007 at 04:17:56PM +0100, Ingo Molnar wrote:
>
> By popular demand, here is release -v24 of the CFS scheduler patch.
>
> It is a full backport of the latest & greatest scheduler code to
> v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> downloaded from the usual place:
>
> http://people.redhat.com/mingo/cfs-scheduler/

Doesn't compile with 2.6.21.7:

kernel/sched.c: In function ‘cpu_to_core_group’:
kernel/sched.c:6056: error: ‘per_cpu__cpu_sibling_map’ undeclared (first use in this function)
kernel/sched.c:6056: error: (Each undeclared identifier is reported only once
kernel/sched.c:6056: error: for each function it appears in.)
kernel/sched.c:6056: warning: type defaults to ‘int’ in declaration of ‘type name’
kernel/sched.c:6056: error: invalid type argument of ‘unary *’
kernel/sched.c: In function ‘build_sched_domains’:
kernel/sched.c:6319: error: ‘per_cpu__cpu_sibling_map’ undeclared (first use in this function)
kernel/sched.c:6319: warning: type defaults to ‘int’ in declaration of ‘type name’
kernel/sched.c:6319: error: invalid type argument of ‘unary *’
kernel/sched.c:6330: warning: type defaults to ‘int’ in declaration of ‘type name’
kernel/sched.c:6330: error: invalid type argument of ‘unary *’
make[1]: *** [kernel/sched.o] Error 1
make[1]: *** Waiting for unfinished jobs....

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

2007-11-19 18:52:54

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Erik Andersen <[email protected]> wrote:

> On Mon Nov 19, 2007 at 04:17:56PM +0100, Ingo Molnar wrote:
> >
> > By popular demand, here is release -v24 of the CFS scheduler patch.
> >
> > It is a full backport of the latest & greatest scheduler code to
> > v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> > downloaded from the usual place:
> >
> > http://people.redhat.com/mingo/cfs-scheduler/
>
> Doesn't compile with 2.6.21.7:

does the patch below help?

Ingo

Index: linux-cfs-2.6.21.7.q/include/linux/sched.h
===================================================================
--- linux-cfs-2.6.21.7.q.orig/include/linux/sched.h
+++ linux-cfs-2.6.21.7.q/include/linux/sched.h
@@ -10,7 +10,7 @@
#define COMPAT_REGISTER_SYSCTL

/* backporting helper macro: */
-#define cpu_sibling_map(cpu) per_cpu(cpu_sibling_map, cpu)
+#define cpu_sibling_map(cpu) cpu_sibling_map[cpu]

/*
* cloning flags:

2007-11-19 18:59:35

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8,v2.6.22.13, v2.6.21.7


* Fortier,Vincent [Montreal] <[email protected]> wrote:

> Hi Ingo,
>
> Thnx a lot for theses backports...
>
> Ran into this while compiling a 2.6.22.13 with CFS v24
>
> CC kernel/sched.o
> kernel/sched.c: In function 'cpu_to_core_group':
> kernel/sched.c:6056: error: 'per_cpu__cpu_sibling_map' undeclared (first

find below the current set of fixlets - it should fix the
cpu_sibling_map build error too. (or re-download the v24 patch again to
get the fixes)

Ingo

--- linux.orig/kernel/sysctl.c
+++ linux/kernel/sysctl.c
@@ -309,6 +309,7 @@ static struct ctl_table kern_table[] = {
.mode = 644,
.proc_handler = &proc_dointvec,
},
+#ifdef CONFIG_FAIR_GROUP_SCHED
{
.ctl_name = CTL_UNNUMBERED,
.procname = "sched_min_bal_int_shares",
@@ -326,6 +327,7 @@ static struct ctl_table kern_table[] = {
.proc_handler = &proc_dointvec,
},
#endif
+#endif
{
.ctl_name = CTL_UNNUMBERED,
.procname = "sched_compat_yield",
--- linux.orig/include/linux/sched.h
+++ linux/include/linux/sched.h
@@ -10,7 +10,7 @@
#define COMPAT_REGISTER_SYSCTL

/* backporting helper macro: */
-#define cpu_sibling_map(cpu) per_cpu(cpu_sibling_map, cpu)
+#define cpu_sibling_map(cpu) cpu_sibling_map[cpu]

/*
* cloning flags:

2007-11-19 19:24:25

by Erik Andersen

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

On Mon Nov 19, 2007 at 07:52:37PM +0100, Ingo Molnar wrote:
>
> * Erik Andersen <[email protected]> wrote:
>
> > On Mon Nov 19, 2007 at 04:17:56PM +0100, Ingo Molnar wrote:
> > >
> > > By popular demand, here is release -v24 of the CFS scheduler patch.
> > >
> > > It is a full backport of the latest & greatest scheduler code to
> > > v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> > > downloaded from the usual place:
> > >
> > > http://people.redhat.com/mingo/cfs-scheduler/
> >
> > Doesn't compile with 2.6.21.7:
>
> does the patch below help?

Yes, compiles and boots with that change,

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

2007-11-19 19:46:48

by Vincent Fortier

[permalink] [raw]
Subject: RE: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3,v2.6.23.8, v2.6.22.13, v2.6.21.7

> -----Message d'origine-----
> De : [email protected]
> [mailto:[email protected]] De la part de Ingo Molnar
>
>
> * Erik Andersen <[email protected]> wrote:
>
> > On Mon Nov 19, 2007 at 04:17:56PM +0100, Ingo Molnar wrote:
> > >
> > > By popular demand, here is release -v24 of the CFS
> scheduler patch.
> > >
> > > It is a full backport of the latest & greatest scheduler code to
> > > v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> > > downloaded from the usual place:
> > >
> > > http://people.redhat.com/mingo/cfs-scheduler/
> >
> > Doesn't compile with 2.6.21.7:
>
> does the patch below help?
>

Looks good. Compiling.

- vin

2007-11-20 11:58:44

by Damien Wyart

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

Hello,

* Ingo Molnar <[email protected]> [2007-11-19 16:17]:
> By popular demand, here is release -v24 of the CFS scheduler patch. It
> is a full backport of the latest & greatest scheduler code to
> v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> downloaded from the usual place:

> http://people.redhat.com/mingo/cfs-scheduler/

> As usual, any sort of feedback, bugreport, fix and suggestion is more
> than welcome!

Testing sched-cfs-v2.6.24-rc3-v24.patch on top of 2.6.24-rc3-git1
(ignored the two "already applied" messages coming from git1 commits),
I get a 1.00 minimum load in top, coming from the load_balance_mo thread
staying in D-state. I get this on 2 different computers with similar
configs, so I am attaching one of them here.

--
Damien Wyart


Attachments:
(No filename) (783.00 B)
config (36.78 kB)
Download all attachments

2007-11-20 12:28:54

by Srivatsa Vaddagiri

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

On Tue, Nov 20, 2007 at 12:24:59PM +0100, Damien Wyart wrote:
> Testing sched-cfs-v2.6.24-rc3-v24.patch on top of 2.6.24-rc3-git1
> (ignored the two "already applied" messages coming from git1 commits),
> I get a 1.00 minimum load in top, coming from the load_balance_mo thread
> staying in D-state. I get this on 2 different computers with similar
> configs, so I am attaching one of them here.

Ingo,
I am reworking the fair-group load balance patches. Can you drop
what you have until I resubmit?

--
Regards,
vatsa

2007-11-20 12:56:21

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Damien Wyart <[email protected]> wrote:

> > As usual, any sort of feedback, bugreport, fix and suggestion is
> > more than welcome!
>
> Testing sched-cfs-v2.6.24-rc3-v24.patch on top of 2.6.24-rc3-git1
> (ignored the two "already applied" messages coming from git1 commits),
> I get a 1.00 minimum load in top, coming from the load_balance_mo
> thread staying in D-state. I get this on 2 different computers with
> similar configs, so I am attaching one of them here.

does the patch below fix that problem for you?

Ingo

Index: linux/kernel/sched.c
===================================================================
--- linux.orig/kernel/sched.c
+++ linux/kernel/sched.c
@@ -7138,7 +7138,7 @@ static int load_balance_monitor(void *un
else if (timeout < sysctl_sched_max_bal_int_shares)
timeout *= 2;

- msleep(timeout);
+ msleep_interruptible(timeout);
}

return 0;

2007-11-20 13:22:29

by Damien Wyart

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

> > Testing sched-cfs-v2.6.24-rc3-v24.patch on top of 2.6.24-rc3-git1
> > (ignored the two "already applied" messages coming from git1
> > commits), I get a 1.00 minimum load in top, coming from the
> > load_balance_mo thread staying in D-state. I get this on 2 different
> > computers with similar configs, so I am attaching one of them here.

* Ingo Molnar <[email protected]> [071120 13:55]:
> does the patch below fix that problem for you?

Yes, it does. Thanks for your answer.

Damien

> Index: linux/kernel/sched.c
> ===================================================================
> --- linux.orig/kernel/sched.c
> +++ linux/kernel/sched.c
> @@ -7138,7 +7138,7 @@ static int load_balance_monitor(void *un
> else if (timeout < sysctl_sched_max_bal_int_shares)
> timeout *= 2;
>
> - msleep(timeout);
> + msleep_interruptible(timeout);
> }
>
> return 0;
>

2007-11-20 20:00:38

by Vincent Fortier

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

On Mon, 2007-11-19 at 16:17 +0100, Ingo Molnar wrote:
> By popular demand, here is release -v24 of the CFS scheduler patch.
>
> It is a full backport of the latest & greatest scheduler code to
> v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> downloaded from the usual place:
>
> http://people.redhat.com/mingo/cfs-scheduler/
>
> There's tons of changes since v22 was released:
>
> 36 files changed, 2359 insertions(+), 1082 deletions(-)
>
> that's 187 individual commits from 32 authors.
>
> So even if CFS v22 worked well for you, please try this release too and
> report regressions (if any).
>
Hi Ingo,

Tested latest v24 patch against 686 & x86_64 and it runs smoothly, at
least as well as previous v22 release.

As always, Thnx very much for this backport.

- vin


2007-11-20 20:38:56

by Fabio Comolli

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

Hi.
First, thanks for this patch. I don't have numbers but working with my
laptop feels much better.
Just a question: does the patch include the fix (divide by zero....)
you just posted in the stable review for 2.6.23.9?

Thanks and regards,
Fabio



On Nov 19, 2007 4:17 PM, Ingo Molnar <[email protected]> wrote:
>
> By popular demand, here is release -v24 of the CFS scheduler patch.
>
> It is a full backport of the latest & greatest scheduler code to
> v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> downloaded from the usual place:
>
> http://people.redhat.com/mingo/cfs-scheduler/
>
> There's tons of changes since v22 was released:
>
> 36 files changed, 2359 insertions(+), 1082 deletions(-)
>
> that's 187 individual commits from 32 authors.
>
> So even if CFS v22 worked well for you, please try this release too and
> report regressions (if any).
>
> There are countless improvements in -v24 (see the shortlog further below
> for details), but here are a few highlights:
>
> - improved interactivity via Peter Ziljstra's "virtual slices" feature.
> As load increases, the scheduler shortens the virtual timeslices that
> tasks get, so that applications observe the same constant latency for
> getting on the CPU. (This goes on until the slices reach a minimum
> granularity value)
>
> - CONFIG_FAIR_USER_SCHED is now available across all backported
> kernels and the per user weights are configurable via
> /sys/kernel/uids/. Group scheduling got refined all around.
>
> - performance improvements
>
> - bugfixes
>
> 99% of these changes are already upstream in Linus's git tree and they
> will be released as part of v2.6.24. (there are 4 pending commits that
> are in the small 2.6.24-rc3-v24 patch.)
>
> As usual, any sort of feedback, bugreport, fix and suggestion is more
> than welcome!
>
> Ingo
>
> ------------------>
> Adrian Bunk (3):
> sched: make kernel/sched.c:account_guest_time() static
> sched: proper prototype for kernel/sched.c:migration_init()
> sched: make sched_nr_latency static
>
> Alexey Dobriyan (1):
> sched: uninline scheduler
>
> Andi Kleen (5):
> sched: cleanup: remove unnecessary gotos
> sched: cleanup: refactor common code of sleep_on / wait_for_completion
> sched: cleanup: refactor normalize_rt_tasks
> sched: remove stale comment from sched_group_set_shares()
> sched: fix return value of wait_for_completion_interruptible()
>
> Arjan van de Ven (1):
> Make scheduler debug file operations const
>
> Balbir Singh (1):
> sched: fix delay accounting regression
>
> Christian Borntraeger (1):
> sched: fix accounting of interrupts during guest execution on s390
>
> Cliff Wickman (1):
> hotplug cpu: migrate a task within its cpuset
>
> Dhaval Giani (1):
> sched: group scheduling, sysfs tunables
>
> Dmitry Adamushko (16):
> sched: clean up struct load_stat
> sched: clean up schedstat block in dequeue_entity()
> sched: sched_setscheduler() fix
> sched: add set_curr_task() calls
> sched: do not keep current in the tree and get rid of sched_entity::fair_key
> sched: optimize task_new_fair()
> sched: simplify sched_class::yield_task()
> sched: rework enqueue/dequeue_entity() to get rid of set_curr_task()
> sched: yield fix
> sched: fix __pick_next_entity()
> sched: tidy up SCHED_RR
> sched: cleanup, remove calc_weighted()
> sched: cleanup, make dequeue_entity() and update_stats_wait_end() similar
> sched: fix group scheduling for SCHED_BATCH
> sched: fix __set_task_cpu() SMP race
> sched: remove activate_idle_task()
>
> Eric Dumazet (1):
> sched: cleanup, use NSEC_PER_MSEC and NSEC_PER_SEC
>
> Eugene Teo (1):
> Fix tsk->exit_state usage
>
> Gautham R Shenoy (1):
> sched: fix rt ptracer monopolizing CPU
>
> Hiroshi Shimamoto (1):
> sched: clean up sched_fork()
>
> Ingo Molnar (80):
> sched: fix sysctl_sched_child_runs_first flag
> sched: resched task in task_new_fair()
> sched: small sched_debug cleanup
> sched: debug: track maximum 'slice'
> sched: uniform tunings
> sched: use constants if !CONFIG_SCHED_DEBUG
> sched: remove stat_gran
> sched: remove precise CPU load
> sched: remove precise CPU load calculations #2
> sched: track cfs_rq->curr on !group-scheduling too
> sched: cleanup: simplify cfs_rq_curr() methods
> sched: uninline __enqueue_entity()/__dequeue_entity()
> sched: speed up update_load_add/_sub()
> sched: clean up calc_weighted()
> sched: introduce se->vruntime
> sched: move sched_feat() definitions
> sched: optimize vruntime based scheduling
> sched: simplify check_preempt() methods
> sched: wakeup granularity increase
> sched: add se->vruntime debugging
> sched: remove SCHED_FEAT_SKIP_INITIAL
> sched: add more vruntime statistics
> sched: debug: update exec_clock only when SCHED_DEBUG
> sched: remove wait_runtime limit
> sched: remove wait_runtime fields and features
> sched: fix delay accounting performance regression
> sched: prettify /proc/sched_debug output
> sched: enhance debug output
> sched: kernel/sched_fair.c whitespace cleanups
> sched debug: BKL usage statistics
> sched: remove unneeded tunables
> sched debug: print settings
> sched debug: more width for parameter printouts
> sched: entity_key() fix
> sched: remove condition from set_task_cpu()
> sched: remove last_min_vruntime effect
> sched: undo some of the recent changes
> sched: fix sign check error in place_entity()
> sched: fix sched_fork()
> sched: remove set_leftmost()
> sched: clean up schedstats, cnt -> count
> sched: cleanup, remove stale comment
> sched: mark scheduling classes as const
> sched: whitespace cleanups
> sched: vslice fixups for non-0 nice levels
> sched: optimize schedule() a bit on SMP
> sched: tweak wakeup granularity
> sched: run sched_domain_debug() if CONFIG_SCHED_DEBUG=y
> sched: break out if printing a warning in sched_domain_debug()
> sched: style cleanup
> sched: kfree(NULL) is valid
> sched: cleanup: rename SCHED_FEAT_USE_TREE_AVG to SCHED_FEAT_TREE_AVG
> sched: cleanup: rename task_grp to task_group
> sched: cleanup: function prototype cleanups
> sched: fix: move the CPU check into ->task_new_fair()
> sched: update comment
> sched: clean up is_migration_thread()
> sched: do not normalize kernel threads via SysRq-N
> sched: do not wakeup-preempt with SCHED_BATCH tasks
> sched: speed up context-switches a bit
> sched: reintroduce cache-hot affinity
> sched: debug: increase width of debug line
> sched: debug, improve migration statistics
> sched: allow the immediate migration of cache-cold tasks
> sched: affine sync wakeups
> sched: sync wakeups preempt too
> sched: cleanup, fix spacing
> sched: cleanup, make struct rq comments more consistent
> sched: add KERN_CONT annotation
> sched: fix fastcall mismatch in completion APIs
> sched: clean up sched_domain_debug()
> sched: fix style of swap() macro in kernel/sched_fair.c
> sched: fix style in kernel/sched.c
> sched: reintroduce SMP tunings again
> sched: turn off PREEMPT_RESTRICT
> sched: remove PREEMPT_RESTRICT
> sched: wakeup preemption fix
> sched: clean up the wakeup preempt check
> sched: clean up the wakeup preempt check, #2
> sched: reorder SCHED_FEAT_ bits
>
> James Bottomley (1):
> sched: fix incorrect assumption that cpu 0 exists
>
> Ken Chen (2):
> sched: fix improper load balance across sched domain
> sched: reduce schedstat variable overhead a bit
>
> Laurent Vivier (2):
> sched: guest CPU accounting: maintain stats in account_system_time()
> sched: don't clear PF_VCPU in scheduler
>
> Matthias Kaehlcke (1):
> sched: use list_for_each_entry_safe() in __wake_up_common()
>
> Michael Neuling (2):
> Add scaled time to taskstats based process accounting
> kernel/sched.c: remove bogus comment from account_user_time
>
> Mike Galbraith (3):
> sched: fix SMP migration latencies
> sched: fix formatting of /proc/sched_debug
> sched: prevent wakeup over-scheduling
>
> Milton Miller (7):
> sched: domain sysctl fixes: use kcalloc()
> sched: domain sysctl fixes: use for_each_online_cpu()
> sched: domain sysctl fixes: unregister the sysctl table before domains
> sched: domain sysctl fixes: do not crash on allocation failure
> sched: domain sysctl fixes: add terminator comment
> sched: more robust sd-sysctl entry freeing
> sched: fix sched_domain sysctl registration again
>
> Oleg Nesterov (3):
> do CPU_DEAD migrating under read_lock(tasklist) instead of write_lock_irq(tasklist)
> migration_call(CPU_DEAD): use spin_lock_irq() instead of task_rq_lock()
> sched: fix SCHED_FIFO tasks & FAIR_GROUP_SCHED
>
> Paul E. McKenney (1):
> sched: export cpu_clock()
>
> Paul Jackson (2):
> cpuset: remove sched domain hooks from cpusets
> cpuset sched_load_balance flag
>
> Paul Menage (4):
> Task Control Groups: example CPU accounting subsystem
> Fix cpusets update_cpumask
> sched: clean up some control group code
> sched: report CPU usage in CFS cgroup directories
>
> Pavel Emelyanov (3):
> pid namespaces: changes to show virtual ids to user
> Uninline find_task_by_xxx set of functions
> Use helpers to obtain task pid in printks
>
> Peter Williams (2):
> sched: reduce balance-tasks overhead
> sched: isolate SMP balancing code a bit more
>
> Peter Zijlstra (21):
> sched: simplify SCHED_FEAT_* code
> sched: new task placement for vruntime
> sched: simplify adaptive latency
> sched: clean up new task placement
> sched: add tree based averages
> sched: handle vruntime 64-bit overflow
> sched: better min_vruntime tracking
> sched: add vslice
> sched debug: check spread
> sched: max_vruntime() simplification
> sched: clean up min_vruntime use
> sched: speed up and simplify vslice calculations
> sched: another wakeup_granularity fix
> sched: disable sleeper_fairness on SCHED_BATCH
> sched: disable forced preemption by default
> sched: activate task_hot() only on fair-scheduled tasks
> sched: fix unconditional irq lock
> sched: fix vslice
> sched: documentation: place_entity() comments
> sched: reintroduce the sched_min_granularity tunable
> sched: avoid large irq-latencies in smp-balancing
>
> S.Caglar Onur (1):
> sched debug: BKL usage statistics, fix
>
> Satyam Sharma (1):
> sched: use show_regs() to improve __schedule_bug() output
>
> Srivatsa Vaddagiri (16):
> sched: group-scheduler core
> sched: revert recent removal of set_curr_task()
> sched: fix minor bug in yield
> sched: print nr_running and load in /proc/sched_debug
> sched: print &rq->cfs stats
> sched: clean up code under CONFIG_FAIR_GROUP_SCHED
> sched: add fair-user scheduler
> sched: group scheduler wakeup latency fix
> sched: group scheduler SMP migration fix
> sched: group scheduler, fix coding style issues
> sched: group scheduler, fix bloat
> sched: group scheduler, fix latency
> sched: fix new task startup crash
> Hook up group scheduler with control groups
> sched: move rcu_head to task_group struct
> sched: fix copy_namespace() <-> sched_fork() dependency in do_fork
>
> Zou Nan hai (1):
> sched: some proc entries are missed in sched_domain sys_ctl debug code
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2007-11-20 20:41:25

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Fabio Comolli <[email protected]> wrote:

> Hi.
> First, thanks for this patch. I don't have numbers but working with my
> laptop feels much better.

curious: what scheduler/kernel version have you used before?

> Just a question: does the patch include the fix (divide by zero....)
> you just posted in the stable review for 2.6.23.9?

yeah.

Ingo

2007-11-20 20:48:01

by Fabio Comolli

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

Hi

On Nov 20, 2007 9:41 PM, Ingo Molnar <[email protected]> wrote:
>
> * Fabio Comolli <[email protected]> wrote:
>
> > Hi.
> > First, thanks for this patch. I don't have numbers but working with my
> > laptop feels much better.
>
> curious: what scheduler/kernel version have you used before?

I was using 2.6.23.1 with ck patches.

>
> > Just a question: does the patch include the fix (divide by zero....)
> > you just posted in the stable review for 2.6.23.9?
>
> yeah.

OK, thanks. Have you already tried to apply the patch to 2.6.23.9?
Shall we expect some other rejects?

>
> Ingo
>

Regards,
Fabio

2007-11-20 21:53:52

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Fabio Comolli <[email protected]> wrote:

> Hi
>
> On Nov 20, 2007 9:41 PM, Ingo Molnar <[email protected]> wrote:
> >
> > * Fabio Comolli <[email protected]> wrote:
> >
> > > Hi.
> > > First, thanks for this patch. I don't have numbers but working with my
> > > laptop feels much better.
> >
> > curious: what scheduler/kernel version have you used before?
>
> I was using 2.6.23.1 with ck patches.

are you sure? The last -ck patch i can find is for .22:

http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/

or have they been forward ported? (if yes, do you have an URL for that)

(my guess is you used the 2.6.23.1 scheduler (CFS), so the improvement
you felt on the laptop is relative to 2.6.23-vanilla in essence - which
is good news.)

> > > Just a question: does the patch include the fix (divide by
> > > zero....) you just posted in the stable review for 2.6.23.9?
> >
> > yeah.
>
> OK, thanks. Have you already tried to apply the patch to 2.6.23.9?
> Shall we expect some other rejects?

hm:

The latest stable version of the Linux kernel is: 2.6.23.8

so there's no 23.9 yet.

Ingo

2007-11-20 22:48:44

by Matt

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

> are you sure? The last -ck patch i can find is for .22:

> http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/

> or have they been forward ported? (if yes, do you have an URL for that)

> (my guess is you used the 2.6.23.1 scheduler (CFS), so the improvement
> you felt on the laptop is relative to 2.6.23-vanilla in essence - which
> is good news.)

waninkoko over at forums.gentoo.org tries to forward port ck-patchset
as long as he can, as you can see
http://forums.gentoo.org/viewtopic-t-577970.html
(see later kamikaze-sources / zen-sources / klight-sources for more info)
ck-patchset & cfs has been an excellent combination so far ...

Cheers

Mat

2007-11-20 22:59:31

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Matthew <[email protected]> wrote:

> > are you sure? The last -ck patch i can find is for .22:
>
> > http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/
>
> > or have they been forward ported? (if yes, do you have an URL for that)
>
> > (my guess is you used the 2.6.23.1 scheduler (CFS), so the improvement
> > you felt on the laptop is relative to 2.6.23-vanilla in essence - which
> > is good news.)
>
> waninkoko over at forums.gentoo.org tries to forward port ck-patchset
> as long as he can, as you can see
> http://forums.gentoo.org/viewtopic-t-577970.html
> (see later kamikaze-sources / zen-sources / klight-sources for more info)
> ck-patchset & cfs has been an excellent combination so far ...

ok, it's great that people are picking up -ck. What i was trying to
figure out here was whether this is a vanilla-O(1)->CFS, SD->CFS or
CFS.23->CFS.24 comparison. It seems it's probably a CFS.23->CFS.24
comparison. I'd also be interested in those people's impressions about
CFSv24 who previously found SD to be superior to earlier CFS versions.
If there's still any regression (interactivity or other) we might still
be able to fix it before 2.6.24 is released.

Ingo

2007-11-21 08:50:41

by Fabio Comolli

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

Hi

On Nov 20, 2007 10:53 PM, Ingo Molnar <[email protected]> wrote:
<snip>
> > >
> > > curious: what scheduler/kernel version have you used before?
> >
> > I was using 2.6.23.1 with ck patches.
>
> are you sure? The last -ck patch i can find is for .22:
>
> http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/
>
> or have they been forward ported? (if yes, do you have an URL for that)
>
> (my guess is you used the 2.6.23.1 scheduler (CFS), so the improvement
> you felt on the laptop is relative to 2.6.23-vanilla in essence - which
> is good news.)

Well, it seems that Matthew has been faster than me :-) Yes, I got -ck
patches from kamikaze sources.

>
> > > > Just a question: does the patch include the fix (divide by
> > > > zero....) you just posted in the stable review for 2.6.23.9?
> > >
> > > yeah.
> >
> > OK, thanks. Have you already tried to apply the patch to 2.6.23.9?
> > Shall we expect some other rejects?
>
> hm:
>
> The latest stable version of the Linux kernel is: 2.6.23.8
>
> so there's no 23.9 yet.
>
> Ingo
>

I meant the 23.9 patches that Greg KH posted some hours ago for
review. I guess I will find myself in a couple of days.

To answer your latest mail, again I don't have numbers but from my
point ov view:

23.1 < 23.1+ck < 23.8+cfs.24

where 23.1+ck is slightly better than vanilla 23.1 and 23.8+cfs.24 is
much better than 23.1+ck.

My workload is pretty standard, F8+kde+firefox+kmail+amsn, some kernel
compile and so on.

Regards,
Fabio

2007-11-21 10:40:11

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7


* Fabio Comolli <[email protected]> wrote:

> To answer your latest mail, again I don't have numbers but from my
> point ov view:
>
> 23.1 < 23.1+ck < 23.8+cfs.24
>
> where 23.1+ck is slightly better than vanilla 23.1 and 23.8+cfs.24 is
> much better than 23.1+ck.
>
> My workload is pretty standard, F8+kde+firefox+kmail+amsn, some kernel
> compile and so on.

do you remember which particular workload improved the most, and in
roughly whatway? (no need for numbers - subjective impressions are
accurate enough for this)

Ingo

2007-11-21 11:11:59

by Fabio Comolli

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7

Hi

On Nov 21, 2007 11:39 AM, Ingo Molnar <[email protected]> wrote:
>
> * Fabio Comolli <[email protected]> wrote:
>
> > To answer your latest mail, again I don't have numbers but from my
> > point ov view:
> >
> > 23.1 < 23.1+ck < 23.8+cfs.24
> >
> > where 23.1+ck is slightly better than vanilla 23.1 and 23.8+cfs.24 is
> > much better than 23.1+ck.
> >
> > My workload is pretty standard, F8+kde+firefox+kmail+amsn, some kernel
> > compile and so on.
>
> do you remember which particular workload improved the most, and in
> roughly whatway? (no need for numbers - subjective impressions are
> accurate enough for this)
>

Firefox, especially when multiple tabs are open. Sometimes I used to
experience some latencies that with cfs.24 are gone.

> Ingo
>

Regards,
Fabio

2007-11-26 21:00:37

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch/backport] CFS scheduler, -v24, for v2.6.24-rc3, v2.6.23.9, v2.6.22.13, v2.6.21.7


* Ingo Molnar <[email protected]> wrote:

> By popular demand, here is release -v24 of the CFS scheduler patch.
>
> It is a full backport of the latest & greatest scheduler code to
> v2.6.24-rc3, v2.6.23.8, v2.6.22.13, v2.6.21.7. The patches can be
> downloaded from the usual place:
>
> http://people.redhat.com/mingo/cfs-scheduler/

update: there's now a CFS-v24 patch available for v2.6.23.9 as well.

Ingo