It maybe involve three CPUs in the try_to_wake_up(), for example:
Current CPU: #0
task_cpu(p): #1
Target CPU : #? <-- get by select_task_rq()
ttwu_local: record times try_to_wake_up() was called to wake up
the local cpu.
1.) Note ttwu_local also includes some migrations, such as a task is
migrated from CPU #1 to Target CPU #0.
ttwu_wake_remote: record times in this domain try_to_wake_up() awoke
a task that last ran on a different cpu in this domain.
1.) Part of the migrations are recorded in ttwu_wake_remote, such as
a task is migrated from CPU #1 to Target CPU #2.
2.) If a task is put in CPU #1 wake_list, the ttwu_wake_remote will
record it too.
Current try_to_wake_up() schedstat does not have a migration counter.
And we cannot get the migration number from the ttwu_local, ttwu_count
and ttwu_wake_remote correctly.
But we really want to know the try_to_wake_up migration number for
analyzing some benchmark tests.
This patch adds the ttwu_migration counter to record the migrations.
Put it at the end, do not break some tools.
Signed-off-by: Huang Shijie <[email protected]>
---
Documentation/scheduler/sched-stats.rst | 4 +++-
kernel/sched/core.c | 4 ++++
kernel/sched/debug.c | 1 +
kernel/sched/sched.h | 1 +
kernel/sched/stats.c | 5 +++--
5 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/Documentation/scheduler/sched-stats.rst b/Documentation/scheduler/sched-stats.rst
index 03c062915998..3220dbe220d5 100644
--- a/Documentation/scheduler/sched-stats.rst
+++ b/Documentation/scheduler/sched-stats.rst
@@ -36,7 +36,7 @@ to write their own scripts, the fields are described here.
CPU statistics
--------------
-cpu<N> 1 2 3 4 5 6 7 8 9
+cpu<N> 1 2 3 4 5 6 7 8 9 10
First field is a sched_yield() statistic:
@@ -61,6 +61,8 @@ Next three are statistics describing scheduling latency:
nanoseconds)
9) # of timeslices run on this cpu
+Next new item for try_to_wake_up() statistics:
+ 10) # of times try_to_wake_up() was called to do the task migration.
Domain statistics
-----------------
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e0fc8f7c7487..f0038c103843 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4288,11 +4288,15 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
cpu = select_task_rq(p, p->wake_cpu, wake_flags | WF_TTWU);
if (task_cpu(p) != cpu) {
+ struct rq *rq = this_rq();
+
if (p->in_iowait) {
delayacct_blkio_end(p);
atomic_dec(&task_rq(p)->nr_iowait);
}
+ schedstat_inc(rq->ttwu_migration);
+
wake_flags |= WF_MIGRATED;
psi_ttwu_dequeue(p);
set_task_cpu(p, cpu);
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 0b2340a79b65..f0fd4e2795c9 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -809,6 +809,7 @@ do { \
P(sched_goidle);
P(ttwu_count);
P(ttwu_local);
+ P(ttwu_migration);
}
#undef P
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 060616944d7a..48b552f6b87f 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1123,6 +1123,7 @@ struct rq {
/* try_to_wake_up() stats */
unsigned int ttwu_count;
unsigned int ttwu_local;
+ unsigned int ttwu_migration;
#endif
#ifdef CONFIG_CPU_IDLE
diff --git a/kernel/sched/stats.c b/kernel/sched/stats.c
index 857f837f52cb..5651d4c0a63f 100644
--- a/kernel/sched/stats.c
+++ b/kernel/sched/stats.c
@@ -133,12 +133,13 @@ static int show_schedstat(struct seq_file *seq, void *v)
/* runqueue-specific stats */
seq_printf(seq,
- "cpu%d %u 0 %u %u %u %u %llu %llu %lu",
+ "cpu%d %u 0 %u %u %u %u %llu %llu %lu %u",
cpu, rq->yld_count,
rq->sched_count, rq->sched_goidle,
rq->ttwu_count, rq->ttwu_local,
rq->rq_cpu_time,
- rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount);
+ rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount,
+ rq->ttwu_migration);
seq_printf(seq, "\n");
--
2.39.2