These two patches clean up the process of scanning the CPU.
Patch 1 stops checking for a new idle core in time
if an idle core has already been found.
Patch 2 tries to minimize false attempts by adjusting the order
of scanning CPU.
v3->v2:
- Add "Acked-by: Mel Gorman <[email protected]>" and
"Reviewed-by: Vincent Guittot <[email protected]>"
for patch2
v1->v2:
- Simplified patch1 code and add
"Acked-by: Mel Gorman <[email protected]>" for patch1
- Modify commit description to make it more clear
[v1] https://lore.kernel.org/all/[email protected]
[v2] https://lore.kernel.org/all/[email protected]
Hao Jia (2):
sched/numa: Stop an exhastive search if an idle core is found
sched/core: Adjusting the order of scanning CPU
kernel/sched/core.c | 2 +-
kernel/sched/fair.c | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
--
2.37.0
In update_numa_stats() we try to find an idle cpu on the NUMA node,
preferably an idle core. we can stop looking for the next idle core
or idle cpu after finding an idle core. But we can't stop the
whole loop of scanning the CPU, because we need to calculate
approximate NUMA stats at a point in time. For example,
the src and dst nr_running is needed by task_numa_find_cpu().
Signed-off-by: Hao Jia <[email protected]>
Acked-by: Mel Gorman <[email protected]>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c36aa54ae071..643cbcb61b49 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1804,7 +1804,7 @@ static void update_numa_stats(struct task_numa_env *env,
ns->nr_running += rq->cfs.h_nr_running;
ns->compute_capacity += capacity_of(cpu);
- if (find_idle && !rq->nr_running && idle_cpu(cpu)) {
+ if (find_idle && idle_core < 0 && !rq->nr_running && idle_cpu(cpu)) {
if (READ_ONCE(rq->numa_migrate_on) ||
!cpumask_test_cpu(cpu, env->p->cpus_ptr))
continue;
--
2.37.0
When select_idle_capacity() starts scanning for an idle CPU, it starts
with target CPU that has already been checked in select_idle_sibling().
So we start checking from the next CPU and try the target CPU at the end.
Similarly for task_numa_assign(), we have just checked numa_migrate_on
of dst_cpu, so start from the next CPU. This also works for
steal_cookie_task(), the first scan must fail and start directly
from the next one.
Signed-off-by: Hao Jia <[email protected]>
Acked-by: Mel Gorman <[email protected]>
Reviewed-by: Vincent Guittot <[email protected]>
---
kernel/sched/core.c | 2 +-
kernel/sched/fair.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 25b582b6ee5f..40149ff68f14 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6206,7 +6206,7 @@ static bool steal_cookie_task(int cpu, struct sched_domain *sd)
{
int i;
- for_each_cpu_wrap(i, sched_domain_span(sd), cpu) {
+ for_each_cpu_wrap(i, sched_domain_span(sd), cpu + 1) {
if (i == cpu)
continue;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 643cbcb61b49..e910df06b779 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1836,7 +1836,7 @@ static void task_numa_assign(struct task_numa_env *env,
int start = env->dst_cpu;
/* Find alternative idle CPU. */
- for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start) {
+ for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start + 1) {
if (cpu == env->best_cpu || !idle_cpu(cpu) ||
!cpumask_test_cpu(cpu, env->p->cpus_ptr)) {
continue;
@@ -6811,7 +6811,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target)
util_min = uclamp_eff_value(p, UCLAMP_MIN);
util_max = uclamp_eff_value(p, UCLAMP_MAX);
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
unsigned long cpu_cap = capacity_of(cpu);
if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))
--
2.37.0
On Fri, Dec 16, 2022 at 02:24:04PM +0800, Hao Jia wrote:
> Hao Jia (2):
> sched/numa: Stop an exhastive search if an idle core is found
> sched/core: Adjusting the order of scanning CPU
>
> kernel/sched/core.c | 2 +-
> kernel/sched/fair.c | 6 +++---
> 2 files changed, 4 insertions(+), 4 deletions(-)
Thanks!
The following commit has been merged into the sched/core branch of tip:
Commit-ID: feaed76376fd612b9450ad1251aec20a63ccfe92
Gitweb: https://git.kernel.org/tip/feaed76376fd612b9450ad1251aec20a63ccfe92
Author: Hao Jia <[email protected]>
AuthorDate: Fri, 16 Dec 2022 14:24:05 +08:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Tue, 27 Dec 2022 12:52:16 +01:00
sched/numa: Stop an exhastive search if an idle core is found
In update_numa_stats() we try to find an idle cpu on the NUMA node,
preferably an idle core. we can stop looking for the next idle core
or idle cpu after finding an idle core. But we can't stop the
whole loop of scanning the CPU, because we need to calculate
approximate NUMA stats at a point in time. For example,
the src and dst nr_running is needed by task_numa_find_cpu().
Signed-off-by: Hao Jia <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Acked-by: Mel Gorman <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d0e2a48..aaff209 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1804,7 +1804,7 @@ static void update_numa_stats(struct task_numa_env *env,
ns->nr_running += rq->cfs.h_nr_running;
ns->compute_capacity += capacity_of(cpu);
- if (find_idle && !rq->nr_running && idle_cpu(cpu)) {
+ if (find_idle && idle_core < 0 && !rq->nr_running && idle_cpu(cpu)) {
if (READ_ONCE(rq->numa_migrate_on) ||
!cpumask_test_cpu(cpu, env->p->cpus_ptr))
continue;
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 8589018acc65e5ddfd111f0a7ee85f9afde3a830
Gitweb: https://git.kernel.org/tip/8589018acc65e5ddfd111f0a7ee85f9afde3a830
Author: Hao Jia <[email protected]>
AuthorDate: Fri, 16 Dec 2022 14:24:06 +08:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Tue, 27 Dec 2022 12:52:17 +01:00
sched/core: Adjusting the order of scanning CPU
When select_idle_capacity() starts scanning for an idle CPU, it starts
with target CPU that has already been checked in select_idle_sibling().
So we start checking from the next CPU and try the target CPU at the end.
Similarly for task_numa_assign(), we have just checked numa_migrate_on
of dst_cpu, so start from the next CPU. This also works for
steal_cookie_task(), the first scan must fail and start directly
from the next one.
Signed-off-by: Hao Jia <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Reviewed-by: Vincent Guittot <[email protected]>
Acked-by: Mel Gorman <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
kernel/sched/core.c | 2 +-
kernel/sched/fair.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1f3259c..048ec24 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6209,7 +6209,7 @@ static bool steal_cookie_task(int cpu, struct sched_domain *sd)
{
int i;
- for_each_cpu_wrap(i, sched_domain_span(sd), cpu) {
+ for_each_cpu_wrap(i, sched_domain_span(sd), cpu + 1) {
if (i == cpu)
continue;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index aaff209..e9d906a 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1836,7 +1836,7 @@ static void task_numa_assign(struct task_numa_env *env,
int start = env->dst_cpu;
/* Find alternative idle CPU. */
- for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start) {
+ for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start + 1) {
if (cpu == env->best_cpu || !idle_cpu(cpu) ||
!cpumask_test_cpu(cpu, env->p->cpus_ptr)) {
continue;
@@ -6935,7 +6935,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target)
util_min = uclamp_eff_value(p, UCLAMP_MIN);
util_max = uclamp_eff_value(p, UCLAMP_MAX);
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
unsigned long cpu_cap = capacity_of(cpu);
if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))