2020-05-04 07:56:26

by Vincent Guittot

[permalink] [raw]
Subject: Re: [RFC PATCH] sched/fair: correct llc shared domain's number of busy CPUs

On Mon, 4 May 2020 at 03:57, Hillf Danton <[email protected]> wrote:
>
>
> The comment says, if there is an imbalance between LLC domains (IOW we
> could increase the overall cache use), we need some less-loaded LLC
> domain to pull some load.
>
> To show that imbalance, record busy CPUs as they come and go by doing
> a minor cleanup for sd::nohz_idle.

Your comment failed to explain why we can get rid of sd->nohz_idle

>
> Cc: Mel Gorman <[email protected]>
> Cc: Vincent Guittot <[email protected]>
> Cc: Valentin Schneider <[email protected]>
> Cc: Dietmar Eggemann <[email protected]>
> Signed-off-by: Hillf Danton <[email protected]>
> ---
>
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10138,13 +10138,8 @@ static void set_cpu_sd_state_busy(int cp
>
> rcu_read_lock();
> sd = rcu_dereference(per_cpu(sd_llc, cpu));
> -
> - if (!sd || !sd->nohz_idle)
> - goto unlock;
> - sd->nohz_idle = 0;

you remove the use of sd->nohz_idle but you don't remove it from
struct sched_domain

> -
> - atomic_inc(&sd->shared->nr_busy_cpus);
> -unlock:
> + if (sd)
> + atomic_inc(&sd->shared->nr_busy_cpus);
> rcu_read_unlock();
> }
>
> @@ -10168,13 +10163,8 @@ static void set_cpu_sd_state_idle(int cp
>
> rcu_read_lock();
> sd = rcu_dereference(per_cpu(sd_llc, cpu));
> -
> - if (!sd || sd->nohz_idle)
> - goto unlock;
> - sd->nohz_idle = 1;
> -
> - atomic_dec(&sd->shared->nr_busy_cpus);
> -unlock:
> + if (sd)
> + atomic_dec(&sd->shared->nr_busy_cpus);
> rcu_read_unlock();
> }
>
>