Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sat, 12 Mar 2022 13:05:18 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     Qing Wang <wangqing@vivo.com>
Cc:     Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        Sudeep Holla <sudeep.holla@arm.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH V2] sched: topology: make cache topology separate from
 cpu topology
Message-ID: <20220312120518.GC6235@worktop.programming.kicks-ass.net>
References: <1646969135-26647-1-git-send-email-wangqing@vivo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1646969135-26647-1-git-send-email-wangqing@vivo.com>
Precedence: bulk

On Thu, Mar 10, 2022 at 07:25:33PM -0800, Qing Wang wrote:
> From: Wang Qing <wangqing@vivo.com>
> 
> Some architectures(e.g. ARM64), caches are implemented like below:
> SD(Level 1):          ************ DIE ************
> SD(Level 0):          **** MC ****    **** MC *****
> cluster:              **cluster 0**   **cluster 1**
> cores:                0   1   2   3   4   5   6   7
> cache(Level 1):       C   C   C   C   C   C   C   C
> cache(Level 2):  	  **C**   **C**   **C**   **C**
> cache(Level 3):       *******shared Level 3********
> sd_llc_id(current):   0   0   0   0   4   4   4   4
> sd_llc_id(should be): 0   0   2   2   4   4   6   6
> 
> Caches and cpus have different topology, this causes cpus_share_cache()
> return the wrong value in sd, which will affect the CPU load balance.

Then fix your SD_flags already.

> diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
> index cce6136b..3048fa6
> --- a/include/linux/arch_topology.h
> +++ b/include/linux/arch_topology.h
> @@ -82,6 +82,8 @@ extern struct cpu_topology cpu_topology[NR_CPUS];
>  #define topology_cluster_cpumask(cpu)	(&cpu_topology[cpu].cluster_sibling)
>  #define topology_llc_cpumask(cpu)	(&cpu_topology[cpu].llc_sibling)
>  void init_cpu_topology(void);
> +void init_cpu_cache_topology(void);
> +void fix_cpu_llc(int cpu, int *first_cpu, int *cpu_num);
>  void store_cpu_topology(unsigned int cpuid);
>  const struct cpumask *cpu_coregroup_mask(int cpu);
>  const struct cpumask *cpu_clustergroup_mask(int cpu);
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index d201a70..d894ced
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -661,6 +661,9 @@ static void update_top_cache_domain(int cpu)
>  	if (sd) {
>  		id = cpumask_first(sched_domain_span(sd));
>  		size = cpumask_weight(sched_domain_span(sd));
> +#ifdef CONFIG_GENERIC_ARCH_TOPOLOGY
> +		fix_cpu_llc(cpu, &id, &size);
> +#endif
>  		sds = sd->shared;
>  	}

NAK on that.