Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Fri, 11 Mar 2022 10:25:30 -0800
From:   Darren Hart <darren@os.amperecomputing.com>
To:     Qing Wang <wangqing@vivo.com>
Cc:     Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        Sudeep Holla <sudeep.holla@arm.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH V2] sched: topology: make cache topology separate from
 cpu topology
Message-ID: <YiuUGi0cFyLHZhAI@fedora>
References: <1646969135-26647-1-git-send-email-wangqing@vivo.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1646969135-26647-1-git-send-email-wangqing@vivo.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1v7lrfl0klKrII5bqymyB47Rz+pOoFHwwUDiFhjoxhc5qSZctt7llQJqrm2S?=
 =?us-ascii?Q?oLwFy8yLP+KUaznqsKk/YjNecwh4FhoqxXjPcsDGpaQpXqXgRvrrrIkuVAed?=
 =?us-ascii?Q?F8Pugy3tL3EtZxnTZDiIGmO1OXVbQT3wBYI5O77UHJNXI3JhLJ0Cy3BhI9OT?=
 =?us-ascii?Q?mn/hH6q+jYjbL5OOujkNzn/XeXyGp7gr3w+n9iDQI84+hp0P8hHasl5qxbkR?=
 =?us-ascii?Q?H1WgfWGkHMZOo58fyYVOqdMj3q0eq56N4Xehn8oVMrKwGRZJeXbn6NB7s8Lh?=
 =?us-ascii?Q?5TrGGKELEISqBPW6r+IFY1M0lIowbB5Df84ylQ6z41s0E4qbFMD2gKHykUDN?=
 =?us-ascii?Q?aIVKYO4jg0Ml6IWh1dPLk2YNICCWouutGe5iDSR98ZCkcSN0iPZOGPOWn5F4?=
 =?us-ascii?Q?R17e124CpdPzrddHcgvbtsOyhGP0jq0fBLm8+/mCiSV+BYj98QkpI7PSORHU?=
 =?us-ascii?Q?L52bmQtXLxiunKQ59QCAKsz58qxwOpLBO8Beo2gcjPJpLvNnfIc3E0WCBTbJ?=
 =?us-ascii?Q?VTdr/v2pLtEZzON/BDB5JHA2j234hw0Gxt315rEFQFDoXAKh1zyok1hBLLqv?=
 =?us-ascii?Q?dFLuADYUfyyZZLWm9dpOAygV4pjgjioTWsSHh64EHJ9NB1MjaLyR1nDIgaDB?=
 =?us-ascii?Q?T2HhmZ+WRivkB0hVMxEwJjC7mHJuP7j7qdSCrSKspDMATUKhmab1sd5zeQVe?=
 =?us-ascii?Q?ULbrfEhRp/rCfWsIIVENhdxkOinvzHdGse7RgdQcb7ocvS+4BJ4CgNXCVi+/?=
 =?us-ascii?Q?9tu8rWjBBpIc1fkK1VNxzCB1EV8Tt0DJLj3FbAwhQxlU0znmSc36N5EkmlA2?=
 =?us-ascii?Q?9ZsquLo+JJccgsoMGNM6n8ECm+TSjzIInLiJfdQUT8UmxZVORhQnSPwZ5ZZQ?=
 =?us-ascii?Q?zvW5RErkB1f6QfU/nRfEYidXq9544M6miyOsO+Oc48lEjTD3c1GTooj8FyWL?=
 =?us-ascii?Q?Dmm2UI0n3dScI048Uu40kkMaDRr9TqiLaGR8LwDQV7K3lnZJFKNOkgb3EZbx?=
 =?us-ascii?Q?r8NHfcJZDKL4Op0R0uwdrOBb8R5KA5I+Fx7dsimuWRCwiQZSZ3unUi3r7kbe?=
 =?us-ascii?Q?1cPGPzaORSg8PzA3Ja3RWowvTtIJzQRkwgPlAT7HOmfAiFPyJwZe0CtPdb1w?=
 =?us-ascii?Q?kubAb7keHtUhu8u4QtY7mutDeaZXmwCUs2cwbbobnOrYx+Cbgr0DV0AcZie7?=
 =?us-ascii?Q?ey66H2BuOYK8MavANNkJjC965hyHdUpW78By+uwlQoenY45eqlF09iR+QSxp?=
 =?us-ascii?Q?2fJIIoR2ruA1BPD954YXg6E7QdZY3077382zKVFdoh14FUnBpoOaMrf4xb7i?=
 =?us-ascii?Q?+m1iha+IbiB9hVny0v+SGsociQofIC0kKw01V7e78+3H/b2s8f46ppjzURUa?=
 =?us-ascii?Q?EHcXnFJXIyIkR6ZTuVW76PSABEqcIZj4hWFHwLNuGuRTIM4FzVVMtEp75e8p?=
 =?us-ascii?Q?HoQDEbVJYG411mFwtlqA7aHMeF/qbnt9Tw+tC3EkiIPf7JI3NDx7bSCKJztv?=
 =?us-ascii?Q?xWaTgzEZ/0yo8FLnJW8qas1ZNX3opduUrJkNrBR84nPXfqeo2t1bnO8ggqdd?=
 =?us-ascii?Q?HM4v2cWjJ+o2ivVIEiTWBLx4W1l0hR7qvfs+8K3PdbD9xv6JAPhST0cOKw7r?=
 =?us-ascii?Q?xtyr4M8rG/p1jn9TkZzA5kuexi18/P2724En7qUT+1O40imj6cjSBWbdAx1O?=
 =?us-ascii?Q?2vHYumWelKFkmoRx9UPoW7b6D5eV0yg+LdjIKfnMNf66RneurmgeLVeREE0V?=
 =?us-ascii?Q?koQbThs0Qg=3D=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eaec79e5-289b-498b-3140-08da038c870f
X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB6326.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2022 18:25:33.6958
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 20KGy+m8uqmYcickgPuvtqVWpx3tDcbPXsMkh2QBtzAWtBpGFkM7J7WyGrSnnqp6ciNnBQgp6/RD/J+A3fT2bExQWWg8z7v+/keLKLz6ECK0vlrE3gJA1kPHz5XEFIYX
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR0101MB3043
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 10, 2022 at 07:25:33PM -0800, Qing Wang wrote:
> From: Wang Qing <wangqing@vivo.com>
> 
> Some architectures(e.g. ARM64), caches are implemented like below:
> SD(Level 1):          ************ DIE ************
> SD(Level 0):          **** MC ****    **** MC *****
> cluster:              **cluster 0**   **cluster 1**
> cores:                0   1   2   3   4   5   6   7
> cache(Level 1):       C   C   C   C   C   C   C   C
> cache(Level 2):       **C**   **C**   **C**   **C**
> cache(Level 3):       *******shared Level 3********
> sd_llc_id(current):   0   0   0   0   4   4   4   4
> sd_llc_id(should be): 0   0   2   2   4   4   6   6

Should cluster 0 and 1 span the same cpu mask as the MCs? Based on how
you describe the cache above, it seems like what you are looking for
would be:

(SD DIE level removed in favor of the same span MC)
SD(Level 1):          ************ MC  ************
SD(Level 0):          *CLS0*  *CLS1*  *CLS2*  *CLS3* (CONFIG_SCHED_CLUSTER)
cores:                0   1   2   3   4   5   6   7
cache(Level 1):       C   C   C   C   C   C   C   C
cache(Level 2):       **C**   **C**   **C**   **C**
cache(Level 3):       *******shared Level 3********

Provided cpu_coregroup_mask and cpu_clustergroup_mask return the
corresponding cpumasks, this should work with the default sched domain
topology.

It looks to me like the lack of nested cluster support in
parse_cluster() in drivers/base/arch_topology.c is what needs to be
updated to accomplish the above. With cpu_topology[cpu].cluster_sibling and
core_sibling updated to reflect the topology you describe, the rest of
the sched domains construction would work with the default sched domain
topology.

I'm not very familiar with DT, especially the cpu-map. Does your DT
reflect the topology you want to build?


-- 
Darren Hart
Ampere Computing / OS and Kernel