2023-04-10 16:43:33

by K Prateek Nayak

[permalink] [raw]
Subject: [PATCH 2/2] x86/Documentation: Add documentation about cluster

x86 processors map cluster to the L2 cache. Add documentation stating
the same, and provide more information on the values and API related to
CPU clusters exposed by the kernel.

Signed-off-by: K Prateek Nayak <[email protected]>
---
Documentation/x86/topology.rst | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)

diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst
index 7f58010ea86a..35991d8cdef1 100644
--- a/Documentation/x86/topology.rst
+++ b/Documentation/x86/topology.rst
@@ -33,6 +33,7 @@ historical nature and should be cleaned up.
The topology of a system is described in the units of:

- packages
+ - cluster
- cores
- threads

@@ -90,6 +91,27 @@ Package-related topology information in the kernel:
Cache. In general, it is a number identifying an LLC uniquely on the
system.

+
+Clusters
+========
+A cluster consists of 1 or more threads. It does not matter whether the threads
+are SMT- or CMT-type threads. All the threads of a cluster share the same L2
+cache.
+
+Cluster-related topology information in the kernel:
+
+ - cluster_id:
+
+ A per-CPU variable containing:
+
+ - On Intel, the common upper bits of APIC ID of the list of CPUs sharing
+ the L2 Cache with lower bits set to 0.
+
+ - On AMD, with Topology Extension, the common upper bits of the Extended
+ APIC ID of the list of CPUs sharing the L2 Cache, left shifted to
+ remove trailing 0s.
+
+
Cores
=====
A core consists of 1 or more threads. It does not matter whether the threads
@@ -125,6 +147,11 @@ Thread-related topology information in the kernel:

The number of online threads is also printed in /proc/cpuinfo "siblings."

+ - topology_cluster_cpumask():
+
+ The cpumask contains all online threads in the cluster to which a thread
+ belongs.
+
- topology_sibling_cpumask():

The cpumask contains all online threads in the core to which a thread
@@ -138,6 +165,10 @@ Thread-related topology information in the kernel:

The physical package ID to which a thread belongs.

+ - topology_cluster_id();
+
+ The ID of the cluster to which a thread belongs.
+
- topology_core_id();

The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo
--
2.34.1


2023-04-11 04:18:08

by Bagas Sanjaya

[permalink] [raw]
Subject: Re: [PATCH 2/2] x86/Documentation: Add documentation about cluster

On Mon, Apr 10, 2023 at 10:05:27PM +0530, K Prateek Nayak wrote:
> diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst
> index 7f58010ea86a..35991d8cdef1 100644
> --- a/Documentation/x86/topology.rst
> +++ b/Documentation/x86/topology.rst
> @@ -33,6 +33,7 @@ historical nature and should be cleaned up.
> The topology of a system is described in the units of:
>
> - packages
> + - cluster
> - cores
> - threads
>
> @@ -90,6 +91,27 @@ Package-related topology information in the kernel:
> Cache. In general, it is a number identifying an LLC uniquely on the
> system.
>
> +
> +Clusters
> +========
> +A cluster consists of 1 or more threads. It does not matter whether the threads
> +are SMT- or CMT-type threads. All the threads of a cluster share the same L2
> +cache.
> +
> +Cluster-related topology information in the kernel:
> +
> + - cluster_id:
> +
> + A per-CPU variable containing:
> +
> + - On Intel, the common upper bits of APIC ID of the list of CPUs sharing
> + the L2 Cache with lower bits set to 0.
> +
> + - On AMD, with Topology Extension, the common upper bits of the Extended
> + APIC ID of the list of CPUs sharing the L2 Cache, left shifted to
> + remove trailing 0s.
> +
> +
> Cores
> =====
> A core consists of 1 or more threads. It does not matter whether the threads
> @@ -125,6 +147,11 @@ Thread-related topology information in the kernel:
>
> The number of online threads is also printed in /proc/cpuinfo "siblings."
>
> + - topology_cluster_cpumask():
> +
> + The cpumask contains all online threads in the cluster to which a thread
> + belongs.
> +
> - topology_sibling_cpumask():
>
> The cpumask contains all online threads in the core to which a thread
> @@ -138,6 +165,10 @@ Thread-related topology information in the kernel:
>
> The physical package ID to which a thread belongs.
>
> + - topology_cluster_id();
> +
> + The ID of the cluster to which a thread belongs.
> +
> - topology_core_id();
>
> The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo

The doc LGTM, thanks!

Reviewed-by: Bagas Sanjaya <[email protected]>

--
An old man doll... just what I always wanted! - Clara


Attachments:
(No filename) (2.32 kB)
signature.asc (235.00 B)
Download all attachments

2023-04-11 08:29:18

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH 2/2] x86/Documentation: Add documentation about cluster

On Mon, Apr 10, 2023 at 10:05:27PM +0530, K Prateek Nayak wrote:
> x86 processors map cluster to the L2 cache. Add documentation stating
> the same, and provide more information on the values and API related to
> CPU clusters exposed by the kernel.
>
> Signed-off-by: K Prateek Nayak <[email protected]>
> ---
> Documentation/x86/topology.rst | 31 +++++++++++++++++++++++++++++++
> 1 file changed, 31 insertions(+)
>
> diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst
> index 7f58010ea86a..35991d8cdef1 100644
> --- a/Documentation/x86/topology.rst
> +++ b/Documentation/x86/topology.rst
> @@ -33,6 +33,7 @@ historical nature and should be cleaned up.
> The topology of a system is described in the units of:
>
> - packages
> + - cluster
> - cores
> - threads
>
> @@ -90,6 +91,27 @@ Package-related topology information in the kernel:
> Cache. In general, it is a number identifying an LLC uniquely on the
> system.
>
> +
> +Clusters
> +========
> +A cluster consists of 1 or more threads. It does not matter whether the threads
> +are SMT- or CMT-type threads. All the threads of a cluster share the same L2
> +cache.

I'm not quite sure that's a correct discription of what a cluster is.

Yes, SMT will fundamentally share core-level caches (and should we not
always have SMT share all cache topoligies?)

But there is also x86 where L2 is shared between multiple cores -- while
the above seems to suggest L2 is single core only.

2023-04-11 11:01:16

by K Prateek Nayak

[permalink] [raw]
Subject: Re: [PATCH 2/2] x86/Documentation: Add documentation about cluster

Hello Peter,

Thank you for reviewing the patches.

On 4/11/2023 1:55 PM, Peter Zijlstra wrote:
> On Mon, Apr 10, 2023 at 10:05:27PM +0530, K Prateek Nayak wrote:
>> x86 processors map cluster to the L2 cache. Add documentation stating
>> the same, and provide more information on the values and API related to
>> CPU clusters exposed by the kernel.
>>
>> Signed-off-by: K Prateek Nayak <[email protected]>
>> ---
>> Documentation/x86/topology.rst | 31 +++++++++++++++++++++++++++++++
>> 1 file changed, 31 insertions(+)
>>
>> diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst
>> index 7f58010ea86a..35991d8cdef1 100644
>> --- a/Documentation/x86/topology.rst
>> +++ b/Documentation/x86/topology.rst
>> @@ -33,6 +33,7 @@ historical nature and should be cleaned up.
>> The topology of a system is described in the units of:
>>
>> - packages
>> + - cluster
>> - cores
>> - threads
>>
>> @@ -90,6 +91,27 @@ Package-related topology information in the kernel:
>> Cache. In general, it is a number identifying an LLC uniquely on the
>> system.
>>
>> +
>> +Clusters
>> +========
>> +A cluster consists of 1 or more threads. It does not matter whether the threads
>> +are SMT- or CMT-type threads. All the threads of a cluster share the same L2
>> +cache.
>
> I'm not quite sure that's a correct discription of what a cluster is.
>
> Yes, SMT will fundamentally share core-level caches (and should we not
> always have SMT share all cache topoligies?)

I can reword the cluster description as follows:

"A cluster consists of threads of one or more cores sharing the same
L2 cache."

>
> But there is also x86 where L2 is shared between multiple cores -- while
> the above seems to suggest L2 is single core only.

I hope the above rewording solves this confusion. Let me know otherwise.
--
Thanks and Regards,
Prateek

2023-04-11 11:02:12

by K Prateek Nayak

[permalink] [raw]
Subject: Re: [PATCH 2/2] x86/Documentation: Add documentation about cluster

Hello Bagas,

On 4/11/2023 9:28 AM, Bagas Sanjaya wrote:
> On Mon, Apr 10, 2023 at 10:05:27PM +0530, K Prateek Nayak wrote:
>> diff --git a/Documentation/x86/topology.rst b/Documentation/x86/topology.rst
>> index 7f58010ea86a..35991d8cdef1 100644
>> --- a/Documentation/x86/topology.rst
>> +++ b/Documentation/x86/topology.rst
>> @@ -33,6 +33,7 @@ historical nature and should be cleaned up.
>> The topology of a system is described in the units of:
>>
>> - packages
>> + - cluster
>> - cores
>> - threads
>>
>> @@ -90,6 +91,27 @@ Package-related topology information in the kernel:
>> Cache. In general, it is a number identifying an LLC uniquely on the
>> system.
>>
>> +
>> +Clusters
>> +========
>> +A cluster consists of 1 or more threads. It does not matter whether the threads
>> +are SMT- or CMT-type threads. All the threads of a cluster share the same L2
>> +cache.
>> +
>> +Cluster-related topology information in the kernel:
>> +
>> + - cluster_id:
>> +
>> + A per-CPU variable containing:
>> +
>> + - On Intel, the common upper bits of APIC ID of the list of CPUs sharing
>> + the L2 Cache with lower bits set to 0.
>> +
>> + - On AMD, with Topology Extension, the common upper bits of the Extended
>> + APIC ID of the list of CPUs sharing the L2 Cache, left shifted to
>> + remove trailing 0s.
>> +
>> +
>> Cores
>> =====
>> A core consists of 1 or more threads. It does not matter whether the threads
>> @@ -125,6 +147,11 @@ Thread-related topology information in the kernel:
>>
>> The number of online threads is also printed in /proc/cpuinfo "siblings."
>>
>> + - topology_cluster_cpumask():
>> +
>> + The cpumask contains all online threads in the cluster to which a thread
>> + belongs.
>> +
>> - topology_sibling_cpumask():
>>
>> The cpumask contains all online threads in the core to which a thread
>> @@ -138,6 +165,10 @@ Thread-related topology information in the kernel:
>>
>> The physical package ID to which a thread belongs.
>>
>> + - topology_cluster_id();
>> +
>> + The ID of the cluster to which a thread belongs.
>> +
>> - topology_core_id();
>>
>> The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo
>
> The doc LGTM, thanks!
>
> Reviewed-by: Bagas Sanjaya <[email protected]>

Thank you for reviewing the patch :)

--
Thanks and Regards,
Prateek