Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through
sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online
which let the arch specific functions handle (at least on x86)
populating the shared_cpu_map. However, with the changes in the
aforementioned commit, populate_cache_leaves() is not called when a CPU
comes online as a result of hotplug since last_level_cache_is_valid()
returns true as the cacheinfo data is not discarded. The CPU coming
online is not present in shared_cpu_map, however, it will not be added
since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in
populate_cache_leaves() when cacheinfo is first populated for x86)
This can lead to inconsistencies in the shared_cpu_map when an offlined
CPU comes online again. Example below depicts the inconsistency in the
shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on
a 3rd Generation EPYC processor:
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
/sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# echo 0 > /sys/devices/system/cpu/cpu8/online
# echo 1 > /sys/devices/system/cpu/cpu8/online
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
/sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8
/sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8
/sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8
/sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8
# cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
136
# cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
9-15,136-143
Clear the flag when the CPU is removed from shared_cpu_map when
cache_shared_cpu_map_remove() is called during CPU hotplug. This will
allow cache_shared_cpu_map_setup() to add the CPU coming back online in
the shared_cpu_map. Set the flag again when the shared_cpu_map is setup.
Following are results of performing the same test as described above with
the changes:
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
/sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# echo 0 > /sys/devices/system/cpu/cpu8/online
# echo 1 > /sys/devices/system/cpu/cpu8/online
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
/sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
/sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
8,136
# cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
8-15,136-143
Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs")
Signed-off-by: K Prateek Nayak <[email protected]>
---
drivers/base/cacheinfo.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
index d1ae443fd7a0..cbae8be1fe52 100644
--- a/drivers/base/cacheinfo.c
+++ b/drivers/base/cacheinfo.c
@@ -410,11 +410,14 @@ static int cache_shared_cpu_map_setup(unsigned int cpu)
coherency_max_size = this_leaf->coherency_line_size;
}
+ /* shared_cpu_map is now populated for the cpu */
+ this_cpu_ci->cpu_map_populated = true;
return 0;
}
static void cache_shared_cpu_map_remove(unsigned int cpu)
{
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
struct cacheinfo *this_leaf, *sib_leaf;
unsigned int sibling, index, sib_index;
@@ -447,6 +450,9 @@ static void cache_shared_cpu_map_remove(unsigned int cpu)
}
}
}
+
+ /* cpu is no longer populated in the shared map */
+ this_cpu_ci->cpu_map_populated = false;
}
static void free_cache_attributes(unsigned int cpu)
--
2.34.1
Hi Prateek,
On 2023/5/8 16:41, K Prateek Nayak wrote:
> Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through
> sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online
> which let the arch specific functions handle (at least on x86)
> populating the shared_cpu_map. However, with the changes in the
> aforementioned commit, populate_cache_leaves() is not called when a CPU
> comes online as a result of hotplug since last_level_cache_is_valid()
> returns true as the cacheinfo data is not discarded. The CPU coming
Yes in free_cache_attributes() we only update the shared_cpu_map but make
other attributes remained. From my feelings we should do all the work
opposite to detect_cache_attributes(), including free the memory allocated.
> online is not present in shared_cpu_map, however, it will not be added
> since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in
> populate_cache_leaves() when cacheinfo is first populated for x86)
>
> This can lead to inconsistencies in the shared_cpu_map when an offlined
> CPU comes online again. Example below depicts the inconsistency in the
> shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on
> a 3rd Generation EPYC processor:
>
> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>
> # echo 0 > /sys/devices/system/cpu/cpu8/online
> # echo 1 > /sys/devices/system/cpu/cpu8/online
>
> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8
> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8
> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8
> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8
>
> # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
> 136
>
> # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
> 9-15,136-143
>
> Clear the flag when the CPU is removed from shared_cpu_map when
> cache_shared_cpu_map_remove() is called during CPU hotplug. This will
> allow cache_shared_cpu_map_setup() to add the CPU coming back online in
> the shared_cpu_map. Set the flag again when the shared_cpu_map is setup.
> Following are results of performing the same test as described above with
> the changes:
>
> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>
> # echo 0 > /sys/devices/system/cpu/cpu8/online
> # echo 1 > /sys/devices/system/cpu/cpu8/online
>
> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>
> # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
> 8,136
>
> # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
> 8-15,136-143
>
> Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs")
It's ok for me to have this tag but I don't think this is the root cause,
the commit happens to expose the problem. Other arthitectures like arm64
never updates the this_cpu_ci->cpu_map_populated even after the cpumap is
populated.
> Signed-off-by: K Prateek Nayak <[email protected]>
Thanks for fixing this!
Reviewed-by: Yicong Yang <[email protected]>
> ---
> drivers/base/cacheinfo.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
> index d1ae443fd7a0..cbae8be1fe52 100644
> --- a/drivers/base/cacheinfo.c
> +++ b/drivers/base/cacheinfo.c
> @@ -410,11 +410,14 @@ static int cache_shared_cpu_map_setup(unsigned int cpu)
> coherency_max_size = this_leaf->coherency_line_size;
> }
>
> + /* shared_cpu_map is now populated for the cpu */
> + this_cpu_ci->cpu_map_populated = true;
> return 0;
> }
>
> static void cache_shared_cpu_map_remove(unsigned int cpu)
> {
> + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
> struct cacheinfo *this_leaf, *sib_leaf;
> unsigned int sibling, index, sib_index;
>
> @@ -447,6 +450,9 @@ static void cache_shared_cpu_map_remove(unsigned int cpu)
> }
> }
> }
> +
> + /* cpu is no longer populated in the shared map */
> + this_cpu_ci->cpu_map_populated = false;
> }
>
> static void free_cache_attributes(unsigned int cpu)
>
Hello Yicong,
On 5/20/2023 12:26 PM, Yicong Yang wrote:
> Hi Prateek,
>
> On 2023/5/8 16:41, K Prateek Nayak wrote:
>> Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through
>> sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online
>> which let the arch specific functions handle (at least on x86)
>> populating the shared_cpu_map. However, with the changes in the
>> aforementioned commit, populate_cache_leaves() is not called when a CPU
>> comes online as a result of hotplug since last_level_cache_is_valid()
>> returns true as the cacheinfo data is not discarded. The CPU coming
>
> Yes in free_cache_attributes() we only update the shared_cpu_map but make
> other attributes remained. From my feelings we should do all the work
> opposite to detect_cache_attributes(), including free the memory allocated.
In fact, when free_cache_attributes() was first added in
commit 246246cbde5e ("drivers: base: support cpu cache information
interface to userspace via sysfs"), it did exactly that. It was later
changed in commit 5944ce092b97 ("arch_topology: Build cacheinfo from
primary CPU")
>
>> online is not present in shared_cpu_map, however, it will not be added
>> since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in
>> populate_cache_leaves() when cacheinfo is first populated for x86)
>>
>> This can lead to inconsistencies in the shared_cpu_map when an offlined
>> CPU comes online again. Example below depicts the inconsistency in the
>> shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on
>> a 3rd Generation EPYC processor:
>>
>> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
>> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>>
>> # echo 0 > /sys/devices/system/cpu/cpu8/online
>> # echo 1 > /sys/devices/system/cpu/cpu8/online
>>
>> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
>> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8
>> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8
>> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8
>> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8
>>
>> # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
>> 136
>>
>> # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
>> 9-15,136-143
>>
>> Clear the flag when the CPU is removed from shared_cpu_map when
>> cache_shared_cpu_map_remove() is called during CPU hotplug. This will
>> allow cache_shared_cpu_map_setup() to add the CPU coming back online in
>> the shared_cpu_map. Set the flag again when the shared_cpu_map is setup.
>> Following are results of performing the same test as described above with
>> the changes:
>>
>> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
>> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>>
>> # echo 0 > /sys/devices/system/cpu/cpu8/online
>> # echo 1 > /sys/devices/system/cpu/cpu8/online
>>
>> # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done
>> /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136
>> /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
>>
>> # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list
>> 8,136
>>
>> # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list
>> 8-15,136-143
>>
>> Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs")
>
> It's ok for me to have this tag but I don't think this is the root cause,
> the commit happens to expose the problem. Other arthitectures like arm64
> never updates the this_cpu_ci->cpu_map_populated even after the cpumap is
> populated.
I agree. I added the tag to indicate where the behavior changed for x86.
Fun fact, "cpu_map_populated" was added specifically for x86 in commit
fac51482577d5 ("drivers: base: cacheinfo: fix x86 with CONFIG_OF
enabled") back in 2016. There seems to be a long history between the
cacheinfo driver and the arch specific methods :)
>
>> Signed-off-by: K Prateek Nayak <[email protected]>
>
> Thanks for fixing this!
>
> Reviewed-by: Yicong Yang <[email protected]>
Thank you for reviewing the changes.
>
>> ---
>> drivers/base/cacheinfo.c | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
>> index d1ae443fd7a0..cbae8be1fe52 100644
>> --- a/drivers/base/cacheinfo.c
>> +++ b/drivers/base/cacheinfo.c
>> @@ -410,11 +410,14 @@ static int cache_shared_cpu_map_setup(unsigned int cpu)
>> coherency_max_size = this_leaf->coherency_line_size;
>> }
>>
>> + /* shared_cpu_map is now populated for the cpu */
>> + this_cpu_ci->cpu_map_populated = true;
>> return 0;
>> }
>>
>> static void cache_shared_cpu_map_remove(unsigned int cpu)
>> {
>> + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
>> struct cacheinfo *this_leaf, *sib_leaf;
>> unsigned int sibling, index, sib_index;
>>
>> @@ -447,6 +450,9 @@ static void cache_shared_cpu_map_remove(unsigned int cpu)
>> }
>> }
>> }
>> +
>> + /* cpu is no longer populated in the shared map */
>> + this_cpu_ci->cpu_map_populated = false;
>> }
>>
>> static void free_cache_attributes(unsigned int cpu)
>>
--
Thanks and Regards,
Prateek