Received-SPF: pass (google.com: domain of linux-kernel+bounces-142897-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <2cde00c8-7878-45c0-8621-fca4e70c75e7@arm.com>
Date: Fri, 12 Apr 2024 10:03:14 -0500
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [External] Re: [PATCH 1/2] ACPI: PPTT: Populate cacheinfo
 entirely with PPTT
Content-Language: en-US
To: yunhui cui <cuiyunhui@bytedance.com>
Cc: rafael@kernel.org, lenb@kernel.org, linux-acpi@vger.kernel.org,
 linux-kernel@vger.kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com,
 aou@eecs.berkeley.edu, linux-riscv@lists.infradead.org
References: <20240407123829.36474-1-cuiyunhui@bytedance.com>
 <abd135fa-c432-4e37-9792-07a0e17e93d5@arm.com>
 <CAEEQ3w=+C2J0ZS227-1P-B+pe_NRp_3i4c4CxGssiKqbpXx_qw@mail.gmail.com>
From: Jeremy Linton <jeremy.linton@arm.com>
In-Reply-To: <CAEEQ3w=+C2J0ZS227-1P-B+pe_NRp_3i4c4CxGssiKqbpXx_qw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,


On 4/10/24 21:28, yunhui cui wrote:
> Hi Jeremy,
> 
> On Wed, Apr 10, 2024 at 9:30 AM Jeremy Linton <jeremy.linton@arm.com> wrote:
>>
>> Hi,
>>
>>
>> First thanks for working on this.
>>
>> On 4/7/24 07:38, Yunhui Cui wrote:
>>> When the type and level information of this_leaf cannot be obtained
>>> from arch, cacheinfo is completely filled in with the content of PPTT.
>>
>> I started reviewing this, based on what I understood to be the need to
>> generate the topology entirely from the PPTT. But, it was raising more
>> questions than answers because the PPTT is far too flexable in its
>> ability to represent cache hierachies that arn't logically useful. For
>> example multiple I or D caches at the same level, or I or D caches
>> higher in the topology than unified ones.
>>
>> At least for arm64 (and I think others) there is an understood
>> simplification that there will be N levels of split I/D caches and M
>> unified levels. And from that, the number of cache leaves are computed
>> and allocated, and then we go in and largly skip PPTT cache nodes which
>> don't make sense in view of a generic topology like that. (see the
>> comment in cacheinfo.c:506)
>>
>> Both of those pieces of information are available in
>> acpi_get_cache_info(). The missing part is marking those N levels of I/D
>> cache as such.
>>
>> Looking at this code I don't really see all the error/allocation
>> logic/etc that assures the cache leaf indexing is allocated correctly
>> which worries me, although admidditly I could be missing something
>> important.
>>
>> In summary, did you consider just allocating matching I/D caches from
>> the number of split levels in acpi_get_cache_info() then removing or
>> invalidating the ones that don't have matching PPTT entries after
>> running cache_setup_acpi()? Thats a fairly trivial change AFAIK if the
>> decision is based on the lack of a cache_id or just changing the
>> this_leaf->type = CACHE_TYPE_UNIFIED assignment to the correct type and
>> assuring left over CACHE_TYPE_NOCACHE entries are removed. I think much
>> of the "significant work" is likely fixed for that to work. Just
>> tweaking detect_cache_level()/get_cache_type() to set
>> CACHE_TYPE_SEPERATE if the level is less than the acpi_get_cache_info()
>> split_level value probably also does the majority of what you need
>> outside of having unequal counts of I and D caches.
>>
>> There are probably other choices as well, thoughts?
>>
> 
> First, I think the current state of the ACPI PPTT specification meets
> the requirements and is logically complete. Otherwise, the PPTT
> specification needs to be updated, right?

The specification is capable of representing all the actual physical 
topologies I'm aware of, its also capable of representing quite a number 
of topologies that don't make sense to (cant be) build and are largely 
nonsense. There are some further details around replacement and 
allocation which could be represented but were AFAIK not considered 
worth the effort because it makes the representation far more complex.

So, except for minor clarifications I'm not aware of a need to update it.

> Our discussion is best focused on the existing and usual case, even on
> ARM64, which is as you say "N-level separated I/D cache, M-level
> unified".

I understood this was for RISC-V when I reviewed it, but i'm fairly 
certain the general cache topologies of RISC-V machines don't vary in 
their core concepts and layout much from the wide variety of 
possibilities available elsewhere. So its roughly the same problem, you 
will likely have one or two layers of split I/D caches and then a number 
of unified private or shared caches above that.


> 
> And then, the problem we have now is that the RISC-V architecture does
> not have a set of registers to describe the cache level and type like
> ARM64 does, so we need to fully trust the contents of the PPTT table.
> Please check the patch:
> https://patchwork.kernel.org/project/linux-riscv/patch/20240407123829.36474-2-cuiyunhui@bytedance.com/

I don't think "fully trust" is something you really want, or is helpful 
as I suggested above.

I suspect all that is needed is the construction of the missing I/D 
layers, as the existing code will create the Unified layers above as it 
does today. On Arm64 the unified system level caches aren't usually 
described by the core cache registers either, which is why there is that 
existing bit of code to change unknown caches to unified. But it also 
turns out the pptt walking code is capable of providing the information 
to create the I/D layers as well, if they are matched (ex there exists 
an I cache for every D cache at a certain level).

So, more clarity about the kinds of topologies you expect is helpful to 
understand where the current code is failing.

AKA, As I mentioned I'm fairly certain that its possible to construct 
the entire cache topology from the PPTT by changing get_cache_type() to 
return "CACHE_TYPE_SEPERATE" when the level is less than the value 
returned as "split_level" from acpi_get_cache_info(). That tends to 
ignore some edge cases, but I might put them in the "don't to that" bucket.


> .
> 
> Judging from this requirement, can you help review this patch? Thanks.

If so, can you describe the edge case you care about that isn't solved 
with a change like that?

> 
> 
> Thanks,
> Yunhui