Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <67c2621f-4cad-2495-9785-7737246d3e90@quicinc.com>
Date:   Wed, 12 Jul 2023 11:09:10 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.2
Subject: Re: [PATCH] arm64: Add the arm64.nolse_atomics command line option
To:     Will Deacon <will@kernel.org>
CC:     <corbet@lwn.net>, <catalin.marinas@arm.com>, <maz@kernel.org>,
        <quic_pkondeti@quicinc.com>, <quic_kaushalk@quicinc.com>,
        <quic_satyap@quicinc.com>, <quic_shashim@quicinc.com>,
        <quic_songxue@quicinc.com>, <linux-doc@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>,
        <linux-arm-kernel@lists.infradead.org>
References: <20230710055955.36551-1-quic_aiquny@quicinc.com>
 <20230710093751.GC32673@willie-the-truck>
 <5cf15f85-0397-96f7-4110-13494551b53b@quicinc.com>
 <20230711082226.GA1554@willie-the-truck>
 <84f0994a-26de-c20a-a32f-ec8fe41df3a3@quicinc.com>
 <20230711102510.GA1809@willie-the-truck>
From:   "Aiqun(Maria) Yu" <quic_aiquny@quicinc.com>
In-Reply-To: <20230711102510.GA1809@willie-the-truck>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 7/11/2023 6:25 PM, Will Deacon wrote:
> On Tue, Jul 11, 2023 at 06:15:49PM +0800, Aiqun(Maria) Yu wrote:
>> On 7/11/2023 4:22 PM, Will Deacon wrote:
>>> On Tue, Jul 11, 2023 at 12:02:22PM +0800, Aiqun(Maria) Yu wrote:
>>>> On 7/10/2023 5:37 PM, Will Deacon wrote:
>>>>> On Mon, Jul 10, 2023 at 01:59:55PM +0800, Maria Yu wrote:
>>>>>> In order to be able to disable lse_atomic even if cpu
>>>>>> support it, most likely because of memory controller
>>>>>> cannot deal with the lse atomic instructions, use a
>>>>>> new idreg override to deal with it.
>>>>>
>>>>> This should not be a problem for cacheable memory though, right?
>>>>>
>>>>> Given that Linux does not issue atomic operations to non-cacheable mappings,
>>>>> I'm struggling to see why there's a problem here.
>>>>
>>>> The lse atomic operation can be issued on non-cacheable mappings as well.
>>>> Even if it is cached data, with different CPUECTLR_EL1 setting, it can also
>>>> do far lse atomic operations.
>>>
>>> Please can you point me to the place in the kernel sources where this
>>> happens? The architecture doesn't guarantee that atomics to non-cacheable
>>> mappings will work, see "B2.2.6 Possible implementation restrictions on
>>> using atomic instructions". Linux, therefore, doesn't issue atomics
>>> to non-cacheable memory.
>>
>> We encounter the issue on third party kernel modules and third party apps
>> instead of linux kernel itself.
> 
> Great, so there's nothing to do in the kernel then!
> 
> The third party code needs to be modified not to use atomic instructions
> with non-cacheable mappings. No need to involve us with that.

> 
>> This is a tradeoff of performance and stability. Per my understanding,
>> options can be used to enable the lse_atomic to have the most performance
>> cared system, and disable the lse_atomic by stability cared most system.
> 
> Where do livelock and starvation fit in with "stability"? Disabling LSE
> atomics for things like qspinlock and the scheduler just because of some
> badly written third-party code isn't much of a tradeoff.
We also have requirement to have cpus/system fully support lse atomic 
and cpus/system not fully support lse atomic with a generic kernel image.
Same kernel module wanted to be used by lse atomic fully support cpu and 
not fully support cpu/system as well.

That's why we want to have a runtime option here.

> 
> Will

-- 
Thx and BRs,
Aiqun(Maria) Yu