Received-SPF: pass (google.com: domain of linux-kernel+bounces-92992-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
Message-ID: <efd2475c-3748-48cc-9918-1cb305f3f581@arm.com>
Date: Tue, 5 Mar 2024 21:07:44 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arm64/mm: Add memory barrier for mm_cid
Content-Language: en-US
To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
 catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com,
 peterz@infradead.org
Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
 nd@arm.com, stable@vger.kernel.org, Aaron Lu <aaron.lu@intel.com>
References: <20240305145335.2696125-1-yeoreum.yun@arm.com>
 <b0a3e152-22bb-4502-a0a0-4b2513bfbec8@efficios.com>
From: "levi.yun" <yeoreum.yun@arm.com>
In-Reply-To: <b0a3e152-22bb-4502-a0a0-4b2513bfbec8@efficios.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
NoDisclaimer: true
X-Microsoft-Antispam-Message-Info-Original:
 fS7fuojPhZU7V/+uc/1w/Ae7IVC+vdAl/WGmtEWRjLRU79+9mXGrOY4VGDX2eMZCHYd8BLxUp7mWIEsrFQr7rKNvJwa9PdmUgEVEcMRvbKy3gY13Mq6OlDeHGsdJhoGzQeMCXHJXcjka+eeVcLFb5cy9MrfFbUbCm+YWXS7aohZuTeCmMafcnLMw/H4Q2Hrna2Lt5KaWa5iznWBuF8uWdbexRYiJWnRspay8n44yC9qseurme1BwHZaCYi1UZTQape4edn3g4dLy9KMHlv2KsTlQ139gQxJm4jeWnYmYP5c8f+AKVm/la3QS04Z0AICrDbxMqJqNb01d0bSlJxODf35K+PUtHNq72Hu/3cqZLhkyrOkkI+N1ib9mbvyMRMt3WJ04PdyAimOIGw5dEIxHsSCuESkfPikcQ5AB+BXWVQiA6iLKu8WaXDoZOFV0giwf5WroalTMAJg09hGxYRMCH/jZ705C33z7Qn2VbJ+qMWJ5zkUPXvkhJS68J51EEyhHmdqdBtIvonLr+1H0LQEJXBtWAWRkmKd/KwCnFnEAJpkjvP5Jp0itTiJ7TMFgwy1ng3+9JtqdyTrMJB3BeXQ+NrzokeZQfCQJYO0LaSvF8ebRczuEoU869WFKsnBWEoI7kqagghdT1xiN/rSOVOciO4Z0f92Pkcyno820wRilg9Y=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR08MB4289.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6178
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5PEPF00014B90.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d1d8fd2b-1f2d-4982-fec7-08dc3d584f3d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dgXtbNV+tQZoDm2MV8czXBWFHHzHRzQYSOBAdllsWlyqfUy2N2yG6hgLcx2N+IBYRLrzhWRBwqCi9+keM6nfPqUbRuO3stiovjtG15aPoBc3CbNH2K4cj43NaiMGNet3D7aqhPoVcAJX/h4hL9JecNFdbydfmoiukGK/o50Yk7pk3q5Jb2bZwUu9Z0A6e56nE9llkiajPUscEyMSWBoz08xJRWAzRyqCB388CtkC84hqXTiz12RY+4020YEPLiWSpo/PBbLUShL8KtRQ/Xfa6JiWYE/7JgYfevEiocwWbIPerpTmSXvt2wRk0qINf5KuH1wJeo23SOnfHYuwphxa2QRcedJyBPYtLo/Cjx8GRcDXGDg16A3dYmqnXvSo6PkyeLS8n9JLzLNX2XUp0isHAjM9TAxdYZIeCse07PeFdNA/spB0NP887JtXtVDh2Nwvg5Do84QB2XZeTLRBb7yvh4bdViq4GMgMiKSqgPHTPtrYOoe7UOgBcV+APtQBga/jTsqT1SNPt5CheY7fhfCOuxJrkZlLkEUm9c6jjWYaLnzkZWnOYJkU4THql7sTTavxQ984HTRnyN3plHG82x0kUpNKH18DlhGVxSo24jY7QCk3V7R7Pq+0se08orC1jceRAEAh/cuBiHP8FR3s8/f25itJ+ww2wqvAuYIlh8YEyBWJDXBfHVhwyr+oz6tvUvpGDhcN04on4sdENINdMYx015wFIqG0VZw0ZyvgNuR+ouu6dvwpvru+/yCv/70e0uRR
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(376005)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Mar 2024 21:07:56.4245
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 957e4012-4f8a-413e-0bcf-08dc3d58548d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5PEPF00014B90.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5693

Hi Mathieu!

On 05/03/2024 20:01, Mathieu Desnoyers wrote:
> On 2024-03-05 09:53, levi.yun wrote:
>> Currently arm64's switch_mm() doesn't always have an smp_mb()
>> which the core scheduler code has depended upon since commit:
>>
>>      commit 223baf9d17f25 ("sched: Fix performance regression 
>> introduced by mm_cid")
>>
>> If switch_mm() doesn't call smp_mb(), sched_mm_cid_remote_clear()
>> can unset the activly used cid when it fails to observe active task 
>> after it
>> sets lazy_put.
>>
>> By adding an smp_mb() in arm64's check_and_switch_context(),
>> Guarantee to observe active task after sched_mm_cid_remote_clear()
>> success to set lazy_put.
>
> This comment from the original implementation of membarrier
> MEMBARRIER_CMD_PRIVATE_EXPEDITED states that the original need from
> membarrier was to have a full barrier between storing to rq->curr and
> return to userspace:
>
> commit 22e4ebb9758 ("membarrier: Provide expedited private command")
>
> commit message:
>
>     * Our TSO archs can do RELEASE without being a full barrier. Look at
>       x86 spin_unlock() being a regular STORE for example.  But for those
>       archs, all atomics imply smp_mb and all of them have atomic ops in
>       switch_mm() for mm_cpumask(), and on x86 the CR3 load acts as a 
> full
>       barrier.
>         * From all weakly ordered machines, only ARM64 and PPC can do 
> RELEASE,
>       the rest does indeed do smp_mb(), so there the spin_unlock() is 
> a full
>       barrier and we're good.
>         * ARM64 has a very heavy barrier in switch_to(), which suffices.
>         * PPC just removed its barrier from switch_to(), but appears 
> to be
>       talking about adding something to switch_mm(). So add a
>       smp_mb__after_unlock_lock() for now, until this is settled on 
> the PPC
>       side.
>
> associated code:
>
> +               /*
> +                * The membarrier system call requires each architecture
> +                * to have a full memory barrier after updating
> +                * rq->curr, before returning to user-space. For TSO
> +                * (e.g. x86), the architecture must provide its own
> +                * barrier in switch_mm(). For weakly ordered machines
> +                * for which spin_unlock() acts as a full memory
> +                * barrier, finish_lock_switch() in common code takes
> +                * care of this barrier. For weakly ordered machines for
> +                * which spin_unlock() acts as a RELEASE barrier (only
> +                * arm64 and PowerPC), arm64 has a full barrier in
> +                * switch_to(), and PowerPC has
> +                * smp_mb__after_unlock_lock() before
> +                * finish_lock_switch().
> +                */
>
> Which got updated to this by
>
> commit 306e060435d ("membarrier: Document scheduler barrier 
> requirements")
>
>                 /*
>                  * The membarrier system call requires each architecture
>                  * to have a full memory barrier after updating
> +                * rq->curr, before returning to user-space.
> +                *
> +                * Here are the schemes providing that barrier on the
> +                * various architectures:
> +                * - mm ? switch_mm() : mmdrop() for x86, s390, sparc, 
> PowerPC.
> +                *   switch_mm() rely on membarrier_arch_switch_mm() 
> on PowerPC.
> +                * - finish_lock_switch() for weakly-ordered
> +                *   architectures where spin_unlock is a full barrier,
> +                * - switch_to() for arm64 (weakly-ordered, spin_unlock
> +                *   is a RELEASE barrier),
>                  */
>
> However, rseq mm_cid has stricter requirements: the barrier needs to be
> issued between store to rq->curr and switch_mm_cid(), which happens
> earlier than:
>
> - spin_unlock(),
> - switch_to().
>
> So it's fine when the architecture switch_mm happens to have that barrier
> already, but less so when the architecture only provides the full barrier
> in switch_to() or spin_unlock().
>
> The issue is therefore not specific to arm64, it's actually a bug in the
> rseq switch_mm_cid() implementation. All architectures that don't have
> memory barriers in switch_mm(), but rather have the full barrier 
> either in
> finish_lock_switch() or switch_to() have them too late for the needs of
> switch_mm_cid().

Thanks for the great detail explain!


>
> I would recommend one of three approaches here:
>
> A) Add smp_mb() in switch_mm_cid() for all architectures that lack that
>    barrier in switch_mm().
>
> B) Figure out if we can move switch_mm_cid() further down in the 
> scheduler
>    without breaking anything (within switch_to(), at the very end of
>    finish_lock_switch() for instance). I'm not sure we can do that though
>    because switch_mm_cid() touches the "prev" which is tricky after
>    switch_to().
>
> C) Add barriers in switch_mm() within all architectures that are 
> missing it.
>
> Thoughts ?
IMHO, A) is look good to me.

Because, In case of B), If you assume spin_unlock() for rq->lock has 
full memory barrier,
I'm not sure about the architecture which using queued_spin_unlock().

When I see the queued_spin_unlock()'s implementation, It implements 
using smp_store_relasse().
But, when we see the memory_barrier.txt describing MULTICOPY ATOMICITY,
If smp_mb__after_atomic() is implemented with smp_mb(), There might fail 
to observe.

Am I wrong?

Many thanks!