Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 17 Feb 2021 09:54:14 -0500 (EST)
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To:     Nadav Amit <nadav.amit@gmail.com>
Cc:     linux-kernel <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>
Message-ID: <330895069.24567.1613573654866.JavaMail.zimbra@efficios.com>
In-Reply-To: <74F1E842-4A84-47BF-B6C2-5407DFDD4A4A@gmail.com>
References: <74F1E842-4A84-47BF-B6C2-5407DFDD4A4A@gmail.com>
Subject: Re: Local execution of ipi_sync_rq_state() on
 sync_runqueues_membarrier_state()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Thread-Topic: Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state()
Thread-Index: aQ9FvzemhUrCAb4Wf2CU2pP04nc+YQ==
Precedence: bulk

----- On Feb 16, 2021, at 4:35 PM, Nadav Amit nadav.amit@gmail.com wrote:

> Hello Mathieu,
> 
> While trying to find some unrelated by, something in
> sync_runqueues_membarrier_state() caught my eye:
> 
> 
>  static int sync_runqueues_membarrier_state(struct mm_struct *mm)
>  {
>        if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) {
>                this_cpu_write(runqueues.membarrier_state, membarrier_state);
> 
>                /*
>                 * For single mm user, we can simply issue a memory barrier
>                 * after setting MEMBARRIER_STATE_GLOBAL_EXPEDITED in the
>                 * mm and in the current runqueue to guarantee that no memory
>                 * access following registration is reordered before
>                 * registration.
>                 */
>                smp_mb();
>                return 0;
>        }
> 
> [ snip ]
> 
>  	smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1);
> 
> 
> And ipi_sync_rq_state() does:
> 
>	this_cpu_write(runqueues.membarrier_state,
>                       atomic_read(&mm->membarrier_state));
> 
> 
> So my question: are you aware smp_call_function_many() would not run
> ipi_sync_rq_state() on the local CPU?

Generally, yes, I am aware of it, but it appears that when I wrote that
code, I missed that important fact. See

commit 227a4aadc75b ("sched/membarrier: Fix p->mm->membarrier_state racy load")

> Is that the intention of the code?

Clearly not! If we look at sync_runqueues_membarrier_state(), there is even a
special-case for mm_users==1 || num online cpus == 1 where it writes the membarrier
state into the current cpu runqueue. I'll prepare a fix, thanks a bunch for spotting
this.

Mathieu

> 
> Thanks,
> Nadav

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com