Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Sun, 17 Feb 2019 13:48:00 -0500
From:   Rich Felker <dalias@libc.org>
To:     linux-kernel@vger.kernel.org
Cc:     Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Alexander Viro <viro@zeniv.linux.org.uk>
Subject: Regression in SYS_membarrier expedited
Message-ID: <20190217184800.GA16118@brightrain.aerifal.cx>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

commit a961e40917fb14614d368d8bc9782ca4d6a8cd11 made it so that the
MEMBARRIER_CMD_PRIVATE_EXPEDITED command cannot be used without first
registering intent to use it. However, registration is an expensive
operation since commit 3ccfebedd8cf54e291c809c838d8ad5cc00f5688, which
added synchronize_sched() to it; this means it's no longer possible to
lazily register intent at first use, and it's unreasonably expensive
to preemptively register intent for possibly extremely-short-lived
processes that will never use it. (My usage case is in libc (musl),
where I can't know if the process will be short- or long-lived;
unnecessary and potentially expensive syscalls can't be made
preemptively, only lazily at first use.)

Can we restore the functionality of MEMBARRIER_CMD_PRIVATE_EXPEDITED
to work even without registration? The motivation of requiring
registration seems to be:

    "Registering at this time removes the need to interrupt each and
    every thread in that process at the first expedited
    sys_membarrier() system call."

but interrupting every thread in the process is exactly what I expect,
and is not a problem. What does seem like a big problem is waiting for
synchronize_sched() to synchronize with an unboundedly large number of
cores (vs only a few threads in the process), especially in the
presence of full_nohz, where it seems like latency would be at least a
few ms and possibly unbounded.

Short of a working SYS_membarrier that doesn't require expensive
pre-registration, I'm stuck just implementing it in userspace with
signals...

Rich