Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932300AbbFDR5L (ORCPT ); Thu, 4 Jun 2015 13:57:11 -0400 Received: from mail-wi0-f171.google.com ([209.85.212.171]:34030 "EHLO mail-wi0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932131AbbFDR5G (ORCPT ); Thu, 4 Jun 2015 13:57:06 -0400 Message-ID: <5570916D.4070008@colorfullife.com> Date: Thu, 04 Jun 2015 19:57:01 +0200 From: Manfred Spraul User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Davidlohr Bueso , Andrew Morton CC: linux-kernel@vger.kernel.org, Davidlohr Bueso Subject: Re: [PATCH 2/2] ipc,msg: provide barrier pairings for lockless receive References: <1432944186-7305-1-git-send-email-dave@stgolabs.net> <1432944186-7305-2-git-send-email-dave@stgolabs.net> In-Reply-To: <1432944186-7305-2-git-send-email-dave@stgolabs.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3375 Lines: 91 On 05/30/2015 02:03 AM, Davidlohr Bueso wrote: > We currently use a full barrier on the sender side to > to avoid receiver tasks disappearing on us while still > performing on the sender side wakeup. We lack however, > the proper CPU-CPU interactions pairing on the receiver > side which busy-waits for the message. Similarly, we do > not need a full smp_mb, and can relax the semantics for > the writer and reader sides of the message. This is safe > as we are only ordering loads and stores to r_msg. And in > both smp_wmb and smp_rmb, there are no stores after the > calls _anyway_. I like the idea, the pairing in ipc is not good. Another one is still open in sem. Perhaps we should formalize it a bit more, so that it is easy to find which barrier pair belongs together. It is only an idea, but right now there are too many bugs. > This obviously applies for pipelined_send and expunge_all, > for EIRDM when destroying a queue. > > Signed-off-by: Davidlohr Bueso > --- > ipc/msg.c | 31 ++++++++++++++++++++++--------- > 1 file changed, 22 insertions(+), 9 deletions(-) > > diff --git a/ipc/msg.c b/ipc/msg.c > index 2b6fdbb..ac5116e 100644 > --- a/ipc/msg.c > +++ b/ipc/msg.c > @@ -196,7 +196,7 @@ static void expunge_all(struct msg_queue *msq, int res) > * or dealing with -EAGAIN cases. See lockless receive part 1 > * and 2 in do_msgrcv(). > */ > - smp_mb(); > + smp_wmb(); Idea for improvement: Add here /* smp_barrier_pair: ipc_msg_01 */ > msr->r_msg = ERR_PTR(res); > } > } > @@ -580,7 +580,7 @@ static inline int pipelined_send(struct msg_queue *msq, struct msg_msg *msg) > /* initialize pipelined send ordering */ > msr->r_msg = NULL; > wake_up_process(msr->r_tsk); > - smp_mb(); /* see barrier comment below */ > + smp_wmb(); /* see barrier comment below */ > msr->r_msg = ERR_PTR(-E2BIG); > } else { > msr->r_msg = NULL; > @@ -589,11 +589,12 @@ static inline int pipelined_send(struct msg_queue *msq, struct msg_msg *msg) > wake_up_process(msr->r_tsk); > /* > * Ensure that the wakeup is visible before > - * setting r_msg, as the receiving end depends > - * on it. See lockless receive part 1 and 2 in > - * do_msgrcv(). > + * setting r_msg, as the receiving can otherwise > + * exit - once r_msg is set, the receiver can > + * continue. See lockless receive part 1 and 2 > + * in do_msgrcv(). > */ > - smp_mb(); > + smp_wmb(); Idea for improvement: Add here /* smp_barrier_pair: ipc_msg_02 */ > msr->r_msg = msg; > > return 1; > @@ -934,10 +935,22 @@ long do_msgrcv(int msqid, void __user *buf, size_t bufsz, long msgtyp, int msgfl > * wake_up_process(). There is a race with exit(), see > * ipc/mqueue.c for the details. > */ > - msg = (struct msg_msg *)msr_d.r_msg; > - while (msg == NULL) { > - cpu_relax(); > + for (;;) { > + /* > + * Pairs with writer barrier in pipelined_send > + * or expunge_all > + */ > + smp_rmb(); And here again /* smp_barrier_pair: for ipc_msg_01 and ipc_msg_02 */ -- Manfred -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/