Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 12 Jul 2019 09:03:28 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Bernard Metzler <BMT@zurich.ibm.com>
Cc:     Arnd Bergmann <arnd@arndb.de>, Doug Ledford <dledford@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] rdma/siw: avoid smp_store_mb() on a u64
Message-ID: <20190712120328.GB27512@ziepe.ca>
References: <20190712085212.3901785-1-arnd@arndb.de>
 <OF05C1A780.433E36D1-ON00258435.003381DA-00258435.003F847E@notes.na.collabserv.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <OF05C1A780.433E36D1-ON00258435.003381DA-00258435.003F847E@notes.na.collabserv.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Jul 12, 2019 at 11:33:46AM +0000, Bernard Metzler wrote:
> >diff --git a/drivers/infiniband/sw/siw/siw_verbs.c
> >b/drivers/infiniband/sw/siw/siw_verbs.c
> >index 32dc79d0e898..41c5ab293fe1 100644
> >+++ b/drivers/infiniband/sw/siw/siw_verbs.c
> >@@ -1142,10 +1142,11 @@ int siw_req_notify_cq(struct ib_cq *base_cq,
> >enum ib_cq_notify_flags flags)
> > 
> > 	if ((flags & IB_CQ_SOLICITED_MASK) == IB_CQ_SOLICITED)
> > 		/* CQ event for next solicited completion */
> >-		smp_store_mb(*cq->notify, SIW_NOTIFY_SOLICITED);
> >+		WRITE_ONCE(*cq->notify, SIW_NOTIFY_SOLICITED);
> > 	else
> > 		/* CQ event for any signalled completion */
> >-		smp_store_mb(*cq->notify, SIW_NOTIFY_ALL);
> >+		WRITE_ONCE(*cq->notify, SIW_NOTIFY_ALL);
> >+	smp_wmb();
> > 
> > 	if (flags & IB_CQ_REPORT_MISSED_EVENTS)
> > 		return cq->cq_put - cq->cq_get;
> 
> 
> Hi Arnd,
> Many thanks for pointing that out! Indeed, this CQ notification
> mechanism does not take 32 bit architectures into account.
> Since we have only three flags to hold here, it's probably better
> to make it a 32bit value. That would remove the issue w/o
> introducing extra smp_wmb(). 

I also prefer not to see smp_wmb() in drivers..

> I'd prefer smp_store_mb(), since on some architectures it shall be
> more efficient.  That would also make it sufficient to use
> READ_ONCE.

The READ_ONCE is confusing to me too, if you need store_release
semantics then the reader also needs to pair with load_acquite -
otherwise it doesn't work.

Still, we need to do something rapidly to fix the i386 build, please
revise right away..

Jason