Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: [PATCH] SUNRPC: Don't allow compiler optimisation of
 svc_xprt_release_slot()
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <20190125203001.GA5972@fieldses.org>
Date:   Fri, 25 Jan 2019 13:32:58 -0800
Cc:     Trond Myklebust <trondmy@hammerspace.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <40A18BC9-E3F9-45D0-98FF-7BCA171A3E62@oracle.com>
References: <4077991d3d3acee4c37c7c8c6dc2b76930c9584e.camel@hammerspace.com>
 <20190109165142.GB32189@fieldses.org>
 <300445038b75d5efafe9391eb4b8e83d9d6e3633.camel@hammerspace.com>
 <20190111211235.GA27206@fieldses.org>
 <BB7240BE-3F20-43AD-B709-7F18CC33CC02@oracle.com>
 <6F5B73B7-E9F8-4FDB-8381-E5C02772C6A5@oracle.com>
 <20190111221030.GA28794@fieldses.org>
 <A33F8CE5-3E5D-4359-8061-8DB8214B012B@oracle.com>
 <20190112005613.GA29181@fieldses.org>
 <F21839E5-A5DB-4BC8-8F3B-2AD02D4B2652@oracle.com>
 <20190125203001.GA5972@fieldses.org>
To:     Bruce Fields <bfields@fieldses.org>
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk


> On Jan 25, 2019, at 12:30 PM, Bruce Fields <bfields@fieldses.org> =
wrote:
>=20
> On Mon, Jan 14, 2019 at 12:24:24PM -0500, Chuck Lever wrote:
>>=20
>>=20
>>> On Jan 11, 2019, at 7:56 PM, Bruce Fields <bfields@fieldses.org> =
wrote:
>>>=20
>>> On Fri, Jan 11, 2019 at 05:27:30PM -0500, Chuck Lever wrote:
>>>>=20
>>>>=20
>>>>> On Jan 11, 2019, at 5:10 PM, Bruce Fields <bfields@fieldses.org> =
wrote:
>>>>>=20
>>>>> On Fri, Jan 11, 2019 at 04:54:01PM -0500, Chuck Lever wrote:
>>>>>>> On Jan 11, 2019, at 4:52 PM, Chuck Lever =
<chuck.lever@oracle.com> wrote:
>>>>>>>> So, I think we need your patch plus something like this.
>>>>>>>>=20
>>>>>>>> Chuck, maybe you could help me with the "XXX: Chuck:" parts?
>>>>>>>=20
>>>>>>> I haven't been following. Why do you think those are necessary?
>>>>>=20
>>>>> I'm worried something like this could happen:
>>>>>=20
>>>>> 	CPU 1				CPU 2
>>>>> 	-----				-----
>>>>>=20
>>>>> 	set XPT_DATA			dec xpt_nr_rqsts
>>>>>=20
>>>>> 	svc_xprt_enqueue		svc_xprt_enqueue
>>>>>=20
>>>>> And both decide nothing should be done if neither sees the change =
that
>>>>> the other made.
>>>>>=20
>>>>> Maybe I'm still missing some reason that couldn't happen.
>>>>>=20
>>>>> Even if it can happen, it's an unlikely race that will likely be =
fixed
>>>>> when another event comes along a little later, which would explain =
why
>>>>> we've never seen any reports.
>>>>>=20
>>>>>>> We've had set_bit and atomic_{inc,dec} in this code for ages,
>>>>>>> and I've never noticed a problem.
>>>>>>>=20
>>>>>>> Rather than adding another CPU pipeline bubble in the RDMA code,
>>>>>>> though, could you simply move the set_bit() call site inside the
>>>>>>> critical sections?
>>>>>>=20
>>>>>> er, inside the preceding critical section. Just reverse the order
>>>>>> of the spin_unlock and the set_bit.
>>>>>=20
>>>>> That'd do it, thanks!
>>>>=20
>>>> I can try that here and see if it results in a performance =
regression.
>>>=20
>>> Thanks, I've got a version with a typo fixed at
>>>=20
>>> 	git://linux-nfs.org/~bfields/linux.git nfsd-next
>>=20
>> Applied all four patches here. I don't see any performance =
regressions,
>> but my server has only a single last-level CPU cache.
>=20
> Thanks!
>=20
> I'm adding a Tested-by: for you if that's OK.

Sorry, yes! that's fine with me.


--
Chuck Lever