Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC PATCH 5/5] locking/rwsem: Remove reader optimistic spinning
To:     Davidlohr Bueso <dave@stgolabs.net>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>, Will Deacon <will@kernel.org>,
        linux-kernel@vger.kernel.org, Phil Auld <pauld@redhat.com>
References: <20201118030429.23017-1-longman@redhat.com>
 <20201118030429.23017-6-longman@redhat.com>
 <20201118053556.3fmmtat7upv6dtvd@linux-p48b.lan>
From:   Waiman Long <longman@redhat.com>
Organization: Red Hat
Message-ID: <ee34bc01-9fef-23ff-ada1-1ec2d39533c9@redhat.com>
Date:   Thu, 19 Nov 2020 13:40:05 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.0
MIME-Version: 1.0
In-Reply-To: <20201118053556.3fmmtat7upv6dtvd@linux-p48b.lan>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Precedence: bulk

On 11/18/20 12:35 AM, Davidlohr Bueso wrote:
> On Tue, 17 Nov 2020, Waiman Long wrote:
>
>> The column "CS Load" represents the number of pause instructions issued
>> in the locking critical section. A CS load of 1 is extremely short and
>> is not likey in real situations. A load of 20 (moderate) and 100 (long)
>> are more realistic.
>>
>> It can be seen that the previous patches in this series have reduced
>> performance in general except in highly contended cases with moderate
>> or long critical sections that performance improves a bit. This change
>> is mostly caused by the "Prevent potential lock starvation" patch that
>> reduce reader optimistic spinning and hence reduce reader fragmentation.
>>
>> The patch that further limit reader optimistic spinning doesn't seem to
>> have too much impact on overall performance as shown in the benchmark
>> data.
>>
>> The patch that disables reader optimistic spinning shows reduced
>> performance at lightly loaded cases, but comparable or slightly better
>> performance on with heavier contention.
>
> I'm not overly worried about the lightly loaded cases here as the users
> (mostly thinking mmap_sem) most likely won't care for real workloads,
> not, ie: will-it-scale type things.
I am not that worry about the lightly loaded cases either. I just state 
the fact that some workloads may see a slightly reduced performance 
because of that.
>
> So at SUSE we also ran into this very same problem with reader optimistic
> spinning and considering the fragmentation went with disabling it, much
> like this patch - but without the reader optimistic lock stealing bits
> you have. So far nothing has really shown to fall out in our performance
> automation. And per your data a single reader spinner does not seem to be
> worth the added complexity of keeping reader spinning vs ripping it out. 

My own testing also show not too much performance difference when 
removing reader spinning except in the lightly loaded cases.

Cheers,
Longman