Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 10 Sep 2018 11:31:56 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Waiman Long <longman@redhat.com>
Cc:     Ingo Molnar <mingo@redhat.com>, Will Deacon <will.deacon@arm.com>,
        linux-kernel@vger.kernel.org, Davidlohr Bueso <dave@stgolabs.net>
Subject: Re: [PATCH] locking/rwsem: Make owner store task pointer of last
 owning reader
Message-ID: <20180910093156.GS24082@hirez.programming.kicks-ass.net>
References: <1536265114-10842-1-git-send-email-longman@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1536265114-10842-1-git-send-email-longman@redhat.com>
User-Agent: Mutt/1.10.0 (2018-05-17)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Sep 06, 2018 at 04:18:34PM -0400, Waiman Long wrote:
> Currently, when a reader acquires a lock, it only sets the
> RWSEM_READER_OWNED bit in the owner field. The other bits are simply
> not used. When debugging hanging cases involving rwsems and readers,
> the owner value does not provide much useful information at all.
> 
> This patch modifies the current behavior to always store the task_struct
> pointer of the last rwsem-acquiring reader in a reader-owned rwsem. This
> may be useful in debugging rwsem hanging cases especially if only one
> reader is involved. However, the task in the owner field may not the
> real owner or one of the real owners at all when the owner value is
> examined, for example, in a crash dump. So it is just an additional
> hint about the past history.
> 
> If CONFIG_DEBUG_RWSEMS is enabled, the owner field will be checked at
> unlock time too to make sure the task pointer value is valid. That does
> have a slight performance cost and so is only enabled as part of that
> debug option.
> 
> From the performance point of view, it is expected that the changes
> shouldn't have any noticeable performance impact. A rwsem microbenchmark
> (with 48 worker threads and 1:1 reader/writer ratio) was ran on a
> 2-socket 24-core 48-thread Haswell system.  The locking rates on a
> 4.19-rc1 based kernel were as follows:
> 
>   1) Unpatched kernel:				543.3 kops/s
>   2) Patched kernel:				549.2 kops/s
>   3) Patched kernel (CONFIG_DEBUG_RWSEMS on):	546.6 kops/s
> 
> There was actually a slight increase in performance (1.1%) in this
> particular case. Maybe it was caused by the elimination of a branch or
> just a testing noise. Turning on the CONFIG_DEBUG_RWSEMS option also
> had less than the expected impact on performance.
> 
> The least significant 2 bits of the owner value are now used to designate
> the rwsem is readers owned and the owners are anonymous.

So no immediate objection; but I'm still hoping to some day rewrite the
whole rwsem thing along the lines we did mutex and merge the 'count' and
'owner' fields into one.

[ RT has something along those lines, and I have half a patch that
  starts doing that too, but I never found enough time to really work on
  it :-( ]

Anyway, when we do something like that, this goes out the window of
course.