We have incorporated various suggestions from Ingo for version 5 of this patchset
and will like to have it merged if there are no objections.
In this patchset, we introduce two categories of optimizations to read
write semaphore. The first four patches from Alex Shi reduce cache bouncing of the
sem->count field by doing a pre-read of the sem->count and avoid cmpxchg
if possible.
The last two patches introduce similar optimistic spinning logic as
the mutex code for the writer lock acquisition of rwsem.
Without these optimizations, Davidlohr Bueso saw a -8% regression to
aim7's shared and high_systime workloads when he switched i_mmap_mutex to
rwsem. Tests were on 8 socket 80 cores system. With the patchset, he
get significant improvements to the aim7 suite instead of regressions:
alltests (+16.3%), custom (+20%), disk (+19.5%), high_systime (+7%),
shared (+18.4%) and short (+6.3%).
Tim Chen also got a +5% improvements to exim mail server workload on a 40 core system.
Thanks to Ingo Molnar, Peter Hurley and Peter Zijlstra for reviewing this patchset.
Regards,
Tim Chen
Changelog:
v5:
1. Try optimistic spinning before we put the writer on the wait queue
to avoid bottlenecking at wait queue. This provides 5% boost to exim workload
and between 2% to 8% boost to aim7.
2. Put MCS locking code into its own mcslock.h file for better reuse
between mutex.c and rwsem.c
3. Remove the configuration RWSEM_SPIN_ON_WRITE_OWNER and make the
operations default per Ingo's suggestions.
v4:
1. Fixed a bug in task_struct definition in rwsem_can_spin_on_owner
2. Fix another typo for RWSEM_SPIN_ON_WRITE_OWNER config option
v3:
1. Added ACCESS_ONCE to sem->count access in rwsem_can_spin_on_owner.
2. Fix typo bug for RWSEM_SPIN_ON_WRITE_OWNER option in init/Kconfig
v2:
1. Reorganize changes to down_write_trylock and do_wake into 4 patches and fixed
a bug referencing &sem->count when sem->count is intended.
2. Fix unsafe sem->owner de-reference in rwsem_can_spin_on_owner.
the option to be on for more seasoning but can be turned off should it be detrimental.
3. Various patch comments update
Alex Shi (4):
rwsem: check the lock before cpmxchg in down_write_trylock
rwsem: remove 'out' label in do_wake
rwsem: remove try_reader_grant label do_wake
rwsem/wake: check lock before do atomic update
Tim Chen (2):
MCS Lock: Restructure the MCS lock defines and locking code into its
own file
rwsem: do optimistic spinning for writer lock acquisition
include/asm-generic/rwsem.h | 8 +-
include/linux/rwsem.h | 8 +-
kernel/mutex.c | 58 ++----------
kernel/rwsem.c | 19 ++++-
lib/rwsem.c | 218 ++++++++++++++++++++++++++++++++++++++-----
5 files changed, 228 insertions(+), 83 deletions(-)
--
1.7.4.4
On Tue, 2013-09-24 at 15:22 -0700, Tim Chen wrote:
> We have incorporated various suggestions from Ingo for version 5 of this patchset
> and will like to have it merged if there are no objections.
>
> In this patchset, we introduce two categories of optimizations to read
> write semaphore. The first four patches from Alex Shi reduce cache bouncing of the
> sem->count field by doing a pre-read of the sem->count and avoid cmpxchg
> if possible.
>
> The last two patches introduce similar optimistic spinning logic as
> the mutex code for the writer lock acquisition of rwsem.
Right. We address the general 'mutexes out perform writer-rwsems'
situations that has been seen in more than one case. Users now need not
worry about performance issues when choosing between these two locking
mechanisms.
Thanks,
Davidlohr