For qspinlocks on ARM64, we would like to use WFE instead
of purely spinning. Qspinlocks internally have lock
contenders spin on an MCS lock.
Update arch_mcs_spin_lock_contended() such that it uses
the new smp_cond_load_acquire() so that ARM64 can also
override this spin loop with its own implementation using WFE.
On x86, it can also cheaper to use this than spinning on
smp_load_acquire().
Signed-off-by: Jason Low <[email protected]>
---
Changes from v1:
- Pass l instead of &l to smp_cond_load_acquire() since
l is already a pointer to the lock variable.
kernel/locking/mcs_spinlock.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/kernel/locking/mcs_spinlock.h b/kernel/locking/mcs_spinlock.h
index c835270..1c55987 100644
--- a/kernel/locking/mcs_spinlock.h
+++ b/kernel/locking/mcs_spinlock.h
@@ -22,13 +22,13 @@ struct mcs_spinlock {
#ifndef arch_mcs_spin_lock_contended
/*
- * Using smp_load_acquire() provides a memory barrier that ensures
- * subsequent operations happen after the lock is acquired.
+ * Using smp_cond_load_acquire() provides the acquire semantics
+ * required so that subsequent operations happen after the
+ * lock is acquired.
*/
#define arch_mcs_spin_lock_contended(l) \
do { \
- while (!(smp_load_acquire(l))) \
- cpu_relax_lowlatency(); \
+ smp_cond_load_acquire(l, VAL); \
} while (0)
#endif
--
2.1.4
On Tue, Apr 12, 2016 at 08:02:17PM -0700, Jason Low wrote:
> For qspinlocks on ARM64, we would like to use WFE instead
> of purely spinning. Qspinlocks internally have lock
> contenders spin on an MCS lock.
>
> Update arch_mcs_spin_lock_contended() such that it uses
> the new smp_cond_load_acquire() so that ARM64 can also
> override this spin loop with its own implementation using WFE.
>
> On x86, it can also cheaper to use this than spinning on
> smp_load_acquire().
>
> Signed-off-by: Jason Low <[email protected]>
FWIW, we just override arch_mcs_spin_lock_contended entirely for arch/arm/
and use wfe there so we could do the same for arm64 in mainline already.
Will
On Wed, 2016-04-13 at 10:43 -0700, Will Deacon wrote:
> On Tue, Apr 12, 2016 at 08:02:17PM -0700, Jason Low wrote:
> > For qspinlocks on ARM64, we would like to use WFE instead
> > of purely spinning. Qspinlocks internally have lock
> > contenders spin on an MCS lock.
> >
> > Update arch_mcs_spin_lock_contended() such that it uses
> > the new smp_cond_load_acquire() so that ARM64 can also
> > override this spin loop with its own implementation using WFE.
> >
> > On x86, it can also cheaper to use this than spinning on
> > smp_load_acquire().
> >
> > Signed-off-by: Jason Low <[email protected]>
>
> FWIW, we just override arch_mcs_spin_lock_contended entirely for arch/arm/
> and use wfe there so we could do the same for arm64 in mainline already.
Right, I was also thinking about that, although when we use
smp_cond_load_acquire() in the generic implementation, would we just end
up overriding it for the arch/arm64 version with the same thing? :)
Jason
On Wed, Apr 13, 2016 at 01:49:04PM -0700, Jason Low wrote:
> On Wed, 2016-04-13 at 10:43 -0700, Will Deacon wrote:
> > On Tue, Apr 12, 2016 at 08:02:17PM -0700, Jason Low wrote:
> > > For qspinlocks on ARM64, we would like to use WFE instead
> > > of purely spinning. Qspinlocks internally have lock
> > > contenders spin on an MCS lock.
> > >
> > > Update arch_mcs_spin_lock_contended() such that it uses
> > > the new smp_cond_load_acquire() so that ARM64 can also
> > > override this spin loop with its own implementation using WFE.
> > >
> > > On x86, it can also cheaper to use this than spinning on
> > > smp_load_acquire().
> > >
> > > Signed-off-by: Jason Low <[email protected]>
> >
> > FWIW, we just override arch_mcs_spin_lock_contended entirely for arch/arm/
> > and use wfe there so we could do the same for arm64 in mainline already.
>
> Right, I was also thinking about that, although when we use
> smp_cond_load_acquire() in the generic implementation, would we just end
> up overriding it for the arch/arm64 version with the same thing? :)
Sure, and we can convert to smp_cond_load_acquire then. I was just thinking
that you can avoid the immediate dependency on Peter's stuff, that's all.
Will