Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030758AbbD1Rni (ORCPT ); Tue, 28 Apr 2015 13:43:38 -0400 Received: from casper.infradead.org ([85.118.1.10]:54443 "EHLO casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030375AbbD1Rnf (ORCPT ); Tue, 28 Apr 2015 13:43:35 -0400 Date: Tue, 28 Apr 2015 19:43:23 +0200 From: Peter Zijlstra To: Chris Metcalf Cc: "Paul E. McKenney" , Manfred Spraul , Oleg Nesterov , Kirill Tkhai , linux-kernel@vger.kernel.org, Ingo Molnar , Josh Poimboeuf Subject: Re: [PATCH 2/2] [PATCH] sched: Add smp_rmb() in task rq locking cycles Message-ID: <20150428174323.GL5029@twins.programming.kicks-ass.net> References: <20150218224317.GC5029@twins.programming.kicks-ass.net> <20150219141905.GA11018@redhat.com> <54E77CC0.5030401@colorfullife.com> <20150220184551.GQ2896@worktop.programming.kicks-ass.net> <20150425195602.GA26676@linux.vnet.ibm.com> <20150426105213.GA27191@linux.vnet.ibm.com> <20150428143357.GF23123@twins.programming.kicks-ass.net> <553FACF1.2020405@ezchip.com> <20150428164033.GJ5029@twins.programming.kicks-ass.net> <553FBC4F.3070104@ezchip.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <553FBC4F.3070104@ezchip.com> User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2373 Lines: 87 On Tue, Apr 28, 2015 at 12:58:55PM -0400, Chris Metcalf wrote: > On 04/28/2015 12:40 PM, Peter Zijlstra wrote: > >On Tue, Apr 28, 2015 at 11:53:21AM -0400, Chris Metcalf wrote: > > > >>The reason we use two 32-bit fields on tilepro is that the only available > >>atomic instruction is tns (test and set), which sets a 32-bit "1" value > >>into the target memory and returns the old 32-bit value. > >And you want a ticket lock as opposed to the test-and-set lock because > >with 64 tiles starvation under contention is a real worry? > > We see substantial unfairness under load with a plain spinlock, > basically because nearer cores on the mesh network can exponentially > crowd out further cores. The ticket lock avoids that, though we > have to be careful to do backoff when checking the lock to avoid > DDoS in the mesh network. Does your arch have 16bit atomic load/stores ? If so, would something like the below not make sense? typedef struct { union { struct { unsigned short head; unsigned short tail; }; unsigned int tickets; }; unsigned int lock; } arch_spinlock_t; static inline void ___tns_lock(unsigned int *lock) { while (tns(lock)) cpu_relax(); } static inline void ___tns_unlock(unsigned int *lock) { WRITE_ONCE(*lock, 0); } static inline void arch_spin_lock(arch_spinlock_t *lock) { unsigned short head, tail; ___tns_lock(&lock->lock); /* XXX does the TNS imply a ___sync? */ head = lock->head; lock->head++; ___tns_unlock(&lock->lock); while (READ_ONCE(lock->tail) != head) cpu_relax(); } static inline void arch_spin_unlock(arch_spinlock_t *lock) { /* * can do with regular load/store because the lock owner * is the only one going to do stores to the tail */ unsigned short tail = READ_ONCE(lock->tail); smp_mb(); /* MB is stronger than RELEASE */ WRITE_ONCE(lock->tail, tail + 1); } static inline void arch_spin_unlock_wait(arch_spinlock_t *lock) { union { struct { unsigned short head; unsigned short tail; }; unsigned int tickets; } x; for (;;) { x.tickets = READ_ONCE(lock->tickets); if (x.head == x.tail) break; cpu_relax(); } } -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/