New in V3:
* Handle rearrangement of some arch's include/asm directories.
New in V2:
* get rid of ugly #ifdef's in kernel/spinlock.h
* convert __raw_{read|write}_lock_flags to an inline func
SGI has observed that on large systems, interrupts are not serviced for
a long period of time when waiting for a rwlock. The following patch
series re-enables irqs while waiting for the lock, resembling the code
which is already there for spinlocks.
I only made the ia64 version, because the patch adds some overhead to
the fast path. I assume there is currently no demand to have this for
other architectures, because the systems are not so large. Of course,
the possibility to implement raw_{read|write}_lock_flags for any
architecture is still there.
Robin Holt
On Tue, 04 Nov 2008 06:24:05 -0600
[email protected] wrote:
> New in V3:
> * Handle rearrangement of some arch's include/asm directories.
>
> New in V2:
> * get rid of ugly #ifdef's in kernel/spinlock.h
> * convert __raw_{read|write}_lock_flags to an inline func
>
> SGI has observed that on large systems, interrupts are not serviced for
> a long period of time when waiting for a rwlock. The following patch
> series re-enables irqs while waiting for the lock, resembling the code
> which is already there for spinlocks.
>
> I only made the ia64 version, because the patch adds some overhead to
> the fast path. I assume there is currently no demand to have this for
> other architectures, because the systems are not so large. Of course,
> the possibility to implement raw_{read|write}_lock_flags for any
> architecture is still there.
>
The patches seem reasonable. I queued all three with the intention of
merging #1 and #2 into 2.6.29. At that stage, architectures can decide
whether or not they want to do this. I shall then spam Tony with #3 so
you can duke it out with him.
It's a bit regrettable to have different architectures behaving in
different ways. It would be interesting to toss an x86_64
implementation into the grinder, see if it causes any problems, see if
it produces any tangible benefits. Then other architectures might
follow. Or not, depending on the results ;)
> It's a bit regrettable to have different architectures behaving in
> different ways. It would be interesting to toss an x86_64
> implementation into the grinder, see if it causes any problems, see if
> it produces any tangible benefits. Then other architectures might
> follow. Or not, depending on the results ;)
I personally expect SGI to work on this for x86_64 in the future.
Once we actually start testing systems with 128 and above cpus, I
would expect to see these performance issues needing to be addressed.
Until then, it is just a theoretical.
Thank you for your help,
Robin Holt
On Wed, 2008-12-03 at 05:37 -0600, Robin Holt wrote:
> > It's a bit regrettable to have different architectures behaving in
> > different ways. It would be interesting to toss an x86_64
> > implementation into the grinder, see if it causes any problems, see if
> > it produces any tangible benefits. Then other architectures might
> > follow. Or not, depending on the results ;)
>
> I personally expect SGI to work on this for x86_64 in the future.
> Once we actually start testing systems with 128 and above cpus, I
> would expect to see these performance issues needing to be addressed.
> Until then, it is just a theoretical.
Personally I consider this a ugly hack and would love to see people
solve the actual problem and move away from rwlock_t, its utter rubbish.
Peter Zijlstra píše v St 03. 12. 2008 v 13:25 +0100:
> On Wed, 2008-12-03 at 05:37 -0600, Robin Holt wrote:
> > > It's a bit regrettable to have different architectures behaving in
> > > different ways. It would be interesting to toss an x86_64
> > > implementation into the grinder, see if it causes any problems, see if
> > > it produces any tangible benefits. Then other architectures might
> > > follow. Or not, depending on the results ;)
> >
> > I personally expect SGI to work on this for x86_64 in the future.
> > Once we actually start testing systems with 128 and above cpus, I
> > would expect to see these performance issues needing to be addressed.
> > Until then, it is just a theoretical.
>
> Personally I consider this a ugly hack and would love to see people
> solve the actual problem and move away from rwlock_t, its utter rubbish.
Me too, but we don't have that clean and nice solution today, but what
we _do_ have today are the machines which break badly when interrupts
are disabled for the whole duration of taking a rwlock_t. :(
Feel free to rewrite all users of rwlock_t. I'll appreciate it, oh so
very much.
Petr
On Wed, 2008-12-03 at 13:36 +0100, Petr Tesarik wrote:
> Peter Zijlstra píše v St 03. 12. 2008 v 13:25 +0100:
> > On Wed, 2008-12-03 at 05:37 -0600, Robin Holt wrote:
> > > > It's a bit regrettable to have different architectures behaving in
> > > > different ways. It would be interesting to toss an x86_64
> > > > implementation into the grinder, see if it causes any problems, see if
> > > > it produces any tangible benefits. Then other architectures might
> > > > follow. Or not, depending on the results ;)
> > >
> > > I personally expect SGI to work on this for x86_64 in the future.
> > > Once we actually start testing systems with 128 and above cpus, I
> > > would expect to see these performance issues needing to be addressed.
> > > Until then, it is just a theoretical.
> >
> > Personally I consider this a ugly hack and would love to see people
> > solve the actual problem and move away from rwlock_t, its utter rubbish.
>
> Me too, but we don't have that clean and nice solution today, but what
> we _do_ have today are the machines which break badly when interrupts
> are disabled for the whole duration of taking a rwlock_t. :(
Right, which creates an incentive to work on it. So we then either to a
quick and ugly hack and relieve the stress and never again get around to
looking at that particular issue (because management or something
doesn't see the problem anymore) or we do the right thing and fix it
properly.
I prefer the second, but I understand its not always an option.
> Feel free to rewrite all users of rwlock_t. I'll appreciate it, oh so
> very much.
Hehe, I wish! its on my todo list though, along with some other things,
but very unlikely I'll ever get around to it.
The good news is that Eric Dumazet recently got rid of a few prominent
users in the net code.
On Tue, 04 Nov 2008 06:24:05 -0600
[email protected] wrote:
> New in V3:
> * Handle rearrangement of some arch's include/asm directories.
>
> New in V2:
> * get rid of ugly #ifdef's in kernel/spinlock.h
> * convert __raw_{read|write}_lock_flags to an inline func
>
> SGI has observed that on large systems, interrupts are not serviced for
> a long period of time when waiting for a rwlock. The following patch
> series re-enables irqs while waiting for the lock, resembling the code
> which is already there for spinlocks.
>
> I only made the ia64 version, because the patch adds some overhead to
> the fast path. I assume there is currently no demand to have this for
> other architectures, because the systems are not so large. Of course,
> the possibility to implement raw_{read|write}_lock_flags for any
> architecture is still there.
>
I'm not seeing any Tony Luck acks on this work?
> > SGI has observed that on large systems, interrupts are not serviced for
> > a long period of time when waiting for a rwlock. The following patch
> > series re-enables irqs while waiting for the lock, resembling the code
> > which is already there for spinlocks.
> I'm not seeing any Tony Luck acks on this work?
I was initially concerned about how to get enough testing
for this to gain confidence that it doesn't unexpectedly
break something. But Robin pointed out that Novell has
been shipping an equivalent patch in SLES10 without seeing
anything nasty.
Acked-by: Tony Luck <[email protected]>