Absolutely you should minimize the locking contention.
However, that isn't always possible, such as when you
have 64 processors contending on the same resource.
With the current kernel, the trivial example with reader/
writer locks was having them all call gettimeofday().
But try having 64 processors fstat() the same file,
which I have also seen happen (application looping,
waiting for another process to finish setting up the
file so they can all mmap it).
What MCS locks do is they reduce the number of times
the cacheline has to be flung around the system in
order to get work done: they "scale" much better with
the number of processors: O(N) instead of O(N^2).
Kevin
-----Original Message-----
From: Matthew Wilcox
To: Van Maren, Kevin
Cc: 'Linus Torvalds '; 'Jeremy Fitzhardinge '; 'William Lee Irwin III ';
'[email protected] '; 'Linux Kernel List '; '[email protected] ';
'[email protected] '; '[email protected] '
Sent: 11/8/02 12:52 PM
Subject: Re: [Linux-ia64] reader-writer livelock problem
On Fri, Nov 08, 2002 at 11:41:57AM -0600, Van Maren, Kevin wrote:
> processor to acquire/release the lock once. So with 32 processors
> contending for the lock, at 1us per cache-cache transfer (picked
if you have 32 processors contending for the same spinlock, you have
bigger problems.
--
Revolutions do not require corporate support.
On Fri, Nov 08, 2002 at 12:05:30PM -0600, Van Maren, Kevin wrote:
> Absolutely you should minimize the locking contention.
> However, that isn't always possible, such as when you
> have 64 processors contending on the same resource.
if you've got 64 processors contending on the same resource, maybe you
need to split that resource up so they can have a copy each. all that
cacheline bouncing can't do your numa boxes any good.
> With the current kernel, the trivial example with reader/
> writer locks was having them all call gettimeofday().
i hear x86-64 has a lockless gettimeofday. maybe that's the solution.
> But try having 64 processors fstat() the same file,
> which I have also seen happen (application looping,
> waiting for another process to finish setting up the
> file so they can all mmap it).
umm.. the call trace:
sys_fstat
|-> vfs_fstat
| |-> fget
| |-> read_lock(&files->file_lock)
| |-> vfs_getattr
| |-> inode->i_op->getattr
| |-> generic_fillattr
|-> cp_new_stat64
|-> memset
|-> copy_to_user
so you're talking about contention on files->file_lock, right? it's really
not the kernel's fault that your app is badly written. that lock's private
to process & children, so it's not like another application can hurt you.
> What MCS locks do is they reduce the number of times
> the cacheline has to be flung around the system in
> order to get work done: they "scale" much better with
> the number of processors: O(N) instead of O(N^2).
yes, but how slow are they in the uncontended case?
--
Revolutions do not require corporate support.
>>>>> On Fri, 8 Nov 2002 19:19:07 +0000, Matthew Wilcox <[email protected]> said:
Matthew> On Fri, Nov 08, 2002 at 12:05:30PM -0600, Van Maren, Kevin
Matthew> wrote:
>> Absolutely you should minimize the locking contention. However,
>> that isn't always possible, such as when you have 64 processors
>> contending on the same resource.
Matthew> if you've got 64 processors contending on the same
Matthew> resource, maybe you need to split that resource up so they
Matthew> can have a copy each. all that cacheline bouncing can't do
Matthew> your numa boxes any good.
Matthew, please understand that this is NOT a performance problem.
It's a correctness problem. If livelock can resolut from read-write locks,
it's a huge security problem. Period.
--david