DomainKey-Signature: a=rsa-sha1; s=beta; d=google.com; c=nofws; q=dns;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to:
	cc:content-type:content-transfer-encoding:x-system-of-record;
	b=Y44vccontBWaGHtjwpy5IkGLOTIvJEcPdBZqBhopbpmSxHLC8z1QJsuT5zuyWm86E
	9V6SQxQzja4wOdYxkQYmQ==
MIME-Version: 1.0
In-Reply-To: <20090724155041.GF5878@count0.beaverton.ibm.com>
References: <20090724032033.2463.79256.stgit@hastromil.mtv.corp.google.com>
	 <20090724032200.2463.82408.stgit@hastromil.mtv.corp.google.com>
	 <20090724155041.GF5878@count0.beaverton.ibm.com>
Date: Fri, 24 Jul 2009 09:01:46 -0700
Message-ID: <6599ad830907240901w4fc02097k83d0c1012708e6ee@mail.gmail.com>
Subject: Re: [PATCH 5/6] Makes procs file writable to move all threads by tgid 
	at once
From: Paul Menage <menage@google.com>
To: Matt Helsley <matthltc@us.ibm.com>
Cc: Ben Blum <bblum@google.com>, linux-kernel@vger.kernel.org,
       containers@lists.linux-foundation.org, akpm@linux-foundation.org,
       serue@us.ibm.com, lizf@cn.fujitsu.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1921
Lines: 46

On Fri, Jul 24, 2009 at 8:50 AM, Matt Helsley<matthltc@us.ibm.com> wrote:
>
> There is much ado about not taking additional "global locks" in fork()
> paths.
>
> * The fork and exit callbacks cgroup_fork() and cgroup_exit(), don't
> * (usually) take cgroup_mutex. ?These are the two most performance
> * critical pieces of code here.
> ...
>
> and as I recall cgroup_fork() doesn't ever take cgroup_mutex because it is
> so performance critical.

cgroup_mutex is a much bigger and heavier mutex than the new rwsem
being introduced in this patch. It's sort of the BKL of cgroups,
although where possible I'm encouraging use of finer-grained
alternatives (such as subsystem-specific locks, the per-hierarchy
lock, etc).

> Assuming the above comments in kernel/cgroup.c
> are correct then this patch adds a performance regression by introducing a
> global mutex in the fork path, doesn't it?

Yes, although to what magnitude isn't clear.

Alternatives that we looked at were:

- add a clone_rwsem to task_struct, and require that a clone operation
that's adding to the same thread group take a read lock on the
leader's clone_rwsem; then the effect would be localised to a single
process; but for a system that has one big multi-threaded server on
it, the effect would still be similar to a global lock

- move the housekeeping done by cgroup_fork() inside the tasklist_lock
critical section in do_fork(); then cgroup_attach_proc() can rely on
the existing global tasklist_lock to provide the necessary
synchronization, rather than introducing a second global lock; the
downside is that it slightly increases the size of the section where
tasklist_lock is held for write.

Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/