Date: Tue, 17 Oct 2006 12:18:23 -0700
From: Paul Jackson <pj@sgi.com>
To: "Siddha, Suresh B" <suresh.b.siddha@intel.com>
Cc: dino@in.ibm.com, suresh.b.siddha@intel.com, menage@google.com,
       Simon.Derr@bull.net, linux-kernel@vger.kernel.org, mbligh@google.com,
       rohitseth@google.com, dipankar@in.ibm.com
Subject: Re: [RFC] Cpuset: explicit dynamic sched domain control flags
Message-Id: <20061017121823.e6f695aa.pj@sgi.com>
In-Reply-To: <20061017114306.A19690@unix-os.sc.intel.com>
References: <20061016230351.19049.29855.sendpatchset@jackhammer.engr.sgi.com>
	<20061017114306.A19690@unix-os.sc.intel.com>
Organization: SGI
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1881
Lines: 44

> What happens when the job in the cpuset with no sched domain
> becomes active? In this case, scheduler can't make use of all cpus
> that this cpuset is allowed to use.

What happens then is that the job manager marks the cpuset of this
newly activated job as being a sched_domain.

And if the job manager doesn't do that, and sets up a situation in
which the scheduler domains don't line up with the active jobs, then
they can't get scheduler load balancing across all the CPUs in those
jobs cpusets.  That's exactly what they asked for -- that's exactly
what they got.

(Actually, is that right?  I thought load balancing would still occur
at higher levels in the sched domain/group hierarchy, just not as
often.)

It is not the kernels job to make it impossible for user code to do
stupid things.  It's the kernels job to offer up various mechanisms,
and let user space code decide what to do when.

And, anyhow, how does this differ from overloading the cpu_exclusive
flag to define sched domains.  One can setup the same thing there,
where a job can't balance across all its CPUs:

	/dev/cpuset/cs1		cpu_exclusive = 1; cpus = 0-7
	/dev/cpuset/cs1/suba	cpu_exclusive = 1; cpus = 0-3
	/dev/cpuset/cs1/subb	cpu_exclusive = 1; cpus = 4-7

(sched_domain_enabled = 0 in all cpusets)

If you put a task in cpuset "cs1" (not in one of the sub cpusets)
then it can't load balance between CPUs 0-3 and CPUs 4-7 (or can't
load balance as often - depending on how this works.)

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/