Date: Tue, 30 Oct 2012 07:34:36 +0100
From: Andrew Jones <drjones@redhat.com>
To: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>, "H. Peter Anvin" <hpa@zytor.com>,
        Avi Kivity <avi@redhat.com>, Ingo Molnar <mingo@redhat.com>,
        Marcelo Tosatti <mtosatti@redhat.com>, Rik van Riel <riel@redhat.com>,
        Srikar <srikar@linux.vnet.ibm.com>,
        "Nikunj A. Dadhania" <nikunj@linux.vnet.ibm.com>,
        KVM <kvm@vger.kernel.org>, Jiannan Ouyang <ouyang@cs.pitt.edu>,
        Chegu Vinod <chegu_vinod@hp.com>,
        "Andrew M. Theurer" <habanero@linux.vnet.ibm.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Srivatsa Vaddagiri <srivatsa.vaddagiri@gmail.com>,
        Gleb Natapov <gleb@redhat.com>
Subject: Re: [PATCH V2 RFC 3/3] kvm: Check system load and handle different
 commit cases accordingly
Message-ID: <20121030063436.GA2224@turtle.usersys.redhat.com>
References: <20121029140621.15448.92083.sendpatchset@codeblue>
 <20121029140717.15448.83182.sendpatchset@codeblue>
 <1351533280.24721.46.camel@twins>
 <508F6C60.1050202@linux.vnet.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <508F6C60.1050202@linux.vnet.ibm.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3102
Lines: 94

On Tue, Oct 30, 2012 at 11:27:52AM +0530, Raghavendra K T wrote:
> On 10/29/2012 11:24 PM, Peter Zijlstra wrote:
> >On Mon, 2012-10-29 at 19:37 +0530, Raghavendra K T wrote:
> >>+/*
> >>+ * A load of 2048 corresponds to 1:1 overcommit
> >>+ * undercommit threshold is half the 1:1 overcommit
> >>+ * overcommit threshold is 1.75 times of 1:1 overcommit threshold
> >>+ */
> >>+#define COMMIT_THRESHOLD (FIXED_1)
> >>+#define UNDERCOMMIT_THRESHOLD (COMMIT_THRESHOLD >> 1)
> >>+#define OVERCOMMIT_THRESHOLD ((COMMIT_THRESHOLD << 1) -
> >>(COMMIT_THRESHOLD >> 2))
> >>+
> >>+unsigned long kvm_system_load(void)
> >>+{
> >>+       unsigned long load;
> >>+
> >>+       load = avenrun[0] + FIXED_1/200;
> >>+       load = load / num_online_cpus();
> >>+
> >>+       return load;
> >>+}
> >
> >ARGH.. no that's wrong.. very wrong.
> >
> >  1) avenrun[] EXPORT_SYMBOL says it should be removed, that's not a
> >joke.
> 
> Okay.
> 
> >  2) avenrun[] is a global load, do not ever use a global load measure
> 
> This makes sense. Using a local optimization that leads to near global
> optimization is the way to go.
> 
> >
> >  3) avenrun[] has nothing what so ever to do with runqueue lengths,
> >someone with a gazillion tasks in D state will get a huge load but the
> >cpu is very idle.
> >
> 
> I used loadavg as an alternative measure. But the above condition
> poses a concern for that.
> 
> Okay, now IIUC, usage of *any* global measure is bad?
> 
> Because I was also thinking to use nrrunning()/ num_online_cpus(), to
> get an idea of global overcommit sense. (ofcourse since, this involves
> iteration over per CPU nrrunning, I wanted to calculate this
> periodically)
> 
> The overall logic, of having overcommit_threshold,
> undercommit_threshold, I wanted to use for even dynamic ple_window
> tuning purpose.
> 
> so logic was:
> < undercommit_threshold => 16k ple_window
> > overcommit_threshold  => 4k window.
> for in between case scale the ple_window accordingly.
> 
> The alternative was to decide depending on how ple handler succeeded in
> yield_to. But I thought, that is too sensitive and more overhead.
> 
> This topic may deserve different thread, but thought I shall table it here.
> 
> So, Thinking about the alternatives to implement, logic such as
> 
> (a) if(undercommitted)
>     just go back and spin rather than going for yield_to iteration.
> (b) if (overcommitted)
>    better to yield rather than  spinning logic
> 
>    of current patches..
> 
> [ ofcourse, (a) is already met to large extent by your patches..]
> 
> So I think everything boils down to
> 
> "how do we measure these two thresholds without much overhead in a
> compliant way"
> 
> Ideas welcome..
> 

What happened to Avi's preempt notifier idea for determining
under/overcommit? If nobody has picked that up yet, then I'll go ahead and
try to prototype it.

Drew
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/