Subject: Re: lmbench lat_mmap slowdown with CONFIG_PARAVIRT
From: Zachary Amsden <zach@vmware.com>
To: Ingo Molnar <mingo@elte.hu>
Cc: Nick Piggin <npiggin@suse.de>,
       Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
       Linus Torvalds <torvalds@linux-foundation.org>,
       "hpa@zytor.com" <hpa@zytor.com>,
       "jeremy@xensource.com" <jeremy@xensource.com>,
       "chrisw@sous-sol.org" <chrisw@sous-sol.org>,
       "rusty@rustcorp.com.au" <rusty@rustcorp.com.au>
In-Reply-To: <20090120112634.GA20858@elte.hu>
References: <20090120110542.GE19505@wotan.suse.de>
	 <20090120112634.GA20858@elte.hu>
Content-Type: text/plain
Date: Tue, 20 Jan 2009 11:05:18 -0800
Message-Id: <1232478318.16317.160.camel@bodhitayantram.eng.vmware.com>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1299
Lines: 30

On Tue, 2009-01-20 at 03:26 -0800, Ingo Molnar wrote:

> Jeremy, any ideas where this slowdown comes from and how it could be 
> fixed?

Well I'm early responding to this thread before reading on, but I looked
at the generated assembly for some common mm paths and it looked awful.
The biggest loser was probably having functions to convert pte_t back
and forth to pteval_t, which makes most potential mask / shift
optimizations impossible - indeed, because the compiler doesn't even
understand pte_val(X) = Y is static over the lifetime of the function,
it often calls these same conversions back and forth several times, and
because this is often done inside hidden macros, it's not even possible
to save a cached value in most places.

The bulk of state required to keep this extra conversion around ties up
a lot of registers and as a result heavily limits potential further
optimizations.

The code did not look more branchy to me, however, and gcc seemed to do
a good job with lining up a nice branch structure in the few paths I
looked at.

Zach

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/