Message-ID: <4B217803.6080407@am.sony.com>
Date: Thu, 10 Dec 2009 14:36:51 -0800
From: Tim Bird <tim.bird@am.sony.com>
User-Agent: Thunderbird 2.0.0.14 (X11/20080501)
MIME-Version: 1.0
To: "rostedt@goodmis.org" <rostedt@goodmis.org>
CC: "Bird, Tim" <Timothy.Bird@am.sony.com>, Ingo Molnar <mingo@elte.hu>,
       Andrew Morton <akpm@linux-foundation.org>,
       Peter Zijlstra <a.p.zijlstra@chello.nl>,
       Arnaldo Carvalho de Melo <acme@redhat.com>,
       Li Zefan <lizf@cn.fujitsu.com>, Thomas Gleixner <tglx@linutronix.de>,
       linux kernel <linux-kernel@vger.kernel.org>,
       Frederic Weisbecker <fweisbec@gmail.com>
Subject: Re: [PATCH 2/4] ftrace - add function_duration tracer
References: <4B202778.4030801@am.sony.com> <20091210070800.GB16874@elte.hu>	 <4B216478.3080601@am.sony.com> <1260482698.2146.234.camel@gandalf.stny.rr.com>
In-Reply-To: <1260482698.2146.234.camel@gandalf.stny.rr.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1700
Lines: 41

Steven Rostedt wrote:
>> I'm a little worried about this.  ftrace is already an order
>> of magnitude more overhead than the previous tracer I was using.
> 
> Which tracer was that? I know you used the -finstrument-functions for
> tracing before, and that has quite a bit overhead as well. The ring
> buffer has gone through a lot to speed it up.

KFT.  It used -finstrument-functions, which did result in
a few more instructions per call site than mcount does.  However,
the code paths in KFT are much shorter than in ftrace.
And the fact that both function exit and entry are instrumented
meant that I didn't have to do funky trampoline junk, or
manage a separate structure, in order to capture function
exit events.

It is somewhat to be expected that ftrace would have higher
overhead, because ftrace does more. But in the version
of code I was using (which is probably quite old), there
was lots of extraneous code in the hot path.
For example, there are at least 4 checks in the code
to detect whether tracing is active or not.

The ring buffer code was particularly slow.  I was using
a simple fixed-size array in KFT (on a uniprocessor), and
the time to save an event was under 1 microsecond.  With
ftrace on the same hardware, it costs about 6 microseconds
to save an event.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/