Message-ID: <1488504931.7212.41.camel@linux.intel.com>
Subject: Re: [PATCH] ftrace: add ftrace_graph_max_depth kernel parameter
From: Todd Brandt <todd.e.brandt@linux.intel.com>
Reply-To: todd.e.brandt@linux.intel.com
To: Steven Rostedt <rostedt@goodmis.org>
Cc: linux-kernel@vger.kernel.org, todd.e.brandt@intel.com
Date: Thu, 02 Mar 2017 17:35:31 -0800
In-Reply-To: <20170302200950.6d16c91a@gandalf.local.home>
References: <1488499935-23216-1-git-send-email-todd.e.brandt@linux.intel.com>
         <20170302193827.6f5ee2eb@gandalf.local.home>
         <20170302193943.3117fca7@gandalf.local.home>
         <1488502497.7212.24.camel@linux.intel.com>
         <20170302200950.6d16c91a@gandalf.local.home>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2174
Lines: 55

On Thu, 2017-03-02 at 20:09 -0500, Steven Rostedt wrote:
> On Thu, 02 Mar 2017 16:54:57 -0800
> Todd Brandt <todd.e.brandt@linux.intel.com> wrote:
> 
> 
> > BTW I had one other question for you, is it possible to move the ftrace
> > module's initialization further up the queue? ATM it initializes at
> > about 600ms into boot, which is fine since most of the performance
> > issues are happening at 1000ms+. But eventually it might be nice to
> > trace some of the earlier init code.
> 
> module initialization? Or you mean boot up initialization?
oh right, yes. 

> 
> For boot up, we have this:
> 
> 	mm_init();
> 	sched_init();
> 	idr_init_cache();
> 	workqueue_init_early();
> 	rcu_init();
> 	trace_init();
> 
> Now, going before mm_init() will definitely be quite a pain, as we
> that would require making the ring buffer out of early boot memory.
> 
> We could add a early_trace_init() just before sched_init(), with some
> limited tracing. Tracepoints require rcu, although we could make hooks
> to just do hard coded changes (no rcu synchronization) with a flag that
> denotes that trace_init() hasn't been called yet.
> 
> I could toy with the idea, implement it, and see what breaks.
If it was possible that would be great. The reason I ask is because I've
implemented a tool called analyze_boot which I intend to upgrade to use
early callgraph trace (it just uses initcall_debug now):

https://marc.info/?l=linux-pm&m=148667508500967&w=2

I'll have it do the same thing for boot that analyze_suspend does for
suspend/resume: display the callgraphs for each initcall instance in
html. The max_graph_depth param is necessary to keep the html outputs
manageable. Most laptop/desktop machines running standard linux seem to
take at least 5 seconds for boot, but we've got some servers optimized
to boot in less than a second (which means I'd only be able to trace
half of the boot timeline or less). The more coverage I can get the
better.

P.S. I submitted the analyze_boot tool on the pm list but Raphael didn't
think it would fit in the PM tree. Would you be willing to accept this
script? (I can resubmit but nothing's changed since the RESEND).

> 
> -- Steve