Date: Fri, 20 Nov 2015 10:36:58 -0800 (PST)
From: Brian Robbins <brianrob@microsoft.com>
To: Stephane Eranian <eranian@google.com>
cc: Brian Robbins <brianrob@microsoft.com>,
        Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF
 binaries.
In-Reply-To: <CABPqkBQeKJpUf+Y_00ghRb7yZAq+ccqbCKt4C1pHzaLz+VAgAQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.10.1511201032170.1537@BRIANROB-ZB-1>
References: <1447960147-2681-1-git-send-email-brianrob@microsoft.com> <20151119193101.GQ3816@twins.programming.kicks-ass.net> <SN1PR0301MB15982C2861CC002B4EAB1A47C81B0@SN1PR0301MB1598.namprd03.prod.outlook.com>
 <CABPqkBQeKJpUf+Y_00ghRb7yZAq+ccqbCKt4C1pHzaLz+VAgAQ@mail.gmail.com>
User-Agent: Alpine 2.10 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Nov 2015 18:37:20.1652
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0301MB1586
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3558
Lines: 77


On Fri, 20 Nov 2015, Stephane Eranian wrote:

> On Thu, Nov 19, 2015 at 3:45 PM, Brian Robbins <brianrob@microsoft.com> wrote:
> >
> > From: Peter Zijlstra <peterz@infradead.org>
> > Sent: Thursday, November 19, 2015 11:31 AM
> > To: Brian Robbins
> > Cc: Ingo Molnar; Arnaldo Carvalho de Melo; linux-kernel@vger.kernel.org; Stephane Eranian
> > Subject: Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.
> >
> > > On Thu, Nov 19, 2015 at 11:09:07AM -0800, Brian Robbins wrote:
> > > > Programs can execute code contained in files that don't conform
> > > > to the ELF standard.  This is common for cross-platform runtimes
> > > > that support ahead-of-time compilation.  In this case, perf is
> > > > unable to resolve addresses to symbolic names, as the format of
> > > > the mmap'd file is unknown to perf.
> >
> > > I'm assuming you're using PECOFF? Why not teach perf to read those?
> >
> > > Surely there's a library that can parse that stuff, it seems to me Wine
> > > for one needs fairly decent support for this, also GDB (and other
> > > toolchain bits) on Windows targets.
> >
> > > Its just that I was hoping we could move towards getting rid of those
> > > /tmp files now that we have much better JIT support.
> >
> > Hi Peter,
> >
> > Thank you for the feedback.  The file format is similar to PE, but is not identical.  So, we would be implementing something very scoped, which doesn't feel right to me.
> >
> > I am interested in the new JIT support, however my understanding from the information that I've read is that it requires kernel support in 4.x, though I can't seem to find where I read that.  I want to make sure that this works on older kernels (3.x) as well.
> >
> You need 4.1 if you have a situation where your JIT may recompile code
> multiple times. The key difference is that
> with 4.1 you can use the same timestamp clock source for both the JIT
> runtime and the kernel perf_event subsystem.
> That makes it possible to correlate samples from the JIT with samples
> from the hardware. Without that, you may not
> get correct symbolization if the code is re-jitted or moved.
> 

Thanks Stephane.  That's good to know.

Unfortunately, because the scenario here is not actually a JIT, it 
requires a bit more control around when the data is generated.  
Specifically, the requirement is that the data be generated offline as 
opposed to when the process is running.  Rather than using virtual 
addresses, we use offsets from the beginning of the file, and then update 
the data adding in the base address of the loaded file to get the full 
virtual address after the data has been collected.

Thus, the scenario is a bit different than a JIT scenario, but can re-use 
some of the plumbing.  Knowing this information, does this scenario sound 
OK?

I'd be happy to look at seeing what it would take to make this a bit 
cleaner (and deprecate the tmp file), but am keen to enable the scenario 
sooner rather than later.

Thanks again for your consideration.

-Brian


> >
> > The reason I went with this approach is because it is simple for runtimes to implement and has no requirement that perf understand the file format.  I am open to feedback if there is a preferred solution that would still work for older kernels as well.
> >
> > Thanks.
> > -Brian
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/