2009-06-10 12:44:18

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

On Thu, 2009-05-07 at 09:25 +0000, tip-bot for Markus Metzger wrote:
> Commit-ID: 7e0bfad24d85de7cf2202a7b0ce51de11a077b21
> Gitweb: http://git.kernel.org/tip/7e0bfad24d85de7cf2202a7b0ce51de11a077b21
> Author: Markus Metzger <[email protected]>
> AuthorDate: Fri, 24 Apr 2009 09:44:48 +0200
> Committer: Ingo Molnar <[email protected]>
> CommitDate: Fri, 24 Apr 2009 10:18:51 +0200
>
> x86, bts: reenable ptrace branch trace support
>
> The races found by Oleg Nesterov have been fixed.
>
> Reenable branch trace support.
>
> Signed-off-by: Markus Metzger <[email protected]>
> Acked-by: Oleg Nesterov <[email protected]>
> LKML-Reference: <[email protected]>
> Signed-off-by: Ingo Molnar <[email protected]>

OK, this whole BTS thing worries me because it seems to expose too much
of the Intel Debug Store thingy to the outside world.

The thing is, once we do PEBS from inside the kernel, we'll have to
share the debug store, and the way this thing sets it up isn't really
going to work.

So what I'd like to see is having this debug store removed from the
interface and abstracted away as cpu resource -- a single page when in
use should suffice I think.

If you want to expose a buffer to userspace, use the regular mmap()
interface and fill those pages from the DS interrupt handler.


2009-06-10 12:51:30

by Ingo Molnar

[permalink] [raw]
Subject: Re: [tip:tracing/core] x86, bts: reenable ptrace branch trace support


* Peter Zijlstra <[email protected]> wrote:

> On Thu, 2009-05-07 at 09:25 +0000, tip-bot for Markus Metzger wrote:
> > Commit-ID: 7e0bfad24d85de7cf2202a7b0ce51de11a077b21
> > Gitweb: http://git.kernel.org/tip/7e0bfad24d85de7cf2202a7b0ce51de11a077b21
> > Author: Markus Metzger <[email protected]>
> > AuthorDate: Fri, 24 Apr 2009 09:44:48 +0200
> > Committer: Ingo Molnar <[email protected]>
> > CommitDate: Fri, 24 Apr 2009 10:18:51 +0200
> >
> > x86, bts: reenable ptrace branch trace support
> >
> > The races found by Oleg Nesterov have been fixed.
> >
> > Reenable branch trace support.
> >
> > Signed-off-by: Markus Metzger <[email protected]>
> > Acked-by: Oleg Nesterov <[email protected]>
> > LKML-Reference: <[email protected]>
> > Signed-off-by: Ingo Molnar <[email protected]>
>
> OK, this whole BTS thing worries me because it seems to expose too
> much of the Intel Debug Store thingy to the outside world.
>
> The thing is, once we do PEBS from inside the kernel, we'll have
> to share the debug store, and the way this thing sets it up isn't
> really going to work.
>
> So what I'd like to see is having this debug store removed from
> the interface and abstracted away as cpu resource -- a single page
> when in use should suffice I think.
>
> If you want to expose a buffer to userspace, use the regular
> mmap() interface and fill those pages from the DS interrupt
> handler.

hm, that makes sense indeed. We are in the merge window now -
Markus, what's your suggestion?

Ingo

2009-06-10 13:23:59

by Metzger, Markus T

[permalink] [raw]
Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

>-----Original Message-----
>From: Ingo Molnar [mailto:[email protected]]
>Sent: Wednesday, June 10, 2009 2:51 PM
>To: Peter Zijlstra
>Cc: [email protected]; [email protected]; [email protected]; Metzger, Markus T;
>[email protected]; [email protected]; [email protected]
>Subject: Re: [tip:tracing/core] x86, bts: reenable ptrace branch trace support
>
>
>* Peter Zijlstra <[email protected]> wrote:
>
>> On Thu, 2009-05-07 at 09:25 +0000, tip-bot for Markus Metzger wrote:
>> > Commit-ID: 7e0bfad24d85de7cf2202a7b0ce51de11a077b21
>> > Gitweb: http://git.kernel.org/tip/7e0bfad24d85de7cf2202a7b0ce51de11a077b21
>> > Author: Markus Metzger <[email protected]>
>> > AuthorDate: Fri, 24 Apr 2009 09:44:48 +0200
>> > Committer: Ingo Molnar <[email protected]>
>> > CommitDate: Fri, 24 Apr 2009 10:18:51 +0200
>> >
>> > x86, bts: reenable ptrace branch trace support
>> >
>> > The races found by Oleg Nesterov have been fixed.
>> >
>> > Reenable branch trace support.
>> >
>> > Signed-off-by: Markus Metzger <[email protected]>
>> > Acked-by: Oleg Nesterov <[email protected]>
>> > LKML-Reference: <[email protected]>
>> > Signed-off-by: Ingo Molnar <[email protected]>
>>
>> OK, this whole BTS thing worries me because it seems to expose too
>> much of the Intel Debug Store thingy to the outside world.
>>
>> The thing is, once we do PEBS from inside the kernel, we'll have
>> to share the debug store, and the way this thing sets it up isn't
>> really going to work.
>>
>> So what I'd like to see is having this debug store removed from
>> the interface and abstracted away as cpu resource -- a single page
>> when in use should suffice I think.
>>
>> If you want to expose a buffer to userspace, use the regular
>> mmap() interface and fill those pages from the DS interrupt
>> handler.
>
>hm, that makes sense indeed. We are in the merge window now -
>Markus, what's your suggestion?

The Debug Store interface is completely in-kernel. It does not expose
anything to the outside world.

What we expose is a ptrace interface for branch tracing.
That this is built on top of Debug Store is completely hidden.
The Debug Store interface may be changed without impacting the
user-visible part at any time.

I do think that a ptrace interface makes sense since debuggers are the
targeted users for branch tracing.

I don't see why we should not merge the fixes now and then rework the
in-kernel parts as needed for supporting PEBS.


regards,
markus.

---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

2009-06-10 13:29:30

by Peter Zijlstra

[permalink] [raw]
Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

On Wed, 2009-06-10 at 14:22 +0100, Metzger, Markus T wrote:
>
> The Debug Store interface is completely in-kernel. It does not expose
> anything to the outside world.
>
> What we expose is a ptrace interface for branch tracing.
> That this is built on top of Debug Store is completely hidden.
> The Debug Store interface may be changed without impacting the
> user-visible part at any time.
>
> I do think that a ptrace interface makes sense since debuggers are the
> targeted users for branch tracing.
>
> I don't see why we should not merge the fixes now and then rework the
> in-kernel parts as needed for supporting PEBS.

Ok, so what is all that account_locked_memory() for?

2009-06-10 13:34:59

by Metzger, Markus T

[permalink] [raw]
Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

>-----Original Message-----
>From: Peter Zijlstra [mailto:[email protected]]
>Sent: Wednesday, June 10, 2009 3:29 PM
>To: Metzger, Markus T
>Cc: Ingo Molnar; [email protected]; [email protected]; [email protected]; [email protected];
>[email protected]; [email protected]
>Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support
>
>On Wed, 2009-06-10 at 14:22 +0100, Metzger, Markus T wrote:
>>
>> The Debug Store interface is completely in-kernel. It does not expose
>> anything to the outside world.
>>
>> What we expose is a ptrace interface for branch tracing.
>> That this is built on top of Debug Store is completely hidden.
>> The Debug Store interface may be changed without impacting the
>> user-visible part at any time.
>>
>> I do think that a ptrace interface makes sense since debuggers are the
>> targeted users for branch tracing.
>>
>> I don't see why we should not merge the fixes now and then rework the
>> in-kernel parts as needed for supporting PEBS.
>
>Ok, so what is all that account_locked_memory() for?

To allow debuggers (users) to decide how much memory they want to spend
on branch tracing.

regards,
markus.

---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

2009-06-10 13:43:44

by Peter Zijlstra

[permalink] [raw]
Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

On Wed, 2009-06-10 at 14:32 +0100, Metzger, Markus T wrote:
> >-----Original Message-----
> >From: Peter Zijlstra [mailto:[email protected]]
> >Sent: Wednesday, June 10, 2009 3:29 PM
> >To: Metzger, Markus T
> >Cc: Ingo Molnar; [email protected]; [email protected]; [email protected]; [email protected];
> >[email protected]; [email protected]
> >Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support
> >
> >On Wed, 2009-06-10 at 14:22 +0100, Metzger, Markus T wrote:
> >>
> >> The Debug Store interface is completely in-kernel. It does not expose
> >> anything to the outside world.
> >>
> >> What we expose is a ptrace interface for branch tracing.
> >> That this is built on top of Debug Store is completely hidden.
> >> The Debug Store interface may be changed without impacting the
> >> user-visible part at any time.
> >>
> >> I do think that a ptrace interface makes sense since debuggers are the
> >> targeted users for branch tracing.
> >>
> >> I don't see why we should not merge the fixes now and then rework the
> >> in-kernel parts as needed for supporting PEBS.
> >
> >Ok, so what is all that account_locked_memory() for?
>
> To allow debuggers (users) to decide how much memory they want to spend
> on branch tracing.

But that is the debug store, right? User visible through some mlock
accounting and limit.

Furthermore, it appears there is an interface for setting the size,
that's also user visible and not fixable after the fact.

Once you want to multiplex cpu-wide and per task BTS/PEBS contexts,
there is no choice but to view the DS as a cpu resource, not a task
resource, therefore you cannot specify a size, nor attribute it to
specific tasks mm accounting.

2009-06-10 14:07:30

by Metzger, Markus T

[permalink] [raw]
Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support

>-----Original Message-----
>From: Peter Zijlstra [mailto:[email protected]]
>Sent: Wednesday, June 10, 2009 3:44 PM
>To: Metzger, Markus T
>Cc: Ingo Molnar; [email protected]; [email protected]; [email protected]; [email protected];
>[email protected]; [email protected]
>Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support
>
>On Wed, 2009-06-10 at 14:32 +0100, Metzger, Markus T wrote:
>> >-----Original Message-----
>> >From: Peter Zijlstra [mailto:[email protected]]
>> >Sent: Wednesday, June 10, 2009 3:29 PM
>> >To: Metzger, Markus T
>> >Cc: Ingo Molnar; [email protected]; [email protected]; [email protected];
>[email protected];
>> >[email protected]; [email protected]
>> >Subject: RE: [tip:tracing/core] x86, bts: reenable ptrace branch trace support
>> >
>> >On Wed, 2009-06-10 at 14:22 +0100, Metzger, Markus T wrote:
>> >>
>> >> The Debug Store interface is completely in-kernel. It does not expose
>> >> anything to the outside world.
>> >>
>> >> What we expose is a ptrace interface for branch tracing.
>> >> That this is built on top of Debug Store is completely hidden.
>> >> The Debug Store interface may be changed without impacting the
>> >> user-visible part at any time.
>> >>
>> >> I do think that a ptrace interface makes sense since debuggers are the
>> >> targeted users for branch tracing.
>> >>
>> >> I don't see why we should not merge the fixes now and then rework the
>> >> in-kernel parts as needed for supporting PEBS.
>> >
>> >Ok, so what is all that account_locked_memory() for?
>>
>> To allow debuggers (users) to decide how much memory they want to spend
>> on branch tracing.
>
>But that is the debug store, right? User visible through some mlock
>accounting and limit.

Debug Store is the underlying implementation, that's all.

In the current implementation, we lock the entire buffer. In a future
implementation we might not need to lock the entire buffer, any more.
The only user-visible effect is that he will get -ENOMEM for bigger
buffers than today.


>Furthermore, it appears there is an interface for setting the size,
>that's also user visible and not fixable after the fact.

I don't see how that differs from setting the initial size.


>Once you want to multiplex cpu-wide and per task BTS/PEBS contexts,
>there is no choice but to view the DS as a cpu resource, not a task
>resource, therefore you cannot specify a size, nor attribute it to
>specific tasks mm accounting.

We can still attach a buffer to a task to hold the branch or pebs trace.
When can copy the trace from the limited cpu resource to that per-task buffer
when the Debug Store buffer overflows.

The accounting is done to have the tracer, i.e. the debugger, pay for the
system resources it uses.
If we later on decide to use a fixed-size statically allocated per-cpu
Debug Store buffer, the tracer would no longer need to pay for the
locked memory. It would still have to pay for the memory to hold the bigger
tracer buffer that the debugger requested.


regards,
markus.

---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.