I ran into a few strange issues today using a new perf tool (3.0-rcX)
against a 2.6.38 based kernel. Basic things seem to work:
# perf record -aR -- sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.443 MB perf.data (~19334 samples) ]
But:
# perf record -agR -- sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.558 MB perf.data (~68077 samples) ]
Can't find id 9's machine
Found 1 unknown events!
Is this an older tool processing a perf.data file generated by a more
recent tool?
If that is not the case, consider reporting to [email protected].
I couldn't tell if this is kernel/user space incompatibility or tighter
error checking in the new perf tool. Before I start digging further,
thought I'd ask if this combination is supported. The error message
above doesn't ask me:
Are you using an old kernel?
-Arun
On 06/22/2011 03:39 PM, Arun Sharma wrote:
>
> I ran into a few strange issues today using a new perf tool (3.0-rcX)
> against a 2.6.38 based kernel. Basic things seem to work:
>
> # perf record -aR -- sleep 1
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.443 MB perf.data (~19334 samples) ]
>
> But:
>
> # perf record -agR -- sleep 1
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 1.558 MB perf.data (~68077 samples) ]
> Can't find id 9's machine
> Found 1 unknown events!
>
> Is this an older tool processing a perf.data file generated by a more
> recent tool?
>
> If that is not the case, consider reporting to
> [email protected].
>
> I couldn't tell if this is kernel/user space incompatibility or tighter
> error checking in the new perf tool. Before I start digging further,
> thought I'd ask if this combination is supported. The error message
> above doesn't ask me:
>
> Are you using an old kernel?
I have not seen issues like this using newer perf userspace against
older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
testing patches).
David
>
> -Arun
>
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-perf-users" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On 6/23/11 7:22 AM, David Ahern wrote:
> I have not seen issues like this using newer perf userspace against
> older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
> now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
> testing patches).
I narrowed it down to PERF_SAMPLE_RAW:
perf record -ag -- sleep 1
is fine, but:
perf record -agR -- sleep 1
fails for me most of the time. The reason I needed to use the -R in the
first place is that "perf script" fails on older kernels with:
Samples do not contain timestamps.
With the newer perf, I don't get errors, but the timestamp field is
invalid. So I need to use the -R flag to get valid timestamps +
stacktraces out of "perf script".
-Arun
On 06/23/2011 01:39 PM, Arun Sharma wrote:
> On 6/23/11 7:22 AM, David Ahern wrote:
>
>> I have not seen issues like this using newer perf userspace against
>> older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
>> now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
>> testing patches).
>
> I narrowed it down to PERF_SAMPLE_RAW:
>
> perf record -ag -- sleep 1
>
> is fine, but:
>
> perf record -agR -- sleep 1
>
> fails for me most of the time. The reason I needed to use the -R in the
> first place is that "perf script" fails on older kernels with:
>
> Samples do not contain timestamps.
>
> With the newer perf, I don't get errors, but the timestamp field is
> invalid. So I need to use the -R flag to get valid timestamps +
> stacktraces out of "perf script".
That should have been fixed.
And -T on record gets the timestamps.
David
>
> -Arun
On Thu, Jun 23, 2011 at 12:39:54PM -0700, Arun Sharma wrote:
> On 6/23/11 7:22 AM, David Ahern wrote:
>
> >I have not seen issues like this using newer perf userspace against
> >older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
> >now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
> >testing patches).
>
> I narrowed it down to PERF_SAMPLE_RAW:
>
> perf record -ag -- sleep 1
>
> is fine, but:
>
> perf record -agR -- sleep 1
>
> fails for me most of the time. The reason I needed to use the -R in
> the first place is that "perf script" fails on older kernels with:
>
> Samples do not contain timestamps.
>
> With the newer perf, I don't get errors, but the timestamp field is
> invalid. So I need to use the -R flag to get valid timestamps +
> stacktraces out of "perf script".
I'm confused, you first said it happens with new tools on older kernel.
Can you tell us which combination of kernel/user raises the error? and
which error.
Thanks!
On Thu, Jun 23, 2011 at 02:02:15PM -0600, David Ahern wrote:
>
>
> On 06/23/2011 01:39 PM, Arun Sharma wrote:
> > On 6/23/11 7:22 AM, David Ahern wrote:
> >
> >> I have not seen issues like this using newer perf userspace against
> >> older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
> >> now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
> >> testing patches).
> >
> > I narrowed it down to PERF_SAMPLE_RAW:
> >
> > perf record -ag -- sleep 1
> >
> > is fine, but:
> >
> > perf record -agR -- sleep 1
> >
> > fails for me most of the time. The reason I needed to use the -R in the
> > first place is that "perf script" fails on older kernels with:
> >
> > Samples do not contain timestamps.
> >
> > With the newer perf, I don't get errors, but the timestamp field is
> > invalid. So I need to use the -R flag to get valid timestamps +
> > stacktraces out of "perf script".
>
> That should have been fixed.
>
> And -T on record gets the timestamps.
>
> David
Right, it would be nice to suggest that from perf script when timestamps
are not recorded.
On 6/23/11 5:11 PM, Frederic Weisbecker wrote:
>
> I'm confused, you first said it happens with new tools on older kernel.
>
> Can you tell us which combination of kernel/user raises the error? and
> which error.
>
perf-3.0 + 2.6.38
perf record -agR gives:
Can't find id 9's machine
Found 1 unknown events!
perf-3.0 + 2.6.32
perf record -ag
perf script
samples have bogus timestamps
perf-3.0 + 2.6.32
perf record -agR
perf script
error: Samples do not contain timestamps
perf-3.0 + any kernel
perf record -agT
perf script
is happy everywhere. Thanks!
-Arun
On 06/23/2011 06:43 PM, Arun Sharma wrote:
> On 6/23/11 5:11 PM, Frederic Weisbecker wrote:
>
>>
>> I'm confused, you first said it happens with new tools on older kernel.
>>
>> Can you tell us which combination of kernel/user raises the error? and
>> which error.
>>
I tried to repeat your examples below using perf-core tip (3.0-rc3,
af07ce3e77).
>
> perf-3.0 + 2.6.38
>
> perf record -agR gives:
>
> Can't find id 9's machine
> Found 1 unknown events!
This does not reproduce on 2.6.38.8-32.fc15.x86_64 running bare metal or
2.6.38 in a VM. If you are running bare metal try -e cpu-clock and see
if it is cycles related. I doubt it, but that should be the only
difference in the tests.
That said, I did see that message while working on the gettimeofday
patches a few weeks ago. I believe the root cause was the early mixture
of cycles and tracepoints -- something I fixed before sending out the
patches (sample_type hack as well as how the time-of-day trace events
were added to the event list).
>
> perf-3.0 + 2.6.32
>
> perf record -ag
> perf script
>
> samples have bogus timestamps
I'm surprised by this one. I tried with an older Fedora 12 VM (2.6.32.21
kernel). I don't get timestamps with perf-script and perf-report -D
shows -1 which is what I expect given that SAMPLE_TIME attribute is not
set by default. One difference here is that VM's default to cpu-clock
for the event.
>
> perf-3.0 + 2.6.32
>
> perf record -agR
> perf script
>
> error: Samples do not contain timestamps
And this one did not work out so well with the F-12 VM. It caused a
kernel panic - top line on console is perf_swevent_hrtimer (lines
scrolled off the top).
>
> perf-3.0 + any kernel
>
> perf record -agT
> perf script
>
> is happy everywhere. Thanks!
Which is what I would expect. Glad to see my version of reality apply
elsewhere. ;-)
David
>
> -Arun
>
On 06/23/2011 06:11 PM, Frederic Weisbecker wrote:
> On Thu, Jun 23, 2011 at 02:02:15PM -0600, David Ahern wrote:
>>
>>
>> On 06/23/2011 01:39 PM, Arun Sharma wrote:
>>> On 6/23/11 7:22 AM, David Ahern wrote:
>>>
>>>> I have not seen issues like this using newer perf userspace against
>>>> older kernels. For example, my laptop was running Fedora 14 (2.6.35) and
>>>> now Fedora 15 (2.6.38.8) and I typically use latest perf builds (e.g.,
>>>> testing patches).
>>>
>>> I narrowed it down to PERF_SAMPLE_RAW:
>>>
>>> perf record -ag -- sleep 1
>>>
>>> is fine, but:
>>>
>>> perf record -agR -- sleep 1
>>>
>>> fails for me most of the time. The reason I needed to use the -R in the
>>> first place is that "perf script" fails on older kernels with:
>>>
>>> Samples do not contain timestamps.
>>>
>>> With the newer perf, I don't get errors, but the timestamp field is
>>> invalid. So I need to use the -R flag to get valid timestamps +
>>> stacktraces out of "perf script".
>>
>> That should have been fixed.
>>
>> And -T on record gets the timestamps.
>>
>> David
>
> Right, it would be nice to suggest that from perf script when timestamps
> are not recorded.
Timestamps are enabled by default, but that output option is removed if
the samples do not have timestamps. That message is generated if the
user requests timestamps (-f time) in the perf-script output and the
samples do not have timestamps, but Arun did not request that. Arun's
mileage with perf-3.0 definitely varies from what I've seen.
David