Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
with HEAD as:
1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
It maybe related to this commit:
commit d596299
Author: John Garry <[email protected]>
Date: Thu Mar 8 18:58:29 2018 +0800
perf vendor events: Add support for pmu events vendor subdirectory
Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
problem for me.
@@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
/* model directory, reset topic */
+#if 0
if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
(level == 2 && is_dir)) {
+#else
+ if (level == 1 && is_dir) {
+#endif
The problem is that the tools/perf/pmu-events/pmu-events.c file generated by
the jevents program is missing the "prefix" lines like:
struct pmu_event pme_power8[] = {
and
struct pmu_event pme_power9[] = {
i.e the pmu-events.c begins like:
---
#include "../../pmu-events/pmu-events.h"
{
.name = "pm_data_from_dl2l3_mod",
.event = "event=0x4c048",
.desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
.topic = "cache",
.long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
},
---
Below is the debug and info messages at the time of failure:
---
HOSTCC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
HOSTLD /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
LINK /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
GEN /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
d 0 50 powerpc pmu-events/arch/powerpc
d 1 4096 power8 pmu-events/arch/powerpc/power8
f 2 10275 cache.json pmu-events/arch/powerpc/power8/cache.json
f 2 302 floating-point.json pmu-events/arch/powerpc/power8/floating-point.json
f 2 29283 marked.json pmu-events/arch/powerpc/power8/marked.json
f 2 12028 memory.json pmu-events/arch/powerpc/power8/memory.json
f 2 159271 other.json pmu-events/arch/powerpc/power8/other.json
f 2 10777 pipeline.json pmu-events/arch/powerpc/power8/pipeline.json
f 2 3781 pmc.json pmu-events/arch/powerpc/power8/pmc.json
f 2 6664 translation.json pmu-events/arch/powerpc/power8/translation.json
f 2 25104 frontend.json pmu-events/arch/powerpc/power8/frontend.json
d 1 4096 power9 pmu-events/arch/powerpc/power9
f 2 1096 floating-point.json pmu-events/arch/powerpc/power9/floating-point.json
f 2 5503 cache.json pmu-events/arch/powerpc/power9/cache.json
f 2 14885 frontend.json pmu-events/arch/powerpc/power9/frontend.json
f 2 27807 marked.json pmu-events/arch/powerpc/power9/marked.json
f 2 4820 memory.json pmu-events/arch/powerpc/power9/memory.json
f 2 85387 other.json pmu-events/arch/powerpc/power9/other.json
f 2 23839 pipeline.json pmu-events/arch/powerpc/power9/pipeline.json
f 2 5138 pmc.json pmu-events/arch/powerpc/power9/pmc.json
f 2 9897 translation.json pmu-events/arch/powerpc/power9/translation.json
f 1 440 mapfile.csv pmu-events/arch/powerpc/mapfile.csv
jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
{
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
},
----
On 13/03/2018 19:17, Sukadev Bhattiprolu wrote:
>
>
> Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
> with HEAD as:
>
> 1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
>
> It maybe related to this commit:
>
> commit d596299
> Author: John Garry <[email protected]>
> Date: Thu Mar 8 18:58:29 2018 +0800
>
> perf vendor events: Add support for pmu events vendor subdirectory
>
> Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
> problem for me.
>
> @@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
>
>
> /* model directory, reset topic */
> +#if 0
> if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
> (level == 2 && is_dir)) {
> +#else
> + if (level == 1 && is_dir) {
> +#endif
>
>
> The problem is that the tools/perf/pmu-events/pmu-events.c file generated by
> the jevents program is missing the "prefix" lines like:
>
> struct pmu_event pme_power8[] = {
> and
> struct pmu_event pme_power9[] = {
>
>
> i.e the pmu-events.c begins like:
> ---
>
> #include "../../pmu-events/pmu-events.h"
> {
> .name = "pm_data_from_dl2l3_mod",
> .event = "event=0x4c048",
> .desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
> .topic = "cache",
> .long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
> },
>
> ---
>
I will check this, thanks and sorry for the hassle.
John
> Below is the debug and info messages at the time of failure:
>
> ---
>
> HOSTCC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
> HOSTLD /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
> LINK /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
> GEN /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
> d 0 50 powerpc pmu-events/arch/powerpc
> d 1 4096 power8 pmu-events/arch/powerpc/power8
> f 2 10275 cache.json pmu-events/arch/powerpc/power8/cache.json
> f 2 302 floating-point.json pmu-events/arch/powerpc/power8/floating-point.json
> f 2 29283 marked.json pmu-events/arch/powerpc/power8/marked.json
> f 2 12028 memory.json pmu-events/arch/powerpc/power8/memory.json
> f 2 159271 other.json pmu-events/arch/powerpc/power8/other.json
> f 2 10777 pipeline.json pmu-events/arch/powerpc/power8/pipeline.json
> f 2 3781 pmc.json pmu-events/arch/powerpc/power8/pmc.json
> f 2 6664 translation.json pmu-events/arch/powerpc/power8/translation.json
> f 2 25104 frontend.json pmu-events/arch/powerpc/power8/frontend.json
> d 1 4096 power9 pmu-events/arch/powerpc/power9
> f 2 1096 floating-point.json pmu-events/arch/powerpc/power9/floating-point.json
> f 2 5503 cache.json pmu-events/arch/powerpc/power9/cache.json
> f 2 14885 frontend.json pmu-events/arch/powerpc/power9/frontend.json
> f 2 27807 marked.json pmu-events/arch/powerpc/power9/marked.json
> f 2 4820 memory.json pmu-events/arch/powerpc/power9/memory.json
> f 2 85387 other.json pmu-events/arch/powerpc/power9/other.json
> f 2 23839 pipeline.json pmu-events/arch/powerpc/power9/pipeline.json
> f 2 5138 pmc.json pmu-events/arch/powerpc/power9/pmc.json
> f 2 9897 translation.json pmu-events/arch/powerpc/power9/translation.json
> f 1 440 mapfile.csv pmu-events/arch/powerpc/mapfile.csv
> jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
> CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
> {
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
> },
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
> },
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
> },
>
> ----
>
>
> .
>
John Garry [[email protected]] wrote:
> On 13/03/2018 19:17, Sukadev Bhattiprolu wrote:
> >
> >
> > Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
> > with HEAD as:
> >
> > 1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
> >
> > It maybe related to this commit:
> >
> > commit d596299
> > Author: John Garry <[email protected]>
> > Date: Thu Mar 8 18:58:29 2018 +0800
> >
> > perf vendor events: Add support for pmu events vendor subdirectory
> >
> > Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
> > problem for me.
> >
Hi John,
I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
may not be supported by all file systems.
Not relying on ->d_type seems to fix it:
@@ -873,26 +879,26 @@ static int is_leaf_dir(const char *fpath)
return 0;
while ((dir = readdir(d)) != NULL) {
- if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
- res = 0;
- break;
- } else if (dir->d_type == DT_UNKNOWN) {
- char path[PATH_MAX];
- struct stat st;
+ char path[PATH_MAX];
+ struct stat st;
- sprintf(path, "%s/%s", fpath, dir->d_name);
- if (stat(path, &st))
- break;
+ if (strcmp(dir->d_name, ".") == 0 ||
+ strcmp(dir->d_name, "..") == 0)
+ continue;
- if (S_ISDIR(st.st_mode)) {
- res = 0;
- break;
- }
+ sprintf(path, "%s/%s", fpath, dir->d_name);
+ if (stat(path, &st))
+ break;
+
+ if (S_ISDIR(st.st_mode)) {
+ res = 0;
+ break;
On 13/03/2018 20:10, Sukadev Bhattiprolu wrote:
+
> John Garry [[email protected]] wrote:
>> On 13/03/2018 19:17, Sukadev Bhattiprolu wrote:
>>>
>>>
>>> Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
>>> with HEAD as:
>>>
>>> 1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
>>>
>>> It maybe related to this commit:
>>>
>>> commit d596299
>>> Author: John Garry <[email protected]>
>>> Date: Thu Mar 8 18:58:29 2018 +0800
>>>
>>> perf vendor events: Add support for pmu events vendor subdirectory
>>>
>>> Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
>>> problem for me.
>>>
>
> Hi John,
>
> I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
> entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
> may not be supported by all file systems.
>
> Not relying on ->d_type seems to fix it:
>
Hi Sukadev,
Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did
add the check for d_type == DT_UNKNOWN.
But, if all files have d_type == DT_UNKNOWN, you're code would from
visual observation look to be same as mine (apart from check for '.' or
'..' filename, which I would say is already covered by stat() and
S_ISDIR()). Or is d_type value just unreliable?
Much appreciated,
John
> @@ -873,26 +879,26 @@ static int is_leaf_dir(const char *fpath)
> return 0;
>
> while ((dir = readdir(d)) != NULL) {
> - if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
> - res = 0;
> - break;
> - } else if (dir->d_type == DT_UNKNOWN) {
> - char path[PATH_MAX];
> - struct stat st;
> + char path[PATH_MAX];
> + struct stat st;
>
> - sprintf(path, "%s/%s", fpath, dir->d_name);
> - if (stat(path, &st))
> - break;
> + if (strcmp(dir->d_name, ".") == 0 ||
> + strcmp(dir->d_name, "..") == 0)
> + continue;
>
> - if (S_ISDIR(st.st_mode)) {
> - res = 0;
> - break;
> - }
> + sprintf(path, "%s/%s", fpath, dir->d_name);
> + if (stat(path, &st))
> + break;
> +
> + if (S_ISDIR(st.st_mode)) {
> + res = 0;
> + break;
>
>
>
John Garry [[email protected]] wrote:
> On 13/03/2018 20:10, Sukadev Bhattiprolu wrote:
>
> > Hi John,
> >
> > I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
> > entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
> > may not be supported by all file systems.
> >
> > Not relying on ->d_type seems to fix it:
> >
>
> Hi Sukadev,
>
> Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add
> the check for d_type == DT_UNKNOWN.
>
> But, if all files have d_type == DT_UNKNOWN, you're code would from visual
> observation look to be same as mine (apart from check for '.' or '..'
> filename, which I would say is already covered by stat() and S_ISDIR()). Or
> is d_type value just unreliable?
In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when
it sees the "." or ".." entries right? In the new code, we skip those and
return false only if we find some other directory.
Thanks,
Sukadev
Em Tue, Mar 13, 2018 at 12:17:42PM -0700, Sukadev Bhattiprolu escreveu:
>
>
> Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
> with HEAD as:
>
> 1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
>
> It maybe related to this commit:
Humpf, this should've been caught by one of the powerpc crossbuild
containers I have in place... will check why the jevents.c code is not
being stressed...
- Arnaldo
> commit d596299
> Author: John Garry <[email protected]>
> Date: Thu Mar 8 18:58:29 2018 +0800
>
> perf vendor events: Add support for pmu events vendor subdirectory
>
> Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
> problem for me.
>
> @@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
>
>
> /* model directory, reset topic */
> +#if 0
> if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
> (level == 2 && is_dir)) {
> +#else
> + if (level == 1 && is_dir) {
> +#endif
>
>
> The problem is that the tools/perf/pmu-events/pmu-events.c file generated by
> the jevents program is missing the "prefix" lines like:
>
> struct pmu_event pme_power8[] = {
> and
> struct pmu_event pme_power9[] = {
>
>
> i.e the pmu-events.c begins like:
> ---
>
> #include "../../pmu-events/pmu-events.h"
> {
> .name = "pm_data_from_dl2l3_mod",
> .event = "event=0x4c048",
> .desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
> .topic = "cache",
> .long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
> },
>
> ---
>
> Below is the debug and info messages at the time of failure:
>
> ---
>
> HOSTCC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
> HOSTLD /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
> LINK /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
> GEN /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
> d 0 50 powerpc pmu-events/arch/powerpc
> d 1 4096 power8 pmu-events/arch/powerpc/power8
> f 2 10275 cache.json pmu-events/arch/powerpc/power8/cache.json
> f 2 302 floating-point.json pmu-events/arch/powerpc/power8/floating-point.json
> f 2 29283 marked.json pmu-events/arch/powerpc/power8/marked.json
> f 2 12028 memory.json pmu-events/arch/powerpc/power8/memory.json
> f 2 159271 other.json pmu-events/arch/powerpc/power8/other.json
> f 2 10777 pipeline.json pmu-events/arch/powerpc/power8/pipeline.json
> f 2 3781 pmc.json pmu-events/arch/powerpc/power8/pmc.json
> f 2 6664 translation.json pmu-events/arch/powerpc/power8/translation.json
> f 2 25104 frontend.json pmu-events/arch/powerpc/power8/frontend.json
> d 1 4096 power9 pmu-events/arch/powerpc/power9
> f 2 1096 floating-point.json pmu-events/arch/powerpc/power9/floating-point.json
> f 2 5503 cache.json pmu-events/arch/powerpc/power9/cache.json
> f 2 14885 frontend.json pmu-events/arch/powerpc/power9/frontend.json
> f 2 27807 marked.json pmu-events/arch/powerpc/power9/marked.json
> f 2 4820 memory.json pmu-events/arch/powerpc/power9/memory.json
> f 2 85387 other.json pmu-events/arch/powerpc/power9/other.json
> f 2 23839 pipeline.json pmu-events/arch/powerpc/power9/pipeline.json
> f 2 5138 pmc.json pmu-events/arch/powerpc/power9/pmc.json
> f 2 9897 translation.json pmu-events/arch/powerpc/power9/translation.json
> f 1 440 mapfile.csv pmu-events/arch/powerpc/mapfile.csv
> jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
> CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
> {
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
> },
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
> },
> ^
> /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
> },
>
> ----
On 13/03/2018 21:18, Sukadev Bhattiprolu wrote:
> John Garry [[email protected]] wrote:
>> On 13/03/2018 20:10, Sukadev Bhattiprolu wrote:
>>
>>> Hi John,
>>>
>>> I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
>>> entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
>>> may not be supported by all file systems.
>>>
>>> Not relying on ->d_type seems to fix it:
>>>
>>
>> Hi Sukadev,
>>
>> Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add
>> the check for d_type == DT_UNKNOWN.
>>
>> But, if all files have d_type == DT_UNKNOWN, you're code would from visual
>> observation look to be same as mine (apart from check for '.' or '..'
>> filename, which I would say is already covered by stat() and S_ISDIR()). Or
>> is d_type value just unreliable?
>
> In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when
> it sees the "." or ".." entries right? In the new code, we skip those and
> return false only if we find some other directory.
>
OK, that's the real issue, being the "." and ".." entries.
@Arnaldo, I'll try to send a fix for this today. I just need to make an
xfs to test. Sorry for the hassle.
John
> Thanks,
>
> Sukadev
>
>
>
On 14/03/2018 07:10, John Garry wrote:
> On 13/03/2018 21:18, Sukadev Bhattiprolu wrote:
>> John Garry [[email protected]] wrote:
>>> On 13/03/2018 20:10, Sukadev Bhattiprolu wrote:
>>>
>>>> Hi John,
>>>>
>>>> I have an xfs file system which seems to have d_type == DT_UNKNOWN
>>>> for all
>>>> entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says
>>>> ->d_type
>>>> may not be supported by all file systems.
>>>>
>>>> Not relying on ->d_type seems to fix it:
>>>>
>>>
>>> Hi Sukadev,
>>>
>>> Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I
>>> did add
>>> the check for d_type == DT_UNKNOWN.
>>>
>>> But, if all files have d_type == DT_UNKNOWN, you're code would from
>>> visual
>>> observation look to be same as mine (apart from check for '.' or '..'
>>> filename, which I would say is already covered by stat() and
>>> S_ISDIR()). Or
>>> is d_type value just unreliable?
>>
>> In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when
>> it sees the "." or ".." entries right? In the new code, we skip those and
>> return false only if we find some other directory.
>>
>
> OK, that's the real issue, being the "." and ".." entries.
>
> @Arnaldo, I'll try to send a fix for this today. I just need to make an
> xfs to test. Sorry for the hassle.
>
Hi Sukadev,
Even with the xfs I have created I still don't see DT_UNKNOWN. So can
you kindly test this change for me:
diff --git a/tools/perf/pmu-events/jevents.c
b/tools/perf/pmu-events/jevents.c
index 1c018445e757..9f712dafa0b2 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -873,7 +873,10 @@ static int is_leaf_dir(const char *fpath)
return 0;
while ((dir = readdir(d)) != NULL) {
- if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
+ if (!strcmp(dir->d_name, ".") || !strcmp(dir->d_name, ".."))
+ continue;
+
+ if (dir->d_type == DT_DIR) {
res = 0;
break;
} else if (dir->d_type == DT_UNKNOWN) {
It's quite similar to your change, except keeping the check for DT_DIR.
Much appreciated,
John
> John
>
>> Thanks,
>>
>> Sukadev
>>
>>
>>
>