2015-06-11 15:02:36

by Liang, Kan

[permalink] [raw]
Subject: [PATCH V2 1/1] perf,tools: add time out to force stop endless mmap processing

From: Kan Liang <[email protected]>

perf top reads all threads' /proc/xxx/maps. If there is any threads
which generating a keeping growing huge /proc/xxx/maps, perf will do
infinite loop in perf_event__synthesize_mmap_events.
This patch fixes this issue by adding a time out to force stop this kind
of endless mmap processing.

Reported-by: Huang, Ying <[email protected]>
Signed-off-by: Kan Liang <[email protected]>

---

Changes since V1
- Add warning message for time out.

tools/perf/util/event.c | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c
index 793b150..b4dccf2 100644
--- a/tools/perf/util/event.c
+++ b/tools/perf/util/event.c
@@ -213,6 +213,8 @@ static int perf_event__synthesize_fork(struct perf_tool *tool,
return 0;
}

+#define MMAP_TIMEOUT (50 * 1000000ULL)
+
int perf_event__synthesize_mmap_events(struct perf_tool *tool,
union perf_event *event,
pid_t pid, pid_t tgid,
@@ -222,6 +224,7 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
{
char filename[PATH_MAX];
FILE *fp;
+ unsigned long long t;
int rc = 0;

if (machine__is_default_guest(machine))
@@ -240,6 +243,7 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
}

event->header.type = PERF_RECORD_MMAP2;
+ t = rdclock();

while (1) {
char bf[BUFSIZ];
@@ -253,6 +257,13 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,
if (fgets(bf, sizeof(bf), fp) == NULL)
break;

+ if ((rdclock() - t) > MMAP_TIMEOUT) {
+ pr_warning("Reading %s time out."
+ "The file may be too huge or keep growing.\n",
+ filename);
+ break;
+ }
+
/* ensure null termination since stack will be reused. */
strcpy(execname, "");

--
1.8.3.1


2015-06-11 15:45:50

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH V2 1/1] perf,tools: add time out to force stop endless mmap processing

Em Thu, Jun 11, 2015 at 03:49:04AM -0400, [email protected] escreveu:
> perf top reads all threads' /proc/xxx/maps. If there is any threads
> which generating a keeping growing huge /proc/xxx/maps, perf will do
> infinite loop in perf_event__synthesize_mmap_events.
> This patch fixes this issue by adding a time out to force stop this kind
> of endless mmap processing.
>
> Reported-by: Huang, Ying <[email protected]>
> Signed-off-by: Kan Liang <[email protected]>
>
> ---
>
> Changes since V1
> - Add warning message for time out.

<SNIP>

> + if ((rdclock() - t) > MMAP_TIMEOUT) {
> + pr_warning("Reading %s time out."
> + "The file may be too huge or keep growing.\n",
> + filename);
> + break;
> + }
> +
> /* ensure null termination since stack will be reused. */
> strcpy(execname, "");

Have you tried this? I.e. pr_warning, IIRC, will make it appear in the
logs that this happened, but this will get probably lost in the noise
and only if you have a suspicion that something may have went wrong you
will try to look in the, possibly long, logs for a possible explanation.

So, perhaps we need to do something like what is done in
perf_session__process_events(), i.e. at the end of the synthesizing,
look at a counter for such situations and use ui__warning(), that in the
TUI case will show a window message that will show the warning and wait
for the user to acknowledge it by pressing a Ok button.

This is all done in perf_session__warn_about_errors().

This is how I think this should be done, but as this is such a corner
case, and this patch fixes these long loops, this may be applied now and
then what I suggest may be done on top.

Anyway, please try to reply to David Ahern question about an example for
this case, because he is working on a netlink based replacement to the
synthesizing of PERF_RECORD_ events for existing tasks and will have to
take this case into account there as well...

- Arnaldo