Message-ID: <557B16C4.7000000@gmail.com>
Date: Fri, 12 Jun 2015 11:28:36 -0600
From: David Ahern <dsahern@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: "Liang, Kan" <kan.liang@intel.com>, Andi Kleen <andi@firstfloor.org>
CC: Arnaldo Carvalho de Melo <acme@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Huang, Ying" <ying.huang@intel.com>
Subject: Re: [PATCH 1/1] perf,tools: add time out to force stop endless mmap
 processing
References: <1433922364-22580-1-git-send-email-kan.liang@intel.com> <20150611140614.GC2696@kernel.org> <5579A766.4010504@gmail.com> <20150611184737.GU19417@two.firstfloor.org> <557A28F6.8040603@gmail.com> <37D7C6CF3E00A74B8858931C1DB2F077018767AD@SHSMSX103.ccr.corp.intel.com> <557AFDB1.7030902@gmail.com> <37D7C6CF3E00A74B8858931C1DB2F07701876834@SHSMSX103.ccr.corp.intel.com>
In-Reply-To: <37D7C6CF3E00A74B8858931C1DB2F07701876834@SHSMSX103.ccr.corp.intel.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3223
Lines: 86

On 6/12/15 11:05 AM, Liang, Kan wrote:
>>
>> On 6/12/15 8:42 AM, Liang, Kan wrote:
>>>
>>>>
>>>> On 6/11/15 12:47 PM, Andi Kleen wrote:
>>>>>> Can you elaborate on an example? I don't see how this can happen
>>>>>> reading a maps file. And it does not read maps for all threads only
>>>>>> thread group leaders.
>>>>>
>>>>> This is with a stress test case that generates lots of small
>>>>> mappings at very high speed and frees them again. So the maps file
>>>>> keeps changing faster than the proc reader can keep it and it can
>>>>> end up with a live lock.
>>>>
>>>> Can you pass it along? I'd like to see how the task_diag proposal handles
>> it.
>>>>
>>>> https://github.com/dsahern/linux/commits/task_diag-wip
>>>
>>> Hi David,
>>>
>>> I tried the task_diag on my platform, but it shows error message when
>>> I run perf top. " Message handling failed: rc -1, errno 25".
>>> And it looks perf top failed to get maps information.
>>
>> Not surprising; it's only half-baked. Can you try perf-record? So far that is
>> the only one I have tested.
>>
>
> Perf record cannot reproduce the infinite loop which found in perf top.
> But we can observe that synthesized threads took very long time in perf record.
>
> According to test result as below, current perf cost 13s to read the maps,
> while task_diag cost 14s to synthesized thread.
> (Note: The time will increase with the test run.)
>
> So it looks task_diag doesn't help on this issue.
>
> [perf]$ sudo ./perf record -e instructions:pp --pid 14560
> Reading /proc/14560/maps cost 13.12690599 s
> ^C[ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.108 MB perf.data (2783 samples) ]

so perf was able to read the proc file?

>
> [perf]$ sudo ./perf_task_diag record -e instructions:pp --pid 14560
> synthesized threads took 14.435450 sec
> ^C[ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.035 MB perf.data (885 samples) ]
>
>
>> Also, while running that kernel you can build the test programs under
>> tools/testing/selftests/task_diag/ and try task_diag_all. I am away from my
>> dev box at the moment. As I recall you will want to try 'task_diag_all o $pid'
>> or 'task_diag_all a'
>>
> Neither options work on my platform.
>
> [task_diag]$ sudo ./task_diag_all a
> Unable to receive message: Operation not supported
> [task_diag]$ sudo ./task_diag_all o 14751
> Unable to receive message: Operation not supported

Are you sure task_diag is enabled? There is an option under General I 
believe:
config TASK_DIAG
         bool "Export task/process properties through netlink"
         depends on NET && TASKSTATS
         default n
         help
           Export selected properties for tasks/processes through the
           generic netlink interface. Unlike the proc file system, task_diag
           returns information in a binary format, allows to specify which
           information are required.

           Say N if unsure.

David

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/