Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <030c90fe-a09b-eff2-a1cf-55f6609825fb@intel.com>
Date:   Mon, 5 Sep 2022 14:46:49 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.11.0
Subject: Re: perf top -p broken for multithreaded processes since 5.19
Content-Language: en-US
To:     Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>
Cc:     =?UTF-8?B?VG9tw6HFoSBUcm5rYQ==?= <trnka@scm.com>,
        linux-perf-users <linux-perf-users@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>
References: <10137382.nUPlyArG6x@mintaka.ncbr.muni.cz>
 <2fd9b620-e9a8-9cd9-1b4a-6b13d829b2ae@intel.com>
 <YxJW19MwaMOb8D30@kernel.org>
 <ca905aa0-9d7b-d6d8-c789-2bd22057619b@intel.com>
 <YxNf2/H5Iyz5M8sn@kernel.org>
 <9a7bc232-e647-0667-4402-2f207003854b@intel.com>
 <CA+JHD91ReRGYNiBuO=1CGNZy1egcMjDo+VFO=kmCFrqE0mnK7w@mail.gmail.com>
From:   Adrian Hunter <adrian.hunter@intel.com>
Organization: Intel Finland Oy, Registered Address: PL 281, 00181 Helsinki,
 Business Identity Code: 0357606 - 4, Domiciled in Helsinki
In-Reply-To: <CA+JHD91ReRGYNiBuO=1CGNZy1egcMjDo+VFO=kmCFrqE0mnK7w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 3/09/22 21:42, Arnaldo Carvalho de Melo wrote:
> 
> 
> On September 3, 2022 2:35:09 PM GMT-03:00, Adrian Hunter <adrian.hunter@intel.com <mailto:adrian.hunter@intel.com>> wrote:
>>On 3/09/22 17:08, Arnaldo Carvalho de Melo wrote:
>>> Em Sat, Sep 03, 2022 at 10:14:25AM +0300, Adrian Hunter escreveu:
>>>> On 2/09/22 22:17, Arnaldo Carvalho de Melo wrote:
>>>>> Em Fri, Sep 02, 2022 at 05:50:22PM +0300, Adrian Hunter escreveu:
>>>>>> On 2/09/22 17:46, Tomáš Trnka wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> A bug in perf v5.19 and newer completely breaks monitoring multithreaded
>>>>>>> processes using "perf top -p". The tool fails to start with "Failed to mmap
>>>>>>> with 22 (Invalid argument)". It still seems to work fine on single-threaded
>>>>>>> processes. "perf record" is also unaffected.
>>>>>>
>>>>>> It has been reported here:
>>>>>>
>>>>>>   https://bugzilla.kernel.org/show_bug.cgi?id=216441 <https://bugzilla.kernel.org/show_bug.cgi?id=216441>
>>>>>
>>>>> If I do:
>>>>>
>>>>> ⬢[acme@toolbox perf-urgent]$ git log -2
>>>>> commit dfeb0bc60782471c293938e71b1a1117cfac2cb3 (HEAD -> perf/urgent)
>>>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com <mailto:acme@redhat.com>>
>>>>> Date:   Fri Sep 2 16:15:39 2022 -0300
>>>>>
>>>>>     Revert "libperf evlist: Check nr_mmaps is correct"
>>>>>
>>>>>     This reverts commit 4ce47d842d4c16c07b135b8a7975b8f0672bcc0e.
>>>>>
>>>>>     Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com <mailto:acme@redhat.com>>
>>>>>
>>>>> commit 78cd283f6b8ab701cb35eafd5af8140560a88f16
>>>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com <mailto:acme@redhat.com>>
>>>>> Date:   Fri Sep 2 16:13:41 2022 -0300
>>>>>
>>>>>     Revert "libperf evlist: Allow mixing per-thread and per-cpu mmaps"
>>>>>
>>>>>     This reverts commit ae4f8ae16a07896403c90305d4b9be27f657c1fc.
>>>>>
>>>>>     Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com <mailto:acme@redhat.com>>
>>>>> ⬢[acme@toolbox perf-urgent]$
>>>>>
>>>>> It works again, Tomáš can you please try doing this to see if this works
>>>>> for you?
>>>>>
>>>>
>>>> This is the fix I have so far.  I would like to test it some more though.
>>>
>>> Ok, so I'll leave it for the next pull req, possibly after Linux
>>> Plumbers.
>>>
>>> What do you think about reverting those two patches for v6.0 and then
>>> add this for v6.1?
>>
>>That would break sideband collection with selected CPUs, so I would
>>prefer to avoid that.
>>
>>The fix is a effectively a partial revert of "libperf evlist: Allow
>>mixing per-thread and per-cpu mmaps" anyway.  I just need a few days
>>to do more testing.
> 
> Ok, please add any insight or clarification you find while testing to the end result patch, like this sideband collection with selected CPUs.
> 
> This is all complex, so even comments being prone to drift from code, having them added in the commit log message, associated with the codebase at that time may help us in the future when fixing problems, using 'git blame'.

I finished the testing I wanted to do, and sent V2 of the patch with an expanded commit message:

https://lore.kernel.org/lkml/20220905114209.8389-1-adrian.hunter@intel.com/T/#u