Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Song Liu <songliubraving@fb.com>
To:     Jiri Olsa <jolsa@redhat.com>
CC:     linux-kernel <linux-kernel@vger.kernel.org>,
        Kernel Team <Kernel-team@fb.com>,
        "acme@kernel.org" <acme@kernel.org>,
        "acme@redhat.com" <acme@redhat.com>,
        "namhyung@kernel.org" <namhyung@kernel.org>,
        "jolsa@kernel.org" <jolsa@kernel.org>
Subject: Re: [PATCH] perf-stat: introduce bperf, share hardware PMCs with BPF
Thread-Topic: [PATCH] perf-stat: introduce bperf, share hardware PMCs with BPF
Thread-Index: AQHXFuPifzAWbJ/c0EGfxtA8c/wTpKqAREAAgAA7WYCAAAbkgIAB9duAgAAQFgA=
Date:   Sat, 13 Mar 2021 23:03:41 +0000
Message-ID: <4FD8EC9E-0B89-439C-865C-A245A2F2886F@fb.com>
References: <20210312020257.197137-1-songliubraving@fb.com>
 <YEtawFnompBDKpK0@krava> <45BCAC33-1626-42D1-A170-92DC8D7BAAF8@fb.com>
 <B7934C3F-4414-45AA-BE39-8FE3C64B7E7D@fb.com> <YE03TdSyQz5jXaNA@krava>
In-Reply-To: <YE03TdSyQz5jXaNA@krava>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1DDB6C53B9322141ACFE83815398A77E@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: b2535304-9236-4266-4f41-08d8e6743ee0
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Mar 2021 23:03:41.6179
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: pT/6qqfWF7tyQCn+rwaKdI1mUrcnN4zZh/kJM7i4AOwHkFXsPEifQj5ph5SVA0b6eLfxWJ2VmanKTFNASmKXCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR15MB4757
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761
 definitions=2021-03-13_11:2021-03-12,2021-03-13 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 phishscore=0
 clxscore=1015 mlxscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0
 spamscore=0 suspectscore=0 adultscore=0 priorityscore=1501 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103130179
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Mar 13, 2021, at 2:06 PM, Jiri Olsa <jolsa@redhat.com> wrote:
>=20
> On Fri, Mar 12, 2021 at 04:09:53PM +0000, Song Liu wrote:
>>=20
>>=20
>>> On Mar 12, 2021, at 7:45 AM, Song Liu <songliubraving@fb.com> wrote:
>>>=20
>>>=20
>>>=20
>>>> On Mar 12, 2021, at 4:12 AM, Jiri Olsa <jolsa@redhat.com> wrote:
>>>>=20
>>>> On Thu, Mar 11, 2021 at 06:02:57PM -0800, Song Liu wrote:
>>>>> perf uses performance monitoring counters (PMCs) to monitor system
>>>>> performance. The PMCs are limited hardware resources. For example,
>>>>> Intel CPUs have 3x fixed PMCs and 4x programmable PMCs per cpu.
>>>>>=20
>>>>> Modern data center systems use these PMCs in many different ways:
>>>>> system level monitoring, (maybe nested) container level monitoring, p=
er
>>>>> process monitoring, profiling (in sample mode), etc. In some cases,
>>>>> there are more active perf_events than available hardware PMCs. To al=
low
>>>>> all perf_events to have a chance to run, it is necessary to do expens=
ive
>>>>> time multiplexing of events.
>>>>>=20
>>>>> On the other hand, many monitoring tools count the common metrics (cy=
cles,
>>>>> instructions). It is a waste to have multiple tools create multiple
>>>>> perf_events of "cycles" and occupy multiple PMCs.
>>>>>=20
>>>>> bperf tries to reduce such wastes by allowing multiple perf_events of
>>>>> "cycles" or "instructions" (at different scopes) to share PMUs. Inste=
ad
>>>>> of having each perf-stat session to read its own perf_events, bperf u=
ses
>>>>> BPF programs to read the perf_events and aggregate readings to BPF ma=
ps.
>>>>> Then, the perf-stat session(s) reads the values from these BPF maps.
>>>>>=20
>>>>> Please refer to the comment before the definition of bperf_ops for th=
e
>>>>> description of bperf architecture.
>>>>>=20
>>>>> bperf is off by default. To enable it, pass --use-bpf option to perf-=
stat.
>>>>> bperf uses a BPF hashmap to share information about BPF programs and =
maps
>>>>> used by bperf. This map is pinned to bpffs. The default address is
>>>>> /sys/fs/bpf/bperf_attr_map. The user could change the address with op=
tion
>>>>> --attr-map.
>>>>=20
>>>> nice, I recall the presentation about that and was wondering
>>>> when this will come up ;-)
>>>=20
>>> The progress is slower than I expected. But I finished some dependencie=
s of=20
>>> this in the last year:=20
>>>=20
>>> 1. BPF_PROG_TEST_RUN for raw_tp event;
>>> 2. perf-stat -b, which introduced skeleton and bpf_counter;
>>> 3. BPF task local storage, I didn't use it in this version, but it coul=
d,
>>>    help optimize bperf in the future.=20
>>>=20
>>>>=20
>>>>>=20
>>>>> ---
>>>>> Known limitations:
>>>>> 1. Do not support per cgroup events;
>>>>> 2. Do not support monitoring of BPF program (perf-stat -b);
>>>>> 3. Do not support event groups.
>>>>>=20
>>>>> The following commands have been tested:
>>>>>=20
>>>>> perf stat --use-bpf -e cycles -a
>>>>> perf stat --use-bpf -e cycles -C 1,3,4
>>>>> perf stat --use-bpf -e cycles -p 123
>>>>> perf stat --use-bpf -e cycles -t 100,101
>>>>=20
>>>> I assume the output is same as standard perf?
>>=20
>> Btw, please give it a try. :)=20
>>=20
>> It worked pretty well in my tests. If it doesn't work for some combinati=
on=20
>> of options, please let me know.=20
>=20
> heya, can't compile
>=20
>  CLANG    /home/jolsa/linux-perf/tools/perf/util/bpf_skel/.tmp/bperf_foll=
ower.bpf.o
> util/bpf_skel/bperf_follower.bpf.c:8:10: fatal error: 'bperf_u.h' file no=
t found
> #include "bperf_u.h"
>         ^~~~~~~~~~~

Oops, I forgot git-add. :(=20

The file is very simple:

tools/perf/util/bpf_skel/bperf_u.h:


// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
// Copyright (c) 2021 Facebook

#ifndef __BPERF_STAT_U_H
#define __BPERF_STAT_U_H

enum bperf_filter_type {
        BPERF_FILTER_GLOBAL =3D 1,
        BPERF_FILTER_CPU,
        BPERF_FILTER_PID,
        BPERF_FILTER_TGID,
};

#endif /* __BPERF_STAT_U_H */

Thanks,
Song