Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Song Liu <songliubraving@fb.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Netdev <netdev@vger.kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Kernel Team <Kernel-team@fb.com>,
        "ast@kernel.org" <ast@kernel.org>,
        "daniel@iogearbox.net" <daniel@iogearbox.net>,
        "acme@kernel.org" <acme@kernel.org>
Subject: Re: [RFC perf,bpf 1/5] perf, bpf: Introduce PERF_RECORD_BPF_EVENT
Thread-Topic: [RFC perf,bpf 1/5] perf, bpf: Introduce PERF_RECORD_BPF_EVENT
Thread-Index: AQHUdhLRNu0EDS9AC0KZ+9MuHYh4daVD/w6AgACjMoCAAVksAIAAM3wA
Date:   Thu, 8 Nov 2018 18:04:44 +0000
Message-ID: <C858C862-E523-4CE8-AB39-CC9B27BE2538@fb.com>
References: <20181106205246.567448-1-songliubraving@fb.com>
 <20181106205246.567448-2-songliubraving@fb.com>
 <20181107084057.GG9781@hirez.programming.kicks-ass.net>
 <31067290-4B66-4AA1-8027-607397BC0264@fb.com>
 <20181108150028.GU9761@hirez.programming.kicks-ass.net>
In-Reply-To: <20181108150028.GU9761@hirez.programming.kicks-ass.net>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: fb.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7ED8500B60234A4285502EC3EE9CD718@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 329fd542-917a-4fff-6268-08d645a4aa22
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Nov 2018 18:04:44.8505
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1726
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-11-08_08:,,
 signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Peter,

> On Nov 8, 2018, at 7:00 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>=20
> On Wed, Nov 07, 2018 at 06:25:04PM +0000, Song Liu wrote:
>>=20
>>=20
>>> On Nov 7, 2018, at 12:40 AM, Peter Zijlstra <peterz@infradead.org> wrot=
e:
>>>=20
>>> On Tue, Nov 06, 2018 at 12:52:42PM -0800, Song Liu wrote:
>>>> For better performance analysis of BPF programs, this patch introduces
>>>> PERF_RECORD_BPF_EVENT, a new perf_event_type that exposes BPF program
>>>> load/unload information to user space.
>>>>=20
>>>>       /*
>>>>        * Record different types of bpf events:
>>>>        *   enum perf_bpf_event_type {
>>>>        *      PERF_BPF_EVENT_UNKNOWN          =3D 0,
>>>>        *      PERF_BPF_EVENT_PROG_LOAD        =3D 1,
>>>>        *      PERF_BPF_EVENT_PROG_UNLOAD      =3D 2,
>>>>        *   };
>>>>        *
>>>>        * struct {
>>>>        *      struct perf_event_header header;
>>>>        *      u16 type;
>>>>        *      u16 flags;
>>>>        *      u32 id;  // prog_id or map_id
>>>>        * };
>>>>        */
>>>>       PERF_RECORD_BPF_EVENT                   =3D 17,
>>>>=20
>>>> PERF_RECORD_BPF_EVENT contains minimal information about the BPF progr=
am.
>>>> Perf utility (or other user space tools) should listen to this event a=
nd
>>>> fetch more details about the event via BPF syscalls
>>>> (BPF_PROG_GET_FD_BY_ID, BPF_OBJ_GET_INFO_BY_FD, etc.).
>>>=20
>>> Why !? You're failing to explain why it cannot provide the full
>>> information there.
>>=20
>> Aha, I missed this part. I will add the following to next version. Pleas=
e
>> let me know if anything is not clear.
>=20
>>=20
>> This design decision is picked for the following reasons. First, BPF=20
>> programs could be loaded-and-jited and/or unloaded before/during/after=20
>> perf-record run. Once a BPF programs is unloaded, it is impossible to=20
>> recover details of the program. It is impossible to provide the=20
>> information through a simple key (like the build ID). Second, BPF prog
>> annotation is under fast developments. Multiple informations will be=20
>> added to bpf_prog_info in the next few releases. Including all the
>> information of a BPF program in the perf ring buffer requires frequent=20
>> changes to the perf ABI, and thus makes it very difficult to manage=20
>> compatibility of perf utility.=20
>=20
> So I don't agree with that reasoning. If you want symbol information
> you'll just have to commit to some form of ABI. That bpf_prog_info is an
> ABI too.

At the beginning of the perf-record run, perf need to query bpf_prog_info=20
of already loaded BPF programs. Therefore, we need to commit to the=20
bpf_prog_info ABI. If we also include full information of the BPF program=20
in the perf ring buffer, we will commit to TWO ABIs.=20

Also, perf-record write the event to perf.data file, so the data need to be=
=20
serialized. This is implemented in patch 4/5. To include the data in the=20
ring buffer, we will need another piece of code in the kernel to do the
same serialization work.  =20

On the other hand, processing BPF load/unload events synchronously should
not introduce too much overhead for meaningful use cases. If many BPF progs
are being loaded/unloaded within short period of time, it is not the steady
state that profiling works care about.=20

Would these resolve your concerns?=20

Thanks,
Song