Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Song Liu <songliubraving@fb.com>
To:     Okash Khawaja <osk@fb.com>
CC:     Daniel Borkmann <daniel@iogearbox.net>, Martin Lau <kafai@fb.com>,
        "Alexei Starovoitov" <ast@kernel.org>, Yonghong Song <yhs@fb.com>,
        Quentin Monnet <quentin.monnet@netronome.com>,
        Jakub Kicinski <jakub.kicinski@netronome.com>,
        "David S. Miller" <davem@davemloft.net>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        Kernel Team <Kernel-team@fb.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH bpf-next 2/3] bpf: btf: add btf json print functionality
Thread-Topic: [PATCH bpf-next 2/3] bpf: btf: add btf json print functionality
Thread-Index: AQHUCNbpU62yBE7bRkiOmLHhXAVRLaRpxzYA
Date:   Wed, 20 Jun 2018 23:14:33 +0000
Message-ID: <CB2174F3-11DA-496B-A3AF-9FA857C37DB1@fb.com>
References: <20180620203051.223156973@fb.com>
 <20180620203703.101156292@fb.com>
In-Reply-To: <20180620203703.101156292@fb.com>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: fb.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-ID: <923B46EE0BFCE9478401B8BB6C5CA06E@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: f038d072-44ef-4f79-a6f2-08d5d703955e
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2018 23:14:33.2073
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1279
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-06-20_10:,,
 signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Jun 20, 2018, at 1:30 PM, Okash Khawaja <osk@fb.com> wrote:
>=20
> This consumes functionality exported in the previous patch. It does the
> main job of printing with BTF data. This is used in the following patch
> to provide a more readable output of a map's dump. It relies on
> json_writer to do json printing. Below is sample output where map keys
> are ints and values are of type struct A:
>=20
> typedef int int_type;
> enum E {
>        E0,
>        E1,
> };
>=20
> struct B {
>        int x;
>        int y;
> };
>=20
> struct A {
>        int m;
>        unsigned long long n;
>        char o;
>        int p[8];
>        int q[4][8];
>        enum E r;
>        void *s;
>        struct B t;
>        const int u;
>        int_type v;
>        unsigned int w1: 3;
>        unsigned int w2: 3;
> };
>=20
> $ sudo bpftool map dump -p id 14
> [{
>        "key": 0
>    },{
>        "value": {
>            "m": 1,
>            "n": 2,
>            "o": "c",
>            "p": [15,16,17,18,15,16,17,18
>            ],
>            "q": [[25,26,27,28,25,26,27,28
>                ],[35,36,37,38,35,36,37,38
>                ],[45,46,47,48,45,46,47,48
>                ],[55,56,57,58,55,56,57,58
>                ]
>            ],
>            "r": 1,
>            "s": 0x7ffff6f70568,
>            "t": {
>                "x": 5,
>                "y": 10
>            },
>            "u": 100,
>            "v": 20,
>            "w1": 0x7,
>            "w2": 0x3
>        }
>    }
> ]
>=20
> This patch uses json's {} and [] to imply struct/union and array. More
> explicit information can be added later. For example, a command line
> option can be introduced to print whether a key or value is struct
> or union, name of a struct etc. This will however come at the expense
> of duplicating info when, for example, printing an array of structs.
> enums are printed as ints without their names.
>=20
> Signed-off-by: Okash Khawaja <osk@fb.com>
> Acked-by: Martin KaFai Lau <kafai@fb.com>
>=20
> ---
> tools/bpf/bpftool/btf_dumper.c |  247 +++++++++++++++++++++++++++++++++++=
++++++
> tools/bpf/bpftool/btf_dumper.h |   18 ++
> 2 files changed, 265 insertions(+)
>=20
> --- /dev/null
> +++ b/tools/bpf/bpftool/btf_dumper.c
> @@ -0,0 +1,247 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (c) 2018 Facebook */
> +
> +#include <linux/btf.h>
> +#include <linux/err.h>
> +#include <stdio.h> /* for (FILE *) used by json_writer */
> +#include <linux/bitops.h>
> +#include <string.h>
> +#include <ctype.h>
> +
> +#include "btf.h"
> +#include "json_writer.h"
> +
> +#define BITS_PER_BYTE_MASK (BITS_PER_BYTE - 1)
> +#define BITS_PER_BYTE_MASKED(bits) ((bits) & BITS_PER_BYTE_MASK)
> +#define BITS_ROUNDDOWN_BYTES(bits) ((bits) >> 3)
> +#define BITS_ROUNDUP_BYTES(bits) \
> +	(BITS_ROUNDDOWN_BYTES(bits) + !!BITS_PER_BYTE_MASKED(bits))
> +
> +static int btf_dumper_do_type(const struct btf *btf, uint32_t type_id,
> +		uint8_t bit_offset, const void *data, json_writer_t *jw);
> +
> +static void btf_dumper_ptr(const void *data, json_writer_t *jw)
> +{
> +	jsonw_printf(jw, "%p", *((uintptr_t *)data));
> +}
> +
> +static int btf_dumper_modifier(const struct btf *btf, uint32_t type_id,
> +		const void *data, json_writer_t *jw)
> +{
> +	int32_t actual_type_id =3D btf__resolve_type(btf, type_id);
> +	int ret;
> +
> +	if (actual_type_id < 0)
> +		return actual_type_id;
> +
> +	ret =3D btf_dumper_do_type(btf, actual_type_id, 0, data, jw);
> +
> +	return ret;
> +}
> +
> +static void btf_dumper_enum(const void *data, json_writer_t *jw)
> +{
> +	jsonw_printf(jw, "%d", *((int32_t *)data));
> +}
> +
> +static int btf_dumper_array(const struct btf *btf, uint32_t type_id,
> +		const void *data, json_writer_t *jw)
> +{
> +	const struct btf_type *t =3D btf__type_by_id(btf, type_id);
> +	struct btf_array *arr =3D (struct btf_array *)(t + 1);
> +	int64_t elem_size;
> +	uint32_t i;
> +	int ret;
> +
> +	elem_size =3D btf__resolve_size(btf, arr->type);
> +	if (elem_size < 0)
> +		return elem_size;
> +
> +	jsonw_start_array(jw);
> +	for (i =3D 0; i < arr->nelems; i++) {
> +		ret =3D btf_dumper_do_type(btf, arr->type, 0,
> +				data + (i * elem_size), jw);
> +		if (ret)
> +			return ret;

Shall we call jsonw_end_array() before return here?=20

> +	}
> +
> +	jsonw_end_array(jw);
> +
> +	return 0;
> +}
> +
> +static void btf_dumper_int_bits(uint32_t int_type, uint8_t bit_offset,
> +		const void *data, json_writer_t *jw)
> +{
> +	uint16_t total_bits_offset;
> +	uint32_t bits =3D BTF_INT_BITS(int_type);
> +	uint16_t bytes_to_copy;
> +	uint16_t bits_to_copy;
> +	uint8_t upper_bits;
> +	union {
> +		uint64_t u64_num;
> +		uint8_t u8_nums[8];
> +	} print_num;
> +
> +	total_bits_offset =3D bit_offset + BTF_INT_OFFSET(int_type);
> +	data +=3D BITS_ROUNDDOWN_BYTES(total_bits_offset);
> +	bit_offset =3D BITS_PER_BYTE_MASKED(total_bits_offset);
> +	bits_to_copy =3D bits + bit_offset;
> +	bytes_to_copy =3D BITS_ROUNDUP_BYTES(bits_to_copy);
> +
> +	print_num.u64_num =3D 0;
> +	memcpy(&print_num.u64_num, data, bytes_to_copy);
> +
> +	upper_bits =3D BITS_PER_BYTE_MASKED(bits_to_copy);
> +	if (upper_bits) {
> +		uint8_t mask =3D (1 << upper_bits) - 1;
> +
> +		print_num.u8_nums[bytes_to_copy - 1] &=3D mask;
> +	}
> +
> +	print_num.u64_num >>=3D bit_offset;
> +
> +	jsonw_printf(jw, "0x%llx", print_num.u64_num);
> +}
> +
> +static int btf_dumper_int(const struct btf_type *t, uint8_t bit_offset,
> +		const void *data, json_writer_t *jw)
> +{
> +	uint32_t *int_type =3D (uint32_t *)(t + 1);
> +	uint32_t bits =3D BTF_INT_BITS(*int_type);
> +	int ret =3D 0;
> +
> +	/* if this is bit field */
> +	if (bit_offset || BTF_INT_OFFSET(*int_type) ||
> +			BITS_PER_BYTE_MASKED(bits)) {
> +		btf_dumper_int_bits(*int_type, bit_offset, data, jw);
> +		return ret;
> +	}
> +
> +	switch (BTF_INT_ENCODING(*int_type)) {
> +	case 0:
> +		if (BTF_INT_BITS(*int_type) =3D=3D 64)
> +			jsonw_printf(jw, "%lu", *((uint64_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D 32)
> +			jsonw_printf(jw, "%u", *((uint32_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D 16)
> +			jsonw_printf(jw, "%hu", *((uint16_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D 8)
> +			jsonw_printf(jw, "%hhu", *((uint8_t *)data));
> +		else
> +			btf_dumper_int_bits(*int_type, bit_offset, data, jw);
> +		break;
> +	case BTF_INT_SIGNED:
> +		if (BTF_INT_BITS(*int_type) =3D=3D 64)
> +			jsonw_printf(jw, "%ld", *((int64_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D 32)
> +			jsonw_printf(jw, "%d", *((int32_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D  16)
> +			jsonw_printf(jw, "%hd", *((int16_t *)data));
> +		else if (BTF_INT_BITS(*int_type) =3D=3D  8)
> +			jsonw_printf(jw, "%hhd", *((int8_t *)data));
> +		else
> +			btf_dumper_int_bits(*int_type, bit_offset, data, jw);
> +		break;
> +	case BTF_INT_CHAR:
> +		if (*((char *)data) =3D=3D '\0')
> +			jsonw_null(jw);
> +		else if (isprint(*((char *)data)))
> +			jsonw_printf(jw, "\"%c\"", *((char *)data));
> +		else
> +			jsonw_printf(jw, "%hhx", *((char *)data));
> +		break;
> +	case BTF_INT_BOOL:
> +		jsonw_bool(jw, *((int *)data));
> +		break;
> +	default:
> +		/* shouldn't happen */
> +		ret =3D -EINVAL;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static int btf_dumper_struct(const struct btf *btf, uint32_t type_id,
> +		const void *data, json_writer_t *jw)
> +{
> +	const struct btf_type *t =3D btf__type_by_id(btf, type_id);
> +	struct btf_member *m;
> +	int ret =3D 0;
> +	int i, vlen;
> +
> +	if (t =3D=3D NULL)
> +		return -EINVAL;
> +
> +	vlen =3D BTF_INFO_VLEN(t->info);
> +	jsonw_start_object(jw);
> +	m =3D (struct btf_member *)(t + 1);
> +
> +	for (i =3D 0; i < vlen; i++) {
> +		jsonw_name(jw, btf__name_by_offset(btf, m[i].name_off));
> +		ret =3D btf_dumper_do_type(btf, m[i].type,
> +				BITS_PER_BYTE_MASKED(m[i].offset),
> +				data + BITS_ROUNDDOWN_BYTES(m[i].offset), jw);
> +		if (ret)
> +			return ret;

Shall we call jsonw_end_object() before return here?=20

> +	}
> +
> +	jsonw_end_object(jw);
> +
> +	return 0;
> +}
> +
> +static int btf_dumper_do_type(const struct btf *btf, uint32_t type_id,
> +		uint8_t bit_offset, const void *data, json_writer_t *jw)
> +{
> +	const struct btf_type *t =3D btf__type_by_id(btf, type_id);
> +	int ret =3D 0;
> +
> +	switch (BTF_INFO_KIND(t->info)) {
> +	case BTF_KIND_INT:
> +		ret =3D btf_dumper_int(t, bit_offset, data, jw);
> +		break;
> +	case BTF_KIND_STRUCT:
> +	case BTF_KIND_UNION:
> +		ret =3D btf_dumper_struct(btf, type_id, data, jw);
> +		break;
> +	case BTF_KIND_ARRAY:
> +		ret =3D btf_dumper_array(btf, type_id, data, jw);
> +		break;
> +	case BTF_KIND_ENUM:
> +		btf_dumper_enum(data, jw);
> +		break;
> +	case BTF_KIND_PTR:
> +		btf_dumper_ptr(data, jw);
> +		break;
> +	case BTF_KIND_UNKN:
> +		jsonw_printf(jw, "(unknown)");
> +		break;
> +	case BTF_KIND_FWD:
> +		/* map key or value can't be forward */
> +		ret =3D -EINVAL;
> +		break;
> +	case BTF_KIND_TYPEDEF:
> +	case BTF_KIND_VOLATILE:
> +	case BTF_KIND_CONST:
> +	case BTF_KIND_RESTRICT:
> +		ret =3D btf_dumper_modifier(btf, type_id, data, jw);
> +		break;
> +	default:
> +		jsonw_printf(jw, "(unsupported-kind");

"(unsupported-kind)"   (missing ')').=20

> +		ret =3D -EINVAL;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +int32_t btf_dumper_type(const struct btf *btf, json_writer_t *jw,
> +		uint32_t type_id, const void *data)
> +{
> +	if (!jw)
> +		return -EINVAL;
> +
> +	return btf_dumper_do_type(btf, type_id, 0, data, jw);
nit: btf_dumper_do_type() returns int, while btf_dumper_type() returns int3=
2_t.=20
     This should not matter though.=20

> +}
> --- /dev/null
> +++ b/tools/bpf/bpftool/btf_dumper.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (c) 2018 Facebook */
> +
> +#ifndef BTF_DUMPER_H
> +#define BTF_DUMPER_H
> +
> +/* btf_dumper_type - json print data along with type information
> + * @btf: btf instance initialised via btf__new()
> + * @jw: json writer used for printing
> + * @type_id: index in btf->types array. this points to the type to be du=
mped
> + * @data: pointer the actual data, i.e. the values to be printed
> + *
> + * Returns zero on success and negative error code otherwise
> + */
> +int32_t btf_dumper_type(const struct btf *btf, json_writer_t *jw,
> +		uint32_t type_id, void *data);
> +
> +#endif
>=20