Sender: Tejun Heo <htejun@gmail.com>
Date:   Wed, 12 Jan 2022 12:37:08 -1000
From:   Tejun Heo <tj@kernel.org>
To:     Hao Luo <haoluo@google.com>
Cc:     Alexei Starovoitov <ast@kernel.org>,
        Andrii Nakryiko <andrii@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        KP Singh <kpsingh@kernel.org>,
        Shakeel Butt <shakeelb@google.com>,
        Joe Burton <jevburton.kernel@gmail.com>, bpf@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND RFC bpf-next v1 0/8] Pinning bpf objects outside
 bpffs
Message-ID: <Yd9YFM5kqU9yKXf5@slm.duckdns.org>
References: <20220112193152.3058718-1-haoluo@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220112193152.3058718-1-haoluo@google.com>
Precedence: bulk

Hello, Hao.

On Wed, Jan 12, 2022 at 11:31:44AM -0800, Hao Luo wrote:
> As a concrete usecase of this feature, this patchset introduces a
> simple new program type called 'bpf_view', which can be used to format
> a seq file by a kernel object's state. By pinning a bpf_view program
> into a cgroup directory, userspace is able to read the cgroup's state
> from file in a format defined by the bpf program.

Both kernfs users - sysfs and cgroups - are hard APIs just as procfs, so
allowing bpf programs to add arbitrarily formatted files anywhere likely
isn't a good idea. Even ignoring the hard-defined interface problem,
allowing arbitrary files can cause surprising failures through namespace
collisions (which may be worked around by restricting where these files
reside or how they're named).

While the attraction is understandable, I think this is a misguided
direction. Text file interfaces are okay, or sometimes even good, for
certain things - communicating well established small piece of information.
They're easy to access and as long as the format stays really stable, the
million parsers that they end up spawning are mostly manageable although you
inevitably end up with "I was reading 3rd field of 4th line and you added a
new line above!".

The above also illustrates the problems with using these text file
interfaces. They're good for really static stuff or something really
provisional like for debugging where there's only one or very few consumers.
Outside of those extremes, they become pretty terrible. They are very
inefficient when the data volume isn't trivial. There's no good way to
synchronize accesses to multiple files. There are million ways to parse
them, many of them ever so subtly wrong. There's no good way to version the
interface (not that you can't). And if you throw these flexible files in the
midst of other hard-API files, it'll severely exacerbate confusion.

Also, for something which isn't stable, I think it's better to have some of
the access logic on the reader side. For example, let's say you've been
using data from a cgroup file on the system. One day, the system boots and
the file isn't there. How would you debug that? If it were a, say, py-bcc
script which fetched data through bpf, it wouldn't be difficult to track.
This isn't just happenstance - if you're reaching into a black box to get
data, you better keep that mechanism close to you as it's a fragile
temporary thing prone to breaking.

Yet another argument against it is that the kernel is a really bad place to
process and format data. We can't do real percentiles, or any kind of
non-trivial numeric analysis, or even handle and format real numbers. Given
that bpf already has an a lot more efficient way to transfer data between
kernel and user, it doesn't make sense to push data formatting into the
kernel. Export data to userspace, process and format there.

If there are reasons why this isn't very convenient using bpf. I think the
right thing to do is improving those. One issue that Song raised was that
there's no easy to allow non-root users to run specific bpf programs and
even if we do that with SUID binaries, each execution would be pretty
expensive involving full verification run and so on. But those problems are
solvable - maybe known BPF programs can be cached and made available to
other users - and I think concentrating on such direction would be way more
fruitful for wider purposes than trying to make BPF show text files in
existing fixed interfaces.

Thanks.

-- 
tejun