Received-SPF: pass (google.com: domain of linux-kernel+bounces-126754-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Precedence: bulk
MIME-Version: 1.0
References: <20240329015351.624249-1-drosen@google.com> <CAOQ4uxhLPw9AKBWmUcom3RUrsov0q39tiNhh2Mw7qJbwKr1yRQ@mail.gmail.com>
 <CA+PiJmQR17nwkHaZXUhw=YRM06TfF14bhozc=nM9cw51aiiB6g@mail.gmail.com>
In-Reply-To: <CA+PiJmQR17nwkHaZXUhw=YRM06TfF14bhozc=nM9cw51aiiB6g@mail.gmail.com>
From: Amir Goldstein <amir73il@gmail.com>
Date: Mon, 1 Apr 2024 17:43:28 +0300
Message-ID: <CAOQ4uxiyS9viEpOwT8f2Np=wuMdWwUqqyHFRrBX9+Acy_i3OHw@mail.gmail.com>
Subject: Re: [RFC PATCH v4 00/36] Fuse-BPF and plans on merging with Fuse Passthrough
To: Daniel Rosenberg <drosen@google.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>, bpf@vger.kernel.org, 
	Alexei Starovoitov <ast@kernel.org>, linux-kernel@vger.kernel.org, 
	linux-fsdevel@vger.kernel.org, linux-unionfs@vger.kernel.org, 
	Daniel Borkmann <daniel@iogearbox.net>, John Fastabend <john.fastabend@gmail.com>, 
	Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau <martin.lau@linux.dev>, Song Liu <song@kernel.org>, 
	Eduard Zingerman <eddyz87@gmail.com>, Yonghong Song <yonghong.song@linux.dev>, 
	KP Singh <kpsingh@kernel.org>, Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, 
	Jiri Olsa <jolsa@kernel.org>, Shuah Khan <shuah@kernel.org>, Jonathan Corbet <corbet@lwn.net>, 
	Joanne Koong <joannelkoong@gmail.com>, Mykola Lysenko <mykolal@fb.com>, 
	Christian Brauner <brauner@kernel.org>, kernel-team@android.com, 
	Bernd Schubert <bschubert@ddn.com>
Content-Type: text/plain; charset="UTF-8"

> >
> > That sounds like a good plan, but also, please remember Miklos' request -
> > please split the patch sets for review to:
> > 1. FUSE-passthrough-all-mode
> > 2. Attach BPF program
> >
> > We FUSE developers must be able to review the FUSE/passthough changes
> > without any BPF code at all (which we have little understanding thereof)
> >
> > As a merge strategy, I think we need to aim for merging all the FUSE
> > passthrough infrastructure needed for passthrough of inode operations
> > strictly before merging any FUSE-BPF specific code.
> >
> > In parallel you may get BPF infrastructure merged, but integrating FUSE+BPF,
> > should be done only after all infrastructure is already merged IMO.
> >
>
> Ok. I'll probably mess around with the module stuff at least, in order
> to work out if everything I need is present on the bpf side. Do you
> know if anyone is actively working on extending the file-backing work
> to something like inode-backing? I don't want to duplicate work there,

I am actively *thinking* about working on passthrough for getattr/getxattr.
As soon as I come up with something concrete I will let you know.

> but I'd be happy to start looking at it. Otherwise I'd focus on the
> bpf end for now. I expect we'll want to be able to optionally set the
> bpf program at the same place where we set the backing file/inode.
> Hence the spit into a file and inode program set. I'm still thinking
> over what the best way to address the programs is...
>

My thoughts were doing something similar to FOPEN_PASSTHROUGH
but in response to LOOKUP request and in that case the fuse inode
will enter passthrough mode early and will not leave passthrough mode
until inode is evicted.

> > Please explain what you mean by that.
> > How are fuse-bpf file operations expected to be used and specifically,
> > How are they expected to extend the current FUSE passthrough functionality?
> >
> > Do you mean that an passthrough setup will include a reference to a bpf
> > program that will be used to decide per read/write/splice operation
> > whether it should be passed through to backing file or sent to server
> > direct_io style?
> >
>
> So in the current fuse-bpf setup, the bpf program does two things. It
> can edit certain parameters, and it can indicate what the next action
> should be. That action could be queuing up the post filter after the
> backing operation, deferring to a userspace pre/post filter, or going
> back to normal fuse operations.
> The latter one isn't currently very well fleshed out. Unless you do
> some specific tracking, under existing fuse-bpf you'd have a node id
> of 0, and userspace can't make much out of that. With that aside,

node id 0 sounds weird.
I was wondering if and how a passthrough lookup operation would work.
The only thing I can think of is that in this setup, fuse must use the backing
file st_ino as the fuse node id, so that the kernel can instantiate a fuse inode
before the server knows about it.

> there's all sorts of caching nightmares to deal with there.
>

Yeh...

> We're only using the parameter changing currently in our use cases. I
> wouldn't be opposed to leaving the falling back to fuse for specific
> operations out of v1 of the bpf enhancements, especially if we have
> the userspace pre/post filters available.
> So you'd optionally specify a bpf program to use with the backing
> file. That would allow you to manipulate some data in the files like
> you might in Fuse itself. For instance, data redaction. You could null
> out location metadata in images, provided a map or something with the
> offsets that should be nulled. You could also prepend some data at the
> beginning of a file by adjusting offsets and attrs and whatnot. I
> could imagine having multiple backing files, and the bpf program
> splitting a read into multiple parts to handle parts of it using
> different backing files, although that's not in the current design.
>

Lots of plans ;)

>
> Can you not read/write without interacting with the server? Or do you
> mean FOPEN_DIRECT_IO sends some file ops to the server even in
> passthrough mode?

FOPEN_DIRECT_IO sends write() and read() to the server even in
passthrough mode.

> At the moment I'm tempted to follow the same
> mechanics passthrough is using. The only exception would be possibly
> tossing back to the server, which I mentioned above. That'd only
> happen for, say, read, if we're not under FOPEN_DIRECT_IO. I've not
> looked too closely at FOPEN_DIRECT_IO. In Fuse bpf we currently have
> bpf mode taking priority. Are there any email threads I should look at
> for more background there?

Maybe this patch set:
https://lore.kernel.org/linux-fsdevel/20240208170603.2078871-1-amir73il@gmail.com/

Bernd and I worked on it together as a prerequisite to fuse passthrough.
Benrd has some followup direct_io re-factoring patches.

Thanks,
Amir.