Received: by 2002:a05:6358:53a8:b0:117:f937:c515 with SMTP id z40csp4754609rwe; Mon, 17 Apr 2023 18:41:33 -0700 (PDT) X-Google-Smtp-Source: AKy350Zo8MnYZzLTzWMeeIAJvanHRvuVlojzGYBdBAREYafntIr/Fxte6d6clkyTiXfvcE6/+iw+ X-Received: by 2002:a05:6a20:258e:b0:f0:36a3:706a with SMTP id k14-20020a056a20258e00b000f036a3706amr3610575pzd.10.1681782093360; Mon, 17 Apr 2023 18:41:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681782093; cv=none; d=google.com; s=arc-20160816; b=HPccq6Su6DEA2zg8wLBNy7H8QLoRBxnZosDRlBfNN2USsX4FRaoBONiOzYjDpaJeBP ZqnHzB57sFHkkW5SkTC2cFPUR2PYD1AAgkkAqQuxpFpOMnTJEY6CeOGSMLXJXQYTitC9 i6zU6qx1cBqgAP5eo6QgM+BqKSV42bkkxV/11FeMMRE9J+1+5Xn2PznthcGlhGr/xYp5 +WIFOmRwpSLyZPP25SvNgtJuP4/aNP5HyT2DkhHyxRgQlLR8wn9GlA4jAJZ95cxP5ZrE vWu3hg4hP3k46NuUyIpxyeOJ84RVuSnbvJL3Ngtx+NTiAE2a1tp+Ld15JzFladr6ROFx 8WKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=bMGkpk1YzLLB6Dkq44tjt4SD+YA/2KPf5L1MlIvm+KE=; b=a8cUGKfifrP+1kiWowDV7Lx03oKjdRh65BmKhGeNzKXflNVNSRCMOLAq/m8RTbdpvq +2l711Ifrx5f798SIw1gDHt3QMXCFt7biPjtNAqSQljpXAV8FqWVThj0tNyJOwVI9+7B Mop+Wxv0pZ64Yk3RL4IIKC6zswRPdKDnoRN+BuEMtQ9C0VDIj6LeGKDayWsFgn7YlwMX y5nOmfN/XbUiyp7EmD3/YKPhg+6u6JAHKdeP68gER+X/tdK6IbWYXAjL+bSc2nWp/q3W swGyxsuXVlP4iRZRKKyM0wB3che83Ydyxr83AW/QUN33e6a4EVx7ywu1Onr+lT5ssB1+ qNqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Udx3W0F5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t190-20020a6381c7000000b00517a80b4245si11995422pgd.756.2023.04.17.18.41.20; Mon, 17 Apr 2023 18:41:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Udx3W0F5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229960AbjDRBk5 (ORCPT + 99 others); Mon, 17 Apr 2023 21:40:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229741AbjDRBky (ORCPT ); Mon, 17 Apr 2023 21:40:54 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1131855AC for ; Mon, 17 Apr 2023 18:40:52 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 63-20020a250042000000b00b924691cef2so3637900yba.6 for ; Mon, 17 Apr 2023 18:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681782051; x=1684374051; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=bMGkpk1YzLLB6Dkq44tjt4SD+YA/2KPf5L1MlIvm+KE=; b=Udx3W0F5kzh1CXU8g7LWKjoladX92jD4LTjvnTrKNMLe/kB98XFpP5SRZx2G+LODDa 732pusGPSRXAVbNv1DIlX0Ix/mgkMpYkAhIev8U1x0W6iV79Tc3pCVNmRId45lQ0WAA/ EZ99SloXabv8y9/w4zJ3OdFk7ysy68VPmNybl0mKuWWfNA7Y7xoxMkq0sjahR5oZ0MuY DPHiauTUOQRsk4baUh8TGdku8HKB7uhASof5VCYaVS/6IfbdCc+Mj7ZxVPXsXJxIxLH2 P3qKzoLwPbp1hjacowQrK83FuYlcxOI/l+epJA+5jIXk+89QC8PPxLUm62Sti+tAzBt8 vFHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681782051; x=1684374051; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bMGkpk1YzLLB6Dkq44tjt4SD+YA/2KPf5L1MlIvm+KE=; b=CqqdwZTdZkoaipRYL7FG15ZKtGxAaEyIr54envp/jkZkyCj+DTOTm0acdX2HEN9aH6 ZjkOPnl7jaw+6K9dopdU4ZMjs7ONJjly3Rl2i6fhXTQOafuAXCIEiZq1ApiXWAQiE6yp M8cLbBhIbBEXVRgj9k73v4YwjpQLLg1qWYltHJTG7N5Fc1Du/67AslExg/BbcRlLRJif wtJLBnF3cPUVMUEUli/r8hd9UyFEhyDLVOhGNe5OvhFGt8/tgof/O5+UJS1ZqS1HD3n/ 8Fr+l9/VUCxr0j2Wxz0ZafZLg96eRLJs4Ssy/yOf1IzujoKVZAcCkNhbmvkTz2fohb/0 CnLQ== X-Gm-Message-State: AAQBX9eZU8qD1A0u6dqatu+6zXLyRF6PLJA265AEW81v2IGKIiiIErn0 lVW9pGknfGfxvYY5tAsWD0tpUZu7bsE= X-Received: from drosen.mtv.corp.google.com ([2620:15c:211:201:e67a:98b0:942d:86aa]) (user=drosen job=sendgmr) by 2002:a81:ac5d:0:b0:54f:b95f:8999 with SMTP id z29-20020a81ac5d000000b0054fb95f8999mr10086901ywj.6.1681782051211; Mon, 17 Apr 2023 18:40:51 -0700 (PDT) Date: Mon, 17 Apr 2023 18:40:00 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230418014037.2412394-1-drosen@google.com> Subject: [RFC PATCH bpf-next v3 00/37] FUSE BPF: A Stacked Filesystem Extension for FUSE From: Daniel Rosenberg To: Miklos Szeredi , bpf@vger.kernel.org, Alexei Starovoitov Cc: Amir Goldstein , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-unionfs@vger.kernel.org, Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Shuah Khan , Jonathan Corbet , Joanne Koong , Mykola Lysenko , kernel-team@android.com, Daniel Rosenberg Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org These patches extend FUSE to be able to act as a stacked filesystem. This allows pure passthrough, where the fuse file system simply reflects the lower filesystem, and also allows optional pre and post filtering in BPF and/or the userspace daemon as needed. This can dramatically reduce or even eliminate transitions to and from userspace. In this patch set, I've reworked the bpf code to add a new struct_op type instead of a new program type, and used new kfuncs in place of new helpers. Additionally, it now uses dynptrs for variable sized buffers. The first three patches are repeats of a previous patch set which I have not yet adjusted for comments. I plan to adjust those and submit them separately with fixes, but wanted to have the current fuse-bpf code visible before then. Patches 4-7 mostly rearrange existing code to remove noise from the main patch. Patch 8 contains the main sections of fuse-bpf Patches 9-25 implementing most FUSE functions as operations on a lower filesystem. From patch 25, you can run fuse as a passthrough filesystem. Patches 26-32 provide bpf functionality so that you can alter fuse parameters via fuse_op programs. Patch 33 extends this to userspace, and patches 34-37 add some testing functionality. There's definitely a lot of cleanup and some restructuring I would like to do. In the current form, I could get rid of the large macro in place of a function that takes a struct that groups a bunch of function pointers, although I'm not sure a function that takes three void*'s is much better than the macro... I'm definitely open to suggestions on how to clean that up. This changes the format of adding a backing file/bpf slightly from v2. fuse_op programs are specified by name, limited to 15 characters. The block added to fuse_bpf_entires has been increased to compensate. This adds one more unused field when specifying the backing file. Lookups responses that add a backing file must go through an ioctl interface. This is to prevent any attempts at fooling priveledged programs with fd trickery. Currently, there are two types of fuse_bpf_entry. One for passing the fuse_op program you wish to use, specified by name, and one for passing the fd of the backing file you'd like to associate with the given lookup. In the future, this may be extended to a more complicated system allowing for multiple bpf programs or backing files. This would come with kfuncs for bpf to indicate which backing file should be acted upon. Multiple bpf programs would allow chaining existing programs to extend functionality without requiring an entirely new program set. You can run this without needing to set up a userspace daemon by adding these mount options: root_dir=[fd],no_daemon where fd is an open file descriptor pointing to the folder you'd like to use as the root directory. The fd can be immediately closed after mounting. You may also set a root_bpf program by setting root_bpf=[fuse_op name] after registering a fuse_op program. This is useful for running various fs tests. This patch set is against bpf-next The main changes for v3: Restructured around struct_op programs Using dynptrs instead of packets Using kfuncs instead of new helpers Selftests now use skel for loading Alessio Balsini (1): fs: Generic function to convert iocb to rw flags Daniel Rosenberg (36): bpf: verifier: Accept dynptr mem as mem in herlpers bpf: Allow NULL buffers in bpf_dynptr_slice(_rw) selftests/bpf: Test allowing NULL buffer in dynptr slice fuse-bpf: Update fuse side uapi fuse-bpf: Add data structures for fuse-bpf fuse-bpf: Prepare for fuse-bpf patch fuse: Add fuse-bpf, a stacked fs extension for FUSE fuse-bpf: Add ioctl interface for /dev/fuse fuse-bpf: Don't support export_operations fuse-bpf: Add support for access fuse-bpf: Partially add mapping support fuse-bpf: Add lseek support fuse-bpf: Add support for fallocate fuse-bpf: Support file/dir open/close fuse-bpf: Support mknod/unlink/mkdir/rmdir fuse-bpf: Add support for read/write iter fuse-bpf: support readdir fuse-bpf: Add support for sync operations fuse-bpf: Add Rename support fuse-bpf: Add attr support fuse-bpf: Add support for FUSE_COPY_FILE_RANGE fuse-bpf: Add xattr support fuse-bpf: Add symlink/link support fuse-bpf: allow mounting with no userspace daemon bpf: Increase struct_op limits fuse-bpf: Add fuse-bpf constants WIP: bpf: Add fuse_ops struct_op programs fuse-bpf: Export Functions fuse: Provide registration functions for fuse-bpf fuse-bpf: Set fuse_ops at mount or lookup time fuse-bpf: Call bpf for pre/post filters fuse-bpf: Add userspace pre/post filters WIP: fuse-bpf: add error_out tools: Add FUSE, update bpf includes fuse-bpf: Add selftests fuse: Provide easy way to test fuse struct_op call Documentation/bpf/kfuncs.rst | 23 +- fs/fuse/Kconfig | 8 + fs/fuse/Makefile | 1 + fs/fuse/backing.c | 4241 +++++++++++++++++ fs/fuse/bpf_register.c | 209 + fs/fuse/control.c | 2 +- fs/fuse/dev.c | 85 +- fs/fuse/dir.c | 344 +- fs/fuse/file.c | 63 +- fs/fuse/fuse_i.h | 495 +- fs/fuse/inode.c | 360 +- fs/fuse/ioctl.c | 2 +- fs/fuse/readdir.c | 5 + fs/fuse/xattr.c | 18 + fs/overlayfs/file.c | 23 +- include/linux/bpf.h | 2 +- include/linux/bpf_fuse.h | 283 ++ include/linux/fs.h | 5 + include/uapi/linux/bpf.h | 12 + include/uapi/linux/fuse.h | 41 + kernel/bpf/Makefile | 4 + kernel/bpf/bpf_fuse.c | 241 + kernel/bpf/bpf_struct_ops.c | 6 +- kernel/bpf/bpf_struct_ops_types.h | 4 + kernel/bpf/btf.c | 1 + kernel/bpf/helpers.c | 32 +- kernel/bpf/verifier.c | 32 + tools/include/uapi/linux/bpf.h | 12 + tools/include/uapi/linux/fuse.h | 1135 +++++ .../testing/selftests/bpf/prog_tests/dynptr.c | 1 + .../selftests/bpf/progs/dynptr_success.c | 21 + .../selftests/filesystems/fuse/.gitignore | 2 + .../selftests/filesystems/fuse/Makefile | 189 + .../testing/selftests/filesystems/fuse/OWNERS | 2 + .../selftests/filesystems/fuse/bpf_common.h | 51 + .../selftests/filesystems/fuse/bpf_loader.c | 597 +++ .../testing/selftests/filesystems/fuse/fd.txt | 21 + .../selftests/filesystems/fuse/fd_bpf.bpf.c | 397 ++ .../selftests/filesystems/fuse/fuse_daemon.c | 300 ++ .../selftests/filesystems/fuse/fuse_test.c | 2412 ++++++++++ .../filesystems/fuse/struct_op_test.bpf.c | 642 +++ .../selftests/filesystems/fuse/test.bpf.c | 996 ++++ .../filesystems/fuse/test_framework.h | 172 + .../selftests/filesystems/fuse/test_fuse.h | 494 ++ 44 files changed, 13755 insertions(+), 231 deletions(-) create mode 100644 fs/fuse/backing.c create mode 100644 fs/fuse/bpf_register.c create mode 100644 include/linux/bpf_fuse.h create mode 100644 kernel/bpf/bpf_fuse.c create mode 100644 tools/include/uapi/linux/fuse.h create mode 100644 tools/testing/selftests/filesystems/fuse/.gitignore create mode 100644 tools/testing/selftests/filesystems/fuse/Makefile create mode 100644 tools/testing/selftests/filesystems/fuse/OWNERS create mode 100644 tools/testing/selftests/filesystems/fuse/bpf_common.h create mode 100644 tools/testing/selftests/filesystems/fuse/bpf_loader.c create mode 100644 tools/testing/selftests/filesystems/fuse/fd.txt create mode 100644 tools/testing/selftests/filesystems/fuse/fd_bpf.bpf.c create mode 100644 tools/testing/selftests/filesystems/fuse/fuse_daemon.c create mode 100644 tools/testing/selftests/filesystems/fuse/fuse_test.c create mode 100644 tools/testing/selftests/filesystems/fuse/struct_op_test.bpf.c create mode 100644 tools/testing/selftests/filesystems/fuse/test.bpf.c create mode 100644 tools/testing/selftests/filesystems/fuse/test_framework.h create mode 100644 tools/testing/selftests/filesystems/fuse/test_fuse.h base-commit: 49859de997c3115b85544bce6b6ceab60a7fabc4 -- 2.40.0.634.g4ca3ef3211-goog