Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210125153057.3623715-1-balsini@android.com> <20210125153057.3623715-5-balsini@android.com>
 <CAJfpegvL2kOCkbP9bBL8YD-YMFKiSazD3_wet2-+emFafA6y5A@mail.gmail.com>
 <CAOQ4uxjOGx8gZ2biTEb4a54gw5c_aDn+FFkUvRpY+cmgEEh=sA@mail.gmail.com> <YKJVUUUapNSijV38@google.com>
In-Reply-To: <YKJVUUUapNSijV38@google.com>
From:   Amir Goldstein <amir73il@gmail.com>
Date:   Mon, 17 May 2021 16:21:28 +0300
Message-ID: <CAOQ4uxjpHTerNq70gp+GQP26RijzWOJR1pB+9GxVBSdJyjN1mQ@mail.gmail.com>
Subject: Re: [PATCH RESEND V12 4/8] fuse: Passthrough initialization and release
To:     Alessio Balsini <balsini@android.com>
Cc:     Miklos Szeredi <miklos@szeredi.hu>,
        Akilesh Kailash <akailash@google.com>,
        Antonio SJ Musumeci <trapexit@spawn.link>,
        David Anderson <dvander@google.com>,
        Giuseppe Scrivano <gscrivan@redhat.com>,
        Jann Horn <jannh@google.com>, Jens Axboe <axboe@kernel.dk>,
        Martijn Coenen <maco@android.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Paul Lawrence <paullawrence@google.com>,
        Peng Tao <bergwolf@gmail.com>,
        Stefano Duo <duostefano93@gmail.com>,
        Zimuzo Ezeozue <zezeozue@google.com>, wuyan <wu-yan@tcl.com>,
        fuse-devel <fuse-devel@lists.sourceforge.net>,
        kernel-team <kernel-team@android.com>,
        linux-fsdevel <linux-fsdevel@vger.kernel.org>,
        linux-kernel <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

> I have an ugly patch which uses IDR as Miklos asked, but unfortunately
> I'm facing some performance issues due to the locking mechanisms to keep
> guarantee the RCU consistency. I can post the new patch set as an RFC
> soon for the community to take a look.
> At a glance what happens is:
> - the IDR, one for each fuse_conn, contains pointers to "struct
>   fuse_passthrough" containing:
>   - fuse_file *: which is using passthrough,
>   - file *: native file system file target,
>   - cred of the FUSE server,
> - ioctl(PASSTHROUGH_OPEN): updates IDR, requires spinlock:
>   - kmalloc(fuse_passthrough), update file and cred,
>   - ID = idr_alloc(),
>   - return ID,
> - fuse_open reply from FUSE server with passthrough ID: updates IDR,
>   requires spinlock:
>   - pt = idr_find(ID),
>   - update fuse_file with the current fuse_file,
>   - update fuse_file->passthrough_id = ID,
>   - idr_replace(),
> - read/write/mmap: lock-free IDR read:
>   - idr_find(fuse_file::passthrough_id),
>   - forward request to lower file system as for the current FUSE
>     passthrough patches.
> - ioctl(PASSTHROUGH_CLOSE): updates IDR, requires spinlock:
>   - idr_remove();
>   - call_rcu(): tell possible open fuse_file user that the ID is no more
>     valid and kfree() the allocated struct;
> - close(fuse_file): updates IDR, requires spinlock:
>   - ID = fuse_file::passthrough_id
>   - idr_find(ID),
>   - fuse_passthrough::fuse_file = NULL,
>   - idr_replace().
>
> This would be fine if passthrough is activated for a few, big files,
> where the passthrough overhead is dominated by the direct access
> benefits, but if passthrough is enabled for many small files which just
> do one or two read/writes (as what I did for my benchmark of total build
> time for the kernel, where I was passing-through every opened file), the
> overhead becomes a real issue.
>
> If you have any thoughts on how to make this simpler, I'm absolutely
> open to fix this.
>

This IDR namespace usually serves a single process. Right?
It sounds a bit more like a file table, more specifically, like io_file_table.
I may be way off, but this sounds a bit like IORING_REGISTER_FILES.
Is there anything that can be learned from this UAPI?
Maybe even reuse of some of the io_uring file register code?

Thanks,
Amir.