Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
MIME-Version: 1.0
References: <6ba14287-336d-cdcd-0d39-680f288ca776@ddn.com>
In-Reply-To: <6ba14287-336d-cdcd-0d39-680f288ca776@ddn.com>
From:   Miklos Szeredi <miklos@szeredi.hu>
Date:   Fri, 22 Apr 2022 14:25:12 +0200
Message-ID: <CAJfpegt=KZJKExpxPgGXoBEzWpzepL9cyaqS=dwW5AN6y2up_Q@mail.gmail.com>
Subject: Re: RFC fuse waitq latency
To:     Bernd Schubert <bschubert@ddn.com>
Cc:     Linux-FSDevel <linux-fsdevel@vger.kernel.org>,
        Linux Kernel <linux-kernel@vger.kernel.org>,
        Dharmendra Singh <dsingh@ddn.com>,
        Boaz Harrosh <boaz@plexistor.com>,
        Sagi Manole <sagim@netapp.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, 28 Mar 2022 at 15:21, Bernd Schubert <bschubert@ddn.com> wrote:
>
> I would like to discuss the user thread wake up latency in
> fuse_dev_do_read(). Profiling fuse shows there is room for improvement
> regarding memory copies and splice. The basic profiling with flame graphs
> didn't reveal, though, why fuse is so much
> slower (with an overlay file system) than just accessing the underlying
> file system directly and also didn't reveal why a single threaded fuse
> uses less than 100% cpu, with the application on top of use also using
> less than 100% cpu (simple bonnie++ runs with 1B files).
> So I started to suspect the wait queues and indeed, keeping the thread
> that reads the fuse device for work running for some time gives quite
> some improvements.

Might be related: I experimented with wake_up_sync() that didn't meet
my expectations.  See this thread:

https://lore.kernel.org/all/1638780405-38026-1-git-send-email-quic_pragalla@quicinc.com/#r

Possibly fuse needs some wake up tweaks due to its special scheduling
requirements.

Thanks,
Miklos