Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <YnEeuw6fd1A8usjj@miu.piliscsaba.redhat.com> <20220509124815.vb7d2xj5idhb2wq6@wittgenstein>
 <CAJfpegveWaS5pR3O1c_7qLnaEDWwa8oi26x2v_CwDXB_sir1tg@mail.gmail.com>
 <8ab7f51cf18ba62e3f5bfdf5d9933895413f4806.camel@themaw.net> <CAJfpegv3MCHMzur9R+K+yZC3Z_Wmbq3=pQwuQ=+kQSrihg0c9g@mail.gmail.com>
In-Reply-To: <CAJfpegv3MCHMzur9R+K+yZC3Z_Wmbq3=pQwuQ=+kQSrihg0c9g@mail.gmail.com>
From:   Miklos Szeredi <miklos@szeredi.hu>
Date:   Tue, 10 May 2022 10:07:55 +0200
Message-ID: <CAJfpegutUQOTa71NTy_iaz5Kus2ma16ALqrHtvV2+cQ0tFiaxA@mail.gmail.com>
Subject: Re: [RFC PATCH] getting misc stats/attributes via xattr API
To:     Ian Kent <raven@themaw.net>
Cc:     Christian Brauner <brauner@kernel.org>,
        linux-fsdevel@vger.kernel.org, Dave Chinner <david@fromorbit.com>,
        "Theodore Ts'o" <tytso@mit.edu>, Karel Zak <kzak@redhat.com>,
        Greg KH <gregkh@linuxfoundation.org>,
        linux-kernel@vger.kernel.org,
        Linux API <linux-api@vger.kernel.org>,
        linux-man <linux-man@vger.kernel.org>,
        LSM <linux-security-module@vger.kernel.org>,
        David Howells <dhowells@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Christian Brauner <christian@brauner.io>,
        Amir Goldstein <amir73il@gmail.com>,
        James Bottomley <James.Bottomley@hansenpartnership.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, 10 May 2022 at 10:06, Miklos Szeredi <miklos@szeredi.hu> wrote:
>
> On Tue, 10 May 2022 at 06:27, Ian Kent <raven@themaw.net> wrote:
>
> > > Was there ever a test patch for systemd using fsinfo(2)?  I think
> > > not.
> >
> > Mmm ... I'm hurt you didn't pay any attention to what I did on this
> > during the original fsinfo() discussions.
>
> I can't find anything related to this in my mailbox.  Maybe you
> mentioned it at some point, but I have not been involved with the
> actual systemd changes.  So not meant to belittle your work at all.
>
> > > Until systemd people start to reengineer the mount handing to allow
> > > for retrieving a single mount instead of the complete mount table we
> > > will never know where the performance bottleneck lies.
> >
> > We didn't need the systemd people to do this only review and contribute
> > to the pr for the change and eventually merge it.
> >
> > What I did on this showed that using fsinfo() allone about halved the
> > CPU overhead (from around 4 processes using about 80%) and once the
> > mount notifications was added too it went down to well under 10% per
> > process. The problem here was systemd is quite good at servicing events
> > and reducing event processing overhead meant more events would then be
> > processed. Utilizing the mount notifications queueing was the key to
> > improving this and that was what I was about to work on at the end.
> >
> > But everything stopped before the work was complete.
> >
> > As I said above it's been a long time since I looked at the systemd
> > work and it definitely was a WIP so "what you see is what you get"
> > at https://github.com/raven-au/systemd/commits/. It looks like the
> > place to look to get some idea of what was being done is branch
> > notifications-devel or notifications-rfc-pr. Also note that this
> > uses the libmount fsinfo() infrastrucure that was done by Karal Zak
> > (and a tiny bit by me) at the time.
>
> Looks great as a first step.
>
> What do you mean by "Utilizing the mount notifications queueing"?
>
> Do you mean batching of notifications?   I think that's a very
> important issue: processing each individual notifcation may not make
> sense when there are lots of them.  For example, doing ceate
> mount+remote mount in a loop a million times will result in two

s/remote/remove/

> million notification messages (with high likelyhood of queue
> overflow), but in the end the mount table will end up being the
> same...
>
> Thanks,
> Miklos