Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <7e8e4b1b1e282a11757252dbe02445171f6f9675.camel@themaw.net>
Subject: Re: [PATCH 00/17] VFS: Filesystem information and notifications
 [ver #17]
From:   Ian Kent <raven@themaw.net>
To:     Miklos Szeredi <miklos@szeredi.hu>,
        Steven Whitehouse <swhiteho@redhat.com>
Cc:     David Howells <dhowells@redhat.com>,
        Christian Brauner <christian.brauner@ubuntu.com>,
        James Bottomley <James.Bottomley@hansenpartnership.com>,
        Miklos Szeredi <mszeredi@redhat.com>,
        viro <viro@zeniv.linux.org.uk>,
        Christian Brauner <christian@brauner.io>,
        Jann Horn <jannh@google.com>,
        "Darrick J. Wong" <darrick.wong@oracle.com>,
        Linux API <linux-api@vger.kernel.org>,
        linux-fsdevel <linux-fsdevel@vger.kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Tue, 03 Mar 2020 19:09:31 +0800
In-Reply-To: <CAJfpegsW5S3dRhhfGyAnhLEDjBxMQRBda5fsnXQ+=S=4YR0MCA@mail.gmail.com>
References: <158230810644.2185128.16726948836367716086.stgit@warthog.procyon.org.uk>
         <1582316494.3376.45.camel@HansenPartnership.com>       
 <CAOssrKehjnTwbc6A1VagM5hG_32hy3mXZenx_PdGgcUGxYOaLQ@mail.gmail.com>   
 <1582556135.3384.4.camel@HansenPartnership.com>        
 <CAJfpegsk6BsVhUgHNwJgZrqcNP66wS0fhCXo_2sLt__goYGPWg@mail.gmail.com>   
 <a657a80e-8913-d1f3-0ffe-d582f5cb9aa2@redhat.com>      
 <1582644535.3361.8.camel@HansenPartnership.com>        
 <20200228155244.k4h4hz3dqhl7q7ks@wittgenstein> 
 <107666.1582907766@warthog.procyon.org.uk>     
 <CAJfpegu0qHBZ7iK=R4ajmmHC4g=Yz56otpKMy5w-y0UxJ1zO+Q@mail.gmail.com>   
 <0403cda7345e34c800eec8e2870a1917a8c07e5c.camel@themaw.net>    
 <CAJfpegtu6VqhPdcudu79TX3e=_NZaJ+Md3harBGV7Bg_-+fR8Q@mail.gmail.com>   
 <1509948.1583226773@warthog.procyon.org.uk>    
 <CAJfpegtOwyaWpNfjomRVOt8NKqT94O5n4-LOHTR7YZT9fadVHA@mail.gmail.com>   
 <CAJfpegtemv64mpmTRT6ViHmsWq4nNE4KQvuHkNCYozRU7dQd8Q@mail.gmail.com>   
 <06d2dbf0-4580-3812-bb14-34c6aa615747@redhat.com>      
 <CAJfpegsW5S3dRhhfGyAnhLEDjBxMQRBda5fsnXQ+=S=4YR0MCA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.32.5 (3.32.5-1.fc30) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 2020-03-03 at 11:32 +0100, Miklos Szeredi wrote:
> On Tue, Mar 3, 2020 at 11:22 AM Steven Whitehouse <
> swhiteho@redhat.com> wrote:
> > Hi,
> > 
> > On 03/03/2020 09:48, Miklos Szeredi wrote:
> > > On Tue, Mar 3, 2020 at 10:26 AM Miklos Szeredi <miklos@szeredi.hu
> > > > wrote:
> > > > On Tue, Mar 3, 2020 at 10:13 AM David Howells <
> > > > dhowells@redhat.com> wrote:
> > > > > Miklos Szeredi <miklos@szeredi.hu> wrote:
> > > > > 
> > > > > > I'm doing a patch.   Let's see how it fares in the face of
> > > > > > all these
> > > > > > preconceptions.
> > > > > Don't forget the efficiency criterion.  One reason for going
> > > > > with fsinfo(2) is
> > > > > that scanning /proc/mounts when there are a lot of mounts in
> > > > > the system is
> > > > > slow (not to mention the global lock that is held during the
> > > > > read).
> > > BTW, I do feel that there's room for improvement in userspace
> > > code as
> > > well.  Even quite big mount table could be scanned for *changes*
> > > very
> > > efficiently.  l.e. cache previous contents of
> > > /proc/self/mountinfo and
> > > compare with new contents, line-by-line.  Only need to parse the
> > > changed/added/removed lines.
> > > 
> > > Also it would be pretty easy to throttle the number of updates so
> > > systemd et al. wouldn't hog the system with unnecessary
> > > processing.
> > > 
> > > Thanks,
> > > Miklos
> > > 
> > 
> > At least having patches to compare would allow us to look at the
> > performance here and gain some numbers, which would be helpful to
> > frame
> > the discussions. However I'm not seeing how it would be easy to
> > throttle
> > updates... they occur at whatever rate they are generated and this
> > can
> > be fairly high. Also I'm not sure that I follow how the
> > notifications
> > and the dumping of the whole table are synchronized in this case,
> > either.
> 
> What I meant is optimizing current userspace without additional
> kernel
> infrastructure.   Since currently there's only the monolithic
> /proc/self/mountinfo, it's reasonable that if the rate of change is
> very high, then we don't re-read this table on every change, only
> within a reasonable time limit (e.g. 1s) to provide timely updates.
> Re-reading the table on every change would (does?) slow down the
> system so that the actual updates would even be slower, so throttling
> in this case very much  makes sense.

Optimizing user space is a huge task.

For example, consider this (which is related to a recent upstream
discussion I had):
https://blog.janestreet.com/troubleshooting-systemd-with-systemtap/

Working on improving libmount is really useful but that can't help
with inherently inefficient approaches to keeping info. current
which is actually needed at times.

> 
> Once we have per-mount information from the kernel, throttling
> updates
> probably does not make sense.

And can easily lead to application problems. Throttling will
lead to an inability to have up to date information upon which
application decisions are made.

I don't think it's a viable solution to the separate problem
of a large number of notifications either.

Ian