Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
From:   Florian Weimer <fweimer@redhat.com>
To:     Miklos Szeredi <miklos@szeredi.hu>
Cc:     Miklos Szeredi <mszeredi@redhat.com>,
        Christian Brauner <brauner@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-api@vger.kernel.org, linux-man@vger.kernel.org,
        linux-security-module@vger.kernel.org, Karel Zak <kzak@redhat.com>,
        Ian Kent <raven@themaw.net>,
        David Howells <dhowells@redhat.com>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Christian Brauner <christian@brauner.io>,
        Amir Goldstein <amir73il@gmail.com>
Subject: Re: [RFC PATCH 2/3] add statmnt(2) syscall
References: <20230913152238.905247-1-mszeredi@redhat.com>
        <20230913152238.905247-3-mszeredi@redhat.com>
        <20230914-salzig-manifest-f6c3adb1b7b4@brauner>
        <CAJfpegs-sDk0++FjSZ_RuW5m-z3BTBQdu4T9QPtWwmSZ1_4Yvw@mail.gmail.com>
        <20230914-lockmittel-verknallen-d1a18d76ba44@brauner>
        <CAJfpegt-VPZP3ou-TMQFs1Xupj_iWA5ttC2UUFKh3E43EyCOQQ@mail.gmail.com>
        <20230918-grafik-zutreffen-995b321017ae@brauner>
        <CAOssrKfS79=+F0h=XPzJX2E6taxAPmEJEYPi4VBNQjgRR5ujqw@mail.gmail.com>
        <871qeloxj0.fsf@oldenburg.str.redhat.com>
        <CAJfpegupTzdG4=UwguL02c08ZaoX+UK7+=9XQ9D1G4wLMxuqFA@mail.gmail.com>
Date:   Tue, 26 Sep 2023 16:19:46 +0200
In-Reply-To: <CAJfpegupTzdG4=UwguL02c08ZaoX+UK7+=9XQ9D1G4wLMxuqFA@mail.gmail.com>
        (Miklos Szeredi's message of "Tue, 26 Sep 2023 16:06:05 +0200")
Message-ID: <87wmwdnhj1.fsf@oldenburg.str.redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

* Miklos Szeredi:

>> Try-and-resize interfaces can be quite bad for data obtained from the
>> network.
>
> In this particular case it's all local information.

That's good.

>>  If the first call provides the minimum buffer size (like
>> getgroups, but unlike readlink or the glibc *_r interfaces for NSS),
>> this could at least allow us to avoid allocating too much.  In
>> userspace, we cannot reduce the size of the heap allocation without
>> knowing where the pointers are and what they mean.
>
> Does it matter if the heap allocation is say 32k instead of 589bytes?
>  The returned strings are not limited in size, but are quite unlikely
> to be over PATH_MAX.

It matters if the application needs to keep a copy.

> E.g. getdents apparently uses 32k buffers, which is really a tiny
> amount of heap these days, but more than enough for the purpose.  Not
> sure if this is hard coded into libc or if it's the result of some
> heuristic based on available memory, but I don't see why similar
> treatment couldn't be applied to the statmount(2) syscall.

getdents gets away with this buffer size because applications can copy
out all the data from struct dirent if they need long-term storage.
They have to do that because the usual readdir interface overwrites the
buffer, potentially at the next readdir call.  This means the buffer
size does not introduce an amount of memory fragmention that is
dependent on the directory size.

With an opaque, pointer-carrying struct, copying out the data is not
possible in a generic fashion.  Only the parts that the application
knows about can be copied out.  So I think it's desirable to have a
fairly exact allocation.

>> I also don't quite understand the dislike of variable-sized records.
>> Don't getdents, inotify, Netlink all use them?  And I think at least for
>> Netlink, more stuff is added all the time?
>
> What do you mean by variable sized records?

Iterating through d_reclen-sized subojects (for getdents).

Thanks,
Florian