Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
From:   Florian Weimer <fweimer@redhat.com>
To:     Miklos Szeredi <miklos@szeredi.hu>
Cc:     Miklos Szeredi <mszeredi@redhat.com>,
        Christian Brauner <brauner@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-api@vger.kernel.org, linux-man@vger.kernel.org,
        linux-security-module@vger.kernel.org, Karel Zak <kzak@redhat.com>,
        Ian Kent <raven@themaw.net>,
        David Howells <dhowells@redhat.com>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Christian Brauner <christian@brauner.io>,
        Amir Goldstein <amir73il@gmail.com>
Subject: Re: [RFC PATCH 2/3] add statmnt(2) syscall
References: <20230913152238.905247-1-mszeredi@redhat.com>
        <20230913152238.905247-3-mszeredi@redhat.com>
        <20230914-salzig-manifest-f6c3adb1b7b4@brauner>
        <CAJfpegs-sDk0++FjSZ_RuW5m-z3BTBQdu4T9QPtWwmSZ1_4Yvw@mail.gmail.com>
        <20230914-lockmittel-verknallen-d1a18d76ba44@brauner>
        <CAJfpegt-VPZP3ou-TMQFs1Xupj_iWA5ttC2UUFKh3E43EyCOQQ@mail.gmail.com>
        <20230918-grafik-zutreffen-995b321017ae@brauner>
        <CAOssrKfS79=+F0h=XPzJX2E6taxAPmEJEYPi4VBNQjgRR5ujqw@mail.gmail.com>
        <871qeloxj0.fsf@oldenburg.str.redhat.com>
        <CAJfpegupTzdG4=UwguL02c08ZaoX+UK7+=9XQ9D1G4wLMxuqFA@mail.gmail.com>
        <87wmwdnhj1.fsf@oldenburg.str.redhat.com>
        <CAJfpegvKECAFNhWYKfGbSWVX8pycQxsHnCr6KSqrQrR+u77yAg@mail.gmail.com>
Date:   Tue, 26 Sep 2023 16:39:03 +0200
In-Reply-To: <CAJfpegvKECAFNhWYKfGbSWVX8pycQxsHnCr6KSqrQrR+u77yAg@mail.gmail.com>
        (Miklos Szeredi's message of "Tue, 26 Sep 2023 16:33:50 +0200")
Message-ID: <87bkdpngmw.fsf@oldenburg.str.redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

* Miklos Szeredi:

> On Tue, 26 Sept 2023 at 16:19, Florian Weimer <fweimer@redhat.com> wrote:
>
>> getdents gets away with this buffer size because applications can copy
>> out all the data from struct dirent if they need long-term storage.
>> They have to do that because the usual readdir interface overwrites the
>> buffer, potentially at the next readdir call.  This means the buffer
>> size does not introduce an amount of memory fragmention that is
>> dependent on the directory size.
>>
>> With an opaque, pointer-carrying struct, copying out the data is not
>> possible in a generic fashion.  Only the parts that the application
>> knows about can be copied out.  So I think it's desirable to have a
>> fairly exact allocation.
>
> Okay, so let's add a 'size' field to the struct, which is set to the
> size used (as opposed to the size of the buffer).   That should solve
> copying without wasting a single byte of memory.

That would be helpful.

> Otherwise the format is fully copyable, since the strings are denoted
> with an offset, which doesn't change after the buffer is copied.

I missed the development in that direction.  Yes, offsets would work
nicely in this context.  They help with compat syscalls, too.

If the buffer is relocatable like that, we can even try first with a
reasonably sized on-stack buffer and create an exactly-sized heap
allocation from that.

Thanks,
Florian