Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sun, 06 Jun 2021 21:57:27 +1000
From:   Nicholas Piggin <npiggin@gmail.com>
Subject: Re: [PATCH v4 00/15] Add futex2 syscalls
To:     =?iso-8859-1?b?QW5kcuk=?= Almeida <andrealmeid@collabora.com>,
        Andrey Semashev <andrey.semashev@gmail.com>
Cc:     acme@kernel.org, Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        corbet@lwn.net, Davidlohr Bueso <dave@stgolabs.net>,
        Darren Hart <dvhart@infradead.org>, fweimer@redhat.com,
        joel@joelfernandes.org, kernel@collabora.com,
        krisman@collabora.com, libc-alpha@sourceware.org,
        linux-api@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-kselftest@vger.kernel.org, malteskarupke@fastmail.fm,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        pgriffais@valvesoftware.com, Peter Oskolkov <posk@posk.io>,
        Steven Rostedt <rostedt@goodmis.org>, shuah@kernel.org,
        Thomas Gleixner <tglx@linutronix.de>, z.figura12@gmail.com
References: <20210603195924.361327-1-andrealmeid@collabora.com>
        <1622799088.hsuspipe84.astroid@bobo.none>
        <fb85fb20-5421-b095-e68b-955afa105467@collabora.com>
        <1622853816.mokf23xgnt.astroid@bobo.none>
        <6d8e3bb4-0cef-b991-9a16-1f03d10f131d@gmail.com>
In-Reply-To: <6d8e3bb4-0cef-b991-9a16-1f03d10f131d@gmail.com>
MIME-Version: 1.0
Message-Id: <1622980258.cfsuodze38.astroid@bobo.none>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Excerpts from Andrey Semashev's message of June 5, 2021 6:56 pm:
> On 6/5/21 4:09 AM, Nicholas Piggin wrote:
>> Excerpts from Andr=C3=A9 Almeida's message of June 5, 2021 6:01 am:
>>> =C3=80s 08:36 de 04/06/21, Nicholas Piggin escreveu:
>>=20
>>>> I'll be burned at the stake for suggesting it but it would be great if
>>>> we could use file descriptors. At least for the shared futex, maybe
>>>> private could use a per-process futex allocator. It solves all of the
>>>> above, although I'm sure has many of its own problem. It may not play
>>>> so nicely with the pthread mutex API because of the whole static
>>>> initialiser problem, but the first futex proposal did use fds. But it'=
s
>>>> an example of an alternate API.
>>>>
>>>
>>> FDs and futex doesn't play well, because for futex_wait() you need to
>>> tell the kernel the expected value in the futex address to avoid
>>> sleeping in a free lock. FD operations (poll, select) don't have this
>>> `value` argument, so they could sleep forever, but I'm not sure if you
>>> had taken this in consideration.
>>=20
>> I had. The futex wait API would take a fd additional. The only
>> difference is the waitqueue that is used when a sleep or wake is
>> required is derived from the fd, not from an address.
>>=20
>> I think the bigger sticking points would be if it's too heavyweight an
>> object to use (which could be somewhat mitigated with a simpler ida
>> allocator although that's difficult to do with shared), and whether libc
>> could sanely use them due to the static initialiser problem of pthread
>> mutexes.
>=20
> The static initialization feature is not the only benefit of the current=20
> futex design, and probably not the most important one. You can work=20
> around the static initialization in userspace, e.g. by initializing fd=20
> to an invalid value and creating a valid fd upon the first use. Although=20
> that would still incur a performance penalty and add a new source of=20
> failure.

Sounds like a serious problem, but maybe it isn't. On the other hand,
maybe we don't have to support pthread mutexes as they are anyway=20
because futex already does that fairly well.

> What is more important is that waiting on fd always requires a kernel=20
> call. This will be terrible for performance of uncontended locks, which=20
> is the majority of time.

No. As I said just before, it would be the same except the waitqueue is=20
derived from fd rather than address.

>=20
> Another important point is that a futex that is not being waited on=20
> consumes zero kernel resources while fd is a limited resource even when=20
> not used. You can have millions futexes in userspace and you are=20
> guaranteed not to exhaust any limit as long as you have memory. That is=20
> an important feature, and the current userspace is relying on it by=20
> assuming that creating mutexes and condition variables is cheap.

Is it an important feture? Would 1 byte of kernel memory per uncontended=20
futex be okay? 10? 100?

I do see it's very nice the current design that requires no=20
initialization for uncontended, I'm just asking questions to get an idea=20
of what constraints we're working with. We have a pretty good API=20
already which can support unlimited uncontended futexes, so I'm=20
wondering do we really need another very very similar API that doesn't
fix the really difficult problems of the existing one?

Thanks,
Nick

> Having futex fd would be useful in some cases to be able to integrate=20
> futexes with IO. I did have use cases where I would have liked to have=20
> FUTEX_FD in the past. These cases arise when you already have a thread=20
> that operates on fds and you want to avoid having a separate thread that=20
> blocks on futexes in a similar fashion. But, IMO, that should be an=20
> optional opt-in feature. By far, not every futex needs to have an fd.=20
> For just waiting on multiple futexes, the native support that futex2=20
> provides is superior.
>=20
> PS: I'm not asking FUTEX_FD to be implemented as part of futex2 API.=20
> futex2 would be great even without it.