Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
User-Agent: Cyrus-JMAP/3.5.0-alpha0-1291-gc66fc0a3a2-fm-20210913.001-gc66fc0a3
Mime-Version: 1.0
Message-Id: <f6fdecfe-963d-4669-ae05-1d7192467a19@www.fastmail.com>
In-Reply-To: <YUDlzxLjNsW+oYGC@hirez.programming.kicks-ass.net>
References: <20210908184905.163787-1-posk@google.com>
 <20210908184905.163787-3-posk@google.com>
 <CAG48ez2LyLNkH4iVbeKJUuH=oh57WECkKYCW+G9mtheoh7Fsvg@mail.gmail.com>
 <CAPNVh5eaW7r_Nv-wHEyxQiFkXngmONwPyZSFvtTEhk3TxJ+iMA@mail.gmail.com>
 <CAG48ez0mgCXpXnqAUsa0TcFBPjrid-74Gj=xG8HZqj2n+OPoKw@mail.gmail.com>
 <d656e605-4f89-4ea2-8baf-f7786f0630d9@www.fastmail.com>
 <YUDlzxLjNsW+oYGC@hirez.programming.kicks-ass.net>
Date:   Tue, 14 Sep 2021 11:40:01 -0700
From:   "Andy Lutomirski" <luto@kernel.org>
To:     "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc:     "Jann Horn" <jannh@google.com>, "Peter Oskolkov" <posk@google.com>,
        "Peter Oskolkov" <posk@posk.io>, "Ingo Molnar" <mingo@redhat.com>,
        "Thomas Gleixner" <tglx@linutronix.de>,
        "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
        "Linux API" <linux-api@vger.kernel.org>,
        "Paul Turner" <pjt@google.com>, "Ben Segall" <bsegall@google.com>,
        "Andrei Vagin" <avagin@google.com>,
        "Thierry Delisle" <tdelisle@uwaterloo.ca>
Subject: Re: [PATCH 2/4 v0.5] sched/umcg: RFC: add userspace atomic helpers
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk


On Tue, Sep 14, 2021, at 11:11 AM, Peter Zijlstra wrote:
> On Tue, Sep 14, 2021 at 09:52:08AM -0700, Andy Lutomirski wrote:
> > With a custom mapping, you don=E2=80=99t need to pin pages at all, I=
 think.
> > As long as you can reconstruct the contents of the shared page and
> > you=E2=80=99re willing to do some slightly careful synchronization, =
you can
> > detect that the page is missing when you try to update it and skip t=
he
> > update. The vm_ops->fault handler can repopulate the page the next
> > time it=E2=80=99s accessed.
>=20
> The point is that the moment we know we need to do this user-poke, is
> schedule(), which could be called while holding mmap_sem (it being a
> preemptable lock). Which means we cannot go and do faults.

That=E2=80=99s fine. The page would be in one or two states: present and=
 writable by kernel or completely gone. If its present, the scheduler wr=
ites it. If it=E2=80=99s gone, the scheduler skips the write and the nex=
t fault fills it in.

>=20
> > All that being said, I feel like I=E2=80=99m missing something. The =
point of
> > this is to send what the old M:N folks called =E2=80=9Cscheduler act=
ivations=E2=80=9D,
> > right?  Wouldn=E2=80=99t it be more efficient to explicitly wake som=
ething
> > blockable/pollable and write the message into a more efficient data
> > structure?  Polling one page per task from userspace seems like it
> > will have inherently high latency due to the polling interval and wi=
ll
> > also have very poor locality.  Or am I missing something?
>=20
> The idea was to link the user structures together in a (single) linked
> list. The server structure gets a list of all the blocked tasks. This
> avoids having to a full N iteration (like Java, they're talking stupid
> number of N).
>=20
> Polling should not happen, once we run out of runnable tasks, the serv=
er
> task gets ran again and it can instantly pick up all the blocked
> notifications.
>=20

How does the server task know when to read the linked list?  And what=E2=
=80=99s wrong with a ring buffer or a syscall?