Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210128224819.2651899-1-axelrasmussen@google.com>
 <20210128224819.2651899-9-axelrasmussen@google.com> <20210201200654.GI260413@xz-x1>
In-Reply-To: <20210201200654.GI260413@xz-x1>
From:   Axel Rasmussen <axelrasmussen@google.com>
Date:   Tue, 2 Feb 2021 15:07:14 -0800
Message-ID: <CAJHvVci5r5L-ga4QS2WoZ2AqG3HJhBeg3TH6F6nAtu-PmK6+1g@mail.gmail.com>
Subject: Re: [PATCH v3 8/9] userfaultfd: update documentation to describe
 minor fault handling
To:     Peter Xu <peterx@redhat.com>
Cc:     Alexander Viro <viro@zeniv.linux.org.uk>,
        Alexey Dobriyan <adobriyan@gmail.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Chinwen Chang <chinwen.chang@mediatek.com>,
        Huang Ying <ying.huang@intel.com>,
        Ingo Molnar <mingo@redhat.com>, Jann Horn <jannh@google.com>,
        Jerome Glisse <jglisse@redhat.com>,
        Lokesh Gidra <lokeshgidra@google.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Michael Ellerman <mpe@ellerman.id.au>,
        =?UTF-8?Q?Michal_Koutn=C3=BD?= <mkoutny@suse.com>,
        Michel Lespinasse <walken@google.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Mike Rapoport <rppt@linux.vnet.ibm.com>,
        Nicholas Piggin <npiggin@gmail.com>, Shaohua Li <shli@fb.com>,
        Shawn Anastasio <shawn@anastas.io>,
        Steven Rostedt <rostedt@goodmis.org>,
        Steven Price <steven.price@arm.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-fsdevel@vger.kernel.org, Linux MM <linux-mm@kvack.org>,
        Adam Ruprecht <ruprecht@google.com>,
        Cannon Matthews <cannonmatthews@google.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Oliver Upton <oupton@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Feb 1, 2021 at 12:07 PM Peter Xu <peterx@redhat.com> wrote:
>
> On Thu, Jan 28, 2021 at 02:48:18PM -0800, Axel Rasmussen wrote:
> > Reword / reorganize things a little bit into "lists", so new features /
> > modes / ioctls can sort of just be appended.
> >
> > Describe how UFFDIO_REGISTER_MODE_MINOR and UFFDIO_CONTINUE can be used
> > to intercept and resolve minor faults. Make it clear that COPY and
> > ZEROPAGE are used for MISSING faults, whereas CONTINUE is used for MINOR
> > faults.
>
> Bare with me since I'm not native speaker.. but I'm pointing out things that
> reads odd to me.  Feel free to argue. :)

No worries, that is true for many people in the community. I'm happy
to reword to make things as clear as possible. :)

>
> [...]
>
> > +Resolving Userfaults
> > +--------------------
> > +
> > +There are three basic ways to resolve userfaults:
> > +
> > +- ``UFFDIO_COPY`` atomically copies some existing page contents from
> > +  userspace.
> > +
> > +- ``UFFDIO_ZEROPAGE`` atomically zeros the new page.
> > +
> > +- ``UFFDIO_CONTINUE`` maps an existing, previously-populated page.
> > +
> > +These operations are atomic in the sense that they guarantee nothing can
> > +see a half-populated page, since readers will keep userfaulting until the
> > +operation has finished.
> > +
> > +By default, these wake up userfaults blocked on the range in question.
> > +They support a ``UFFDIO_*_MODE_DONTWAKE`` ``mode`` flag, which indicates
> > +that waking will be done separately at some later time.
> > +
> > +Which of these are used depends on the kind of fault:
>
> Maybe:
>
> "We should choose the ioctl depending on the kind of the page fault, and what
>  we'd like to do with it:"
>
> ?
>
> > +
> > +- For ``UFFDIO_REGISTER_MODE_MISSING`` faults, a new page has to be
> > +  provided. This can be done with either ``UFFDIO_COPY`` or
>
> UFFDIO_ZEROPAGE does not need a new page.
>
> > +  ``UFFDIO_ZEROPAGE``. The default (non-userfaultfd) behavior would be to
> > +  provide a zero page, but in userfaultfd this is left up to userspace.
>
> "By default, kernel will provide a zero page for a missing fault.  With
>  userfaultfd, the userspace could decide which content to provide before the
>  faulted thread continues." ?
>
> > +
> > +- For ``UFFDIO_REGISTER_MODE_MINOR`` faults, an existing page already
>
> "page cache existed"?
>
> > +  exists. Userspace needs to ensure its contents are correct (if it needs
> > +  to be modified, by writing directly to the non-userfaultfd-registered
> > +  side of shared memory), and then issue ``UFFDIO_CONTINUE`` to resolve
> > +  the fault.
>
> "... Userspace can modify the page content before asking the faulted thread to
>  continue the fault with UFFDIO_CONTINUE ioctl." ?

I agree with all the comments; these areas can be clarified. I didn't
take the suggestions exactly as-is, but I did reword these parts in my
v4. Let me know if further changes would be useful.

>
> --
> Peter Xu
>