Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 14 Apr 2020 17:45:16 -0400
From:   Peter Xu <peterx@redhat.com>
To:     Brian Geffon <bgeffon@google.com>
Cc:     Andrea Arcangeli <aarcange@redhat.com>,
        linux-mm <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Sonny Rao <sonnyrao@google.com>
Subject: Re: Userfaultfd doesn't seem to break out of poll on fd close
Message-ID: <20200414214516.GA182757@xz-x1>
References: <CADyq12wPW69ovpW4akDY5PGBbrvnwsLO86=sSKTU4CB3dNwG3Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CADyq12wPW69ovpW4akDY5PGBbrvnwsLO86=sSKTU4CB3dNwG3Q@mail.gmail.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sun, Apr 12, 2020 at 01:10:40PM -0700, Brian Geffon wrote:
> Hi,
> It seems that userfaultfd isn't woken from a poll when the file
> descriptor is closed. It seems that it should be from the code in
> userfault_ctx_release, but it appears that's not actually called
> immediately. I have a simple standalone example that shows this
> behavior. It's straight forward: one thread creates a userfaultfd and
> then closes it after a second thread has entered a poll syscall, some
> abbreviated strace output is below showing this and the code can be
> seen here: https://gist.github.com/bgaff/9a8fbbe8af79c0e18502430d416df77e
> 
> Given that it's probably very common to have a dedicated thread remain
> blocked indefinitely in a poll(2) waiting for faults there must be a
> way to break it out early when it's closed. Am I missing something?

Hi, Brian,

I might be wrong below, just to share my understanding...

IMHO a well-behaved userspace should not close() on a file descriptor
if it's still in use within another thread.  In this case, the poll()
thread is still using the userfaultfd handle, so imo it's cleaner that
the main thread should pthread_join() on the poll() thread before it
closes the handle.  It can be easily achieved by attaching another
eventfd to the struct pollfds array, and write to the eventfd when the
main thread wants to quit so that the poll() will return on the write
to the eventfd.

On the other hand I'm thinking whether we can achieve what you said.
IIUC userfaultfd_release() is only called when the file descriptor
destructs itself.  But shouldn't the poll() take a refcount of that
file descriptor too before waiting?  Not sure userfaultfd_release() is
the place to kick then, because if so, close() will only decrease the
fd refcount from 2->1, and I'm not sure userfaultfd_release() will be
triggered.

Thanks,

-- 
Peter Xu