Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <79375b71-db2e-3e66-346b-254c90d915e2@cslab.ece.ntua.gr> <20230719211631.890995-1-axelrasmussen@google.com>
In-Reply-To: <20230719211631.890995-1-axelrasmussen@google.com>
From:   Axel Rasmussen <axelrasmussen@google.com>
Date:   Wed, 19 Jul 2023 14:54:21 -0700
Message-ID: <CAJHvVcj+Sc41mfercqxBii5cqRBEgZxNix2R1YMi04K-5nBh8w@mail.gmail.com>
Subject: Re: Using userfaultfd with KVM's async page fault handling causes
 processes to hung waiting for mmap_lock to be released
To:     Dimitris Siakavaras <jimsiak@cslab.ece.ntua.gr>
Cc:     viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
        linux-kernel@vger.kernel.org, Peter Xu <peterx@redhat.com>,
        linux-mm@kvack.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Jul 19, 2023 at 2:16=E2=80=AFPM Axel Rasmussen <axelrasmussen@googl=
e.com> wrote:
>
> Thanks for the detailed report Dimitris! I've CCed the MM mailing list an=
d some
> folks who work on userfaultfd.

Apologies, I should have quoted the original message for the others I
added to CC: https://lore.kernel.org/lkml/79375b71-db2e-3e66-346b-254c90d91=
5e2@cslab.ece.ntua.gr/T/#u

>
> I took a look at this today, but I haven't quite come up with a solution.
>
> I thought it might be as easy as changing userfaultfd_release() to set re=
leased
> *after* taking the lock. But no such luck, the ordering is what it is to =
deal
> with another subtle case:
>
>
>         WRITE_ONCE(ctx->released, true);
>
>         if (!mmget_not_zero(mm))
>                 goto wakeup;
>
>         /*
>          * Flush page faults out of all CPUs. NOTE: all page faults
>          * must be retried without returning VM_FAULT_SIGBUS if
>          * userfaultfd_ctx_get() succeeds but vma->vma_userfault_ctx
>          * changes while handle_userfault released the mmap_lock. So
>          * it's critical that released is set to true (above), before
>          * taking the mmap_lock for writing.
>          */
>         mmap_write_lock(mm);
>
> I think perhaps the right thing to do is to have handle_userfault() relea=
se
> mmap_lock when it returns VM_FAULT_NOPAGE, and to have GUP deal with that
> appropriately? But, some investigation is required to be sure that's okay=
 to do
> in the other non-GUP ways we can end up in handle_userfault().