Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 12.22.5.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=12.22.5.234; helo=mail.nvidia.com;
From:   Alistair Popple <apopple@nvidia.com>
To:     Peter Xu <peterx@redhat.com>
CC:     <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Nadav Amit <nadav.amit@gmail.com>,
        Mike Rapoport <rppt@linux.vnet.ibm.com>,
        Hugh Dickins <hughd@google.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        "Kirill A . Shutemov" <kirill@shutemov.name>,
        Jerome Glisse <jglisse@redhat.com>,
        Matthew Wilcox <willy@infradead.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Hildenbrand <david@redhat.com>,
        Andrea Arcangeli <aarcange@redhat.com>
Subject: Re: [PATCH v6 06/23] mm/shmem: Handle uffd-wp special pte in page fault handler
Date:   Thu, 16 Dec 2021 17:30:12 +1100
Message-ID: <8160258.WfFtTLUMm5@nvdebian>
In-Reply-To: <YbrZ+p8MPLdk/Lvi@xz-m1.local>
References: <20211115075522.73795-1-peterx@redhat.com> <6587740.tPqSsf18xI@nvdebian> <YbrZ+p8MPLdk/Lvi@xz-m1.local>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Dec 2021 06:30:20.0281
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ea99ec2e-e120-49ef-8dc4-08d9c05d8860
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT061.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1303
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thursday, 16 December 2021 5:17:30 PM AEDT Peter Xu wrote:
> On Thu, Dec 16, 2021 at 04:56:42PM +1100, Alistair Popple wrote:
> > On Monday, 15 November 2021 6:55:05 PM AEDT Peter Xu wrote:
> > 
> > [...]
> > 
> > > diff --git a/mm/memory.c b/mm/memory.c
> > > index d5966d9e24c3..e8557d43a87d 100644
> > > --- a/mm/memory.c
> > > +++ b/mm/memory.c
> > > @@ -3452,6 +3452,43 @@ static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf)
> > >  	return 0;
> > >  }
> > >  
> > > +static vm_fault_t pte_marker_clear(struct vm_fault *vmf)
> > > +{
> > > +	vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
> > > +				       vmf->address, &vmf->ptl);
> > > +	/*
> > > +	 * Be careful so that we will only recover a special uffd-wp pte into a
> > > +	 * none pte.  Otherwise it means the pte could have changed, so retry.
> > > +	 */
> > > +	if (is_pte_marker(*vmf->pte))
> > > +		pte_clear(vmf->vma->vm_mm, vmf->address, vmf->pte);
> > > +	pte_unmap_unlock(vmf->pte, vmf->ptl);
> > > +	return 0;
> > > +}
> > > +
> > > +/*
> > > + * This is actually a page-missing access, but with uffd-wp special pte
> > > + * installed.  It means this pte was wr-protected before being unmapped.
> > > + */
> > > +static vm_fault_t pte_marker_handle_uffd_wp(struct vm_fault *vmf)
> > > +{
> > > +	/* Careful!  vmf->pte unmapped after return */
> > > +	if (!pte_unmap_same(vmf))
> > 
> > Hasn't vmf->pte already been unmapped by do_swap_page() by the time we get
> > here?
> 
> Great catch, thanks!
> 
> It was needed before with the "swap special pte" version because that was
> handled outside do_swap_page().  After the rebase I forgot to remove it.

No worries, and for what it's worth IMHO this version that handles it inside
do_swap_page() along with all the other "special" cases is much nicer.

> I believe it didn't crash simply because we've got commit 2ca99358671a ("mm:
> clear vmf->pte after pte_unmap_same() returns", 2021-11-06) very recently so it
> just became a safe no-op, so all things will still work.
> 
> I'll drop it.
> 
> > 
> > > +		return 0;
> > > +
> > > +	/*
> > > +	 * Just in case there're leftover special ptes even after the region
> > > +	 * got unregistered - we can simply clear them.  We can also do that
> > > +	 * proactively when e.g. when we do UFFDIO_UNREGISTER upon some uffd-wp
> > > +	 * ranges, but it should be more efficient to be done lazily here.
> > > +	 */
> > > +	if (unlikely(!userfaultfd_wp(vmf->vma) || vma_is_anonymous(vmf->vma)))
> > > +		return pte_marker_clear(vmf);
> > > +
> > > +	/* do_fault() can handle pte markers too like none pte */
> > > +	return do_fault(vmf);
> > > +}
> > > +
> > >  static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
> > >  {
> > >  	swp_entry_t entry = pte_to_swp_entry(vmf->orig_pte);
> > > @@ -3465,8 +3502,11 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
> > >  	if (WARN_ON_ONCE(vma_is_anonymous(vmf->vma) || !marker))
> > >  		return VM_FAULT_SIGBUS;
> > >  
> > > -	/* TODO: handle pte markers */
> > > -	return 0;
> > > +	if (marker & PTE_MARKER_UFFD_WP)
> > 
> > Can we make this check `marker == PTE_MARKER_UFFD_WP`? There is currently only
> > one user of pte markers, and from what I can tell pte_marker_handle_uffd_wp()
> > wouldn't do the correct thing if other users were added because it could clear
> > non-uffd-wp markers. I don't think it's worth making it do the right thing now,
> > but a comment noting that would be helpful.
> 
> Sure thing, and yeah I agree it's trivial and shouldn't matter in real-life.
> 
> I'll change it to "marker == PTE_MARKER_UFFD_WP" as you suggested, so if
> there's surprise we'll get a sigbus.
> 
> Thanks,
> 
> > 
> > > +		return pte_marker_handle_uffd_wp(vmf);
> > > +
> > > +	/* This is an unknown pte marker */
> > > +	return VM_FAULT_SIGBUS;
> > >  }
> 
>