Message-ID: <0d96835d-bcf4-1192-536c-0af314405880@redhat.com>
Date:   Fri, 17 Dec 2021 22:20:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.0
Subject: Re: [PATCH v1 06/11] mm: support GUP-triggered unsharing via
 FAULT_FLAG_UNSHARE (!hugetlb)
Content-Language: en-US
To:     Nadav Amit <namit@vmware.com>, Jason Gunthorpe <jgg@nvidia.com>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>,
        David Rientjes <rientjes@google.com>,
        Shakeel Butt <shakeelb@google.com>,
        John Hubbard <jhubbard@nvidia.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Mike Rapoport <rppt@linux.ibm.com>,
        Yang Shi <shy828301@gmail.com>,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
        Matthew Wilcox <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>, Jann Horn <jannh@google.com>,
        Michal Hocko <mhocko@kernel.org>,
        Rik van Riel <riel@surriel.com>,
        Roman Gushchin <guro@fb.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Peter Xu <peterx@redhat.com>,
        Donald Dutile <ddutile@redhat.com>,
        Christoph Hellwig <hch@lst.de>,
        Oleg Nesterov <oleg@redhat.com>, Jan Kara <jack@suse.cz>,
        Linux-MM <linux-mm@kvack.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        "open list:DOCUMENTATION" <linux-doc@vger.kernel.org>
References: <20211217113049.23850-1-david@redhat.com>
 <20211217113049.23850-7-david@redhat.com>
 <CAHk-=wgL5u3XMgfUN6BOqVO0OvPx3-LEri1ju-1TW4dFhHQO4g@mail.gmail.com>
 <CAHk-=wgKft6E_EeLA1GnEXcQBA9vu8m2B-M-U7PuiNa0+9gpHA@mail.gmail.com>
 <54c492d7-ddcd-dcd0-7209-efb2847adf7c@redhat.com>
 <CAHk-=wgjOsHAXttQa=csLG10Cp2hh8Dk8CnNC3_WDpBpTzBESQ@mail.gmail.com>
 <20211217204705.GF6385@nvidia.com>
 <2E28C79D-F79C-45BE-A16C-43678AD165E9@vmware.com>
From:   David Hildenbrand <david@redhat.com>
Organization: Red Hat
In-Reply-To: <2E28C79D-F79C-45BE-A16C-43678AD165E9@vmware.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 17.12.21 22:15, Nadav Amit wrote:
> 
> 
>> On Dec 17, 2021, at 12:47 PM, Jason Gunthorpe <jgg@nvidia.com> wrote:
>>
>> On Fri, Dec 17, 2021 at 12:36:43PM -0800, Linus Torvalds wrote:
>>
>>>> 5. Take a R/O pin (RDMA, VFIO, ...)
>>>> -> refcount > 1
>>>>
>>>> 6. memset(mem, 0xff, pagesize);
>>>> -> Write fault -> COW
>>>
>>> I do not believe this is actually a bug.
>>>
>>> You asked for a R/O pin, and you got one.
>>>
>>> Then somebody else modified that page, and you got exactly what you
>>> asked for - a COW event. The original R/O pin has the original page
>>> that it asked for, and can read it just fine.
>>
>> To remind all, the GUP users, like RDMA, VFIO use
>> FOLL_FORCE|FOLL_WRITE to get a 'r/o pin' specifically because of the
>> COW breaking the coherence. In these case 'r/o pin' does not mean
>> "snapshot the data", but its only a promise not to write to the pages
>> and still desires coherence with the memory map.
>>
>> Eg in RDMA we know of apps asking for a R/O pin of something in .bss
>> then filling that something with data finally doing the actual
>> DMA. Breaking COW after pin breaks those apps.
>>
>> The above #5 can occur for O_DIRECT read and in that case the
>> 'snapshot the data' is perfectly fine as racing the COW with the
>> O_DIRECT read just resolves the race toward the read() direction.
>>
>> IIRC there is some other scenario that motivated this patch?
> 
> I think that there is an assumption that once a page is COW-broken,
> it would never have another write-fault that might lead to COW
> breaking later.
> 
> AFAIK at least after userfaultfd-WP followed by
> userfaultfd-write-unprotect a page might be write-protected and
> go through do_wp_page() a second time to be COW-broken again. In
> such case, I think the FOLL_FORCE|FOLL_WRITE would not help.
> 
> I suspect (not sure) that this might even happen with mprotect()
> since I do not see all code-paths preserving whether the page
> was writable.
> 

uffd-wp and mprotect() are broken as well, yes. But the easiest example
is just swap + read fault.

Section 2 and 3 in [1], along with reproducers.

Note that I didn't mention uffd-wp and mprotect(), because these require
"manual intervention". With swap, it's not your application doing
something "special".

[1]
https://lore.kernel.org/r/3ae33b08-d9ef-f846-56fb-645e3b9b4c66@redhat.com

-- 
Thanks,

David / dhildenb