Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 22 Sep 2020 13:10:46 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Peter Xu <peterx@redhat.com>
Cc:     John Hubbard <jhubbard@nvidia.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Jan Kara <jack@suse.cz>, Michal Hocko <mhocko@suse.com>,
        Kirill Tkhai <ktkhai@virtuozzo.com>,
        Kirill Shutemov <kirill@shutemov.name>,
        Hugh Dickins <hughd@google.com>,
        Christoph Hellwig <hch@lst.de>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Oleg Nesterov <oleg@redhat.com>,
        Leon Romanovsky <leonro@nvidia.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Jann Horn <jannh@google.com>
Subject: Re: [PATCH 1/5] mm: Introduce mm_struct.has_pinned
Message-ID: <20200922161046.GB731578@ziepe.ca>
References: <20200921211744.24758-1-peterx@redhat.com>
 <20200921211744.24758-2-peterx@redhat.com>
 <224908c1-5d0f-8e01-baa9-94ec2374971f@nvidia.com>
 <20200922151736.GD19098@xz-x1>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200922151736.GD19098@xz-x1>
Precedence: bulk

On Tue, Sep 22, 2020 at 11:17:36AM -0400, Peter Xu wrote:

> > But it's admittedly a cosmetic point, combined with my perennial fear that
> > I'm missing something when I look at a READ_ONCE()/WRITE_ONCE() pair. :)
> 
> Yeah but I hope I'm using it right.. :) I used READ_ONCE/WRITE_ONCE explicitly
> because I think they're cheaper than atomic operations, (which will, iiuc, lock
> the bus).

It is worth thinking a bit about racing fork with
pin_user_pages(). The desired outcome is:

  If fork wins the page is write protected, and pin_user_pages_fast()
  will COW it.

  If pin_user_pages_fast() wins then fork must see the READ_ONCE and
  the pin.

As get_user_pages_fast() is lockless it looks like the ordering has to
be like this:

  pin_user_pages_fast()                   fork()
   atomic_set(has_pinned, 1);
   [..]
   atomic_add(page->_refcount)
   ordered check write protect()
                                          ordered set write protect()
                                          atomic_read(page->_refcount)
                                          atomic_read(has_pinned)

Such that in all the degenerate racy cases the outcome is that both
sides COW, never neither.

Thus I think it does have to be atomics purely from an ordering
perspective, observing an increased _refcount requires that has_pinned
!= 0 if we are pinning.

So, to make this 100% this ordering will need to be touched up too.

Jason