Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20200917112538.GD8409@ziepe.ca> <CAHk-=wjtfjB3TqTFRzVmOrB9Mii6Yzc-=wKq0fu4ruDE6AsJgg@mail.gmail.com>
 <20200917193824.GL8409@ziepe.ca> <CAHk-=wiY_g+SSjncZi8sO=LrxXmMox0NO7K34-Fs653XVXheGg@mail.gmail.com>
 <20200918164032.GA5962@xz-x1> <20200918173240.GY8409@ziepe.ca>
 <20200918204048.GC5962@xz-x1> <0af8c77e-ff60-cada-7d22-c7cfcf859b19@nvidia.com>
 <20200919000153.GZ8409@ziepe.ca> <20200921083505.GA5862@quack2.suse.cz>
 <20200921120301.GD8409@ziepe.ca> <CAFCwf10eNmwq2wD71xjUhqkvv5+_pJMR1nPug2RqNDcFT4H86Q@mail.gmail.com>
 <CAHk-=wgLk5xWj8v6mQXaWL0zWAf2cis8EpPkQUuEZ7K3LxVQHQ@mail.gmail.com>
In-Reply-To: <CAHk-=wgLk5xWj8v6mQXaWL0zWAf2cis8EpPkQUuEZ7K3LxVQHQ@mail.gmail.com>
From:   Oded Gabbay <ogabbay@kernel.org>
Date:   Wed, 16 Feb 2022 21:20:49 +0200
Message-ID: <CAFCwf10WEwrZzD7nWkyaZYnnYtZs0aWYwBniEPrb_xdgiQiggw@mail.gmail.com>
Subject: Re: [PATCH 1/4] mm: Trial do_wp_page() simplification
To:     Linus Torvalds <torvalds@linux-foundation.org>
Cc:     David Hildenbrand <david@redhat.com>,
        Jason Gunthorpe <jgg@ziepe.ca>, Jan Kara <jack@suse.cz>,
        John Hubbard <jhubbard@nvidia.com>,
        Leon Romanovsky <leonro@nvidia.com>,
        Linux-MM <linux-mm@kvack.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        "Maya B . Gokhale" <gokhale2@llnl.gov>,
        Yang Shi <yang.shi@linux.alibaba.com>,
        Marty Mcfadden <mcfadden8@llnl.gov>,
        Kirill Shutemov <kirill@shutemov.name>,
        Oleg Nesterov <oleg@redhat.com>, Jann Horn <jannh@google.com>,
        Kirill Tkhai <ktkhai@virtuozzo.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Christoph Hellwig <hch@lst.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        Daniel Vetter <daniel.vetter@ffwll.ch>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Peter Xu <peterx@redhat.com>, dave@sr71.net
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Feb 16, 2022 at 9:04 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> [ Added David Hildenbrand to the participants. David, see
>
>     https://bugzilla.kernel.org/show_bug.cgi?id=215616
>
>   for details ]
>
> On Wed, Feb 16, 2022 at 8:59 AM Oded Gabbay <oded.gabbay@gmail.com> wrote:
> >
> > All the details are in the bug, but the bottom line is that somehow,
> > this patch causes corruption when the numa balancing feature is
> > enabled AND we don't use process affinity AND we use GUP to pin pages
> > so our accelerator can DMA to/from system memory.
>
> Hmm. I thought all the remaining issues were related to THP - and
> David Hildenbrand had a series to fix those up.
>
> The fact that it also shows up with numa balancing is a bit
> unfortunate, because I think that means that that patch series may not
> have caught that case.
>
> That said - what does "we use GUP to pin pages" mean? Does it actually
> use the pinning logic, or just regular old GUP?
>
> I'm assuming this is just the existing pin_user_pages_fast() (ie a
> proper pin) in drivers/misc/habanalabs/common/memory.c. But I wanted
> to confirm that it's not some other situation.
>
>                    Linus
>
[Adding dave hansen as we chatted about it in irc]

It uses the pinning logic, simply calling pin_user_pages_fast with the
relevant gup flags to pin the userspace memory so we can after that
dma map it and give the bus address to the h/w.

And afaik, the gup flags combination we use
(FOLL_FORCE | FOLL_WRITE | FOLL_LONGTERM) is the correct combination,
at least according to the last time it was discussed with Daniel,
Jason and other people.

Oded