Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.112.34 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.112.34; helo=mail.nvidia.com;
Subject: Re: [PATCH v9 07/10] mm: Device exclusive memory access
To:     Andrew Morton <akpm@linux-foundation.org>,
        Alistair Popple <apopple@nvidia.com>
CC:     <linux-mm@kvack.org>, <nouveau@lists.freedesktop.org>,
        <bskeggs@redhat.com>, <rcampbell@nvidia.com>,
        <linux-doc@vger.kernel.org>, <bsingharora@gmail.com>,
        <linux-kernel@vger.kernel.org>, <dri-devel@lists.freedesktop.org>,
        <hch@infradead.org>, <jglisse@redhat.com>, <willy@infradead.org>,
        <jgg@nvidia.com>, <peterx@redhat.com>, <hughd@google.com>,
        Christoph Hellwig <hch@lst.de>
References: <20210524132725.12697-1-apopple@nvidia.com>
 <20210524132725.12697-8-apopple@nvidia.com>
 <20210524151157.2dc5d2bb510ff86dc449bf0c@linux-foundation.org>
From:   John Hubbard <jhubbard@nvidia.com>
Message-ID: <ccedcc53-ead0-a482-1ef0-3702cc82faef@nvidia.com>
Date:   Mon, 24 May 2021 18:31:17 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.10.2
MIME-Version: 1.0
In-Reply-To: <20210524151157.2dc5d2bb510ff86dc449bf0c@linux-foundation.org>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2021 01:31:32.2315
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 68d0f4be-31ec-4eb6-cf32-08d91f1cd3fe
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT030.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3008
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 5/24/21 3:11 PM, Andrew Morton wrote:
>> ...
>>
>>   Documentation/vm/hmm.rst     |  17 ++++
>>   include/linux/mmu_notifier.h |   6 ++
>>   include/linux/rmap.h         |   4 +
>>   include/linux/swap.h         |   7 +-
>>   include/linux/swapops.h      |  44 ++++++++-
>>   mm/hmm.c                     |   5 +
>>   mm/memory.c                  | 128 +++++++++++++++++++++++-
>>   mm/mprotect.c                |   8 ++
>>   mm/page_vma_mapped.c         |   9 +-
>>   mm/rmap.c                    | 186 +++++++++++++++++++++++++++++++++++
>>   10 files changed, 405 insertions(+), 9 deletions(-)
>>
> 
> This is quite a lot of code added to core MM for a single driver.
> 
> Is there any expectation that other drivers will use this code?

Yes! This should work for GPUs (and potentially, other devices) that support
OpenCL SVM atomic accesses on the device. I haven't looked into how amdgpu
works in any detail, but that's certainly at the top of the list of likely
additional callers.

> 
> Is there a way of reducing the impact (code size, at least) for systems
> which don't need this code?

I'll leave this question to others for the moment, in order to answer
the "do we need it at all" points.

> 
> How beneficial is this code to nouveau users?  I see that it permits a
> part of OpenCL to be implemented, but how useful/important is this in
> the real world?
> 

So this is interesting. Right now, OpenCL support in Nouveau is rather new
and so probably not a huge impact yet. However, we've built up enough experience
with CUDA and OpenCL to learn that atomic operations, as part of the user
space programming model, are a super big deal. Atomic operations are so
useful and important that I'd expect many OpenCL SVM users to be uninterested in
programming models that lack atomic operations for GPU compute programs.

Again, this doesn't rule out future, non-GPU accelerator devices that may
come along.

Atomic ops are just a really important piece of high-end multi-threaded
programming, it turns out. So this is the beginning of support for an
important building block for general purpose programming on devices that
have GPU-like memory models.


thanks,
-- 
John Hubbard
NVIDIA