Received: by 2002:a25:ab43:0:0:0:0:0 with SMTP id u61csp8554901ybi; Thu, 6 Jun 2019 14:37:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqzI6bJ1lWBXBMNw9Ave5ry6troaTBd/Xz6Q0UkKzaMYCpf07ohUnBzEsULEbxzSIifQ+Ha+ X-Received: by 2002:a63:5024:: with SMTP id e36mr603988pgb.220.1559857060636; Thu, 06 Jun 2019 14:37:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559857060; cv=none; d=google.com; s=arc-20160816; b=Vrz87qtD5puLYcFVblw+c+akoid+gV2gcdm1DgoBFkjuQgA+vxxqCcXQgCSnOOM4qV ypl1UV2trhdQhZi339bcpmwZUupY/p6Ma6KqFXUpWEXEXQy+E02JvQudzBkvMo9KUj1G /FPEAkcLUIweWvBWiT6aERkZ2oYgWWrH+iUp9KafKEtBIE6dDD5H7klmPOSetsEzwrlk 2fYYzqGcj9wku6+mMh9dd8hMsmqJa9GJXJXEcjBT6EvCd+6c7xadrHlYXhQmZ2UjwkIl vM5kuFCNshXxBMe4omKWlCvdye8pfOtM+sYq8rIOtD2xwhHIMh7hXEP/jQZSg83bZ0zk i19g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:dkim-signature:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=lnWmoYy7kAn4bLm7t9cFbOMXhWsLoa4EfFFlm+lwmig=; b=svJGlIa2orxgGLAESzeTOVKDVCISx1cb3AfFss9utaL1he8NOcQHN367YXcF9lWLud SbV3XvvjvCQJJ1rW1fnOThpQCp9MatYoekNvVJoBTJNGJ5L1EfUSXxDjUJ2Jofe443kq a23zhfpRNF/Pzfpuyo7+/Be24TChNzTncDJGTqzb1a9hlkfRjl1JDJxmIUdhjfia9pWh L3gQGYNDNNCmR50bwJ5qGoj/3ldVsvcHcdwneqHocDz4zIAZD8Q00E67uvFglWofXNiw 2GGXHKnXXtyRfPdeBPjf7I8rQZ5fncAOk/WhEBit53wwZPB05rT68VkY9H9qmklaI6Ac XFqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b="UDAQv/GI"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k193si174241pgc.473.2019.06.06.14.37.25; Thu, 06 Jun 2019 14:37:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b="UDAQv/GI"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728305AbfFFSuS (ORCPT + 99 others); Thu, 6 Jun 2019 14:50:18 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:13057 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727163AbfFFSuR (ORCPT ); Thu, 6 Jun 2019 14:50:17 -0400 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Thu, 06 Jun 2019 11:50:01 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Thu, 06 Jun 2019 11:50:15 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Thu, 06 Jun 2019 11:50:15 -0700 Received: from rcampbell-dev.nvidia.com (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 6 Jun 2019 18:50:15 +0000 Subject: Re: [PATCH 1/5] mm/hmm: Update HMM documentation To: Jason Gunthorpe CC: , , John Hubbard , Ira Weiny , Dan Williams , Arnd Bergmann , Balbir Singh , Dan Carpenter , "Matthew Wilcox" , Souptick Joarder , "Andrew Morton" , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= References: <20190506232942.12623-1-rcampbell@nvidia.com> <20190506232942.12623-2-rcampbell@nvidia.com> <20190606140239.GA21778@ziepe.ca> X-Nvconfidentiality: public From: Ralph Campbell Message-ID: Date: Thu, 6 Jun 2019 11:50:15 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.0 MIME-Version: 1.0 In-Reply-To: <20190606140239.GA21778@ziepe.ca> X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL108.nvidia.com (172.18.146.13) To HQMAIL107.nvidia.com (172.20.187.13) Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1559847001; bh=lnWmoYy7kAn4bLm7t9cFbOMXhWsLoa4EfFFlm+lwmig=; h=X-PGP-Universal:Subject:To:CC:References:X-Nvconfidentiality:From: Message-ID:Date:User-Agent:MIME-Version:In-Reply-To: X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=UDAQv/GI8zoEqSn7mrswFwJ98G2U/yQQD/Aj0+YO7Qf5+Xr8LIE45i/9Kf0DT9xG3 JPPWSe0ZAVdjMHx+2C0ISnDMM3NkNkQqa5NXSEs50jcCB4GiVbS7uUaIN4pcELz3g+ MfSt7NTboICB1mkUbEx3mzD15NkILVLfyY7W7hCxGsr/Vew7TUf7Kead9IDlmuVFZ6 0g3htFBOh79b75YZGU+BAEjA89isoq7X+Uy0Kb8zWa5u3BVH/ZZ7h3FAU9N5uKCy4N M3lK8+o+d2kh73ZkQKQwz0Fm+222atCS0/SR5UcSUSCqkOPa9W54dbdognW0U2lnZN 46ZaurvMxolKw== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 6/6/19 7:02 AM, Jason Gunthorpe wrote: > On Mon, May 06, 2019 at 04:29:38PM -0700, rcampbell@nvidia.com wrote: >> From: Ralph Campbell >> >> Update the HMM documentation to reflect the latest API and make a few mi= nor >> wording changes. >> >> Signed-off-by: Ralph Campbell >> Reviewed-by: J=C3=A9r=C3=B4me Glisse >> Cc: John Hubbard >> Cc: Ira Weiny >> Cc: Dan Williams >> Cc: Arnd Bergmann >> Cc: Balbir Singh >> Cc: Dan Carpenter >> Cc: Matthew Wilcox >> Cc: Souptick Joarder >> Cc: Andrew Morton >> Documentation/vm/hmm.rst | 139 ++++++++++++++++++++------------------- >> 1 file changed, 73 insertions(+), 66 deletions(-) >=20 > Okay, lets start picking up hmm patches in to the new shared hmm.git, > as promised I will take responsibility to send these to Linus. The > tree is here: >=20 > https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git/log/?h=3Dhm= m >=20 > This looks fine to me with one minor comment: >=20 >> diff --git a/Documentation/vm/hmm.rst b/Documentation/vm/hmm.rst >> index ec1efa32af3c..7c1e929931a0 100644 >> +++ b/Documentation/vm/hmm.rst >> =20 >> @@ -151,21 +151,27 @@ registration of an hmm_mirror struct:: >> =20 >> int hmm_mirror_register(struct hmm_mirror *mirror, >> struct mm_struct *mm); >> - int hmm_mirror_register_locked(struct hmm_mirror *mirror, >> - struct mm_struct *mm); >> =20 >> - >> -The locked variant is to be used when the driver is already holding mma= p_sem >> -of the mm in write mode. The mirror struct has a set of callbacks that = are used >> +The mirror struct has a set of callbacks that are used >> to propagate CPU page tables:: >> =20 >> struct hmm_mirror_ops { >> + /* release() - release hmm_mirror >> + * >> + * @mirror: pointer to struct hmm_mirror >> + * >> + * This is called when the mm_struct is being released. >> + * The callback should make sure no references to the mirror occur >> + * after the callback returns. >> + */ >=20 > This is not quite accurate (at least, as the other series I sent > intends), the struct hmm_mirror is valid up until > hmm_mirror_unregister() is called - specifically it remains valid > after the release() callback. >=20 > I will revise it (and the hmm.h comment it came from) to read the > below. Please let me know if you'd like something else: >=20 > /* release() - release hmm_mirror > * > * @mirror: pointer to struct hmm_mirror > * > * This is called when the mm_struct is being released. The callback > * must ensure that all access to any pages obtained from this mirror > * is halted before the callback returns. All future access should > * fault. > */ >=20 > The key task for release is to fence off all device access to any > related pages as the mm is about to recycle them and the device must > not cause a use-after-free. >=20 > I applied it to hmm.git >=20 > Thanks, > Jason >=20 Yes, I agree this is better. Also, I noticed the sample code for hmm_range_register() is wrong. If you could merge this minor change into this patch, that would be appreciated. diff --git a/Documentation/vm/hmm.rst b/Documentation/vm/hmm.rst index dc8fe4241a18..b5fb9bc02aa2 100644 --- a/Documentation/vm/hmm.rst +++ b/Documentation/vm/hmm.rst @@ -245,7 +245,7 @@ The usage pattern is:: hmm_range_wait_until_valid(&range, TIMEOUT_IN_MSEC); goto again; } - hmm_mirror_unregister(&range); + hmm_range_unregister(&range); return ret; } take_lock(driver->update); @@ -257,7 +257,7 @@ The usage pattern is:: // Use pfns array content to update device page table - hmm_mirror_unregister(&range); + hmm_range_unregister(&range); release_lock(driver->update); up_read(&mm->mmap_sem); return 0;